Training Linear Attention Models on AWS: i14's Laplace 3B Case Study

i14 cracked the memory puzzle with Laplace 3B, a linear attention model that keeps its recurrent state at a fixed 29.5 MB per sequence—no bloated KV cache needed. They trained and served it on AWS Trainium2 using native PyTorch, making inference memory predictable and efficient regardless of context length.