A community publication
From kernels to clusters.
An open community for learning, writing, and tinkering on the infrastructure behind modern AI: inference engines, training systems, ML stacks, and everything in between.
A schematic inference step: model operations become scheduled kernels, the GPU executes them, and the resulting scores are used to sample the next token.
Latest article Architecture
A from-scratch tour of what's actually inside an LLM: how a transformer turns tokens into predictions, what Query, Key, and Value really mean, and how generating text one token at a time builds the KV cache — the growing pool of memory that makes inference so expensive.
Read the articleTraining Systems 9 min read
Training Systems 12 min read
Architecture 14 min read
Trainers, optimizers, recipes, debugging large runs.
Transformers, MoE, SSMs, hybrids, and what's next.
More threads to explore
Explore a weight-bandwidth ceiling and a fixed KV-cache memory example across precisions, batch sizes, and GPUs.
Turn a model size and token budget into training FLOPs, GPU-hours, wall-clock days, and cost, using the 6ND rule and real GPU peak numbers.
Per-GPU VRAM breakdown for training and inference — params, gradients, optimizer state, activations, KV cache, with ZeRO/FSDP sharding.
Explore a synthetic attention map illustrating causal masking and recurring patterns.
More to try
Knowledge grows when it's shared.
A useful explanation. A hard-won lesson. A question worth exploring. If you're learning or building ML systems, there's a place for your perspective here.
One email per publication. No spam, unsubscribe anytime.