Machine Learning

2026-05-02KV Cache and PagedAttention: Why vLLM Actually WorksKV cache stores attention keys and values across tokens to avoid recomputation during autoregressive decoding. PagedAttention extends this with virtual memory paging, making vLLM's throughput possible.→2026-05-01LLM Inference Optimization: Stop Paying for Idle SiliconLLM inference is memory-bandwidth bound, not compute bound. Continuous batching, speculative decoding, quantization, and KV cache management — the techniques that actually move the needle.→2026-01-08Eigenvalues of a MatrixEigenvalues and eigenvectors — the linear algebra behind PCA, Google's PageRank, and the stability analysis of dynamical systems. Geometric intuition and computation.→2025-09-12 Parameters in LLMsHow to count parameters in a transformer LLM — attention heads, MLP layers, embeddings, and normalization. Includes a worked example matching published model sizes.→2025-09-06Matrix-Vector Dot ProductMatrix-vector dot products are the inner loop of neural network inference. Understanding the memory access patterns and FLOP counts explains why hardware utilisation matters so much.→