← 🖥️ ML Infrastructure & ServingNEXT IN ML INFRASTRUCTURE & SERVINGPagedAttention→
Core
FlashAttention and IO-Aware Kernels
Naive attention is slow not because of the matmuls but because it writes the full N-by-N attention matrix to GPU high-bandwidth memory and reads it back, which is memory-bandwidth bound. FlashAttention fuses the whole attention computation into one kernel that tiles the inputs in fast on-chip SRAM and never materializes the full matrix, using an online-softmax trick to stay exact. Applied-AI interviews probe it because it is why long-context training and serving became affordable and a clean test of GPU memory-hierarchy reasoning.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
ML Infrastructure & GPUsWhy is standard attention memory-bound, and how does FlashAttention fix it without changing the math?→LLM & GenAI FundamentalsWhy is an LLM's context window limited, and how do models extend it?→ML Infrastructure & GPUsYour LLM decode is slow even though GPU compute utilization looks low. Why is it memory-bandwidth-bound?→ML Infrastructure & GPUsYour activations for one long sequence no longer fit on a GPU. Explain context parallelism and ring attention.→LLM & GenAI FundamentalsWhy do transformers scale attention scores by 1/√d_k, and what breaks if you skip it?→LLM & GenAI FundamentalsEncoder-only vs decoder-only vs encoder-decoder: when do you use each, and why are decoder-only models dominant?→
COMPANIES THAT ASSUME THIS