← 🧠 Foundations of LLMs & GenAINEXT IN FOUNDATIONS OF LLMS & GENAIAttention and Self-Attention→
Core
The Transformer Architecture
The transformer is the architecture behind modern LLMs: stacked blocks that each mix information across tokens with self-attention and then transform each token with a feed-forward network, wrapped in residual connections and normalization. Understanding the two sub-layers (attention mixes across tokens, the feed-forward processes each one) explains where parameters live, why Mixture-of-Experts scales the feed-forward, and why decoder-only models dominate. Applied-AI interviews probe it because it is the mental scaffold for everything else, attention cost, KV cache, MoE, and serving.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
LLM & GenAI FundamentalsWhy do transformers scale attention scores by 1/√d_k, and what breaks if you skip it?→LLM & GenAI FundamentalsEncoder-only vs decoder-only vs encoder-decoder: when do you use each, and why are decoder-only models dominant?→Coding & DSAImplement multi-head self-attention from scratch in NumPy, with a causal mask.→LLM & GenAI FundamentalsWhat is cross-attention, and how does it differ from self-attention?→LLM & GenAI FundamentalsWhat do the feed-forward (MLP) layers in a transformer do, and why are they most of the parameters?→Coding & DSAImplement multi-head attention from scratch.→
COMPANIES THAT ASSUME THIS