AppliedAIPrep logoAppliedAI/Prep

AI glossary, in plain English

214 terms an Applied AI engineer is expected to use correctly in an interview, each defined in a sentence. Search by the abbreviation you would actually say: DP, RAG, cross-encoder, GRPO. Every term links to the concept page that explains it properly, with the trade-off interviewers probe.

These are definitions, not explainers. If you want the intuition, the worked example and the failure mode, follow the term through to its concept page, or see how the whole library fits together on the map.

ABCDEFGHIJKLMNOPQRSTUVW214 shown

A

A/B TestingEvaluation & ML FoundationsFREE
An A/B test randomly splits users between a control and a variant and compares a metric to measure causal impact.
Also called: A/B test, experimentation, controlled experiment, split test
Activation Functions: ReLU, GELU, SwiGLUEvaluation & ML FoundationsFREE
Activation functions are the nonlinearity between linear layers; without them a deep network collapses into a single linear map no matter how many layers it has.
Also called: activation functions, ReLU, GELU, SwiGLU, nonlinearity
Agent Design Patterns: ReAct, Plan-and-Execute, ReflectionRetrieval & Agents
These are the named control-flow architectures for LLM agents: ReAct interleaves reasoning and actions in a tight loop, plan-and-execute decomposes the task up front and then runs the steps, and reflection adds a self-critique pass that revises output.
Also called: ReAct, plan and execute, plan-and-execute, reflection agent, self-critique
Agent Evaluation and Trajectory AnalysisRetrieval & Agents
Agent evaluation scores the full execution trace (tool calls, observations, state changes, recovery) rather than only the final answer, because a correct answer can hide a broken process and a wrong answer can come from one bad step in an otherwise sound run.
Also called: agent eval, trajectory evaluation, trajectory analysis, agent trajectory eval, evaluating agents
Agent GuardrailsAI Security, Privacy & Governance
An agent that can take actions is far riskier than one that only talks, so guardrails must constrain actions, not just text.
Also called: least privilege, human-in-the-loop, blast radius, agent safety
Agent Memory: Short-Term, Long-Term, and Memory StoresRetrieval & Agents
Agent memory is how an agent retains and recalls information across steps and sessions.
Also called: agent memory, short-term memory, long-term memory, working memory, episodic memory
Agent Reliability and Long-Horizon RobustnessRetrieval & Agents
Long-horizon agents fail because per-step success compounds: a 95 percent reliable step is only about 60 percent reliable over ten steps.
Also called: agent reliability, long-horizon robustness, agent robustness, consistent completion, cascading failure
Agent Security: Tool Poisoning, Memory Poisoning, ContainmentAI Security, Privacy & Governance
Agent security covers threats that only exist once an LLM can call tools and act on their results: malicious tool or MCP responses, poisoned long-term memory, privilege escalation through tool misuse, and goal hijacking.
Also called: agent security, tool poisoning, memory poisoning, agent containment, tool misuse
Agentic and Corrective RAGRetrieval & Agents
Standard RAG retrieves once and generates; agentic RAG wraps retrieval in a loop so the model decides whether to retrieve, what to query, evaluates the results, and retrieves again until it has what it needs.
Also called: agentic RAG, corrective RAG, CRAG, self-RAG, adaptive retrieval
Agents and Tool UseRetrieval & AgentsFREE
An agent is an LLM in a loop that can take actions through tools: it reasons, calls a tool (search, a database, code, an API), observes the result, and repeats until done.
Also called: agent, agents, tool use, function calling, ReAct
AI Governance FrameworksAI Security, Privacy & Governance
AI governance is the program that makes deployments safe, fair, and compliant: risk assessment, documentation (model cards, datasheets), human oversight, monitoring, and incident response, structured by frameworks like the NIST AI Risk Management Framework and laws like the EU AI Act (which tiers obligations by risk).
Also called: AI governance, NIST AI RMF, EU AI Act, governance framework, responsible AI
Arrays and HashingCoding & Engineering CraftFREE
The hash map is the workhorse of coding interviews: average O(1) insert and lookup that turns an O(n^2) all-pairs scan into a single O(n) pass.
Also called: hash map, hash table, dictionary, two sum, seen set
Attention and Self-AttentionFoundations of LLMs & GenAI
Attention turns each token into a query, key, and value, scores every query against every key, softmaxes those scores into weights, and returns the weighted sum of values, so each token pulls in information from the others.
Also called: attention, self-attention, multi-head attention, query key value
Attention Variants: MHA, MQA, and GQAFoundations of LLMs & GenAI
Multi-head attention gives every query head its own key and value heads, which is expressive but makes the KV cache large and memory-bandwidth hungry at decode time.
Also called: MHA, MQA, GQA, multi-query attention, grouped-query attention
Audit TrailsAI Security, Privacy & GovernanceFREE
An audit trail logs enough to reconstruct and explain any AI decision: the input, retrieved context, model and prompt version, output, and who/when, plus human overrides and guardrail events.
Also called: audit logging, traceability, audit log
Autoencoders and GANsEvaluation & ML Foundations
Two foundational generative architectures: autoencoders compress input through a bottleneck and reconstruct it, which makes them useful for denoising, anomaly detection, and learning compact representations, while GANs pit a generator against a discriminator in an adversarial game to produce realistic samples.
Also called: autoencoder, denoising autoencoder, variational autoencoder, VAE, GAN

B

Backfills and ReprocessingData & SQL Engineering
A backfill recomputes historical data after a bug fix, a new column, or a logic change, and it is where fragile pipelines break.
Also called: backfill, backfilling, reprocessing history, recompute history, historical reprocessing
Backpropagation, IntuitivelyEvaluation & ML FoundationsFREE
Backpropagation is the algorithm that computes the gradient of the loss with respect to every parameter in a network, by applying the chain rule in reverse from the output back to the inputs.
Also called: backpropagation, backprop, reverse-mode autodiff, automatic differentiation, chain rule
BacktrackingCoding & Engineering Craft
Backtracking is systematic search over a tree of partial solutions: at each step you choose an option, explore deeper, and undo the choice before trying the next (choose, explore, unchoose).
Also called: backtrack, systematic search, constraint search, prune and backtrack
Batch vs StreamingData & SQL Engineering
Batch processes a bounded dataset on a schedule; streaming processes an unbounded flow of events continuously.
Also called: batch processing, stream processing, streaming, event time, processing time
Benchmarks and Their LimitsEvaluation & ML Foundations
Public benchmarks like MMLU give a shared yardstick, but they saturate, leak into training corpora, and stop tracking real ability once labs optimize for them.
Also called: benchmarks, benchmark contamination, leaderboard, MMLU, eval saturation
Bit ManipulationCoding & Engineering Craft
Bit manipulation uses AND, OR, XOR, and shifts to pack flags into integers, test and toggle individual bits, and exploit tricks like XOR-cancellation to find a unique element in O(1) space.
Also called: bitwise operations, bitmask, XOR trick, bit tricks

C

Caching StrategiesSystem Design for AI in ProductionFREE
A cache trades freshness for speed by keeping a copy of hot data closer to the request.
Also called: caching, cache, cache-aside, write-through, write-back
Calibration and UncertaintyEvaluation & ML Foundations
A model is calibrated if its confidence matches reality: among predictions it makes at 0.8, about 80% are correct.
Also called: calibration, calibrated probabilities, temperature scaling, Platt scaling, isotonic regression
CAP and Consistency ModelsSystem Design for AI in Production
The CAP theorem says that during a network partition a distributed system must choose between consistency and availability; you cannot have both while the network is split.
Also called: CAP, CAP theorem, PACELC, linearizability, eventual consistency
Catastrophic Forgetting and Continual LearningEvaluation & ML Foundations
Catastrophic forgetting is when training a neural network on new data erodes capabilities it already had, because gradient updates overwrite the weights that encoded old skills.
Also called: catastrophic forgetting, continual learning, lifelong learning, elastic weight consolidation, EWC
Causal Inference: Confounders and IdentificationEvaluation & ML Foundations
Causal inference is the discipline of estimating what would happen if you intervened, not just what correlates in observed data.
Also called: causal inference, confounding, Simpson's paradox, difference-in-differences, instrumental variables
Chain-of-Thought and In-Context LearningFoundations of LLMs & GenAIFREE
In-context learning is the ability to perform a task from instructions or a few examples in the prompt, with no weight updates.
Also called: chain-of-thought, chain of thought, in-context learning, few-shot learning, self-consistency
Change Data CaptureData & SQL Engineering
Change Data Capture (CDC) streams the inserts, updates, and deletes from a source database so downstream systems stay in sync without expensive full reloads.
Also called: CDC, change stream, incremental load
Choosing and Adapting Embedding ModelsRetrieval & Agents
Picking an embedding model is a decision about retrieval quality, cost, and operational risk on your data, not about who tops a public leaderboard.
Also called: embedding model selection, choosing embeddings, MTEB, embedding fine-tuning, re-embedding
ChunkingRetrieval & Agents
Chunking splits documents into the passages you embed and retrieve, and it is one of the highest-leverage knobs in RAG.
Also called: chunk size, parent-child retrieval, semantic chunking, chunk
CI/CD for ModelsMLOps & LifecycleFREE
Shipping a model safely needs more than software CI/CD because the model depends on data, not just code.
Also called: ML testing, model testing, continuous delivery for ML
Citations and GroundingRetrieval & AgentsFREE
Grounding means the model answers only from provided sources; citations make each claim traceable to the exact passage that supports it.
Also called: citations, grounding, attribution, source attribution, grounded generation
Classic NLP: Bag-of-Words, TF-IDF, and Word2VecFoundations of LLMs & GenAIFREE
Before learned embeddings, text was turned into sparse high-dimensional vectors with bag-of-words and TF-IDF, which count words and weight them by how distinctive they are but ignore meaning and order.
Also called: bag-of-words, BoW, TF-IDF, word2vec, GloVe
CLT, Sampling, and Confidence IntervalsEvaluation & ML Foundations
The central limit theorem says the mean of a sample is approximately normal regardless of the underlying distribution, which is why so much inference uses the normal curve.
Also called: CLT, central limit theorem, confidence interval, standard error, sampling distribution
Clustering: K-Means, Hierarchical, DBSCANEvaluation & ML FoundationsFREE
Clustering groups unlabeled points by similarity.
Also called: clustering, k-means, kmeans, DBSCAN, hierarchical clustering
CNNs: Convolution, Pooling, Receptive FieldsEvaluation & ML Foundations
Convolutional neural networks replace dense layers with small filters slid across an image, sharing weights so the same edge detector applies everywhere.
Also called: CNN, convolutional neural network, convolution, pooling, receptive field
Communicating with Non-Technical StakeholdersBehavioral & Project Deep-DivesFREE
Much of applied-AI work is explaining complex systems to non-technical people: executives, customers, domain experts.
Also called: stakeholder communication, explaining to non-technical, communicating AI, audience
Concurrency and Thread SafetySystem Design for AI in Production
When multiple threads touch shared mutable state, interleavings cause race conditions: lost updates, torn reads, corrupted data.
Also called: concurrency, thread safety, race condition, mutex, lock
Consistent Hashing and Sharding: Hash Ring, Virtual NodesSystem Design for AI in Production
Why hash(key) mod N reshuffles nearly every key when N changes, how a hash ring cuts that to roughly K/N, and what virtual nodes do for load balance.
Also called: consistent hashing, sharding, hash ring, virtual nodes, partitioning
Constitutional AI and RLAIFFoundations of LLMs & GenAI
RLAIF (RL from AI Feedback) replaces human preference labels with AI-generated ones, scaling alignment past the human-labeling bottleneck.
Also called: Constitutional AI, RLAIF, RL from AI feedback, AI feedback
Constrained and Structured DecodingFoundations of LLMs & GenAI
Asking a model nicely for JSON sometimes fails; constrained decoding guarantees valid output by masking, at each generation step, every token that would violate a schema or grammar, so only valid continuations can be sampled.
Also called: constrained decoding, structured decoding, JSON mode, constrained generation, grammar-constrained
Context Engineering for AgentsRetrieval & Agents
Context engineering is the discipline of designing the full information payload that goes into an agent's context window each turn: system instructions, memory, retrieved data, tool definitions and results, and conversation history.
Also called: context engineering, context management, context window management, agent context design
Context Rot and Long-Context Failure ModesFoundations of LLMs & GenAI
Context rot is the practical degradation of model quality as the input window fills up, even when the official window is a million tokens.
Also called: context rot, lost in the middle, long context degradation, attention sink, needle in a haystack
Continuous BatchingML Infrastructure & Serving
GPUs are efficient on batches, but LLM requests arrive at different times and finish after different numbers of tokens, so static batching wastes the GPU waiting for the slowest request.
Also called: in-flight batching, dynamic batching, batching
Contrastive and Metric LearningEvaluation & ML Foundations
Contrastive learning trains embeddings by comparison: pull similar (positive) pairs together and push dissimilar (negative) pairs apart, so distance encodes similarity.
Also called: contrastive learning, metric learning, triplet loss, InfoNCE, negatives
Cross-Validation (Done Right)Evaluation & ML FoundationsFREE
Cross-validation estimates how a model generalizes by training and testing on rotating folds, giving a more reliable estimate than a single split.
Also called: cross-validation, k-fold, stratified k-fold, time-series split, nested cross-validation
CTEs and SubqueriesData & SQL EngineeringFREE
A CTE (the WITH clause) names an intermediate result so a query reads as a top-to-bottom pipeline instead of nested subqueries.
Also called: CTE, common table expression, WITH clause, subquery, correlated subquery
CV Architectures: ResNets, ViT, DetectionEvaluation & ML Foundations
Modern computer vision rests on three pillars: residual connections that let CNNs go hundreds of layers deep without degrading, Vision Transformers that patchify an image and run self-attention instead of convolutions, and detection heads (one-stage vs two-stage) scored by mAP after non-maximum suppression.
Also called: computer vision architectures, ResNet, Vision Transformer, ViT, object detection

D

Data Quality and ContractsData & SQL EngineeringFREE
Models and analytics are only as good as their data, and a silent upstream data change (a renamed column, a units switch, a spike in nulls) corrupts everything downstream with no error.
Also called: data quality, data contracts, data validation, schema validation, data tests
Decision Trees and Splitting CriteriaEvaluation & ML FoundationsFREE
A decision tree recursively splits the feature space by picking the split that most reduces impurity (Gini or entropy), producing a flowchart you can read top to bottom.
Also called: decision tree, CART, splitting criteria, Gini impurity, information gain
Deduplication (Exact and Fuzzy)Data & SQL Engineering
Duplicates creep into data from retries, joins, and multiple sources, and they corrupt counts, training sets, and aggregates.
Also called: deduplication, dedup, near-duplicate, MinHash, LSH
Differential PrivacyAI Security, Privacy & Governance
Differential privacy adds calibrated noise to data, queries, or training so the output is provably insensitive to any single individual's record, bounding what can be learned about any one person.
Also called: DP, DP-SGD, epsilon, membership inference
Diffusion ModelsFoundations of LLMs & GenAI
Diffusion models generate images (and audio/video) by learning to reverse a noising process: training corrupts data into noise step by step, and the model learns to denoise, so at generation it starts from pure noise and iteratively denoises into a sample.
Also called: diffusion, diffusion model, latent diffusion, denoising, text-to-image
Dimensional Modeling and Star SchemasData & SQL Engineering
Dimensional modeling organizes an analytics warehouse into fact tables (the measurable events) surrounded by dimension tables (the descriptive context), forming a star schema.
Also called: dimensional modeling, star schema, snowflake schema, fact and dimension tables, Kimball modeling
Dimensionality Reduction: PCA, t-SNE, UMAPEvaluation & ML Foundations
Dimensionality reduction compresses high-dimensional data into fewer axes.
Also called: dimensionality reduction, PCA, principal component analysis, t-SNE, UMAP
Disaggregated Prefill/Decode and Prefix CachingML Infrastructure & Serving
LLM inference has two phases with opposite hardware profiles: prefill is compute-bound (it processes the whole prompt in parallel) while decode is memory-bandwidth bound (one token at a time).
Also called: disaggregated prefill, prefill decode disaggregation, prefix caching, chunked prefill, PD disaggregation
Distributed Key-Value StoresSystem Design for AI in Production
A distributed KV store spreads keys across many nodes and replicates each key for durability and availability.
Also called: key-value store, KV store, quorum, LSM-tree, hinted handoff
Distributed Training: Parallelism and FSDPML Infrastructure & Serving
Training large models needs many GPUs, and there are distinct ways to split the work: data parallelism replicates the model and splits the batch; FSDP/ZeRO shards the optimizer state, gradients, and parameters across GPUs to fit models that do not; tensor parallelism splits a layer's matrices...
Also called: distributed training, FSDP, ZeRO, data parallelism, tensor parallelism
DPO and Preference-Optimization VariantsFoundations of LLMs & GenAI
Direct Preference Optimization aligns a model directly on preference pairs with a simple classification-style loss, skipping RLHF's separate reward model and RL loop, which makes alignment far simpler and more stable.
Also called: DPO, direct preference optimization, SimPO, KTO, ORPO
Drift DetectionMLOps & Lifecycle
Models decay because the world changes.
Also called: data drift, concept drift, PSI, distribution shift
Dynamic ProgrammingCoding & Engineering Craft
Dynamic programming solves problems that have overlapping subproblems and optimal substructure by defining a state, writing a recurrence, and caching results so each subproblem is computed once.
Also called: DP, memoization, tabulation

E

EmbeddingsFoundations of LLMs & GenAIFREE
An embedding maps text (or an image) to a dense vector so that semantic similarity becomes geometric closeness, similar meanings land near each other, measured by cosine similarity.
Also called: embedding, vector representation, dense vector, semantic similarity
Ensembling: Bagging, Boosting, StackingEvaluation & ML FoundationsFREE
Ensembles combine multiple models to beat any single one, because if their errors are decorrelated, combining cancels mistakes.
Also called: ensembling, ensemble, bagging, boosting, stacking
Eval-Driven Development and Golden DatasetsEvaluation & ML FoundationsFREE
You cannot improve an LLM system you cannot measure, so the first thing to build is an evaluation: a golden dataset of representative inputs with expected behavior, plus metrics, that you run on every change.
Also called: eval-driven development, golden dataset, evaluation set, eval set, regression suite

F

Fairness, Bias, and Model CardsAI Security, Privacy & Governance
Models can perform unequally across groups, inheriting and amplifying bias in the data, which is a harm and, in regulated domains, illegal.
Also called: fairness, bias, model cards, datasheets, disparate impact
Fast and Slow Pointers (Floyd's Cycle Detection)Coding & Engineering CraftFREE
Fast and slow pointers run two cursors through a sequence at different speeds so geometry, not extra memory, reveals structure.
Also called: fast and slow pointers, tortoise and hare, Floyd's algorithm, cycle detection, tortoise hare
Fault Tolerance and Graceful DegradationSystem Design for AI in Production
AI systems depend on flaky, slow dependencies (model providers, vector stores, tools), so they must degrade gracefully rather than fail hard.
Also called: fault tolerance, graceful degradation, circuit breaker, fallback, resilience
Feature Engineering: Encoding, Scaling, SelectionEvaluation & ML FoundationsFREE
Feature engineering is the work of turning raw columns into inputs a model can learn from: encoding categoricals, scaling numerics, and selecting which features to keep.
Also called: feature engineering, categorical encoding, feature scaling, feature selection, one-hot encoding
Feature Store: Online vs Offline and Training-Serving SkewMLOps & Lifecycle
One feature definition materialized to an offline store for training and an online store for serving.
Also called: feature store, training-serving skew, online features, point-in-time correctness
Federated LearningAI Security, Privacy & Governance
Federated learning trains a shared model across many devices or organizations without moving their raw data to a central server: each party computes updates locally and only the updates are aggregated.
Also called: FedAvg, federated averaging, cross-device learning, cross-silo learning
FlashAttention and IO-Aware KernelsML Infrastructure & Serving
Naive attention is slow not because of the matmuls but because it writes the full N-by-N attention matrix to GPU high-bandwidth memory and reads it back, which is memory-bandwidth bound.
Also called: FlashAttention, flash attention, IO-aware attention, fused attention kernel
Foundation Model Selection and BenchmarkingSystem Design for AI in Production
Foundation model selection is the disciplined process of choosing across frontier models on capability, cost, latency, and context window, validated by your own task evals rather than public leaderboards.
Also called: model selection, foundation model selection, benchmarking models, model evaluation for selection
From RNNs to Transformers: RNN, LSTM, Seq2SeqFoundations of LLMs & GenAIFREE
Recurrent networks process sequences one step at a time through a hidden state, which makes them principled but slow and bad at long-range dependencies because gradients vanish across many steps.
Also called: RNN, LSTM, GRU, seq2seq, recurrent neural network
Function Calling and Tool SchemasRetrieval & Agents
Function calling is the protocol behind tool use: you declare tools as JSON schemas, the model emits a structured call (name plus arguments) that your code validates and runs, and the result goes back into the conversation.
Also called: function calling, tool schemas, tool calling, structured tool calls, tool definitions

G

Gaps and Islands (Sessionization)Data & SQL Engineering
Gaps-and-islands is the pattern for grouping consecutive rows into runs (islands) separated by breaks (gaps), the engine behind sessionization, streak detection, and consolidating contiguous ranges.
Also called: gaps and islands, sessionization, islands, streak detection, consecutive runs
Gaussian Mixtures and the EM AlgorithmEvaluation & ML Foundations
A Gaussian mixture model treats data as generated by several Gaussian components and assigns each point a soft, probabilistic membership rather than a hard cluster label.
Also called: GMM, Gaussian mixture model, expectation maximization, EM algorithm, soft clustering
Generative vs Discriminative Models (Naive Bayes)Evaluation & ML Foundations
A discriminative model learns P(y|x) directly, the decision boundary.
Also called: generative vs discriminative, naive bayes, Naive Bayes, generative model, discriminative model
GPU Memory and the Serving StackML Infrastructure & ServingFREE
Serving an LLM is mostly a memory problem: the GPU must hold the model weights plus a KV cache that grows with sequence length and batch size, and inference splits into a compute-bound prefill and a memory-bandwidth-bound decode.
Also called: GPU memory, serving stack, memory math, prefill, decode
Gradient Descent and OptimizersEvaluation & ML FoundationsFREE
Gradient descent is how models learn: compute the gradient of the loss with respect to the parameters and step opposite it to reduce error.
Also called: gradient descent, SGD, stochastic gradient descent, Adam, optimizer
GraphRAG and Knowledge-Graph RetrievalRetrieval & Agents
GraphRAG builds an entity-and-relationship graph over a corpus, then retrieves by traversing that graph instead of (or alongside) flat vector similarity.
Also called: GraphRAG, graph RAG, knowledge graph RAG, knowledge-graph retrieval, graph retrieval
Graphs: BFS, DFS, and Shortest PathsCoding & Engineering Craft
A graph is nodes and edges, and most of the work is recognizing that a problem is a graph in the first place.
Also called: graph traversal, BFS, DFS, breadth-first search, depth-first search
Greedy AlgorithmsCoding & Engineering Craft
Greedy algorithms build a solution by always taking the locally best choice and never reconsidering.
Also called: greedy, greedy algorithm, exchange argument, greedy choice
GROUP BY and AggregationData & SQL EngineeringFREE
GROUP BY collapses rows sharing the same key values into one row per group, and aggregate functions (COUNT, SUM, AVG) compute a single value per group.
Also called: GROUP BY, aggregation, HAVING vs WHERE, conditional aggregation, SUM CASE
GuardrailsSystem Design for AI in ProductionFREE
Guardrails are the runtime safety layer wrapping an LLM: input checks (detect prompt injection, off-topic or disallowed requests, PII) before the model, and output checks (content safety, schema/format validation, grounding, PII/secret leakage) before the user.
Also called: safety layer, input validation, output filtering, content safety

H

HallucinationFoundations of LLMs & GenAIFREE
A hallucination is fluent, confident output that is wrong or unsupported.
Also called: hallucinations, grounding, faithfulness, confabulation
Handling Missing and Corrupted DataEvaluation & ML Foundations
Missing data has three mechanisms (MCAR, MAR, MNAR) and the mechanism decides whether dropping rows is safe or biased and which imputation is valid.
Also called: missing values, imputation, MCAR, MAR, MNAR
Handling the Live Demo (and Recovery)Behavioral & Project Deep-Dives
AI demos fail in front of customers: the model hallucinates, a service times out, an edge case breaks.
Also called: demo recovery, live demo, handling failure, composure
Heaps and Priority QueuesCoding & Engineering Craft
A binary heap keeps a partial order so you can pull the smallest or largest element in O(log n) and peek at it in O(1), without paying to fully sort.
Also called: heap, binary heap, priority queue, min-heap, max-heap
Hyperparameter OptimizationEvaluation & ML Foundations
Hyperparameter optimization is the search for the settings (learning rate, depth, regularization) that a model does not learn on its own, using grid, random, or Bayesian search.
Also called: hyperparameter tuning, HPO, Bayesian optimization, Hyperband, successive halving
Hypothesis Testing and p-valuesEvaluation & ML Foundations
Hypothesis testing asks whether an observed effect is large enough to be unlikely under a null hypothesis of no effect, summarized by a p-value.
Also called: hypothesis testing, p-value, p values, significance testing, null hypothesis

I

Idempotency and Exactly-Once EffectsSystem Design for AI in ProductionFREE
In a distributed system, calls fail and get retried, so the same request can arrive more than once.
Also called: idempotency, idempotent, exactly-once, idempotency key, deduplication
Idempotent Data Pipelines: Reruns Without Duplicate RowsData & SQL Engineering
How to make a pipeline safe to rerun: insert-overwrite by partition, MERGE keyed on a business id, and deterministic transforms instead of blind appends.
Also called: idempotent pipelines, idempotency, insert-overwrite, upsert, MERGE
Imbalanced Data and ResamplingEvaluation & ML Foundations
Imbalanced data is when one class is rare (fraud, churn, disease), so a model that predicts only the majority scores high accuracy while being useless.
Also called: class imbalance, imbalanced classification, rare class, oversampling, undersampling
Implementing ML From Scratch (NumPy Patterns)Coding & Engineering Craft
ML-from-scratch coding rounds test whether you can express a model as vectorized array operations rather than Python loops, structure a clean forward and backward pass, and write a numerically careful softmax and cross-entropy.
Also called: ML from scratch, NumPy patterns, vectorization, implement softmax, forward backward pass
Incremental Models and MERGE/UPSERTData & SQL Engineering
Incremental models process only new or changed rows instead of rebuilding a table from scratch, using a high-watermark to select the delta and a MERGE/UPSERT to apply it.
Also called: incremental models, MERGE, UPSERT, high watermark, incremental load
Indexing Strategies in SQL: B-Tree, Composite, and CoveringData & SQL Engineering
Pick an index by access pattern: B-tree for ranges, hash for equality, covering for index-only scans, plus composite column order and selectivity.
Also called: indexing, indexes, B-tree index, covering index, composite index
Indirect Prompt Injection and the Lethal TrifectaAI Security, Privacy & Governance
Indirect prompt injection plants attacker instructions inside content an agent retrieves or reads (a web page, a PDF, a support ticket) so a benign user triggers an attack.
Also called: indirect prompt injection, lethal trifecta, data exfiltration attack, poisoned retrieval
Inference-Time Compute and Reasoning ModelsFoundations of LLMs & GenAI
Inference-time (test-time) compute is the idea that spending more computation at generation, longer chains of thought, sampling multiple attempts, or search, reliably improves answers on hard problems, a scaling axis distinct from making the model bigger.
Also called: inference-time compute, test-time compute, reasoning models, thinking models
Information Theory for MLEvaluation & ML Foundations
Information theory gives ML its core measures: entropy (uncertainty in a distribution), cross-entropy (the cost of modeling the true distribution with your predicted one, the classification loss), KL divergence (how far one distribution is from another), and mutual information (how much one...
Also called: information theory, entropy, cross-entropy, KL divergence, mutual information

J

Jailbreaks and Red-Teaming TaxonomyAI Security, Privacy & Governance
Jailbreaks are inputs that get a model to produce content its safety training was meant to refuse, using techniques like role-play framing, encoding, many-shot priming, and gradual crescendo escalation.
Also called: jailbreaks, red teaming, jailbreak taxonomy, crescendo attack, many-shot jailbreak

K

kNN and the Curse of DimensionalityEvaluation & ML FoundationsFREE
k-nearest-neighbors is a lazy, instance-based learner that classifies a point by majority vote of its closest training examples under some distance metric.
Also called: k-nearest neighbors, kNN, nearest neighbor, curse of dimensionality, distance concentration
Knowledge DistillationML Infrastructure & Serving
Knowledge distillation trains a small student model to imitate a larger teacher, using the teacher's soft probability distribution (or internal features) as a richer training signal than hard labels.
Also called: model distillation, teacher-student, distillation, soft targets

L

Label Noise and Weak SupervisionEvaluation & ML Foundations
Label noise is errors in your training labels, and it caps the accuracy a model can reach no matter how good the architecture is.
Also called: label noise, weak supervision, programmatic labeling, confident learning, labeling functions
Late-Interaction Retrieval (ColBERT)Retrieval & Agents
Late-interaction retrieval represents each document as one vector per token rather than a single pooled vector, then scores a query by summing the best token-to-token matches (MaxSim).
Also called: late interaction, ColBERT, MaxSim, multi-vector retrieval, token-level retrieval
Latency Budgets and StreamingSystem Design for AI in ProductionFREE
LLM latency is not one number: time-to-first-token (set by prefill and queueing) and inter-token latency (set by decode) feel very different to users.
Also called: latency budget, time to first token, TTFT, streaming, inter-token latency
Linear and Logistic RegressionEvaluation & ML FoundationsFREE
Linear regression fits a weighted sum of features to a continuous target by minimizing squared error; logistic regression squashes that same linear score through a sigmoid and fits it with cross-entropy to produce a probability.
Also called: linear regression, logistic regression, least squares, logit model, ordinary least squares
Linked Lists: Dummy Head, Fast/Slow Pointers, ReversalCoding & Engineering CraftFREE
The three patterns interviews actually test: the dummy head that removes edge cases, fast/slow pointers for cycles and midpoints, and in-place reversal.
Also called: linked list, singly linked list, doubly linked list, Floyd's algorithm, slow and fast pointers
Listwise, Pairwise, Pointwise: Learning to Rank ExplainedSystem Design for AI in Production
The three learning-to-rank setups compared: pointwise scores items alone, pairwise learns which of two wins, listwise optimizes the order against NDCG.
Also called: learning to rank, LTR, pointwise, pairwise, listwise
LLM Cost OptimizationSystem Design for AI in ProductionFREE
LLM systems get expensive fast, and the cost model is mostly tokens and number of model calls.
Also called: cost optimization, token cost, cost per request
LLM-as-a-JudgeEvaluation & ML FoundationsFREE
When outputs are open-ended (summaries, chat answers, generated code), there is no exact match to score against, so you use a strong LLM to grade them against a rubric.
Also called: LLM-as-judge, LLM as a judge, model-graded eval, G-Eval, judge model
Load BalancingSystem Design for AI in ProductionFREE
A load balancer spreads requests across many backend instances so no single server is overwhelmed, and removes failed instances from rotation.
Also called: load balancer, L4, L7, round robin, least connections
LoRA and Parameter-Efficient Fine-TuningFoundations of LLMs & GenAI
Full fine-tuning updates all of a model's weights, which is expensive in compute and memory and produces a full-size copy per task.
Also called: LoRA, PEFT, parameter-efficient fine-tuning, QLoRA, adapters

M

Matrix and Grid Simulation PatternsCoding & Engineering Craft
Grid problems reward a small set of mechanical patterns: walk a spiral by shrinking four boundaries, rotate a square in place with a transpose-then-reverse, and mark state inside the grid itself to keep extra space at O(1).
Also called: matrix simulation, grid simulation, spiral matrix, rotate image, in-place matrix
Mechanistic Interpretability: Features, Circuits, and SAEsAI Security, Privacy & Governance
Reverse-engineering what a network computes: features as directions, circuits that combine them, superposition, and sparse autoencoders that unpack it.
Also called: mechanistic interpretability, mech interp, circuits, superposition, sparse autoencoders
Merge Intervals, Meeting Rooms, and the Sweep Line PatternCoding & Engineering CraftFREE
Merging, inserting, counting overlaps, and minimum meeting rooms all start the same way: sort by start or end, then sweep once instead of all pairs.
Also called: intervals, interval problems, merge intervals, meeting rooms, sweep line
Message Queues and Event StreamingSystem Design for AI in Production
Broker queues (RabbitMQ, SQS) hand each message to one worker, wait for an ack, and delete it: built for distributing jobs.
Also called: message queue, event streaming, Kafka, RabbitMQ, SQS
Mixed-Precision TrainingML Infrastructure & Serving
Mixed-precision training does most computation in 16-bit (FP16 or BF16) instead of 32-bit, roughly halving memory and speeding up training on modern GPUs, while keeping a few numerically-sensitive parts in FP32 for stability.
Also called: mixed precision, FP16, BF16, bfloat16, loss scaling
Mixture-of-ExpertsFoundations of LLMs & GenAI
A Mixture-of-Experts model replaces the dense feed-forward layer with many expert networks and a router that sends each token to only a few of them.
Also called: MoE, sparse model, expert routing
MLE, MAP, and Bayesian vs FrequentistEvaluation & ML Foundations
Maximum likelihood picks the parameters that make the observed data most probable; MAP adds a prior and picks the most probable parameters given the data.
Also called: MLE, MAP, maximum likelihood, maximum a posteriori, Bayesian vs frequentist
Model Context Protocol (MCP)Retrieval & Agents
MCP is an open client-server standard that lets an agent connect to external tools, data, and prompts through a uniform interface, so one integration works across many hosts instead of writing bespoke glue per model.
Also called: MCP, Model Context Protocol, MCP server, MCP client
Model Debugging MethodologyMLOps & Lifecycle
Model debugging is the systematic process of root-causing why a model underperforms: deciding whether the cause is the data, the features, the labels, model capacity, or the evaluation itself, rather than blindly tuning hyperparameters.
Also called: model debugging, error analysis, root-causing underperformance, train val test gap
Model Monitoring in ProductionMLOps & LifecycleFREE
Monitoring an ML model means more than uptime and latency, because a model can be healthy and silently wrong.
Also called: model monitoring, production monitoring, ML monitoring, monitoring layers
Model Registry, Lineage, and PromotionMLOps & Lifecycle
A model registry is the versioned source of truth for trained models: each model has a version, lineage (the data, code, config, and run that produced it), and a stage (staging, production, archived).
Also called: model registry, lineage, model promotion, model versioning, reproducibility
Model Serving FrameworksML Infrastructure & Serving
You rarely build a serving stack from scratch; frameworks handle the production plumbing.
Also called: vLLM, Triton, TGI, TensorRT-LLM, KServe
Monotonic Stack and Queue: Next Greater Element in O(n)Coding & Engineering Craft
The invariant behind monotonic stacks and deques: pop whatever the new element makes useless.
Also called: monotonic stack, monotonic queue, monotonic deque, next greater element, sliding window maximum
Multi-Agent OrchestrationRetrieval & Agents
When a task is too big or varied for one agent, an orchestrator decomposes it and delegates subtasks to focused sub-agents, each with its own clean context and tools, then synthesizes the results.
Also called: multi-agent, orchestration, sub-agents, subagents, orchestrator-worker
Multi-Armed Bandits: Epsilon-Greedy, UCB, Thompson SamplingEvaluation & ML Foundations
How bandits trade exploration against exploitation, what epsilon-greedy, UCB, and Thompson sampling each do, and when a bandit beats a fixed A/B test.
Also called: multi-armed bandits, bandit, exploration-exploitation, Thompson sampling, UCB
Multi-LoRA ServingML Infrastructure & Serving
LoRA adapters are tiny weight deltas on top of a shared base model, so you can serve hundreds of fine-tuned variants from one set of base weights instead of one full model per tenant.
Also called: multi lora, serving LoRA adapters, LoRA multiplexing
Multi-Stage Retrieval and Ranking FunnelsSystem Design for AI in Production
Search, ads, and feed systems are built as a funnel: retrieve a broad candidate set, rank it with a heavier model, re-rank the top with the heaviest model, then filter and blend with business rules.
Also called: multi-stage ranking, ranking funnel, retrieval and ranking funnel, cascade ranking, multi-stage retrieval
Multi-Tenancy and IsolationAI Security, Privacy & Governance
When one AI system serves many customers (tenants), the cardinal rule is that no tenant can see another's data, ever.
Also called: multi-tenancy, tenant isolation, multi-tenant, cross-tenant leak
Multilingual Models and the Tokenization TaxFoundations of LLMs & GenAI
Multilingual LLMs work unevenly: best on high-resource languages (English), worse on low-resource ones, because training data is English-heavy.
Also called: multilingual models, multilingual, low-resource languages, tokenization tax
Multimodal Models and VLMsFoundations of LLMs & GenAI
Multimodal models process more than text, most commonly vision-language models (VLMs) that take images and text together.
Also called: multimodal, VLM, vision-language model, CLIP, multimodal model

N

NULLs and Three-Valued LogicData & SQL Engineering
NULL means unknown, so SQL uses three-valued logic where comparisons with NULL return UNKNOWN, not TRUE or FALSE.
Also called: NULL, three-valued logic, NOT IN trap, COALESCE, NULLIF
Numerical Stability in CodeCoding & Engineering Craft
Numerical stability is writing arithmetic so floating-point error and overflow do not corrupt the result, which matters because naive ML math (softmax, cross-entropy, variance) silently returns NaN or wrong gradients.
Also called: numerical stability, log-sum-exp, logsumexp, floating point precision, catastrophic cancellation

O

Object Detection and SegmentationEvaluation & ML Foundations
Detection finds objects as boxes plus labels; segmentation labels pixels (semantic) or per-object pixels (instance).
Also called: object detection, semantic segmentation, instance segmentation, Faster R-CNN, YOLO
Observability for LLM SystemsSystem Design for AI in ProductionFREE
You cannot operate or improve an LLM system you cannot see.
Also called: observability, LLM observability, tracing, logging, monitoring
Offline vs Online Evaluation: Why Offline Wins Fail to HoldEvaluation & ML FoundationsFREE
What held-out metrics measure, what an A/B test measures, and why they disagree: static data, feedback loops, and shift.
Also called: offline vs online, offline evaluation, online evaluation, offline online gap
Outlier and Anomaly DetectionEvaluation & ML Foundations
Outlier and anomaly detection finds points that do not fit the bulk of the data using statistical, distance/density, or reconstruction-based methods.
Also called: anomaly detection, outlier detection, novelty detection, isolation forest, local outlier factor
Overfitting and RegularizationEvaluation & ML FoundationsFREE
Overfitting is when a model learns the training data's noise instead of its signal, scoring well in training but failing on new data.
Also called: overfitting, regularization, L1, L2, early stopping

P

P2P Content Distribution: BitTorrent, Gossip, and CDNsSystem Design for AI in Production
Why a single source bottlenecks on upload bandwidth, how BitTorrent chunking makes capacity grow with the swarm, and where gossip and CDN caching fit.
Also called: content distribution, P2P, peer-to-peer, BitTorrent, swarm
PagedAttentionML Infrastructure & Serving
The KV cache is the memory bottleneck in LLM serving, and naively reserving a contiguous block per request (sized for the maximum length) wastes most of it to fragmentation and over-allocation.
Also called: paged attention, KV cache paging, vLLM
Parsing Messy Real-World Data: Defensive Parsing PatternsCoding & Engineering CraftFREE
How to read malformed input: validate, decide per record whether to skip, default, or fail, and keep one bad row from killing the whole batch.
Also called: parsing messy data, data parsing, robust parsing, ingestion
Partitioning and ClusteringData & SQL Engineering
Partitioning splits one large table into physically separate chunks by a key (usually date), so a query with a matching filter reads only the relevant partitions instead of the whole table.
Also called: partitioning, clustering, partition pruning, sort keys, cluster keys
PII HandlingAI Security, Privacy & GovernanceFREE
Personal data in prompts, logs, and training sets is a privacy and compliance risk (GDPR, HIPAA), so you must detect and protect it.
Also called: PII, redaction, data minimization, personal data
Pipeline Orchestration and DAGsData & SQL Engineering
Orchestration runs dependent data tasks as a DAG so each task waits for its upstreams, retries safely, backfills history, and alerts when an SLA is missed.
Also called: orchestration, DAG, Airflow, Dagster, data pipeline orchestration
Policy Optimization: PPO and GRPOFoundations of LLMs & GenAI
PPO and GRPO are the reinforcement-learning algorithms that optimize an LLM against a reward, the RL step in RLHF and in training reasoning models.
Also called: PPO, GRPO, policy optimization, proximal policy optimization, group relative policy optimization
Positional Encodings (RoPE and ALiBi)Foundations of LLMs & GenAI
Attention is order-blind, so models inject token position separately.
Also called: positional encoding, RoPE, rotary position embedding, ALiBi, positional encodings
Precision, Recall, and F1: Thresholds and Imbalanced DataEvaluation & ML FoundationsFREE
Precision is how many flagged items were right, recall is how many real positives you caught.
Also called: precision, recall, F1, precision and recall, F1 score
Prefix Sums and Difference ArraysCoding & Engineering CraftFREE
A prefix-sum array precomputes running totals so any range sum answers in O(1), and pairing prefix sums with a hash map counts subarrays whose sum hits a target or a residue mod k.
Also called: prefix sum, prefix sums, cumulative sum, difference array, range update
Probability Distributions: Bernoulli, Normal, and PoissonEvaluation & ML FoundationsFREE
Bernoulli and binomial for yes/no counts, normal for sums and noise, Poisson for events in a window, exponential for waits, and the loss each one implies.
Also called: probability distributions, Bernoulli distribution, binomial distribution, normal distribution, Gaussian
Prompt and Semantic CachingSystem Design for AI in Production
Caching is one of the cheapest, highest-impact LLM optimizations.
Also called: prompt caching, prefix caching, semantic caching, caching, KV cache reuse
Prompt EngineeringFoundations of LLMs & GenAIFREE
Prompting is the cheapest, fastest way to steer an LLM: clear instructions, few-shot examples, explicit output format, and the right context.
Also called: prompting, prompt template, system prompt, few-shot
Prompt InjectionAI Security, Privacy & GovernanceFREE
Prompt injection is the top security risk for LLM apps: malicious instructions override the model's intended behavior.
Also called: indirect prompt injection, jailbreak, injection
Prompt Versioning and ManagementSystem Design for AI in Production
Prompt versioning treats prompts as production artifacts with their own change log, eval-backed releases, and rollback path, instead of string literals buried in application code.
Also called: prompt versioning, prompt management, prompt registry, versioned prompts
Prompting vs RAG vs Fine-TuningFoundations of LLMs & GenAI
Given an LLM use case, the senior move is matching the technique to what is missing rather than defaulting to one.
Also called: RAG vs fine-tuning, prompting vs fine-tuning, when to fine-tune, when to use RAG

Q

Quantization and Low PrecisionML Infrastructure & Serving
Quantization stores and computes model weights (and activations) in fewer bits, FP16/BF16, FP8, INT8, INT4, instead of FP32, cutting memory and speeding inference at some accuracy cost.
Also called: quantization, low precision, INT8, INT4, FP8
Query Execution and OptimizationData & SQL Engineering
A query optimizer turns your SQL into a physical plan: which tables to scan, in what join order, and whether to use a hash join, sort, or index lookup.
Also called: query optimization, EXPLAIN plan, query planner, execution plan, query tuning
Query Transformation and Multi-Hop RetrievalRetrieval & Agents
The user's raw question is often a poor search query: ambiguous, underspecified, or requiring several facts chained together.
Also called: query transformation, query rewriting, HyDE, multi-hop retrieval, query decomposition

R

RAG EvaluationEvaluation & ML FoundationsFREE
Evaluating a RAG system means evaluating retrieval and generation separately, because a bad answer is usually a retrieval failure (the right context was never fetched) and you cannot fix what you cannot localize.
Also called: retrieval evaluation, recall@k, faithfulness, RAGAS
Ranking and Top-N Per GroupData & SQL Engineering
Top-N-per-group is the partition-then-filter idiom: rank rows within each group with a window function, then keep the ranks you want.
Also called: top-N per group, ROW_NUMBER, RANK, DENSE_RANK, partition then filter
RAPTOR and Small-to-Big: Hierarchical Retrieval for RAGRetrieval & Agents
The fix for the chunk-size tradeoff: RAPTOR clusters and summarizes into a tree, small-to-big embeds child chunks but returns the parent for context.
Also called: hierarchical retrieval, RAPTOR, small-to-big, small to big retrieval, parent-child chunking
Rate Limiting, Retries, and BackoffSystem Design for AI in ProductionFREE
LLM systems depend on rate-limited, sometimes-failing providers, so resilient design is essential.
Also called: rate limiting, token bucket, exponential backoff, retries, circuit breaker
Recommendation Systems: Candidate Generation and RankingSystem Design for AI in Production
Industrial recommenders use a two-stage funnel: cheap candidate generation narrows millions of items to a few hundred, then an expensive ranker scores that shortlist.
Also called: recommendation system, recsys, candidate generation, recommender system, two-stage recommender
Recursion and Divide-and-ConquerCoding & Engineering CraftFREE
Recursion solves a problem by calling itself on smaller inputs until a base case stops it; divide-and-conquer is the variant that splits input into independent subproblems, solves each, and combines the results (merge sort, quickselect).
Also called: recursion, divide and conquer, divide-and-conquer, recursive algorithms, call stack
Reproducible and Deterministic PipelinesMLOps & Lifecycle
A reproducible pipeline produces the same model and metrics from the same inputs, achieved by pinning seeds, dependencies, data versions, and code together.
Also called: reproducible pipelines, deterministic training, reproducibility, bit-for-bit reproducibility
Requirements DiscoveryBehavioral & Project Deep-DivesFREE
The most expensive AI mistakes come from building the wrong thing, and the cause is usually skipping discovery.
Also called: discovery, working backwards, problem definition
RerankingRetrieval & Agents
Reranking is a two-stage retrieval design: a fast bi-encoder fetches a broad candidate set for recall, then a slower but more accurate cross-encoder rescoring each (query, document) pair reorders them for precision.
Also called: reranker, cross-encoder, bi-encoder, two-stage retrieval
Retrieval vs Long ContextRetrieval & Agents
When you can fit a whole document in a model's large context window, should you, or should you retrieve only the relevant chunks?
Also called: long context vs RAG, RAG vs long context
Reward ModelsFoundations of LLMs & GenAI
A reward model turns human preference comparisons into a scalar score for any response, the signal RLHF optimizes against.
Also called: reward model, preference model
RLHF: Reinforcement Learning from Human FeedbackFoundations of LLMs & GenAI
RLHF is how a raw next-token predictor becomes a helpful, harmless assistant.
Also called: RLHF, reinforcement learning from human feedback, alignment, instruction tuning

S

Sampling Techniques: Stratified, Reservoir, ImportanceEvaluation & ML Foundations
Sampling techniques decide which subset of data you train on, evaluate on, or stream through, and that choice quietly determines whether your numbers reflect reality.
Also called: sampling techniques, stratified sampling, reservoir sampling, importance sampling, uniform sampling
Scaling LawsFoundations of LLMs & GenAI
Scaling laws say model loss falls predictably as a power law in parameters, data, and compute, which is why bigger models trained on more data reliably get better.
Also called: Chinchilla, compute-optimal, neural scaling laws
Schema Evolution and Data ContractsData & SQL Engineering
Schemas change as products evolve, and adding, altering, or dropping a column can break every downstream consumer at once.
Also called: schema evolution, data contracts, expand-contract migration, schema migration, backward compatibility
Scoping Under AmbiguityBehavioral & Project Deep-DivesFREE
Real AI projects start ambiguous: vague goals, unknown data, shifting requirements.
Also called: scoping, ambiguity, handling ambiguity, MVP, de-risking
Self-Consistency, Tree-of-Thought, and Prompt ChainingFoundations of LLMs & GenAI
Three ways to push past a single linear chain of thought: self-consistency samples many reasoning paths and votes on the answer, tree-of-thought branches and searches over partial reasoning, and prompt chaining splits one hard prompt into a sequence of focused calls.
Also called: self-consistency, tree of thought, ToT, prompt chaining, majority voting reasoning
Slowly Changing Dimensions (SCD)Data & SQL Engineering
Slowly changing dimensions are the patterns for handling dimension attributes that change over time, such as a customer moving cities or a product changing category.
Also called: slowly changing dimensions, SCD, SCD Type 2, dimension history, effective dating
Small vs Large Models and RoutingFoundations of LLMs & GenAI
Bigger is not always better in production: small models are far cheaper and faster, and for many tasks they are good enough, especially when fine-tuned or given retrieval.
Also called: small vs large models, model routing, small language models, model cascade, SLM
Sorting AlgorithmsCoding & Engineering CraftFREE
Sorting algorithms split into comparison sorts (merge, quick, heap) bounded by an O(n log n) lower bound, and linear-time counting and radix sorts that work only when keys are small bounded integers.
Also called: sorting, merge sort, quicksort, radix sort, comparison sort
Speculative DecodingML Infrastructure & Serving
Decoding is sequential and memory-bound, so generating each token one at a time underuses the GPU.
Also called: draft model, speculative sampling
Speech and Voice AI: ASR, TTS, and Voice AgentsFoundations of LLMs & GenAI
Voice agents chain three systems: speech-to-text (ASR), an LLM, and text-to-speech (TTS), all under a hard real-time latency budget that text chat never faces.
Also called: ASR, TTS, speech recognition, text to speech, voice agents
SQL JoinsData & SQL EngineeringFREE
Joins combine rows across tables on a matching condition, and the join type (inner, left, right, full, semi, anti) controls which non-matching rows survive.
Also called: left join, semi join, anti join, join fan-out, hash join
Stacks and QueuesCoding & Engineering CraftFREE
A stack is last-in-first-out and a queue is first-in-first-out, and most interview value comes from recognizing which problems hide one.
Also called: stack, queue, monotonic stack, LIFO, FIFO
Streaming and BackpressureCoding & Engineering Craft
When data is too big to fit in memory or arrives continuously, you process it as a stream, one piece at a time, with bounded memory, rather than loading it all.
Also called: streaming, backpressure, bounded memory, generators, flow control
SVMs and the Kernel TrickEvaluation & ML Foundations
A support vector machine finds the decision boundary with the widest margin to the nearest points (the support vectors), trading hinge loss against margin width.
Also called: support vector machine, SVM, kernel trick, max-margin classifier, hinge loss
Synthetic Data GenerationEvaluation & ML Foundations
Synthetic data is training or eval data produced by a model, a simulator, or a program rather than collected from the real world, used to bootstrap labels, cover rare cases, and distill a larger model into a smaller one.
Also called: synthetic data, generating training data, data synthesis, LLM-generated data, distillation data

T

Temperature and SamplingFoundations of LLMs & GenAI
At each step a model outputs a probability distribution over the next token; how you pick from it is decoding.
Also called: temperature, sampling, top-p, nucleus sampling, top-k
Testable Design for AI SystemsCoding & Engineering Craft
AI systems are hard to test because models are non-deterministic and call external services, so testability has to be designed in: isolate the non-deterministic model behind an interface so you can mock it, separate deterministic logic (parsing, retrieval, formatting) from the model call and test...
Also called: testable design, dependency injection, mocking, testing AI systems
The Bias-Variance TradeoffEvaluation & ML FoundationsFREE
A model's error decomposes into bias (error from being too simple to capture the pattern, underfitting) and variance (error from being too sensitive to the training sample, overfitting).
Also called: bias-variance, bias-variance tradeoff, underfitting, bias variance
The Big-O That Actually MattersCoding & Engineering CraftFREE
Big-O complexity matters most where it bites in real AI systems: avoid accidental O(n^2) (all-pairs comparisons, repeated linear scans), use hash maps for O(1) lookups, and know that vector search is approximate precisely because exact nearest-neighbor is O(n) per query.
Also called: Big-O, time complexity, complexity, quadratic, O(n^2)
The Computer Vision PipelineEvaluation & ML Foundations
A production CV system is a chain: ingest and version images, preprocess and augment, fine-tune a pretrained backbone, attach a task head, evaluate with sliced metrics, post-process, then serve and monitor.
Also called: computer vision pipeline, CV pipeline, image preprocessing, train serve skew, model serving
The Context WindowFoundations of LLMs & GenAIFREE
The context window is the maximum number of tokens a model can attend to at once, prompt plus generation.
Also called: context window, context length, long context, lost in the middle
The KV CacheFoundations of LLMs & GenAI
During autoregressive decoding, a model would recompute attention over the entire history at every step; the KV cache stores each token's key and value vectors so each new token only attends, never recomputes.
Also called: KV cache, KV-cache, key-value cache, kv cache
The LLM GatewaySystem Design for AI in ProductionFREE
An LLM gateway is a single proxy layer between your application and one or more model providers.
Also called: LLM gateway, model gateway, AI gateway, proxy layer
The RAG PipelineRetrieval & AgentsFREE
Retrieval-Augmented Generation grounds an LLM in external knowledge: at query time you retrieve the most relevant chunks from a knowledge base and put them in the prompt, so the model answers from real sources instead of memory.
Also called: RAG, retrieval-augmented generation, RAG pipeline, retrieval augmented generation
The Transformer ArchitectureFoundations of LLMs & GenAI
The transformer is the architecture behind modern LLMs: stacked blocks that each mix information across tokens with self-attention and then transform each token with a feed-forward network, wrapped in residual connections and normalization.
Also called: transformer, transformer architecture, feed-forward network, FFN, decoder-only
TokenizationFoundations of LLMs & GenAIFREE
Models do not read characters or words; they read tokens, subword chunks produced by an algorithm like BPE that maps text to integer IDs.
Also called: tokenizer, tokens, BPE, byte pair encoding, subword
Topological Sort and DAGsCoding & Engineering Craft
A topological sort orders the nodes of a directed acyclic graph so that every edge points forward, which is exactly what dependency resolution needs.
Also called: topological sort, topo sort, DAG, Kahn's algorithm, dependency resolution
Training Neural Nets: Init, Normalization, Dropout, LR SchedulesEvaluation & ML Foundations
The practical recipe that makes deep nets train at all: scale-aware weight initialization (Xavier, He), normalization layers (batch, layer, RMS) that keep activations well-conditioned, dropout as stochastic regularization, and warmup plus cosine learning-rate schedules.
Also called: weight initialization, He initialization, Xavier initialization, batch normalization, layer normalization
Training Reasoning Models: RLVR, PRM vs ORMFoundations of LLMs & GenAI
Reasoning models like o1 and R1 are not just bigger instruct models: they are trained with reinforcement learning where the reward comes from checking whether the final answer is correct, which teaches the model to generate long internal chains of thought.
Also called: RLVR, GRPO, process reward model, PRM, outcome reward model
Transactions, ACID, and Isolation LevelsData & SQL Engineering
A transaction groups several reads and writes so they either all commit or all roll back, with the ACID guarantees of atomicity, consistency, isolation, and durability.
Also called: ACID, isolation levels, transaction isolation, MVCC, optimistic locking
Transfer LearningEvaluation & ML Foundations
Transfer learning reuses a model pretrained on a large general corpus as the starting point for a new task, so you inherit learned features instead of training from scratch.
Also called: feature extraction, freezing layers, fine-tuning a backbone, pretrained model reuse
Translating Technical Trade-offsBehavioral & Project Deep-DivesFREE
Applied-AI engineers constantly translate between technical reality and business stakeholders: explaining the accuracy-latency-cost triangle, why the model cannot be 100% reliable, and what a trade-off means for the user, in the stakeholder's language, not jargon.
Also called: translating tradeoffs, explaining tradeoffs, accuracy latency cost, communicating uncertainty
Trees, BSTs, and TraversalCoding & Engineering Craft
A binary tree links each node to up to two children, and a binary search tree adds the invariant that everything left is smaller and everything right is larger, which gives O(log n) search on a balanced tree.
Also called: binary tree, BST, binary search tree, tree traversal, inorder
Tries and String AlgorithmsCoding & Engineering Craft
A trie is a prefix tree that stores strings by shared prefixes, giving O(length) lookup and natural prefix queries for autocomplete.
Also called: trie, prefix tree, KMP, Knuth-Morris-Pratt, Rabin-Karp
Two Pointers and Sliding Window: O(n) Array PatternsCoding & Engineering CraftFREE
Converging pointers on a sorted array, and an expanding-contracting window with a running invariant.
Also called: two pointers, sliding window, expand contract, converging pointers, fast slow pointers

U

Union-Find (Disjoint Set Union)Coding & Engineering Craft
Union-Find (Disjoint Set Union) tracks a partition of elements into groups and answers 'are these two connected?' in near-constant amortized time using path compression and union by rank.
Also called: union find, disjoint set union, DSU, disjoint-set, connected components dynamic
User Feedback Loops and the Data FlywheelSystem Design for AI in Production
A data flywheel captures implicit and explicit user feedback in production, routes it into eval sets and fine-tuning data, and uses the improved model to attract more usage that generates more feedback.
Also called: data flywheel, user feedback loops, feedback loop, implicit feedback, feedback signals

V

Vanishing and Exploding GradientsEvaluation & ML Foundations
In a deep or recurrent network the backward gradient is a product of many per-layer Jacobians, so its magnitude compounds: factors mostly below one shrink it toward zero (early layers stop learning) and factors above one blow it up (training diverges into NaNs).
Also called: vanishing gradients, exploding gradients, gradient clipping, residual connections

W

Warehouse vs Lake vs LakehouseData & SQL Engineering
A warehouse enforces schema-on-write with tight governance and fast SQL; a data lake stores raw files cheaply with schema-on-read and no transactions; a lakehouse puts an open table format (Iceberg or Delta) on object storage to give ACID, time travel, and schema evolution at lake cost.
Also called: data warehouse, data lake, lakehouse, Iceberg, Delta Lake
Window FunctionsData & SQL EngineeringFREE
Window functions compute across a set of rows related to the current row, without collapsing them like GROUP BY does, so you can rank within groups, compute running totals and moving averages, and compare a row to its neighbors (LAG/LEAD), all in one pass.
Also called: OVER, PARTITION BY, ROW_NUMBER, LAG, running total