Expert Applied AI interview questions

GRPO drops the value model and estimates advantage from a group of samples per prompt. The signal is the exact mechanics, why the group baseline works, and the normalization traps that quietly bias training.

MQA and GQA shrink the KV cache by sharing key/value heads. MLA takes a different route: compress K and V into a low-rank latent and cache that. The signal is knowing it is a cache trick, not a head-sharing trick, and why it keeps quality.

A model trained at 4K tokens can be stretched to 128K without full retraining. The signal is knowing why naive extrapolation fails and how PI, NTK-aware scaling, and YaRN each rescale RoPE frequencies differently.

Speculative decoding speeds up generation without changing the output distribution. The signal is knowing how draft-model, self-drafting (Medusa/EAGLE), and lookahead approaches differ in where the guesses come from.

MoE saves compute but is awkward to serve: experts must be sharded, tokens routed across devices, and batches balanced. The signal is the all-to-all communication and the load-imbalance problem, not the training story.

Grammar-constrained decoding sounds like a solved problem until tokenization misaligns with the grammar. The signal is the token-vs-character boundary mismatch and how prefix automata and token healing fix it.

At-least-once duplicates and at-most-once loses data; everyone wants exactly-once and few can explain how Kafka delivers it. The signal is idempotent producers plus transactional read-process-write plus the read-committed isolation that ties them together.

Long-running agents that loop, call tools, and spawn sub-tasks need durable execution, not a single request handler. Learn how to checkpoint state, recover from failures, bound runaway loops, and observe every step a non-deterministic agent takes.

Live speech-to-speech translation is a latency race across three models (recognize, translate, synthesize) while audio keeps streaming in. Learn the streaming pipeline, how to commit partial results without flip-flopping, and the tradeoff between latency and translation quality.

Round-robin routing wastes a feature that doubles throughput: cache locality. Designing the router means choosing a replica by KV state, not just least-loaded. Here is the two-layer design.

No single parallelism dimension trains a 175B model on its own. The art is layering three of them onto the right hardware so the expensive communication stays on the fast links. Here is how to size each axis.

MoE buys you more parameters for the same FLOPs, but the gating network, the load imbalance, and the all-to-all shuffle introduce failure modes dense models never see. Here is the training playbook.

RLHF with PPO is not one model training, it is four models in the loop at once, three of them on the GPU during every step. The memory math and the generation bottleneck are what trip people up.

FP8 promises to roughly double training throughput over BF16, but with only a handful of mantissa bits the numerics are unforgiving. Per-tensor scaling and a selective recipe are what make it converge.

Gibberish-looking token strings appended to a prompt can reliably break refusals, and they transfer between models. The signal is explaining the gradient search that finds them and why output checks beat input pattern matching.