fine tuning

The single most-asked applied GenAI question, and the one most candidates answer as a definition dump. The interviewer wants a decision framework with a default and the conditions that flip it. Here is the call that scores.

Alignment is now table stakes even outside 'safety' roles. The interviewer wants the three-stage RLHF pipeline and a crisp account of why DPO dropped the reward model entirely. Here is the answer with the mechanism, not the buzzwords.

PEFT is how everyone fine-tunes large models now. The signal is the low-rank insight behind LoRA, why it cuts memory so much, and what 4-bit QLoRA adds. Here is the answer that goes past 'it's efficient fine-tuning.'

Instruction tuning is what turns a raw next-token predictor into something that follows instructions. The signal is placing it correctly in the pretrain to SFT to alignment pipeline and knowing exactly what it does and does not fix.

Fine-tune a model on your domain and it can forget how to do everything else. The signal is explaining why shared weights cause it and naming the concrete mitigations that all reduce to one principle.

Fine-tuning is mostly a data problem, not a hyperparameter one. The signal is naming what makes a good set (quality, diversity, format, dedup) and arguing why a few thousand clean examples beat a million noisy ones.

Synthetic data is how teams get training data when real data is scarce, and it has sharp failure modes. The signal is naming when it helps, how to hold the quality line, and why training on model output across generations narrows the distribution.

Fine-tuning a 65B model used to mean a node of A100s. QLoRA collapses it onto one card with three specific tricks. The signal is knowing what gets quantized, what stays trainable, and why quality barely moves. Here is the answer.

Combine several fine-tunes into one model by doing arithmetic on their weights, with no retraining and no data. The signal is knowing why it works and which method to reach for when merges interfere.

RAG and fine-tuning get framed as either/or. RAFT trains the model to be good at RAG itself. The tell is what goes in the training set: relevant docs mixed with distractors. Here is the answer.

The senior signal is matching the method to what is actually missing instead of defaulting to your favorite. Knowledge, behavior, or actions each point at a different tool. Here is the decision framework that scores highest.

Alignment that adds refusals and politeness often quietly degrades reasoning and coding. The alignment tax is real and measurable. Here is how to keep the safety gains without paying for them in capability.

When a drug name or a Hindi word becomes nine subword tokens, you pay in cost, context, and accuracy. Adding tokens is easy and adding them well is not. Here is the tradeoff.

A model that tops MTEB can still flop on your jargon-heavy corpus, and most teams jump straight to the expensive fix. There is a ladder, and the cheap rungs are the ones people skip. Here is the order to climb it.

Transfer learning is why you rarely train from scratch, and the real question is how much of the pretrained model to reuse versus adapt. The signal is a crisp decision grid over data size and task similarity, plus knowing when a low LR saves you from catastrophic forgetting.

Hosting one full fine-tuned model per customer scales linearly in GPUs and bankrupts you fast. There is a way to put hundreds of variants on one GPU without giving up batching. The interviewer wants to hear how.

AI features open cross-tenant leak paths that classic SaaS isolation never had to think about: shared embeddings, shared caches, and fine-tunes that memorize. One leak ends an enterprise contract. Here is the isolation model.