generation

These are the knobs every LLM product tunes, and the interviewer wants to know what each does to the probability distribution and which setting fits which task. Including why factual and creative tasks want opposite settings.

How you decode from the model's probabilities shapes the output as much as the model. The signal is knowing the deterministic vs sampling methods and when each fits (factual vs creative). Here is the answer.

Neural text degeneration is a known failure of likelihood-maximizing decoding, not a broken model. The fixes are specific decode-time knobs. Here is which lever to pull and what each one costs.

Generation has to stop somewhere, and the wrong stop rule either truncates answers or burns tokens on trailing garbage. Here is how the EOS token, stop strings, and max-token caps actually interact.

For open-ended generation there's no ground-truth string to match, so accuracy is meaningless. The field uses a layered mix of automatic, model-based, and human metrics. Here is how to assemble a credible eval.

Every chat model's 'temperature' and 'top_p' knobs are a few lines of logit surgery. Implementing them shows you understand decoding, not just call it. Here is the from-scratch version.