44Compare LLM decoding strategies: greedy, beam search, temperature, top-k, top-p, and repetition penalties.▼mediumOpenAIGoogleCohere1 replies◆ premiumHow you decode from the model's probabilities shapes the output as much as the model. The signal is knowing the deterministic vs sampling methods and when each fits (factual vs creative). Here is the answer.Open full answer →
07Implement beam search for a sequence model given a next-token log-probability function.▼hardGoogleNVIDIAMeta1 repliesunlockedBeam search is the decoding algorithm everyone references and few can implement correctly. The signal is keeping k hypotheses by cumulative log-prob, summing logs (not multiplying probs), and handling completed sequences and length. Here is a correct implementation and the tradeoffs.Open full answer →
