95What is FP8, and how does it differ from INT8 for LLM training and inference?▼hardNVIDIAGoogle DeepMindMicrosoft2 replies◆ premiumFP8 is the format behind modern H100-class training and serving. The signal is knowing the two FP8 variants, why a floating format beats INT8 for dynamic range, and where scaling still matters.Open full answer →
03Explain quantization for inference: INT8/INT4, GPTQ/AWQ, what breaks, and how you validate it.▼hard★ EssentialNVIDIAOpenAIxAI2 repliesunlockedQuantization is the first lever for fitting and speeding up models, and the interviewer wants more than 'use fewer bits.' The signal is knowing what precision buys you, why outliers break naive quantization, and how you prove quality held. Here is that answer.Open full answer →
