28What is MFU (Model FLOPs Utilization), and why can GPU utilization be misleading?▼hard★ EssentialNVIDIAOpenAIGoogle1 replies◆ premiumnvidia-smi showing 100% can hide that you are using a fraction of the hardware's real compute. The signal is MFU (useful FLOPs vs peak) and the gap between 'the GPU is busy' and 'the GPU is efficient'.Open full answer →
57Beyond basic gradient checkpointing, how do you choose selective activation recomputation to maximize MFU?▼hardNVIDIAGoogleMeta2 replies◆ premiumFull activation checkpointing saves memory but burns a flat 30% extra compute. Selective recomputation recovers most of that by only recomputing the cheap, memory-heavy operations. Here is how to pick what to recompute.Open full answer →
58How do you overlap communication with computation in distributed training, and how do you verify it works?▼hardNVIDIAMetaGoogle1 replies◆ premiumThe collective communication in distributed training is pure overhead unless it runs while the GPU computes. Hiding it is the difference between 30% and 55% MFU. Here is how the overlap actually works and how you confirm it on a trace.Open full answer →
63How does FP8 training work on Hopper GPUs, and how do you keep it numerically stable?▼expertNVIDIAOpenAIMistral1 replies◆ premiumFP8 promises to roughly double training throughput over BF16, but with only a handful of mantissa bits the numerics are unforgiving. Per-tensor scaling and a selective recipe are what make it converge.Open full answer →
