14How do you select GPUs for LLM training and inference, and what specs actually matter?▼mediumNVIDIAOpenAIxAI1 replies○ sign inPicking GPUs is a real applied decision, and 'get the biggest one' misses the point. The signal is knowing which spec binds your workload (VRAM, bandwidth, interconnect) and the train-versus-serve difference that flips the answer.Open full answer →
62Your training collectives are slow. How do you debug the NCCL/interconnect path and find where bandwidth is lost?▼hardNVIDIAMetaMicrosoft2 replies◆ premiumWhen all-reduce is the bottleneck, the cause is almost always a misconfigured path: traffic on the wrong link, a downed NIC, or a topology NCCL never discovered. Here is the systematic way to find the lost bandwidth.Open full answer →
