Your training collectives are slow. How do you debug the NCCL/interconnect path and find where bandwidth is lost?

When all-reduce is the bottleneck, the cause is almost always a misconfigured path: traffic on the wrong link, a downed NIC, or a topology NCCL never discovered. Here is the systematic way to find the lost bandwidth.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.