How do you profile and diagnose LLM inference performance (TTFT, inter-token latency, GPU utilization)?

LLM serving has its own metrics, and one latency number hides the real bottleneck. The signal is splitting prefill from decode and reading GPU utilization as a clue, not a verdict. Here is the diagnostic toolkit.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.