ttft
Applied AI interview questions tagged ttft, across every topic.
2 questions · 0 unlocked for you
Concepts behind "ttft"
The curriculum that explains the ideas these questions test.
Foundational
Latency Budgets and StreamingLLM latency is not one number: time-to-first-token (set by prefill and queueing) and inter-token latency (set by decode) feel very different to users. Streaming tokens as they generate hides total latency by showing progress immediately. Designing to a latency budget means allocating time across retrieval, model, and tools, measuring TTFT and tokens-per-second (not just end-to-end), and using streaming, caching, and routing to hit it. Applied-AI interviews probe it because perceived latency makes or breaks LLM UX.⚙️ System Design for AI in Production
Advanced
Disaggregated Prefill/Decode and Prefix CachingLLM inference has two phases with opposite hardware profiles: prefill is compute-bound (it processes the whole prompt in parallel) while decode is memory-bandwidth bound (one token at a time). Running both on the same GPU pool makes them fight, so long prefills stall ongoing decodes and you miss either the time-to-first-token or the time-per-output-token SLO. Disaggregation runs them on separate GPU pools and transfers the KV cache between them, and prefix caching reuses KV for shared prompt prefixes. Applied-AI interviews probe it because it is the current frontier of serving architecture and a real latency-SLO tradeoff.🖥️ ML Infrastructure & Serving🔒 Premium