vllm
Applied AI interview questions tagged vllm, across every topic.
5 questions · 2 unlocked for you
Concepts behind "vllm"
The curriculum that explains the ideas these questions test.
Core
PagedAttentionThe KV cache is the memory bottleneck in LLM serving, and naively reserving a contiguous block per request (sized for the maximum length) wastes most of it to fragmentation and over-allocation. PagedAttention borrows virtual-memory paging: store the KV cache in fixed-size non-contiguous pages allocated on demand, so memory is used only as tokens are generated. This packs far more concurrent requests onto a GPU, raising throughput. Applied-AI interviews probe it because it is the key memory innovation behind modern serving (vLLM).🖥️ ML Infrastructure & ServingSign in
Core
Model Serving FrameworksYou rarely build a serving stack from scratch; frameworks handle the production plumbing. General servers (Triton, TorchServe, KServe) serve many model types with dynamic batching, multi-model hosting, and versioning. LLM-specific servers (vLLM, TGI, TensorRT-LLM) add the essentials general servers lack: continuous batching, paged KV cache, and token streaming. Applied-AI interviews probe it because knowing what these provide, and that LLM serving needs the specialized ones, is practical deployment knowledge.🖥️ ML Infrastructure & ServingSign in