75Implement a learning-rate scheduler with linear warmup and cosine decay.▼mediumGoogleMetaOpenAI1 replies◆ premiumAlmost every modern training run uses warmup-then-cosine, and getting it wrong destabilizes early training or wastes the tail. It's a closed-form function of the step. Here is the implementation and the why.Open full answer →
48Design the request router and scheduler for a multi-replica LLM serving cluster.▼expertOpenAINVIDIAAWS2 replies◆ premiumRound-robin routing wastes a feature that doubles throughput: cache locality. Designing the router means choosing a replica by KV state, not just least-loaded. Here is the two-layer design.Open full answer →