streaming

A classic that rewards the two-heap insight. A sorted list gives O(n) inserts; two balanced heaps give O(log n) insert and O(1) median. Here is the implementation and the rebalancing detail people get wrong.

A classic that tests whether you reach past sorting for the right structure. The signal is the heap solution (O(n log k)), the bucket-sort O(n) trick, and how it changes for an unbounded stream. Here is the answer.

The textbook variance formula loses catastrophic precision on streaming data and cannot update incrementally. Welford's one-pass update is the standard for online feature normalization. Here it is.

The classic recursive-descent exercise with an applied-AI twist: the JSON your model streams is truncated mid-token for the entire generation. The signal is a clean strict parser plus a small repair layer, not a second parser. Here is the implementation.

Every LLM product streams, and almost nobody can write the server. The interesting part is not the async generator, it is what happens when the user hits stop, the provider dies at token 90, or moderation flags text you have already put on the screen.

Threshold alerts fire all weekend and miss the real outage Monday. The signal is decomposing seasonality first, choosing the right detector, and tuning for alert fatigue. Here is the answer that survives production.

A hard streaming-systems question: dedup at high throughput with bounded state. The signal is a tiered state design (probabilistic filter in front of durable state) plus watermark-driven eviction. Here is the architecture that does not OOM.

A pipeline-design question that rewards matching the architecture to the freshness requirement, not chasing real-time for its own sake. The signal is the latency-vs-complexity tradeoff and knowing why Kappa emerged to kill Lambda's dual codebase. Here is the decision.

You cannot wait for all the data, and buffering an unbounded stream will OOM. The signal is windowed joins with watermarks for stream-stream, lookup joins for stream-table, and a clear story for late and out-of-order events.

At-least-once duplicates and at-most-once loses data; everyone wants exactly-once and few can explain how Kafka delivers it. The signal is idempotent producers plus transactional read-process-write plus the read-committed isolation that ties them together.

Events show up minutes or days after they happened, and a window that already closed gives wrong counts. The signal is event-time vs processing-time, watermarks with allowed lateness, and how you correct already-emitted aggregates.

Real-time transcription at scale tests streaming ASR, latency budgets, and GPU fleet management under heavy concurrency. The signal is streaming chunked inference with partial results, not batch transcription. Here is the design.

A voice assistant chains STT, an LLM, and TTS under a brutal latency budget where every stage adds delay. The signal is streaming and pipelining the stages plus turn-taking, not three blocking calls. Here is the design.

Online models need features that are fresh and computed exactly the way training computed them. The classic failure is training-serving skew that silently degrades the model. The signal is the online/offline split with one shared definition. Here is the design.

A writing assistant has to feel instant while a cursor moves through a live document. Learn how to split fast deterministic checks from LLM rewrites, stream suggestions, and keep edits stable so the text does not flicker as the user types.

Live speech-to-speech translation is a latency race across three models (recognize, translate, synthesize) while audio keeps streaming in. Learn the streaming pipeline, how to commit partial results without flip-flopping, and the tradeoff between latency and translation quality.

Not every prediction needs a low-latency endpoint, and defaulting to one burns money. The signal is matching the serving pattern to the latency and freshness requirement. Here is the decision and when you'd reverse it.

A fraud model fed a feature an hour behind is half-blind, but recomputing everything in real time burns money you don't need to spend. Freshness is a per-feature decision on a real cost curve. Here is how to manage it.