TL;DR: A feature store computes, stores, and serves features for both training (offline, batch) and inference (online, low-latency) from the same definitions. Its main job is to kill training-serving skew: features computed one way in training and another way at serving. It also enforces point-in-time correctness so training never sees future data.
How to approach it. Define it in one line, then pivot fast to the problem it solves, because that is what the interviewer is testing. Name the two access patterns (offline for training, online for serving) and the two correctness properties (parity and point-in-time joins).
A strong answer. A feature store sits between raw data and models with three jobs: a single place to define features, an offline store (a warehouse or lake) that serves large historical feature sets for training, and an online store (Redis, Cassandra, Cosmos) that serves the latest feature values at low latency for inference. The architectural win is that both paths derive from one feature definition.
The core problem it prevents is training-serving skew: the model trains on features computed by a batch job (full history, careful joins) but at serving time a different code path computes them (a cached value, a different default, a slightly different aggregation). The model then meets inputs in production it never saw in training, and quietly underperforms. A feature store enforces parity by materializing both the offline and online stores from shared definitions.
The second property is point-in-time correctness. When you build a training set, each label at time t must join only feature values that were known before t. A naive join grabs the latest feature value, leaks future information, and produces a model that looks great offline and fails live (the classic temporal-leakage trap). Feature stores do as-of (point-in-time) joins so training reflects exactly what would have been available at prediction time.
| Property | The bug it stops | Mechanism |
|---|---|---|
| Parity | Training-serving skew | One definition materialized to both stores |
| Point-in-time | Label leakage from future data | As-of joins gated by label timestamp |
Bonus value: feature reuse across teams and models, versioning and lineage, freshness monitoring, and backfills.
Key takeaways
- One feature definition feeds both an offline (training) and online (serving) store.
- Parity from shared definitions is what kills training-serving skew.
- Point-in-time joins stop future data from leaking into training labels.
- Skip it for a single model with simple features; the value compounds with many models and real-time needs.
What interviewers probe next.
- "How do you guarantee offline/online parity?" Shared transformation code or a declarative definition that materializes to both stores; tests that compare offline-computed and online-served values for the same entity and time.
- "Online store choice?" A low-latency key-value store keyed by entity id; size for p99 read latency and write throughput from the streaming pipeline.
- "Streaming vs batch features?" Batch for slowly-changing aggregates, streaming for real-time counters; both land in the online store with freshness SLAs.
- "Do you always need one?" No. For a single model with simple features it is overhead; the value compounds with many models, shared features, and real-time needs.
Common mistakes.
- Defining it as "a database for features" and missing skew and point-in-time correctness, the reasons it exists.
- Latest-value joins when building training data, leaking the future.
- Two separate code paths for training and serving features, reintroducing the skew the store is meant to remove.
- Adding a feature store before the complexity justifies it.