← 🗄️ Data & SQL EngineeringNEXT IN DATA & SQL ENGINEERINGBatch vs Streaming→
Core
Incremental Models and MERGE/UPSERT
Incremental models process only new or changed rows instead of rebuilding a table from scratch, using a high-watermark to select the delta and a MERGE/UPSERT to apply it. The hard parts are late-arriving data, idempotent re-runs, and choosing a watermark that does not silently drop rows. Applied-AI interviews probe it because full refreshes do not scale, and a subtly wrong incremental quietly loses or double-counts data.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
SQL & Data EngineeringImplement Slowly Changing Dimension Type 2 history tracking in a Delta lakehouse.→SQL & Data EngineeringWhat is a MERGE (upsert), and how do you use it for incremental loads and SCDs?→SQL & Data EngineeringGiven rows with start and end timestamps, merge all overlapping intervals per user in SQL.→SQL & Data EngineeringGroup a stream of user events into sessions in SQL (30-minute inactivity gap) using window functions.→SQL & Data EngineeringDesign a data pipeline that is safe to re-run: idempotent writes, late data, and exactly-once effects.→SQL & Data EngineeringFind the top-N records per group and a running total per group in SQL.→
COMPANIES THAT ASSUME THIS