← 🗄️ Data & SQL EngineeringNEXT IN DATA & SQL ENGINEERINGBackfills and Reprocessing→
Core
Pipeline Orchestration and DAGs
Orchestration runs dependent data tasks as a DAG so each task waits for its upstreams, retries safely, backfills history, and alerts when an SLA is missed. Tools like Airflow, Dagster, and dbt exist because cron cannot express dependencies, recovery, or partial reruns. Applied-AI interviews probe it because candidates reach for cron, then cannot explain what happens when task three of seven fails at 3am or when you need to reprocess last month.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
MLOps & ML EngineeringHow do you orchestrate ML pipelines (Airflow, Kubeflow, etc.), and what makes ML pipelines special?→MLOps & ML EngineeringWhat DAG design patterns make an ML orchestration pipeline reliable in Airflow or Dagster?→SQL & Data EngineeringYou shipped a logic bug three months ago. How do you safely backfill and reprocess the affected data?→SQL & Data EngineeringImplement Slowly Changing Dimension Type 2 history tracking in a Delta lakehouse.→RAG & Agent System DesignWhen do you build an agent instead of a single LLM call, and how do you keep a multi-step agent reliable?→SQL & Data EngineeringGroup a stream of user events into sessions in SQL (30-minute inactivity gap) using window functions.→
COMPANIES THAT ASSUME THIS