Idempotent Data Pipelines

Data pipelines fail and get rerun, so a pipeline must be idempotent: rerunning it produces the same result, not duplicated or corrupted data. You achieve it with insert-overwrite by partition, MERGE/upsert keyed on a business id, and deterministic transforms, rather than blind appends that double-count on retry. Applied-AI interviews probe it because flaky pipelines are the norm, and a non-idempotent pipeline turns a routine retry into duplicated revenue numbers or a corrupted table.