schema evolution

A new column is safe; a rename silently corrupts every dashboard downstream. The signal is knowing which changes are compatible, enforcing data contracts in CI, and running expand-contract migrations for the breaking ones.

Raw clickstream lands in object storage and somehow becomes clean tables, dashboards, and ML features. The design questions are table format, the bronze-silver-gold layering, batch versus streaming, and how you handle schema drift and late data. Here is the lakehouse blueprint.

A data engineer renames a column and three production models silently start scoring on garbage. Features are a shared contract, and changing them needs the same care as a breaking API change. Here is the discipline.