Data Drift in Production Pipelines: The Silent Quality Killer

Your ingestion job ran. Your dbt models compiled. Your tests passed. And yet, the weekly revenue number is off by 4%, and nobody can tell you why.

Welcome to data drift — the slow, statistical decay of your inputs that never trips a hard check but quietly erodes the meaning of every metric built on top of them.

What Drift Looks Like

Drift is not a failure. It is a shift. Common varieties:

Distribution Drift

A categorical column that used to be 60/40 suddenly becomes 80/20. No nulls, no type errors, just a different world.

• Upstream SaaS vendor reclassified a field
• Marketing launched in a new geography
• A product change funneled users into a different path

Volume Drift

Row counts drift upward or downward slowly over weeks. Hard-coded thresholds never fire because no single day exceeds them.

Semantic Drift

A field's meaning changes while the name stays the same. status = "active" used to mean paying. Now it means "trial or paying." Every downstream metric that assumed the old definition is now wrong.

Schema Drift

New columns appear. Old ones get nullable. Types loosen from int to float. Your pipeline swallows it. Your joins silently drop rows.

Why Standard Tests Miss It

Most dbt tests look like this:

select  from orders where total < 0

That catches the obvious. It misses the subtle. Drift requires looking at the shape of your data over time, not just its current state.

What to Monitor

Track these continuously, not once:

Signal	What to watch
Row count	Week-over-week delta outside expected range
Null rate	Sudden increase in any column
Distinct count	New values in low-cardinality columns
Distribution	KS test or PSI between current and baseline window
Schema	Column additions, removals, or type changes

Responding to Drift

Drift detection without a response plan is just noise. When a drift alert fires:

1. Triage — Is this a real change or an upstream mistake?
2. Trace — Which downstream models consume this column?
3. Decide — Update the baseline, fix the source, or update downstream logic?
4. Document — Record the decision so the next drift on the same column has context.

The Bottom Line

Pipelines that pass their tests are not the same as pipelines that produce trustworthy data. Drift monitoring closes the gap between "it ran" and "it's right."

---

Want drift monitoring without building it yourself? Talk to us.*