How does the baseline/drift detection work?

[AppDevIQ]

A few people have asked how DataScreenIQ's drift detection works
under the hood. Here's a quick overview:

First batch: When you screen a source for the first time, the
engine builds a baseline — schema fingerprint (SHA-256 of sorted
field names + types), null rate per column, enum value sets for
low-cardinality fields, and row count.

Subsequent batches: Every batch is compared against the stored
baseline. Drift events are classified:

Drift kind	Severity
field_added	WARN
field_removed	WARN
type_changed	BLOCK
null_spike	WARN or BLOCK
new_enum_value	WARN
row_count_anomaly	WARN or BLOCK

Baseline adaptation: Null rate baselines use exponential moving
average (EMA) so they adapt gradually. If nulls creep from 2% to 5%
over weeks, the baseline adjusts. But a sudden jump from 2% to 60%
fires immediately.

Reset: If your baseline has learned a broken state, reset it:

client.reset_baseline("orders")

Questions? Ask below.