Is Your Model Going Stale? Practical Tests for Data Drift

Quick take. Machine learning models tend to fail silently, so you have to make them fail loudly [1]. Monitor inputs, predictions and outcomes, because each one catches a different kind of drift. Use the population stability index for a readable per-feature score, use the Kolmogorov-Smirnov test with care at production volumes, and tie every alert to a first action.

Silent failure

Rabanser, Günnemann and Lipton opened their study of shift detection with a blunt observation: machine learning systems, which depend strongly on properties of their inputs, tend to fail silently [1]. A pump that is about to fail starts to vibrate. A model that has gone stale keeps producing confident numbers.

The models I carry into production support decisions that operations teams make every day. A silent failure there costs more than a loud one, so drift monitoring belongs in the build, not in a later phase.

Three kinds of drift

A classifier learns the relationship between inputs x and an outcome y from a training window. Moreno-Torres and colleagues proposed a unifying vocabulary for the ways that relationship can break [2]:

Gama and colleagues use “concept drift” for the case where the relation between the input data and the target changes over time [3]. It is the hardest case, because the inputs can look perfectly normal while the model is wrong.

What to monitor

Each layer catches something the others miss:

• Inputs. Per-feature distributions against a reference window, plus schema checks: missing rates, out-of-range values, new categories. Breck and colleagues list “data invariants hold in training and serving inputs” as a core monitoring test [4].
• Predictions. The score distribution and the predicted positive rate. Sculley and colleagues describe prediction bias, comparing the distribution of predicted labels with observed labels, as a surprisingly useful diagnostic [5].
• Outcomes. Precision, recall and cost once labels arrive. Only this layer directly reveals concept shift [4].
• Skew and age. Whether serving features match the training features computed for the same examples, and how old the deployed model is [4].

Test 1: population stability index by hand

The population stability index (PSI) compares how a feature’s values spread across bins now with how they spread in a reference window:

PSI = sum over bins of (a - e) × ln(a / e)

Here e is the reference share in a bin and a is the current share in the same bin. The steps:

1. Fix bin edges on the reference data, for example deciles of the training distribution.
2. Compute e and a for each bin as proportions.
3. Replace any zero share with a small floor, such as 0.0001, so the logarithm is defined.
4. Add up the bin terms.

Illustrative example: a viscosity feature with four bins.

For the first row: (0.15 - 0.25) × ln(0.15 / 0.25) = (-0.10) × (-0.511) = 0.051. Adding the four rows gives a PSI of 0.105.

The usual rules of thumb come from credit scoring. They read a PSI below 0.10 as little shift and above 0.25 as a significant shift that calls for action, with the band between as a zone to watch [6]. Yurdakul points out that these thresholds are used without reference to Type I or Type II error rates [6]. PSI also depends on the number of bins and on sample size. Treat the thresholds as a starting point: compute PSI between consecutive stable weeks to learn your own noise floor, then alert on departures from it.

Test 2: Kolmogorov-Smirnov, and the large-sample trap

The two-sample Kolmogorov-Smirnov (KS) test compares two empirical cumulative distribution functions (CDFs), the share of values at or below each point. Its statistic is:

D = the largest absolute gap between the two CDFs, over all values

Illustrative: if 40% of reference values fall below 12.0 and 52% of current values do, the gap at 12.0 is 0.12. D is the largest such gap anywhere on the scale.

For large samples, the test rejects “same distribution” at the 5% level when D > 1.36 × sqrt((n + m) / (n × m)), where n and m are the two sample sizes. The constant comes from the limiting distribution tabulated by Smirnov [7]. Watch what happens to the critical gap as data grows:

With a day of high-frequency sensor data, a gap of a fraction of a percentage point counts as “significant”. That is statistically correct and operationally useless. Three fixes:

• Alert on D itself, as an effect size, with a threshold that matters for the model.
• Test on fixed-size random samples, so sensitivity stays constant over time.
• When you test many features at once, correct for multiple comparisons, or many alerts will be false.

For high-dimensional inputs, Rabanser and colleagues found that a two-sample-testing approach, using pretrained classifiers for dimensionality reduction, performed best across the shifts they explored [1].

From alert to action

If only the base rate has moved, you may not need to retrain at all. Elkan showed how to adjust a correct probability estimate analytically when the base rate changes, assuming the distributions within each class stay the same [8]. After any of these actions, re-derive the decision threshold, because the costs and the base rate it was built on may have moved too.

Monitoring checklist

• Store the reference window with the model version it trained.
• Per feature, daily or weekly: missing rate, out-of-range rate, new categories and PSI.
• Per model: score distribution, predicted positive rate, and the observed rate once labels arrive.
• Outcome metrics for the decision you care about, with the label delay recorded.
• Training/serving skew on a logged sample of serving inputs.
• Model age, with a retraining trigger.
• An owner and a first action for every alert.

Governance frameworks point the same way. The NIST AI Risk Management Framework asks that an AI system’s functionality and behaviour be monitored in production (subcategory MEASURE 2.4) [9]. The tests above are one practical way to meet that expectation.

Drift is not a defect in the model. It is the world changing. The failure is not noticing.

References

1. S. Rabanser, S. Günnemann, and Z. C. Lipton, “Failing loudly: An empirical study of methods for detecting dataset shift,” in Advances in Neural Information Processing Systems 32 (NeurIPS 2019), 2019.
2. J. G. Moreno-Torres, T. Raeder, R. Alaiz-Rodríguez, N. V. Chawla, and F. Herrera, “A unifying view on dataset shift in classification,” Pattern Recognition, vol. 45, no. 1, pp. 521-530, 2012.
3. J. Gama, I. Žliobaitė, A. Bifet, M. Pechenizkiy, and A. Bouchachia, “A survey on concept drift adaptation,” ACM Computing Surveys, vol. 46, no. 4, art. 44, 2014.
4. E. Breck, S. Cai, E. Nielsen, M. Salib, and D. Sculley, “The ML test score: A rubric for ML production readiness and technical debt reduction,” in Proc. IEEE Int. Conf. Big Data, 2017.
5. D. Sculley et al., “Hidden technical debt in machine learning systems,” in Advances in Neural Information Processing Systems 28 (NIPS 2015), 2015, pp. 2503-2511.
6. B. Yurdakul, “Statistical properties of population stability index,” Ph.D. dissertation, Western Michigan University, 2018.
7. N. Smirnov, “Table for estimating the goodness of fit of empirical distributions,” The Annals of Mathematical Statistics, vol. 19, no. 2, pp. 279-281, 1948.
8. C. Elkan, “The foundations of cost-sensitive learning,” in Proc. 17th Int. Joint Conf. Artificial Intelligence (IJCAI), 2001, pp. 973-978.
9. National Institute of Standards and Technology, “Artificial Intelligence Risk Management Framework (AI RMF 1.0),” NIST AI 100-1, January 2023.