Conformal prediction: uncertainty quantification you can actually ship

My first-author paper (ACM JCSS 2025) applies conformal prediction to energy disaggregation, inferring which appliance is running from a home's aggregate meter signal. The headline: distribution-free uncertainty intervals that beat Bayesian baselines on expected calibration error, at materially lower compute. No priors, no MCMC, no praying your variational approximation behaves.

Conformal prediction's trick is almost insultingly simple: hold out a calibration set, measure how wrong your model is on it, and use those residuals to wrap any point predictor in intervals with guaranteed coverage. The guarantee is distribution-free, it doesn't care what your model is or how the data is shaped, only that calibration and test data are exchangeable.

Why this matters beyond energy: most ML systems ship point estimates with implied confidence they don't have. Calibrated uncertainty changes what you can build, a prediction that says '87-93% likely' is something a downstream system can make decisions with; a bare '90%' from an uncalibrated model is a decoration. Cheap, guaranteed calibration is one of the most underused tools in applied ML.

The through-line to my agent work is direct. Agents constantly act on model outputs, an extraction, a classification, a judgment call. Knowing when the model doesn't know is the difference between an agent that escalates the right cases to humans and one that's confidently wrong at scale. Calibration isn't academic; it's the input to every good escalation policy I've shipped.