FAQ
Objections & FAQ
Straight answers to the questions the guide tends to raise.
So is conformal prediction useless?
Quite the opposite, it is the right tool for every coverage-objective problem in the guide (prediction sets, anomaly detection, retrieval, robotics, language models, risk control), and the only thing that hands you a distribution-free, finite-sample coverage certificate. The single thing it does not do is make a forecast sharper or a density better.
Doesn’t conformalizing my model improve its uncertainty?
It gives the prediction set a coverage guarantee. It does not change your predictive distribution’s sharpness, and a proper score (log-likelihood, CRPS) is unmoved by the conformal step. If the model is overconfident, conformal widens the set to hit coverage, but the density it implies is just your model’s residual shape, re-levelled.
But conformal predictive systems and CQR output whole distributions!
They do, and they can be excellent. The point is attribution: their sharpness comes from the conditional model they wrap, the quantile regressor, the difficulty estimator, the per-region bins, not from the conformal step, which supplies the coverage certificate. A worked example shows raw quantile regression matching its conformalized version.
Can I get conditional (per-\(x\)) coverage?
Not exactly, and not for free: distribution-free, finite-sample conditional coverage forces infinite-length intervals (the price of conditional coverage). You can approach it under assumptions, conformalized quantile regression, normalized/Mondrian conformal, recent pivotal-score and optimal-transport methods, all of which condition on \(x\).
Does it work for time series?
The basic guarantee needs exchangeability, which time series violate. Adaptive methods (ACI, conformal PID, EnbPI) restore a long-run average coverage and are well worth using; just don’t read them as a per-step guarantee (see time series).
Then what should I use to make forecasts better?
Model the conditional distribution (heteroscedastic models, quantile regression, mixture or flow densities) and recalibrate against a proper score. Then conformalize, if you also need a distribution-free coverage certificate. The order matters: estimate first, certify second.
Conformal prediction certifies the coverage of a set; it does not estimate a distribution. Use it, happily, whenever you need a coverage guarantee. Just don’t expect the certificate to make the forecast underneath it any sharper; that work belongs to the model. Genuinely useful, then, but, in the exact sense of that average over inputs, only marginally so.