Conformal Prediction

Literature

Literature

A working bibliography of conformal prediction, by topic. For papers that put it to work where coverage is genuinely the objective, see the applications page. For where it all sits relative to the information gap, see the map and the connections.

Curated with one filter, in keeping with the rest of the site: keep the careful methodology and the foundational theory, and the work that states the guarantee precisely; skip material that pitches conformal prediction as improving a forecast or delivering “reliable uncertainty” with the marginal-coverage caveat left out. Descriptions are our own one-line readings, not the papers’ abstracts. Corrections welcome.

Start here: surveys, books, foundations

The alrw.net working papers

Vovk and collaborators keep the working-paper series behind Algorithmic Learning in a Random World at alrw.net — the field’s primary source, often ahead of the journal versions. Several entries appear elsewhere on this page in their published forms; the complete series is folded here.

The complete series (48 working papers)
  • #1. On-line predictive linear regression. Vovk, Nouretdinov & Gammerman. Annals of Statistics 37:1566–1590 (2009). pdf
  • #2. Hedging predictions in machine learning. Gammerman & Vovk. Computer Journal 50:151–177 (2007). pdf
  • #3. A tutorial on conformal prediction. Shafer & Vovk. JMLR 9:371–421 (2008). pdf
  • #4. Plug-in martingales for testing exchangeability on-line. Fedorova, Gammerman, Nouretdinov & Vovk. ICML 2012. pdf
  • #5. Conditional validity of inductive conformal predictors. Vovk. Machine Learning 92:349–376 (2013). pdf
  • #6. Cross-conformal predictors. Vovk. Ann. Math. Artif. Intell.. pdf
  • #7. Venn–Abers predictors. Vovk & Petej. UAI 2014. pdf
  • #8. Transductive conformal predictors. Vovk. IJAIT 24(6) (2015). pdf
  • #9. Conformal prediction under hypergraphical models. Fedorova, Gammerman, Nouretdinov & Vovk. IJAIT 24(6) (2015). pdf
  • #10. Efficiency of conformalized ridge regression. Burnaev & Vovk. COLT 2014. pdf
  • #11. Criteria of efficiency for conformal prediction. Vovk, Fedorova, Nouretdinov & Gammerman. Ann. Math. Artif. Intell. (COPA 2016). pdf
  • #12. From conformal to probabilistic prediction. Vovk, Petej & Fedorova. COPA 2014. pdf
  • #13. Large-scale probabilistic prediction with and without validity guarantees. Vovk, Petej & Fedorova. NIPS 2015. pdf
  • #14. Universal probability-free prediction. Vovk & Pavlovic. Ann. Math. Artif. Intell. (COPA 2016). pdf
  • #15. On the concept of Bernoulliness. Vovk. pdf
  • #16. Test statistics and p-values. Gurevich & Vovk. COPA 2019. pdf
  • #17. Nonparametric predictive distributions based on conformal prediction. Vovk, Shen, Manokhin & Xie. Machine Learning (2019). pdf
  • #18. Universally consistent conformal predictive distributions. Vovk. Pattern Recognition 126 (2022). pdf
  • #19. Conformal predictive decision making. Vovk & Bendtsen. COPA 2018. pdf
  • #20. Conformal predictive distributions with kernels. Vovk, Nouretdinov, Manokhin & Gammerman. Braverman Readings. pdf
  • #21. Combining p-values via averaging. Vovk & Wang. Biometrika 107:791–808 (2020). pdf
  • #22. Computationally efficient versions of conformal predictive distributions. Vovk, Nouretdinov, Manokhin & Gammerman. Neurocomputing 397 (2020). pdf
  • #23. Conformal calibrators. Vovk, Petej, Toccaceli & Gammerman. COPA 2020. pdf
  • #24. Testing randomness online. Vovk. Statistical Science 36:595–611 (2021). pdf
  • #25. Non-algorithmic theory of randomness. Vovk. Gurevich Festschrift (2020). pdf
  • #26. Conformal e-prediction. Vovk. Pattern Recognition 166 (2025). pdf
  • #27. Confidence and discoveries with e-values. Vovk & Wang. Statistical Science 38:329–354 (2023). pdf
  • #28. Training conformal predictors. Colombo & Vovk. COPA 2020. pdf
  • #29. Conformal e-testing. Vovk, Nouretdinov & Gammerman. Pattern Recognition 168 (2025). pdf
  • #30. Conformal predictive distributions: an approach to nonparametric fiducial prediction. Vovk. Handbook of Bayesian, Fiducial, and Frequentist Inference. pdf
  • #31. Testing for concept shift online. Vovk. pdf
  • #32. Retrain or not retrain: conformal test martingales for change-point detection. Vovk, Petej, Nouretdinov, Ahlberg, Carlsson & Gammerman. COPA 2021. pdf
  • #33. Conformal testing in a binary model situation. Vovk. COPA 2021. pdf
  • #34. Protected probabilistic regression. Vovk. pdf
  • #35. Protected probabilistic classification. Vovk, Petej & Gammerman. pdf
  • #36. Conformal testing: binary case with Markov alternatives. Vovk, Nouretdinov & Gammerman. COPA 2022. pdf
  • #37. The power of forgetting in statistical hypothesis testing. Vovk. COPA 2023. pdf
  • #38. Testing exchangeability in the batch mode with e-values and Markov alternatives. Vovk. Machine Learning 114 (2025). pdf
  • #39. An optimality property of the Bayes–Kelly algorithm. Vovk. pdf
  • #40. Conditionality principle under unconstrained randomness. Vovk. Statistical Science 41:218–221 (2026). pdf
  • #41. Validity and efficiency of the conformal CUSUM procedure. Vovk, Nouretdinov & Gammerman. COPA 2025. pdf
  • #42. Randomness, exchangeability, and conformal prediction. Vovk. Gammerman Festschrift. pdf
  • #43. Universality of conformal prediction under the assumption of randomness. Vovk. ALT 2026. pdf
  • #44. Inductive randomness predictors: beyond conformal. Vovk. COPA 2025. pdf
  • #45. Exchangeability and randomness for infinite and finite sequences. Vovk. pdf
  • #46. Conformal e-prediction in the presence of confounding. Vovk & Wang. pdf
  • #47. Inductive Venn–Abers and related regressors. Petej & Vovk. COPA 2026, to appear. pdf
  • #48. Aggregation in conformal e-classification. Vovk. COPA 2026, to appear. pdf

Origins and the core method

Validity, exchangeability, and the limits

Game-theoretic foundations, e-values, exchangeability testing

Regression and interval methods

Conditional coverage: methods and guarantees

Conditional coverage: diagnostics and calibration tests

Classification

Risk control and beyond coverage

Selection, outliers, and novelty

Conformal training and efficiency

Distribution shift and weighted conformal

Time series and non-exchangeability

A guided tour of this section, which game each result plays and what remains open, is on the time-series pages: the coverage games and conditional coverage and sharpness.

Two threads run through this literature. One modifies the method to survive dependence — blocks, ensembles, weights, fitted residual quantiles. The other leaves split conformal untouched and prices the damage: how much coverage does temporal dependence actually cost? That accounting is now essentially settled, and the answer is remarkably benign: the loss is (dependence range) / (calibration size), linear and tight, with a matching bound against over-coverage. First the accounting:

Background for the bounds above, from the mixing literature:

And the modified methods:

Online and adaptive

Network and relational data

Conformal predictive distributions and systems

Connections: information theory, proper scores, Bayes, fiducial

Causal inference, treatment effects, and survival

Language models and generation

Software

Talks

Applications

Papers that use conformal prediction where coverage or containment is genuinely the objective, selective prediction, anomaly detection, retrieval, language models, robotics and control, risk control, scientific discovery, causal inference, survival analysis, and medical imaging, are listed and discussed, by domain, on the applications page.

Using conformal prediction in your own project? Tell Claude: “Read https://conformalprediction.net/SKILL.md and create a project skill from it.” It adds a check for whether your coverage is conditionally trustworthy.