Demonstration
Conformal prediction as Bayesian quadrature
The calibration scores induce a posterior over your realized coverage, exactly \(\mathrm{Beta}(k,\,n+1-k)\). The conformal guarantee is the mean of that distribution; you could report a quantile of it instead.
Snell & Griffiths (2025) recast the conformal computation as quadrature. Sort the \(n\) calibration scores; they cut the line into \(n+1\) gaps, and by exchangeability the test score is equally likely to land in any of them: each gap carries expected probability \(1/(n+1)\), and jointly the gap masses are \(\mathrm{Dirichlet}(1,\dots,1)\). Any question of the form “how much mass sits below the threshold \(q_n=S_{(k)}\)?” is then an integral against random weights: the realized coverage of your interval is the sum of the first \(k\) gap masses, which is \(\mathrm{Beta}(k,\,n+1-k)\) exactly, whatever the score distribution. The familiar guarantee, coverage \(\ge 1-\alpha\) on average, is nothing but the mean of this posterior, \(k/(n+1)\).
The first panel shows the quadrature picture: the sorted scores, the gap masses, and the threshold. The second panel shows the \(\mathrm{Beta}(k,\,n+1-k)\) posterior over realized coverage, with a Monte Carlo histogram of actual realized coverage across many fresh calibration draws laid over it; they agree for any score distribution, which is the distribution-free point. The mean is marked, and so is the lower quantile you select: “with 90% probability over calibration draws, my coverage exceeds this” is a calibration-conditional statement the mean alone cannot make. Small \(n\) makes the posterior wide; that width is the conformal fan seen from the loss side.
Takeaway. Conformal prediction quietly computes a full posterior over its own realized coverage and then reports only the mean. Snell & Griffiths make the posterior explicit: report its lower quantile (or highest-density region, for general losses) and the same calibration data yields a strictly more informative statement. What the posterior cannot say is anything about which inputs are covered; it sharpens the calibration-conditional statement, and the information gap stands where it stood.
Using conformal prediction in your own project? Tell Claude: “Read https://conformalprediction.net/SKILL.md and create a project skill from it.” It adds a check for whether your coverage is conditionally trustworthy.