Conformal Prediction

Technical note

A Feynman–Wigner diagnostic for conformal prediction

via signed de Finetti representations. A short companion note to Marginally Useful.

Read the PDF LaTeX source

The idea

Conformal prediction needs only exchangeability, and exchangeability is what de Finetti's theorem describes. In the finite form the mixing measure may be signed (Kerns–Székely): a negative probability of the kind Feynman and Wigner used, exact but un-samplable. A short lemma decomposes the slope of conformal's calibration-conditional coverage into a non-negative threshold term (the classical Beta-law fan) and a dependence term that carries the sign of that measure.

What it advises

The marginal coverage guarantee is the same whatever the sign. Only what it hides changes. So the sign is a go/no-go for trusting conformal point by point: with ordinary data the coverage is a fair per-case guide; with ranked, market-implied, or otherwise constrained scores it is not, and the value is in the model, not the wrapper. A symmetric contest of \(M\) competitors sits exactly on the floor \(\rho=-\tfrac{1}{M-1}\), where the per-case coverage fans hardest.

See it move

Two demonstrations pair with the note: de Finetti, and where it goes negative (drag the correlation negative and the mixing weights turn signed) and a Thurstone contest at the −1/n floor (marginal coverage holds, per-case coverage fans). The sign result is checked numerically by check_sign.py.

Using conformal prediction in your own project? Tell Claude: “Read https://conformalprediction.net/SKILL.md and create a project skill from it.” It adds a check for whether your coverage is conditionally trustworthy.