Conformal Prediction

Demonstration

Balanced placements

Conformal placements are a zero-sum population. Order them with the balanced word and the running acceptance average stays within \(1/(2t)\) of the orbit level \(k/(n+1)\) at every prefix, for any scores.

Draw \(N=n+1\) scores from whichever distribution you like; the choice changes nothing here. Rotating the “test” label through the slots covers exactly \(k=\lceil(1-\alpha)N\rceil\) of the \(N\) placements, so the centered residuals \(\chi_i - k/N\) sum to zero: a scalar Steinitz population. The question is the order in which a traversal visits the placements. What the traversal controls is the running placement-acceptance average, the fraction of placements visited so far that are covered; it is a deterministic quantity, not a sequential coverage rate. A random order lets it wander on the \(1/\sqrt t\) scale before the endpoint forces it home. The balanced word \(b_t=\lfloor tk/N+\tfrac12\rfloor-\lfloor(t-1)k/N+\tfrac12\rfloor\) pins every prefix within \(\tfrac12\) of \(tk/N\), so the average sits inside the \(\pm 1/(2t)\) envelope at every step and lands exactly on \(k/N\) at \(t=N\).

The panel plots the deviation of that average, \(\frac1t\sum_{j\le t}\chi_{\pi(j)} - k/N\) for both orders, with the deterministic envelope drawn. Switch the score distribution to adversarial and nothing moves: the balance is a property of the ranks, not the values. This is the duality of the note in one picture: move the residuals through a permutation, or hold them fixed and move the prefix functionals; either way the ledger never exceeds half a unit.

Takeaway. The balanced traversal calibrates the orbit: every prefix of placements has acceptance average within \(1/(2t)\) of the orbit level \(k/N\), deterministically. That is an exact statement about the orbit, not a claim about sequential coverage. It does not replace exchangeability in one-shot conformal prediction, where a single placement is designated as the test; only exchangeability makes that placement an even draw on the orbit. The traversal is for orbit computations and repeated-placement audits, and it is the conformal face of Steinitz balancing.

Using conformal prediction in your own project? Tell Claude: “Read https://conformalprediction.net/SKILL.md and create a project skill from it.” It adds a check for whether your coverage is conditionally trustworthy.