Demonstration
The conformal fan: dependence and coverage variance
Drag the correlation and watch the inequalities of the fan note move. The mean of the realized coverage shifts with the dependence too, but the variance is what the inequalities pin down.
Fix a split-conformal calibration set of \(n\) scores and ask how often a fresh, independent draw lands under the threshold \(U_{(k)}\), \(k=\lceil(n+1)(1-\alpha)\rceil\). That realized coverage \(c=U_{(k)}\) is a random number; for independent scores it follows the \(\mathrm{Beta}(k,n-k+1)\) law, the classical fan. The note asks what cross-sample dependence does to it. Here the scores are an equicorrelated Gaussian copula at correlation \(\rho\); drag \(\rho\) from the floor \(-1/(n-1)\) up through \(0\) to \(+0.9\).
The top panel is the distribution of \(c\) (orange) against the iid fan (blue). At the floor the fan contracts, here to roughly half the iid variance; near \(\rho=0\) it is the Beta fan; at large \(\rho\) it blows up. The readouts track the two quantities the note studies on the negative side: the per-level ratio \(\operatorname{Var}(c)/\text{fan}\le 1\), and the aggregate \(\sum_k\operatorname{Var}(U_{(k)})/\text{iid}\le 1\) (Theorem 4, proved for negative association). On the Gaussian scale the exact affine law \(\operatorname{Var}(Z_{(k)})=v_k+\rho(1-v_k)\) (Proposition 2) is checked against simulation.
The second panel shows the variance of every order statistic \(\operatorname{Var}(U_{(k)})\) (orange) against the iid Beta variance curve (blue). For \(\rho<0\) the orange curve sits entirely under the blue one, a contraction at every level, and its total area is the aggregate bound. For \(\rho>0\) it lifts above. Because the marginals are fixed uniform, the whole picture is a property of the copula alone: the fan is a copula functional, and the two variance bounds depend only on the bivariate copula diagonal at the operating quantile.
The other buttons make that copula viewpoint concrete. Beyond the symmetric Gaussian and the MA(1) copula, the discrete contest/grid floor (variance exactly zero) and a Latin hypercube (continuous marginals, fan of order \(1/n^2\)), there are two Archimedean copulas with one-sided tail dependence: Clayton (lower tail) and Gumbel (upper tail), sampled through their de Finetti frailty \(W\), the directing variable of Theorem 2. They reveal a surprise. Tail dependence pins the order statistics in its own tail and pushes the fan to the opposite one: Clayton (small scores cluster) fattens the upper order statistics, Gumbel the lower, both far above iid. The “tail asymmetry” readout reports the low-\(k\) and high-\(k\) variance ratios so you can watch them split. Since the conformal \(k\) sits high (small \(\alpha\)), lower-tail dependence inflates the conformal fan more than upper-tail dependence, which is the reverse of the naive guess.
Takeaway. The dependence does not, as one might guess, just bias the coverage; it reshapes its whole distribution, and the cleanest summary is the variance. Negative association narrows the fan, to zero at the discrete contest/grid floor, where realized coverage is k/(n+1) on every draw; positive widens it. The note proves this exactly in aggregate and, for these equicorrelated normals, exactly per level via the affine law you can read off the panel. The per-level contraction for general negative association is still a conjecture, but every structure here keeps the orange curve under the blue. It is the same sign the signed-de-Finetti note reads off the finite mixing measure: a genuine prior widens the fan, the signed corner collapses it.
Using conformal prediction in your own project? Tell Claude: “Read https://conformalprediction.net/SKILL.md and create a project skill from it.” It adds a check for whether your coverage is conditionally trustworthy.