Where it shines
Calibrated anomaly detection
A conformal \(p\)-value is a distribution-free hypothesis test, so you can fix the false-alarm rate exactly.
This is conformal prediction at its cleanest, working as a test rather than a forecaster. Score each point by how unusual it is against a clean reference set, then form the conformal \(p\)-value $$p(x) = \frac{1 + \#\{\,i : s_i \ge s(x)\,\}}{n+1}.$$ For a genuine inlier this \(p\)-value is (super-)uniform, so flagging whenever \(p \le \alpha\) controls the false-alarm rate at exactly \(\alpha\), no matter what the inlier distribution is, with a finite calibration set, no training of a threshold. Below, each test point is plotted at its \(p\)-value; everything under the dashed line is flagged. The inliers’ \(p\)-values fill out the uniform histogram, and the false-alarm rate tracks \(\alpha\) as you drag it. Detection power, catching the real anomalies, is the score’s job, and improves as you push the anomalies further out.
The conformal \(p\)-values of the true inliers, which are uniform on \([0,1]\) (dashed line). That uniformity is the whole guarantee: the fraction landing below \(\alpha\) is \(\alpha\), so the false-alarm budget is honoured by construction.
Takeaway. Type-I error control is coverage, and coverage is what conformal prediction certifies, distribution-free and finite-sample. You set the false-alarm budget and it is kept regardless of the unknown normal distribution. The sharper the score, the more anomalies you catch within that budget, but the budget itself is guaranteed. This is the rare setting where the marginal guarantee is the entire point.