Review

Relational conformal prediction

Cini, Jenkins, Mandic, Alippi and Bianchi train a graph neural network on the residuals of a pre-trained point forecaster to output residual quantiles, and take the interval to be the point forecast plus the fitted quantiles. Whether that deserves the name conformal was argued at ICML and is settled below in two sentences with sources. The substance is elsewhere. The paper’s own synthetic experiment shows the network recovering a term the point forecast left out rather than measuring uncertainty, and a rerun of its METR-LA experiment finds that a lookup table of split-conformal quantiles keyed by sensor, hour of day and current speed, with no graph and nothing trained, takes 85% of the distance from split conformal to the network, and that the network with its graph removed loses half a point. Demonstration 36 rebuilds the construction on a case with known truth.

What is proposed

The paper is Cini, Jenkins, Mandic, Alippi and Bianchi, “Relational Conformal Prediction for Correlated Time Series,” ICML 2025. There are \(N\) univariate series observed on a common clock, a point forecaster \(F_\theta\) trained on the first 40% of the record, and residuals \(r^i_t = x^i_t - \hat x^i_t\) on the next 40%, which the paper calls the calibration set. A spatiotemporal graph network \(Q_\psi\) reads a window of recent residuals at every series, plus the series values and time-of-day encodings, passes messages over a learned \(K\)-nearest-neighbour graph, and outputs quantiles of the residual at each series. It is trained on the calibration residuals “by minimizing the pinball loss over a discrete set of quantiles,” 39 levels from 0.025 to 0.975, and the interval is Eq. 14,

$$\hat C^{\alpha}_{i,t} \;=\; \big[\, \hat x^i_{t+H} + \hat q^{\,i,\alpha/2}_{t+H},\;\; \hat x^i_{t+H} + \hat q^{\,i,1-\alpha/2}_{t+H} \,\big].$$

The abstract states that “Our approach provides accurate coverage and achieves state-of-the-art uncertainty quantification in relevant benchmarks,” and the contributions list “A novel, sound, and effective CP method.” The theory is Proposition 3.1: coverage is at least \(1-\alpha\) minus the total variation distance between the fitted residual law and the true one. Its proof is two lines, the definition of total variation and the observation that “For \(\hat C\) in both Eq. 14 and Eq. 16 we have by construction that \(P_\psi(x \in \hat C) = 1-\alpha\).” The experiments report, at \(\alpha = 0.1\), the coverage gap, the mean width and the Winkler score of the intervals on the last 20% of three real datasets and one synthetic one, against split conformal, two weighted variants, SPCI, HopCPT and an ablation called CoRNN, for three base forecasters each.

What has already been said

The OpenReview record settles the naming question. Reviewer uUN5 wrote that “the method lacks a theoretical guarantee of non-asymptotic valid coverage. Therefore, claiming that it falls within the conformal framework is not justified,” that “Proposition 3.1 follows from the definition of the total variation distance,” and that “CoREL consistently fails to achieve the targeted marginal coverage across datasets. As a result, it is unclear whether the smaller prediction interval widths (or lower Winkler scores) reflect the superiority of the proposed method or simply stem from its anti-conservativeness.” Reviewer oqTA: “there is really no guarantees other than ‘let’s hope our quantile regression works well’.” The authors agreed that “Without any strong assumption, CP methods for TS forecasting cannot offer non-asymptotic coverage guarantees,” the paper was accepted, and their own later paper ResCP describes methods of this kind as ones “which must necessarily be updated when time-invariance is lost.” The one published head-to-head is Guo and co-authors (2026), whose spectral method beats CoRel on METR-LA by a small margin at three levels. We take all of that as given.

The synthetic experiment shows a residual forecaster

Section 5.2 simulates a diffusion on a 60-node graph with Gaussian noise of standard deviation 0.4 and states that “the theoretical optimum PI width to obtain (asymptotically) 90% marginal coverage is 1.315.” With a univariate RNN as the base model, split conformal needs width 1.67 and CoRel reaches 1.33; with a graph forecaster as the base model, split conformal alone reaches 1.32 and CoRel adds nothing. The paper presents the first pair as CoRel “effectively able to capture and leverage existing spatiotemporal dependencies.” The second pair says what those dependencies are. The residual of the univariate model contains the neighbours’ contribution to the next value, which is a predictable term in the conditional mean, and the graph quantile network learns to shift its quantiles by it. That is a second-stage point forecast of the residual with a spread attached, and once the base forecaster knows the graph there is nothing left for it to find. The construction is useful when the base model cannot be retrained. It is not evidence that relational structure carries information about uncertainty, and in this experiment it carries none: the noise is homoscedastic and the optimal interval width is the same constant at every node.

One ablation for two designs, and no baseline sees the other series

The comparison in Table 1 sets CoRel against methods that each read one series in isolation. The one control that shares information across series is CoRNN, of which the paper says “we use the same architecture as CoRel but remove message-passing layers and node embeddings. CoRNN is an ablation of the introduced designs.” It removes two designs at once: the learned graph, and the per-series parameters that let the shared network carry a different scale for each sensor. The gap between CoRNN and CoRel, 30.2 against 23.8 on METR-LA with the RNN base, is attributed to the graph. Running the two ablations separately, below, says otherwise. Keeping the message passing and removing the embeddings scores 27.0; keeping the embeddings and removing the message passing scores 24.2, within half a point of the full model. The learned graph on its own is worth three of the six points, the per-sensor parameters are worth all six, and the graph adds under half a point once they are present. SPCI, the closest method in spirit, was run without its retraining, as App. F states: “we trained each SPCI model only once on the calibration data, rather than implementing the time-adaptive approach where models are re-trained as new observations become available.” No baseline is given the two inputs the quantile network receives alongside the residuals, the hour of day and the current value of the series, and both are strong predictors of how far a traffic forecast can be wrong.

How it is calibrated

Nothing is held out. The released code splits the calibration residuals 90/10 in time, trains the quantile network on the first part and picks the checkpoint by the Winkler score at the 90% level on the second. The Winkler score at level \(\alpha\) is \(2/\alpha\) times the sum of the two pinball losses at \(\alpha/2\) and \(1-\alpha/2\), so the model is selected on its own training loss and reported on the same score. That is a perfectly good quantile regression benchmark, and it is what the paper is. The reviewers’ point about coverage is then just what one expects: a fitted quantile sits inside the truth, all nine CoRel cells of Table 1 have a negative gap, from \(-0.9\) on METR-LA to \(-5.0\) on CER-E, Table 4’s width-minimising offset widens every gap, and Table 3’s adaptation leaves CER-E at \(-2.4\) to \(-3.1\). Conformalizing the fitted quantiles on the slice the code already holds out would fix this at a small cost in score, which Demonstration 36 shows on a case with known truth, but it is not where the paper’s result lies.

Coverage is pooled over every sensor and every step

App. E states that “All the metrics are then averaged over all nodes and time steps within the specified set.” Section 2.1 defines conditional coverage, Eq. 4, and the guarantee in Proposition 3.1 is stated for one series at one step, conditional on the past. Neither is measured. A single pooled coverage number hides how it is distributed across the 207 sensors, and in our rerun below the pooled split-conformal interval that the paper reports as covering at \(-1.3\) points covers individual sensors anywhere between 71% and 99.8%. That spread is not a property of conformal prediction; it is what pooling does, and one line of bookkeeping removes it.

Rerun on METR-LA

We ran the authors’ code on METR-LA: their GRU point forecaster with their configuration and splits, 60 minutes ahead on 207 sensors, and their CoRel model trained on the residuals of the calibration slice with their early stopping. The point forecaster differs from theirs by its random draw, so the residuals are not the paper’s residuals, and every interval below is built on the same ones. Alongside the paper’s split-conformal baseline we add intervals that are split conformal within cells of a lookup table: the signed residuals of the calibration slice are binned by sensor, by hour of day, by a weekend flag and by the speed observed at the forecast origin in five bins, the two-sided conformal quantiles are taken within each cell, and cells with fewer than fifty residuals fall back to the next coarser keying. The keying is chosen on the last 10% of the calibration slice by the Winkler score, which is how CoRel chooses its checkpoint. All numbers are on the paper’s test slice at \(\alpha = 0.1\), averaged over sensors and steps as in the paper, with per-sensor coverage added. The first two rows are a floor: laplace, an online univariate distributional forecaster with nothing trained, run at the same 60-minute horizon on each sensor’s own speed history, and again on the GRU’s residual series, scored on the same targets with its own quantiles. Both start at the beginning of the calibration slice, 11,400 steps before the first scored target.

The laplace rows are the only ones that cover at the nominal level, and they score worst. Warm-up is not the reason: on a 40-sensor subset, starting laplace 135 steps or 27,000 steps before the test slice scores 55 to 57 either way, and its score by week is flat from the first week. Sixty minutes ahead, a sensor’s own history does not resolve the daily cycle or the current traffic state, and its median is within a speed unit of the GRU in absolute error, 5.3 against 4.5, so the cost is all in the width. On the residuals the median width is 18.5, at split conformal’s 19.5, and the mean is pulled to 32 by wide intervals after large errors: the residual of a twelve-step forecast is a moving average of order eleven, autocorrelation 0.83 at lag one and zero at lag twelve, and a forecaster that carries the short-lag persistence to the horizon fans out after every shock. Everything below these rows on the table is what the clock and the current speed buy. The CoRel row reproduces the paper’s Table 1 to the first decimal: 23.8 against their 23.78, a gap of \(-0.9\) against their \(-1.29\), width 14.3 against 14.38. CoRNN reproduces too: 30.0 against their 30.24. The table keyed by sensor and hour, with nothing trained, ties CoRNN, and the selected table beats it. Split conformal on our residuals is worse than on theirs, 46.4 against 40.8, so our point forecaster has heavier tails, and the comparison that matters is within the table. The lookup table takes 85% of the distance from split conformal to CoRel. Keying by sensor and hour alone takes 74%. What the network adds on top of a table is real and small, and it comes with a coverage gap of the same sign. The two clean ablations settle what the network’s own gain is made of. With the graph kept and the 207 node embeddings removed it scores 27.0, the same as the table. With the embeddings kept and the graph removed it scores 24.2. The distance from CoRNN to CoRel that the paper credits to relational structure is the per-sensor parameters, which is what keying a table by sensor supplies for free, and the graph, on top of them, is worth under half a point.

The dumped quantiles say where the network’s gain lives. Its median shifts the interval centre by 2.6 speed units on average against a split-conformal width of 19.5. Taking its width alone, centred at zero, moves the score from 46.4 to 34.4; taking its centre alone with the split-conformal width moves it to 39.7; both together, with the asymmetry the two quantiles carry, give 23.8. Roughly half the gain is a width that follows sensor, hour and traffic state, a third is a centre that the point forecaster missed, and the rest is asymmetry. The residual has an autocorrelation of \(-0.05\) at the forecast horizon, so none of this is read from residual history: it is read from the speed at the origin and the clock, which the network receives as covariates and which the table keys on.

Two more numbers. Pooled split conformal covers the 10% least-covered sensors at 78% or less and the 10% best-covered at 98% or more, against a nominal 90%; every construction keyed by sensor puts that spread inside 86% to 91%, CoRel included. And the shift to both ends of CoRel’s interval that would bring its test coverage to 90%, computed after the fact for scale, is 0.12 on a width of 14.3, under one percent. The conformalization the paper omits would have cost that.

What survives

The engineering is sound and the code is clean. A shared quantile network across many series with a learned sparse graph is a sensible way to model residual quantiles when the base model is fixed, it trains in minutes where SPCI takes days, and the paper is right that most conformal time-series methods ignore the other series. Proposition 3.1 is true. The Winkler score is the right way to compare intervals, and the paper is more careful on that point than most of the literature it cites. What does not survive is the word in the title. The intervals are the output of a quantile regression, their coverage is whatever that regression happens to achieve, the paper’s own tables show it achieving less than the target everywhere, and the guarantee that would make it conformal costs one held-out slice the pipeline already has.

The paper is Cini, Jenkins, Mandic, Alippi and Bianchi, Relational Conformal Prediction for Correlated Time Series, ICML 2025, arXiv:2502.09443v2, code at github.com/andreacini/corel; quotations are from the arXiv v2 text, the OpenReview forum wwYDQ1vXcZ, and the repository as of its single commit of July 2025. Conformalized quantile regression is Romano, Patterson and Candès, NeurIPS 2019. ResCP is Neglia, Cini, Bronstein and Bianchi, arXiv:2510.05060. The head-to-head is Guo, Han, Luo, Liu, Gong and Wang, Delving into Non-Exchangeability for Conformal Prediction in Graph-Structured Multivariate Time Series, arXiv:2605.04957, Table 1. The interval score is Winkler (1972) and Gneiting and Raftery (2007), equation (43). The rerun scripts are data/reviews/corel_tables.py, data/reviews/corel_decompose.py and data/reviews/corel_laplace*.py here; they read the residual file that the authors’ run_base_model.py writes and the quantile dump added to their run_corel.py. Their checkpoint reload needed weights_only=False under torch 2.6, which is a version change and not a fault of the code.