Look-Ahead Bias in Pretrained Forecast Models — Does Training on Future Data Pay?

Chen, Chen, Chen, Huang & Zhang (arXiv:2609.20554) ask whether post-origin training data inflate the measured accuracy and investor value of financial forecasts from pretrained time-series foundation models. Numbers below are from their PDF only; this page does not run a new forecast experiment.

The question the paper asks

Historical backtests need two cutoffs, not one. The first is the usual input cutoff: which returns the model is allowed to see at forecast origin t. The second is the training cutoff: which returns were allowed into the parameters that process that history.

Example from their introduction: a January 2010 forecast that only feeds returns through December 2009, but uses a model trained through 2015, already has temporal exposure — even though the numerical input tape looks point-in-time. As pretrained models (LLMs and numerical TSFMs) show up in finance research, that second cutoff is easy to ignore and hard to audit after the fact.

Design in one paragraph

They evaluate five finance-native time-series foundation model variants — Chronos Tiny, Chronos Mini, Chronos Small, TimesFM 8M, TimesFM 20M — each released as independently trained annual vintages under three training environments: US, Global, and Augmented (Global + JKP factors). Forecast targets: the U.S. equity premium plus excess returns in 13 non-U.S. markets (14 markets total) at horizons 1, 3, 6, 12 months. Numerical histories begin January 1994; out-of-sample target-start months begin January 2001.

Every comparison holds the numerical history, target, and inference protocol fixed and only replaces the annual parameter state. The origin-aligned point-in-time (PIT) benchmark for a target-start year Y(t) is the vintage trained through year Y(t)−1 (lead −1). Rolling designs also use leads −2, 0, +1, +2. Fixed-vintage designs hold the 2000, 2009, or 2023 state fixed while target windows move across its cutoff.

They first check that PIT forecasts beat an expanding historical-average benchmark (so later-vs-PIT gaps are not “noise vs noise”), then measure forecast revisions, matched MSFE effects, and a constrained mean–variance allocation on one-month forecasts (relative risk aversion 3, market weight in [0, 1.5], expanding variance).

Result 1 — PIT forecasts are informative

Under US training, origin-aligned Chronos forecasts reduce mean squared forecast error relative to the historical average. Paper figures (all-available U.S. sample):

Source: abstract / §4.1 / Table 2 narrative in the PDF. Positive R²_HA means lower MSFE than the expanding historical average.

So the PIT rule is not an empty baseline. That matters for interpreting later vintages: if a post-origin model loses to PIT, it is losing to something that already cleared a real-time hurdle.

Result 2 — later vintages revise forecasts, often for the worse

Replacing the annual state moves the forecast. For one-month U.S. TimesFM 20M, the first origin-crossing update (vintage T−1 → T) changes the predicted return sign in 27.9% of matched months. Revisions are not cosmetic rescales.

Accuracy, though, generally favors PIT under US training:

• Pooled rolling comparisons (leads 0, +1, +2 vs PIT on common support): higher mean squared forecast errors for later vintages in 18 of 20 U.S. model–horizon combinations.
• Across those 20 combinations, the median MSFE advantage of PIT is 5.94% of historical-average MSFE (equivalently, median pooled later-minus-PIT effect −5.94 HA-MSFE percentage points).
• Internationally (median across 13 markets): pooled rolling effects favor PIT in all 20 model–horizon combinations.

Fixed-vintage evidence lines up: of 60 state–model–horizon estimates (2000 / 2009 / 2023 × 5 models × 4 horizons), 52 are negative vs PIT, overall median −9.29 HA-MSFE pp. In 12 of 20 U.S. model–horizon cells, PIT wins the pooled rolling comparison and both fixed-2009 and fixed-2023 retrospective comparisons; zero cells show the reverse three-way win for later states.

Result 3 — investor value under a common rule

Same constrained allocation on one-month forecasts; exposed strategy averages leads 0, +1, +2; no transaction costs deducted in the headline comparison the abstract emphasises:

Source: abstract and §5.5 / Table 7. Negative ΔCER means the origin-aligned forecast was more valuable under their rule.

Result 4 — why big revisions need not help

An exact identity splits the squared-error change from a revision D relative to PIT error ePIT:

Alignment benefit vs movement penalty. Under US training, revisions are large relative to remaining PIT error, but alignment efficiency κ typically falls short of the break-even κ > r/2. Rolling U.S. summary in their Table 8 narrative: relative revision scale r ≈ 0.273 (break-even ≈ 0.137) with realized κ ≈ −0.014. International rolling: r ≈ 0.312, break-even ≈ 0.156, realized κ ≈ 0.024. Only about 12–17% of corresponding comparison cells show positive net matched predictive value.

Training content shifts the balance: for the U.S. target, Global training moves the median pooled rolling effect from −5.94 to +1.37 HA-MSFE pp; Augmented brings it back to −3.64. Mixed effects are the point — temporal exposure is necessary for look-ahead contamination, not sufficient for a favorable bias in measured accuracy.

How this fits Quant for Free

Three honest failure modes, three different questions:

• Selection / search inflation → Deflated Sharpe and the validation gauntlet (trial count, OOS).
• Engine / cost implementation → Implementation Risk (same strategy, different libraries).
• Parameter-time look-ahead → this paper (same history, different training cutoff).

If you deploy pretrained return models, Chronos/TimesFM-style or otherwise, a Q4F-shaped checklist would still ask: what is the training cutoff relative to each forecast origin; is there a PIT vintage matched on the same inputs; do costs and OOS structure survive separately from “the foundation model looked good in a blog”? Temporal exposure alone does not prove inflated accuracy — but skipping the PIT match is how you fake a backtest without meaning to.

Where to go next