Every row is a frozen record: the projection published before kickoff, the closing spread from the nflverse schedule (the same definition as the backtest), and the final score. Nothing is re-fit or re-priced after a result is known. The closing line is the bar the model is measured against, not an input to it.
Games graded9292 with a closing line
Model MAE9.48points
Market MAE9.39points
Gap to market+0.09points (lower is better)
Model closer41.3%
80% coverage78%target 80%
By prediction source
Live season accuracy by prediction source
Source
Games
MAE
RMSE
Bias
Total MAE
50 / 80 / 90
Gap
Closer
Pure model
92
9.57
11.98
+1.77
11.57
53% / 82% / 96%
+0.18
42.4%
Published (market-informed)
92
9.48
11.84
+1.65
10.52
52% / 78% / 90%
+0.09
41.3%
Closing line
92
9.39
11.90
+1.53
10.59
–
–
–
Published is the market-informed line, total and scores the site shows: the pure margin mixed with the pregame market spread the model saw when it published. Pure model is the independent projection behind it. Closing line is the benchmark graded on its own. Coverage is the share of results inside each source's 50, 80, and 90 percent intervals: the pure model's frozen ones, and for the published line the dispersion picks are priced with. The published total is graded only for games published since it existed. Home win probability from the published pregame line scores Brier 0.234 and log loss 0.662 over 92 games.
By week
By week
Segment
Games
Model MAE
Market MAE
Gap
Model closer
Bias
Week 1thin
16
11.60
11.28
+0.32
31.3%
+1.72
Week 2thin
16
12.02
11.84
+0.17
50.0%
+3.63
Week 3thin
16
8.48
8.09
+0.39
31.3%
-1.96
Week 4thin
16
6.66
7.31
-0.65
62.5%
+2.07
Week 5thin
15
10.03
10.40
-0.37
46.7%
+1.96
Week 6thin
13
7.80
By divisional game
By divisional game
Segment
Games
Model MAE
Market MAE
Gap
Model closer
Bias
Divisionalthin
28
11.32
11.30
+0.02
46.4%
+3.08
Non-divisional
64
8.67
8.55
+0.12
39.1%
+1.03
Data and model diagnostics
These explain forecast error, not betting records. QB adjustment is the larger of the two teams' expected-starter adjustments the model applied before kickoff, so it separates games where the model priced a quarterback change from games where it did not. Projected winning margin is the absolute model margin: under 3 means the model expects either team to win by fewer than 3 points, regardless of the sportsbook line. Favorite, underdog, over and under pick records are in Recommendations.
By QB adjustment (points)
By QB adjustment (points)
Segment
Games
Model MAE
Market MAE
Gap
Model closer
Bias
nonethin
13
7.37
7.15
+0.21
23.1%
-0.23
under 1 point
70
9.48
9.39
+0.09
42.9%
+2.35
1 point or morethin
9
12.54
12.67
-0.12
55.6%
-1.06
By projected winning margin (points)
By projected winning margin (points)
Segment
Games
Model MAE
Market MAE
Gap
Model closer
Segments grade the published line. Market columns use only the games in that segment with a closing spread. A segment marked thin has fewer than 30 games.
Loading historical comparison…
7.04
+0.76
23.1%
+2.71
Bias
under 3
41
9.92
9.74
+0.18
43.9%
+0.50
3 to 7
34
9.04
9.09
-0.05
41.2%
+1.91
7 to 10thin
11
10.54
10.82
-0.27
45.5%
+4.06
10 or morethin
6
6.97
6.08
+0.89
16.7%
+3.65
Historical walk-forward backtest (pure model)
Frozen, 2016–2025
Every prediction below was made walking forward through each season with only the data available at the time, then frozen. Seasons 2016–2021 were used for parameter selection; later seasons are historical evaluations, not the live season record. The market benchmark is the closing spread: the strongest public forecast of a game's margin. Beating it consistently is rare, and the model is measured against it, not against a naive baseline.
Games graded2761
Model MAE9.85points
Market MAE9.78points
Gap to market+0.07points (lower is better)
Model closer46.5%
Pure-model MAE: 10.08 points on the same 2761 games.
NFL backtest accuracy by season
Season
Games
Model MAE
Market MAE
Gap
Model closer
Bias
2016
267
9.06
9.04
+0.02
48.3%
-0.98
2017
267
10.21
10.08
+0.13
46.1%
-0.46
2018
267
9.87
9.87
0.00
47.9%
+0.06
2019
267
10.24
10.18
+0.06
46.4%
+1.65
2020
269
9.72
9.79
-0.07
50.6%
+0.82
2021
285
10.80
10.67
+0.14
49.5%
-0.52
2022
284
8.79
8.78
+0.01
48.6%
-0.12
2023
285
10.17
9.98
+0.18
44.9%
-1.04
2024
285
9.86
9.70
+0.16
40.4%
-0.49
2025
285
9.76
9.67
+0.09
42.8%
-0.50
All seasons
2761
9.85
9.78
+0.07
46.5%
-0.17
Bias is the mean signed error of the model's home margin: positive means the model leans toward home teams. The published line and the in-game projections lean on the market precisely because the closing line remains the better pregame forecast.
