Model Performance
2026 season, graded as played
Every row is a frozen record: the projection published before kickoff, the closing spread from the CFBD lines feed (median across providers, the same definition as the backtest), and the final score. Nothing is re-fit or re-priced after a result is known. The closing line is the bar the model is measured against, not an input to it.
By prediction source
| Source | Games | MAE | RMSE | Bias | Total MAE | 50 / 80 / 90 | Gap | Closer |
|---|---|---|---|---|---|---|---|---|
| Pure model | 53 | 12.86 | 15.58 | -0.25 | 12.64 | 53% / 85% / 96% | +1.05 | 49.1% |
| Market-informed blend | 53 | 12.86 | 15.55 | -0.23 | – | – | +1.06 | 47.2% |
| Closing line | 53 | 11.80 | 14.03 | -0.20 | 11.56 | – | – | – |
Pure model is the independent projection. The market-informed blend mixes the pure margin with the pregame market spread the model saw when it published, so it is a product line, not model skill. Closing line is the benchmark graded on its own. Coverage is the share of results inside the pure model's frozen 50, 80, and 90 percent intervals. Home win probability, taken from the frozen pregame margin distribution before kickoff, scores Brier 0.207 and log loss 0.600 over 53 games.
By week
| Segment | Games | Model MAE | Market MAE | Gap | Model closer | Bias |
|---|---|---|---|---|---|---|
| Week 1 | 53 | 12.86 | 11.80 | +1.05 | 49.1% | -0.25 |
By opponent classification
| Segment | Games | Model MAE | Market MAE | Gap | Model closer | Bias |
|---|---|---|---|---|---|---|
| FBS vs FBSthin | 8 | 13.68 | 13.44 | +0.24 | 50.0% | +1.04 |
| FCS vs FCS | 45 | 12.71 | 11.51 | +1.20 | 48.9% | -0.48 |
By model favorite size
| Segment | Games | Model MAE | Market MAE | Gap | Model closer | Bias |
|---|---|---|---|---|---|---|
| under 7thin | 25 | 10.20 | 10.88 | -0.68 | 56.0% | -2.94 |
| 7 to 14thin | 11 | 15.63 | 11.36 | +4.26 | 27.3% | +3.32 |
| 14 to 21thin | 12 | 11.11 | 10.71 | +0.40 | 50.0% | -2.60 |
| 21 or morethin | 5 | 24.25 | 20.00 | +4.25 | 60.0% | +10.95 |
By missing preseason inputs
| Segment | Games | Model MAE | Market MAE | Gap | Model closer | Bias |
|---|---|---|---|---|---|---|
| 1 to 3thin | 6 | 14.87 | 14.33 | +0.54 | 50.0% | +4.75 |
| 4 or more | 47 | 12.60 | 11.48 | +1.12 | 48.9% | -0.89 |
Segments are pure model rows. Market columns use only the games in that segment with a closing spread. A segment marked thin has fewer than 30 games.
Historical walk-forward backtest
Every prediction below was made walking forward through each season with only the data available at the time, then frozen. These are historical stand-ins for live performance, not live results. The market benchmark is the closing spread: the strongest public forecast of a game's margin. Beating it consistently is rare, and the model is measured against it, not against a naive baseline.
| Season | Games | Model MAE | Market MAE | Gap | Model closer | Bias |
|---|---|---|---|---|---|---|
| 2021 | 760 | 13.60 | 12.49 | +1.10 | 43.9% | -0.28 |
| 2022 | 757 | 13.86 | 12.28 | +1.58 | 40.8% | +0.29 |
| 2023 | 771 | 13.26 | 12.00 | +1.25 | 43.3% | -0.24 |
| 2024 | 773 | 13.46 | 12.02 | +1.44 | 43.6% | -0.76 |
| 2025 | 792 | 12.84 | 11.93 | +0.91 | 45.8% | -0.32 |
| All seasons | 3853 | 13.40 | 12.14 | +1.26 | 43.5% | -0.27 |
Bias is the mean signed error of the model's home margin: positive means the model leans toward home teams. In-game projections anchor on the market closing line precisely because the closing line remains the better pregame forecast.