Methodology

This project predicts a full distribution of runs scored per team per MLB game, then derives win, run-line, and totals probabilities by simulating each matchup roughly 10,000 times. Predictions are graded daily against sportsbook lines. Sports betting markets are used as a calibration benchmark, not as a gambling application. Sharp market participants push lines toward true probabilities quickly, which makes them a higher-quality probability signal than most independently constructed models.

The framework is borrowed from quantitative finance and decision theory: expected value, the Kelly criterion (Kelly 1956, originally an information-theory result), and probability calibration. The domain is baseball; the methods are standard in applied statistics and quant analysis.

The pipeline covers data ingestion from multiple APIs, feature derivation from pitch-level Statcast data, hierarchical Bayesian inference via NUTS, daily evaluation against actual outcomes, and a production frontend that refreshes each morning of the season.

The current model (v2, live since May 12, 2026) is a two-layer system: a hierarchical Bayesian skill model (Dirichlet-Multinomial over the eight plate-appearance outcomes, fit with NUTS via numpyro/JAX) followed by a per-PA Monte Carlo simulator that plays out each inning with rest-aware bullpen rules, an empirical baserunner-advancement table, and per-game posterior draws to propagate parameter uncertainty. It replaced a prior XGBoost regressor (v1, archived) after a 542-game head-to-head backtest in which v2 won on every metric.

Each plate appearance is modeled as a categorical draw over eight outcomes: strikeout, walk, hit-by-pitch, single, double, triple, home run, in-play out. Three separate hierarchical models learn the additive log-odds offsets that each actor (batter, pitcher, venue) contributes to those eight logits, with OUT as the reference category.

Generative model (batter, schematic)

Four implementation details that matter at this scale:

• Non-centered parameterization. Sampling z_i from a unit normal and forming β_i = μ + σ ⊙ z_iavoids Neal's funnel pathology in hierarchical models, where the geometry of the posterior becomes too distorted for HMC to traverse efficiently.
• Multinomial likelihood, not per-PA Categorical. Aggregating each batter's PAs into outcome counts and using pm.Multinomial is mathematically identical to a per-PA pm.Categorical but ~400× faster on 400k PAs (~3 min vs ~19 h projected). Each leapfrog step evaluates a per-actor log-likelihood rather than a per-row one.
• Platoon and role splits. Batters are fit per (batter, vs_LHP) cell; pitchers per (pitcher, role ∈ {SP, RP}). Position-player pitchers are dropped from the pitcher pool.
• Park as residual log-PF. Park is fit last, on the residual wOBA after batter and pitcher effects are accounted for. A per-venue scalar park_log[v]shifts the seven non-OUT logits additively, weighted by each outcome's wOBA coefficient, so HR and 3B move most under park and K is unaffected.

Sampler diagnostics (4 chains, 2000 draws, 2500 tune)

R-hat is the Gelman-Rubin convergence statistic; values near 1.00 indicate the chains have mixed and are sampling from the same posterior. Effective sample size (ESS) measures how many independent draws the autocorrelated chains are equivalent to; minimum ESS > 400 is the gate.

For each scheduled game we draw K=30 random posterior samples and run N inning-level simulations per draw. N defaults to ~333 in production (10,000 total sims) and 33 in the acceptance-gate test (990 total). Every draw is a coherent realization of all batter, pitcher, and park parameters together, so spread across draws becomes a posterior band on the win rate; the inner-loop variance is the aleatoric run-scoring noise within a fixed parameter setting.

Per-PA outcome sampler (vectorized)

Indices: batter b, pitcher p, role ∈ {SP, RP}, venue v, outcome k ∈ 1..7 (K, BB, HBP, 1B, 2B, 3B, HR; OUT is the reference 0 logit).

On contact, baserunner state evolves via an empirical advancement table built from 365k Statcast PAs (2024+25). The table is a flat lookup of P(new_state, runs_scored, outs_added | state, outs, outcome, out_subtype). Cells with fewer than 100 observations are linearly shrunk toward the outcome-conditional marginal P(· | outcome, subtype), with weight n_cell / 100 on the cell and the rest on the marginal. HR, BB, and HBP advances are deterministic forced moves; the empirical estimates were unreliable there because bases-loaded variants were sparse.

Bullpen management is rest-aware in live mode. Starters are pulled at pa_count ≥ 24 (a pitch-count proxy, roughly 95 pitches at 3.95 P/PA). Relievers are drawn from a queue built from the active 26-man roster, ordered by current rest, and any reliever with ≥ 6 outs in the last 1 day or ≥ 9 outs in the last 2 days is skipped. The per-pitcher workload table is refreshed in the daily pipeline from MLB boxscores.

Variance decomposition (per-game total runs)

The form-noise scalar is calibrated against 2025 actuals to narrow the residual underdispersion in the empirical advancement table. It is not a tuning knob to be re-fit on market backtests; doing that mixes concerns and tends to mask modeling errors as noise.

Acceptance gate (200 stratified 2025 games × 990 sims × 2 sides = 396k team-game samples): simulated mean runs/team-game 4.59 vs actual 4.38 (+4.86%, clears the 5% mean gate); simulated variance 9.72 vs actual 10.32 (−5.86%, clears the relaxed 7% variance gate, misses the original 5%). The model is mildly underdispersed in the tails, so blowouts and large totals get slightly less probability mass than they should. Closing that gap is a v2.1 item.

Market probabilities are estimated directly from the simulated run distributions; there is no parametric assumption like Poisson or negative binomial on top of the sim output. Let h_i, a_i be the home and away runs in sim i for i = 1..N:

The p10/p90 win-probability bandis computed per posterior draw, not from the inning-level samples. For each of the K=30 draws we compute that draw's win probability across its inner sims, then take p10 and p90 over the K draw-level probabilities. The band is a measure of posterior parameter uncertainty about the win rate, separated from the aleatoric run variance. Anti-correlation is enforced by construction: P_away(p10) = 1 − P_home(p90).

A play is flagged when the modeled probability exceeds the sportsbook's de-vigged implied probability by more than 4.5% (ML, RL) or 6.5% (totals). The totals bar is higher because totals markets are noisier and respond more slowly to fresh information. Sizing uses the Kelly criterion:

Full Kelly is theoretically optimal for long-run log-wealth growth (Thorp, Shannon, Kelly 1956) but draws down aggressively on losing streaks. Quarter-Kelly trades some growth rate for a much tighter drawdown distribution. A high-variance flag fires when a team's simulated runs stdev exceeds 4.0, surfacing games where the model itself is unusually uncertain about scoring.

Before cutover, v2 was benchmarked against the frozen v1 XGBoost baseline on every completed 2026 game where both models had a prediction and the v2 sim used a clean live-bullpen queue. v1's code is preserved verbatim at SHA a84b4dd in v2/evaluation/baseline_v1/ so the comparison is stable.

Brier and log-loss are both proper scoring rules, so the comparison is on calibration and sharpness together, not just hit rate on winners. v1's 41.9% max calibration gap is partly thin-bin variance (the worst decile had few games in it), but the directional read still holds: v2 reports a tighter and better-calibrated probability. v2 is also more selective on flagged plays (~20-25% fewer flags) yet posts higher ROI on all three markets, which is the expected pattern when a model gets sharper.

Backtest infrastructure is in v2/evaluation/ (replay populates the v2 prediction table for a date range; backtester joins v1, v2, and actuals and emits the head-to-head report). Live performance, including ongoing Brier and calibration, is on the Performance page.