METHODOLOGY

How far each number is checked.

Every model and dataset this site publishes carries one of four labels on its method card. The label says how much checking stands behind the numbers, not how good they are: a Reviewed ranking can be right and a Verified model can still lose to the market.

The four tiers

Verified3 labelled

Passes its invariant tests plus a published backtest.

The engine's maths is property-tested on every push, and its forecasts have been scored against real results on a page you can read.

Tested16 labelled

Passes its invariant tests; no published backtest yet.

The engine is property-tested on every push (ratings conserve their pool, probabilities add up, order does not matter), but its accuracy has not yet been published as a backtest.

Reviewed4 labelled

Checked by hand; no automated test.

A person has read the method, the code and the output against the sources. Nothing re-checks it automatically when the code changes.

Provisional2 labelled

New or unaudited.

New, still being tuned, or our own estimate. Read it as a first draft and expect it to move.

What counts as a test

An invariant test feeds the production engine generated inputs, heavy favourites and edge cases included, and checks a property it must keep whatever the input: an Elo update gives one side exactly what it takes from the other, a set of probabilities adds up to one, a simulated season plays every game once, a board does not change when the leagues load in a different order. The suite runs before any push that changes model code.

A backtest counts only when it is published on the site, where a reader can check it: the Ledger, which scores every priced game against the market, or a backtest board on the model's own page.

The labels cannot claim more than the repository holds. An automated check fails the build if a Tested or Verified label names a test that does not exist, if a Verified label has no published backtest, or if a tested engine has no label at all.

Every label

VerifiedPasses its invariant tests plus a published backtest.

TestedPasses its invariant tests; no published backtest yet.

  • College football Elo model

    Major-game updates are zero-sum and the conference pools never move under property tests; its history is not yet scored against the market on the site.

  • MLB Elo and postseason odds model

    The Elo, the series odds and the bracket pass property tests on the live bracket; the regular-season sim has no invariant row and no backtest is published.

    • Method card: /predictions/mlb
    • Invariant tests: 9 in test_inv_elo.py, test_inv_odds.py, test_inv_devig.py

  • NBA postseason odds (live bracket) model

    Play-in and bracket odds pass property tests on the 2026 playoffs replayed at five dates: the eliminated at zero, rounds summing to their places, series run from their score. The season sim has no invariant row.

    • Method card: /predictions/nba
    • Invariant tests: 9 in test_inv_bracket_postseason.py

  • NHL postseason odds (live bracket) model

    Bracket odds pass property tests on the 2026 playoffs replayed at three dates: the eliminated at zero, rounds summing to their places, series run from their score. The season sim has no invariant row.

    • Method card: /predictions/nhl
    • Invariant tests: 7 in test_inv_bracket_postseason.py

  • Universal Index (clubs) model

    The board does not depend on the order leagues load, one entrant moves no one else and club Elo is zero-sum; no backtest is published.

    • Method card: /sports/universal
    • Invariant tests: 7 in test_inv_universal.py, test_inv_club_elo.py, test_inv_odds.py

  • Universal Index (nations) model

    Tennis and golf replays conserve points, the national Elo engines are zero-sum and the board ignores code order; no backtest is published.

    • Method card: /sports/universal
    • Invariant tests: 8 in test_inv_nations.py, test_inv_elo.py

  • Rugby union world Elo model

    Every test-match update is zero-sum under property tests; the ranking itself has no published backtest.

  • Election seat and coalition simulations model

    The New Zealand and Israel seat draws fill the chamber and every coalition is a majority under property tests; the US and UK swing models are checked by hand.

  • Universal Index data room dataset

    Built by the same engine as the Universal Index, which passes its order and zero-sum property tests.

  • NFL season simulation model

    Each drawn season conserves its games, fills every playoff slot once and never crowns an eliminated club under property tests; the season odds have no published backtest. Its weekly picks are graded in public in the Ledger.

  • College football season simulation model

    Each drawn season conserves its games, fills every playoff slot once and never crowns an eliminated club under property tests; the season odds have no published backtest. Its weekly picks are graded in public in the Ledger.

  • Premier League season simulation model

    Each drawn season conserves its games, fills every playoff slot once and never crowns an eliminated club under property tests; the season odds have no published backtest. Its weekly picks are graded in public in the Ledger.

    • Method card: /predictions/pl
    • Invariant tests: 4 in test_inv_predictions_sims.py

  • NHL season simulation model

    Each drawn season conserves its games, fills every playoff slot once and never crowns an eliminated club under property tests; the season odds have no published backtest.

  • NBA season simulation model

    Each drawn season conserves its games, fills every playoff slot once and never crowns an eliminated club under property tests; the season odds have no published backtest.

  • EuroLeague season simulation model

    Each drawn season conserves its games, fills every playoff slot once and never crowns an eliminated club under property tests; the season odds have no published backtest.

  • Champions League simulation model

    Each drawn league phase conserves its games, fills every knockout stage once and never crowns a club outside the top 24 under property tests; the strength backtest is in the repository, not on the site.

ReviewedChecked by hand; no automated test.

  • Global Metro Power Rankings dataset

    Hand-curated from primary sources and checked against them by hand; the weights are editorial.

  • Metro index data room dataset

    The same hand-curated composite as the rankings, checked by hand against its sources.

  • Champions ledger data room dataset

    Champions are curated by hand from the governing bodies' records; consistency scripts run, but no invariant test.

  • Valuations and ownership data room dataset

    Published valuations and owners are copied from named sources and checked by hand; our own estimates are labelled as ours.

ProvisionalNew or unaudited.

  • Our own team valuation estimates model

    The promoted and relegated division-move model is a trial; its figures are labelled CoN estimate on the board.

  • Zone Zero Cup model

    New, and its pillar weights are still being tuned; no automated test yet.