Recommended Free Tools
A backtest reconstructs how a model would have performed on races that have already happened. A live, or forward, record logs predictions on future races as they occur, using information and prices available at the time. A chronological test on past races is still a backtest—not a live trial. Backtesting helps develop and challenge a model; prospective records test whether it generalizes and can be executed. Neither historical profit nor a short live run proves a durable betting edge.
Backtest and live results answer different questions
| Dimension | Backtest | Forward or live record | Why it matters |
|---|---|---|---|
| When evidence is collected | Reconstructed from historical races | Recorded on future races as they happen | A future record encounters current data feeds and market conditions. |
| Information available | Must reconstruct what was knowable at the model’s decision time | Uses prediction-time data as it arrives | Historical databases may contain final odds, results, payouts, or other information unavailable before the race. |
| Model and rules | Easy to tune repeatedly, creating overfitting risk | Should be frozen for the test period | Changing rules after seeing losses makes the result difficult to interpret. |
| Prices and execution | May rely on stored or assumed odds | Can log offered and obtained prices, rejected or partial bets, and slippage | Betting returns depend on the price actually available and obtained. |
| Main limitation | Leakage, selection bias, overfitting, or unrealistic assumptions | Small samples, variance, changing markets, execution limits, or selective reporting | Neither type of record is reliable without transparent methods and uncertainty-aware evaluation. |
A time-ordered holdout improves a backtest because it tests on races later than the training data. It remains retrospective: those races have already occurred. A genuinely forward test records predictions on races that had not happened when the model and rules were frozen.
As an Amazon Associate I earn from qualifying purchases.
How to audit a backtest
Set the decision-time cutoff and exclude leakage
For every prediction, state the exact point at which the model would have acted, then audit each input: would it really have been available at that moment? Exclude results, payouts, final odds, finishing positions, and other post-event information. Also check whether data preprocessing, feature selection, or model tuning accidentally uses information from the future.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →In Shuichi Sugiura’s 2026 study of Japanese flat racing, predictors were constrained to information available after entries were finalized and before outcomes were known. The study excluded final odds, popularity rankings, finishing positions, payouts, race outcomes, and same-race outcome information; some same-day variables were also excluded where availability or stability at the selected prediction time was uncertain. The authors warn that post-event information can enter feature engineering, preprocessing, model selection, or evaluation and make performance look overly optimistic. Read the study.
Split races in time order and protect the final test
Use earlier races for training, a later period for validation and model choices, and a still later period for a final assessment. A random split can put observations from different points in time on both sides of the split, making future performance look better than it is when data are time-dependent. Once the final test results influence feature engineering, model selection, or betting rules, that period is no longer an untouched test.
Sugiura’s study used 2015–2022 for training, 2023–2024 for validation, and January 5, 2025–May 10, 2026 for independent testing. Its test set contained 63,910 horse-level observations from 4,556 races. Those dates and sample sizes describe that Japanese flat-racing study; they are not a universal split or sample-size prescription. A separate 2026 race-level study also used a broad temporal arrangement and reserved a later test period for final evaluation. See the race-level study.
Rank #2
Challenge apparent profitability
Trying many odds bands, filters, race categories, features, and model settings makes it more likely that one combination will look profitable by chance in historical data. Compare with a clear market benchmark and a simpler baseline, inspect performance across time periods, and disclose sample size. Treat subgroup results chosen after examining outcomes as exploratory rather than independent confirmation.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →How to evaluate a forward or live record
Before the test starts, freeze the model version, features, selection rules, and staking method for a pre-declared period or sample. Record every qualifying selection, including losing ones; do not remove bets after the outcome is known.
- Timestamp and race identifier
- Model probability or rating, plus expected or fair price if calculated
- Price available when the bet was placed and the price actually obtained
- Stake, result, and any rejected or partially filled bet
- Relevant closing price, where available and meaningful
- Changes to data feeds, model version, or execution process
Paper testing can record selections as if bets were placed without risking money. It can check whether the selection process is consistent, but cannot establish real execution quality. A later small-stakes execution check can reveal availability, access, discipline, and slippage issues; it is not a guarantee of future profit. There is no universal number of live bets that proves an edge: the uncertainty depends on odds, variance, and the observed consistency of results.
Measure prediction quality separately from betting returns
Prediction metrics
Choose metrics that fit the prediction target. ROC AUC and PR-AUC describe aspects of discrimination or ranking; Brier score and log loss assess probability quality and penalize inaccurate probabilities. A model can rank likely winners reasonably well while assigning poorly calibrated probabilities, which matters when estimating fair prices or expected value. These metrics do not by themselves measure betting profit.
Rank #4
Betting metrics and prices
If the claim is about betting performance, report the number of bets, total stakes, returns, profit, ROI or yield, average odds, maximum drawdown, and longest losing run. State the benchmark—for example, market-implied probabilities, a margin-adjusted market baseline where possible, a favourite baseline, or a simpler ratings model. Strike rate alone is not enough: its meaning changes with the odds.
Free tools Windows power users keep installed
One-click scans. No signup required.
Use prices that could realistically have been obtained at the strategy’s decision time. Account for exchange commission where relevant, race changes such as non-runners, and slippage between a trigger price and the price actually available. A backtest based on a price the strategy could not secure can overstate returns. Report losses and uncertainty alongside headline returns; a positive ROI in a small sample may reflect variance.
Best Value
- Used Book in Good Condition
What published studies establish—and what they do not
Sugiura’s peer-reviewed 2026 Frontiers study used a later historical test period for Japanese flat-racing predictions. In that test, the matched no-theory model had a win ROC AUC of 0.7543 (95% CI 0.7475–0.7609), compared with 0.7293 (95% CI 0.7224–0.7362) for the augmented current-full model. For the study’s JRA place-rule-compatible outcome, the corresponding AUCs were 0.7513 (95% CI 0.7469–0.7558) and 0.7164 (95% CI 0.7118–0.7212). These are discrimination results from that study’s temporal test—not betting returns, live results, or evidence of performance in other countries or racing codes. Read the full paper.
The separate 2026 race-level paper evaluates an upset-risk diagnostic and says it was not integrated into horse-level prediction scores. A finding about race-level instability therefore does not show that a horse-selection model is profitable. Read that paper.
A 2026 SSRN preprint on French trotting at Vincennes describes chronological evaluation and a retrospective backtest settled at official PMU dividends. Its abstract reports model metrics against a market benchmark. Because it is a preprint and the results are a historical simulation, it is an example of study design—not proof of live realized returns or a general claim that racing markets can be beaten. Read the preprint.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
