The data leaks
One: lookahead, any calculation touching information unavailable at decision time, the classic being signals computed on a bar's close and executed at its open. Two: repainting inputs, indicators whose historical values move after the fact, making history look cleaner than the live experience ever was. Three: survivorship in the window, testing only eras your idea was built by staring at. Each leak flatters results silently; our own replay harness executes decisions strictly candle-by-candle for exactly this reason.
The cost fictions
Four: zero-cost fills, no spread, no slippage, which converts many marginal systems into paper stars, per the cost mechanics. Five: intrabar fantasy, assuming target-before-stop whenever one candle contains both; honest testing marks those trades ambiguous or resolves them pessimistically. Six: perfect execution at news-time prices no live order would receive. The live gold chart draws these zones in real time if you want to see them on today's candles.
The statistical sins
Seven: overfitting, tuning parameters until history submits, then presenting the tuned result as discovery; the antidote is out-of-sample validation, and the humility to report when validation kills the idea, as it killed most of our clever filters. Eight: sample poverty, conclusions from a few dozen trades, streak luck dressed as edge. A result that survives all eight checks has earned cautious belief; anything less has earned curiosity at most.
FAQ
What is the single most common backtest flaw?
Lookahead leakage in some form: decisions using data not yet available at the decision moment. It is subtle, common and devastating.
How much data makes a trustworthy sample?
Hundreds of comparable trades across varied regimes, with a held-out validation segment the tuning never touched.
Should I trust published strategy backtests?
Only with methodology transparent enough to run the eight checks. Absent that, treat results as advertising.
ⓘ See these ideas on real price: open the free XAUUSD live chart.