What backtesting actually does — and what it cannot do

Backtesting runs a trading strategy against historical market data to estimate how it would have performed if it had been live during that period. Done properly, it is one of the most useful tools an investor has for evaluating whether a strategy makes sense before committing real capital. Done improperly — or read uncritically — it produces some of the most misleading numbers in finance.

What backtesting cannot do is predict the future. Markets change, conditions shift, and strategies that performed beautifully in one regime can underperform sharply when the regime changes. The honest framing is: backtests are evidence about a strategy’s behaviour under conditions resembling the test period, useful as one input among many.

The metrics that matter

Sharpe ratio

The Sharpe ratio measures excess return per unit of volatility. Institutional benchmarks generally treat 1.0 as good, 2.0 as excellent, and anything above 3.0 with skepticism. A backtest claiming a Sharpe ratio above 3.0 across a multi-year test should be treated with care — either the test period was unusually favourable, the strategy is overfitted to historical data, or the calculation is flawed.

Maximum drawdown: how bad does it get

Maximum drawdown measures the largest peak-to-trough decline during the test period. It is what determines whether the investor will actually realise the test’s headline return in practice. A strategy with a 30% maximum drawdown is one most investors can hold through; a strategy with a 60% maximum drawdown is one most investors will abandon at the worst possible time, locking in losses they would not have realised had they held. Drawdown matters as much as return for the practical investor.

Win rate vs profit factor

Win rate is the proportion of trades that are profitable. Profit factor is the ratio of total winning trades to total losing trades. A strategy can have a low win rate (40% of trades profitable) but a high profit factor (2.5+) when the winners are much larger than the losers — a typical trend-following profile. A strategy can have a high win rate (70% of trades profitable) but a profit factor near 1.0 when the winners are small and the occasional losers are large — a typical mean-reversion profile under stress.

Calmar ratio

Calmar ratio is annualised return divided by maximum drawdown. It directly captures the trade-off between return and pain. A strategy with 15% annual return and 10% max drawdown (Calmar = 1.5) is a meaningfully better experience than a strategy with 25% annual return and 40% max drawdown (Calmar = 0.625), even though the second has a higher headline return. For practical retail allocation, Calmar is one of the most informative single metrics.

Common backtesting mistakes that produce misleading results

Look-ahead bias

Using information in the strategy that would not have been available at the time of the trade. The classic example is using end-of-day closing prices to make decisions during the day — the closing price is not known until the day ends, so a strategy that uses it for intraday decisions is implicitly using future information.

Survivorship bias

Testing only on companies or assets that survived to the end of the test period. A strategy that “buys and holds” the S&P 500 from 2000 to 2020 looks very different if you test on the actual S&P 500 (which included many companies that failed during the period) versus today’s S&P 500 backwards-projected (which excludes the failures by definition).

Ignored transaction costs and slippage

Real trading involves real costs. Commissions, spreads, market impact, slippage on illiquid assets. A backtest that ignores these costs tests an idealised version of the strategy that no real investor could actually execute. The gap between idealised and realistic returns is largest for high-frequency strategies and for strategies that trade illiquid instruments.

Overfitting

Tuning the strategy parameters so closely to the historical data that the model captures random noise as if it were signal. Overfitted strategies look spectacular in backtests and underperform in live trading. Defenses include out-of-sample testing, cross-validation across overlapping windows, and parsimony in parameter count — fewer parameters that are tuned harder rather than many parameters that have many free variables.

Testing only one regime

Markets behave differently in bull markets, bear markets, low-volatility regimes, high-volatility regimes, rising-rate environments, and falling-rate environments. A strategy tested only in 2017–2019 is a strategy tested in a low-volatility, rising-equities, falling-rates environment — a fairly easy regime for trend-following strategies. The same strategy in 2022 (high volatility, rising rates, falling equities) would have looked very different. Quality backtests span multiple regimes and report performance separately for each.

How to evaluate a published backtest: a six-question checklist

1. What test period was used, and does it span multiple market regimes?
2. What asset universe was tested, and was it constructed without survivorship bias?
3. Were realistic transaction costs and slippage modelled?
4. Is out-of-sample performance reported separately from in-sample fitting?
5. How many parameters were optimised, and were defenses against overfitting documented?
6. How does the strategy compare to a simple buy-and-hold benchmark over the same period?

A platform that can answer all six questions transparently is treating the backtest as analysis. A platform that obscures any of the six is treating the backtest as marketing.

How Senvix presents strategy backtests

Senvix publishes strategy backtests with the full context required for honest evaluation. Test periods span multiple market regimes including bull and bear conditions. Asset universes are documented and include any universe construction caveats. Transaction costs are modelled at realistic platform-specific levels rather than at idealised midprice. Out-of-sample results are reported separately. Performance metrics include Sharpe ratio, maximum drawdown, Calmar ratio, win rate, profit factor, and benchmark comparison. The reporting acknowledges what backtests can and cannot tell investors, and frames results as informational rather than predictive.