Skip to content
Wealthoak.ai

Methodology

What an attractive backtest can leave out

A disclosure checklist for parameter selection, untouched evaluation data and realistic trading assumptions.

Published by Wealthoak Advisory LLP · Updated

Start with the selection process

Suppose a researcher tries many combinations of entry rules, holding periods and sizing parameters, then publishes only the highest return. The reported result describes the winner of a search. Without the number of attempts and the selection rule, a reader cannot assess how much the result may depend on chance or repeated tuning.

A reproducible report records the candidate set, the rule for selecting a candidate and the point at which evaluation data became visible to the researcher. A later design change informed by the test result makes that data part of development.

Preserve a meaningful evaluation boundary

An untouched evaluation window should not influence strategy selection. In walk-forward testing, training and evaluation windows advance according to a specified schedule. Publish the complete sequence and the retraining rule; choosing only favourable windows defeats the purpose.

A boundary in a spreadsheet is insufficient if the underlying features already incorporate later information. Record when each input would have been available in practice. For example, a signal derived from a completed bar cannot assume the bar’s final values were available before it closed.

Make execution assumptions visible

Commissions, spreads, slippage, financing and instrument constraints affect what a hypothetical trade means. Historical bars do not establish the complete sequence of prices inside each bar. Fill assumptions can materially change outcomes.

TradingView’s strategy documentation explains its emulator and testing settings. A result produced with an emulator should retain those settings with its source export.

Publish the weak cases

Show neighbouring parameter values, adverse subperiods, cost sensitivity and concentrations. A strategy that succeeds only under a narrow combination of assumptions needs a narrower claim than one supported by several distinct tests. This is an evaluation principle, not evidence that a particular strategy passed those tests.

Our methodology and downloadable report template turn these questions into a disclosure protocol. The evidence register records the current status of Wealthoak’s published reports.

Published by Wealthoak Advisory LLP. This is methodology content, not an independently reviewed engine performance report or investment recommendation. Authorship and evidence policy.