The term “back-testing” refers to a testing methodology in which a strategy or predictive model is tested using historical data. Portfolio managers and traders usually use backtesting to cross-validate the effectiveness of their trading strategy.
Customers who buy automated trading software, such as Expert Advisors, also look at back-tests on the products to evaluate their effectiveness before making a purchase decision. It seeks to estimate the performance of a strategy during a past period with the assumption that if the strategy worked previously, it has a good chance of succeeding in the future as well.
Conversely, if the concept does not work well on historical data, the strategy may not work well in the future. The main aim of back-testing is to see whether an indicator or a particular trading strategy can replicate the same results in different types of market conditions (up, down, and sideways).
Popularity of backtesting
The popularity of back-testing among traders and investors cannot be understated, especially with the rise of technology and the introduction of various types of automated trading software. Let’s look at some of the reasons why it has become popular in the world of trading.
Helping to evaluate the effectiveness of EAs
Expert Advisors and forex robots have emerged as a powerful tool for novice traders to earn profits with minimal inputs. However, just like any other industry, the current market is filled with a myriad of companies, all claiming their product is the best. However, the market is also home to some fraudulent EAs which just exist to entice unsuspecting customers. In this case, back-testing results can serve as a key factor in deciding whether to go for a particular software or not.
Confirming the effectiveness of a strategy
Normally, traders have the option of testing their own strategies through trading with a demo account. But this is not always effective due to the time required or the unavailability of demo accounts. Luckily back-testing helps us to check the effectiveness of any strategy we are using, allowing us to test multiple strategies across several markets in a week. This can either confirm the effectiveness of a strategy or point out its flaws to work on.
Optimizing a particular trading approach
The most important thing for any trader is to find a trading strategy or approach which is compatible with the most prominent of their personality traits. Backtesting provides a no-cost way of determining whether a strategy is worth investing money in. Back-tests can reveal the drawdown rates for trading strategies. For instance, if a strategy produces a drawdown of 30% but returns 120% in 200 trades, it’s classified as a risky strategy that should be changed.
Back-testing failures to be aware of
In spite of the effectiveness of back-testing, it is prone to limitations and weaknesses. This can take the following forms.
Look ahead bias
A look ahead bias occurs when one uses information or data in a backtest that was not yet known or available during the period being analyzed. If a trader commits a look ahead bias while testing an investment strategy, it will likely return unreasonably positive results which have no basis in reality. Thus it’s important to use “point-in-time” data instead of restated data. This is true for economic as well as financial data.
Limited historical data
In backtesting, the longer the historical data is used for testing the strategy, the more accurate the results are. A long string of historical data improves the model’s stability by reducing the reliance on market conditions in the short-term. The robustness of the strategy increases when the data used in its testing consists of multiple business cycles. Using limited data, on the other hand, will yield the opposite results.
The problem of factor mining
Because of the presence of an enormous amount of data as well as an abundance of computing power available, it’s likely that one would eventually find a strategy that works well in a backtest. However, in many cases, the backtest results could just be the result of making serious correlations between input variables as they may not have a fundamentally sound basis to persist.
Overfitting
Over-fitting refers to predicting the in-sample data so well that it is actually modeling noise instead of the relationship or principle in the data that the tester is hoping to discover. In other words, overfitting occurs when the degrees of freedom of the model rise. It decreases in-sample prediction error and increases out of sample prediction errors. The degree of freedom can take many forms, such as the number of parameters taken in a model or the number of factors used. As a solution, back-testers should always compare in-sample error with an out-of-sample error instead of repeatedly looking to refine their strategy using in-sample data.
Final thoughts
No model or strategy can work in every market condition, regardless if it’s thoroughly back-tested. The aim should be to determine how to identify and systematically make adjustments that capture returns and reduce risk. This can only be done by effective backtesting and modeling. Thus, it’s best to view backtesting as a method for rejecting strategies rather than validating them.
While it’s true that a strategy not successful in a backtest has no chances of being effective in a real market environment, the converse is not true. In other words, just because a strategy works in a backtest doesn’t necessarily guarantee its effectiveness in a live trading environment.
