The devil in Algo Trading: Overfitting !
As you enter the profession of algorithmic-trading, all you hear are the anecdotes of everyone who fell into the trap of looking at a beautiful optimised set of backtesting results and charts - that performed miraculously well - only to fail in a live environment with real capital. The catastrophic causal factor is imprinted in the brain early in one's algorithmic trading development career - overfitting.
So now as an aspiring algorithmic trader or strategy developer you're always looking over your back to gauge whether your solution is overfitted. How can you know? If the results are really good - does this mean its necessarily due to overfilling - what if your strategy is just very good. I've spent many nights in the last eight months sweating about this very topic.
From my errors, reading many papers, listening to algorithmic trading podcasts and just plain learning from failure - here is the set of rules that we have developed at Aieden Technologies to avoid overfitting:
1) Develop your strategy based on first principles
2) Independently optimise your filters - not together
3) Run live with a small amount of capital for six months+
This means that you develop your strategy and use first principles data analysis to set the parameters for said strategy. If you develop a digital filter - you use digital filtering techniques to determine the frequency of your data and where your filter frequencies need to be. This way you avoid using only optimisation as a way to determine your filter frequencies - an approach that would be prone to data overfitting.
If you have multiple filters as part of your strategy (say trend and mean-reversal filters) - ensure you develop them from first principles and optimise them independently (i.e when you optimise your trend filter - it should be agnostic that the mean-reversal filter even exists and vice-versa). This way you avoid finding those non-causal best fit parameters that simply combine to fit the historical data.
Well this is self explanatory. If in six months - you're not seeing three months of generating 80%+ of your typical backtesting performance - you should probably be concerned that overfitting could be taking place.
The challenge off course is that the market is always changing - so sometimes you may suspect overfitting while in fact your strategy simply stopped working as the pattern that gave you the edge disappeared. We're finding out that the right level of patience is critical in algorithmic trading - give your strategy a chance to prove itself while at the same time being aware of how the market changes could be affecting your strategy first principles approach.
