My Backtest, so Beautiful !

Most of us working on data-science related projects use historical data to predict future values. The vast majority of this group have fallen for the allure of the good-looking historical backtest chart. How can one resist? I admit it, I fell for it too. Why would anyone pick a choppy, rugged, ugly looking chart – yielding less profit with a higher drawdown - when they could choose one that is smooth, exponential and generates untold wealth in the same period? The answer is that in this case (as perhaps others) beauty can be skin deep and that the beautiful curve can have zero or negative live performance merit while the rugged one may have live results that match the backtest performance (which is favourable otherwise you’d never go live with it). The challenge is that you can’t tell simply by looking at the symmetry, beauty, smoothness, ruggedness, or any other look and feel aspect. Believe me I’ve tried, for countless hours – you can’t tell. The ugly one may perform well, badly, or neutrally on live data and so can the pretty one. The one that performs more recently can perform well, badly, or neutrally live and the one that didn’t perform well recently the same. Looking at the chart for visual traits, picking one with more recent high performance – have almost zero causal link to forward performance. I’ve looked at these charts for thousands of hours – the only thing I can tell you is that if the chart is going down and to the right there is a very high probability that it will do that in a live environment. Ok we established not to invest in an algorithm where the historical chart is down and to the right.

I know you know this blog article would be depressing if I left you here. It could discourage people from entering algorithmic trading and perhaps save them a lot of sweat, effort, disappointment, and discouragement but this is hardly how one wants to contribute to the world. In my first blog article I explained some techniques to increase quality and robustness of your algorithm – today I will discuss the most important one – parameter robustness optimization. Intuition tells you to invest in the parameters that generate the perfect historical backtest. Even though I told you, even though you read it elsewhere and even though I will tell you what to do – remember me – you will be susceptible to the charms of the perfect chart with the perfect historical performance. You have been warned.