As I have gained more experience in backtesting strategies, I have come across various performance metrics. Some of them are provided in the standard AB backtest report (e.g. Calmar ratio: CAR/MaxDD), some of them I have picked up and added to by Custom Backtest logic (e.g. David Varadi’s DVR).
I have noticed that I have developed a preference for the Calmar Ratio, especially when I am doing my first analysis of a new strategy. I like the measurement taking the MaxDD as an important factor, this fits my thinking well. One of the downsides of the standard implementation is that it calculates the CAR and MaxDD on the basis of one equity curve and one test only. This doesn’t allow for any statistical validation.
Some months ago I found a very interesting variant of the Calmar Ratio: Pessimistic Gain/MaxDD. The concept applies bootstrap-testing to the Calmar Ratio. In essence, the algorithm takes a list of trades and builds a distribution curve of the Gain and the MaxDD of x new equity curves that are created by drawing from the original list of trades (resampling). When x>number of trades, the mean and std of the distributions of the Gain and the MaxDD may be used to build confidence intervals of the Gain and the MaxDD of the original list of trades (If you want to learn more about bootstrap-testing I higly recommend this post by Jez Liberty). The Pessimistic Gain/MaxDD is then calculated by subtracting 1xSD from the Gain and adding 1xSD to the MaxDD : (Gain-SD_Gain)/ MaxDD-SD_MaxDD.
I take no credit for this application as I happened to come across it in the Amibroker group at Yahoo. Here I found an amibroker plugin dll that contains the implementation of the-concept (resample.dll). The plug-in – with its usage instructions – was posted by a user named dloyer123. I tried to email this user, but the user name is no longer in use at Yahoo. As the information is accessible for everybody in the group, I see no harm to post it here on the website as well. I have added a link at the download page.
At the end, some words of caution. When using the resample.dl I found that it relies on the AB internal quotes array to store trades. When the # trades exceeds the # quotes not all trades will be passed on to the resample.dll. The measure then is not based on the full list of original trades. A second finding is that the resample.dll uses a generic % of equity positionsize. It cannot work with other equity allocation schemes.