MK:
I was wondering if you could change the criteria for your Best Futures systems screen.
You see, my systems are now about 4 months old. During the first say couple of months, none of my systems were allowed to be seen, because the “Risk of 20% Account Loss” stat had to be < 5%.
That had been modified to < 20%, that has allowed for one of my systems - [LINKSYSTEM_31082140] - to be included;)
My system has recently been excluded as my stat now = 20%. You see some successful higher-annual average systems (with higher DD) will unfairly be excluded from this “Best systems” screen.
In looking at some of my other systems (a simple screen from “The Grid” for futures systems at least 4 months old sorted by annual return):
KC Elite #11
Ann Return 231.6%
Max Drawdown 53.3%
Ratio: 4.35X
KC E-mini #17
Ann Return 188.7%
Max Drawdown 26.5%
Ratio: 7.12X
KC Hedge #19
Ann Return 157.7%
Max Drawdown 53.3%
Ratio: 2.95X
KC Futures #26
Ann Return 115.7%
Max Drawdown 35.9%
Ratio: 3.22X
On a side note, it may be a bit more useful to use “Compound Annual %” with this screen.
But my point is that you are in effect excluding systems that may have a drawdown entirely in keeping with their annual average. Do a ratio and you will see what I mean.
Take for instance MPV-3 “The most popular system on C2” that of course shows up on all the screens and has a large amount of views for a very short period of time:
mpv-3
Ann Return 172.5%
Max Drawdown 25.0%
Ratio: 6.90X
KC E-mini is on par for this criteria.
Here are a few others, that “made” the list, to consider:
Wildcard #30
Ann Return 104.4%
Max Drawdown 23.76%
Ratio: 4.39X
Turning Points #16
Ann Return 104.9%
Max Drawdown 16.68%
Ratio: 6.29X
This may be a way to better proliferate your screen with “successful” systems. If you look, oftentimes the list comes up empty or has some new, briefly shown systems with odd- (looking) equity curves.
Now is as good a time to present this. I recently had a bit more than the norm for my backtested drawdown and expect that my annual return figures will soon come into alignment with my compounded annual figures to bring all of my ratios significantly up - yet still be left off this list.
I know this isn’t the only criteria that C2 uses, might as well improve the standard to not exclude good systems.
Regards,
Gilbert
So basically you want him to change the criteria so that it will include your systems? Good luck with that.
If you think that by most statistics you are matching other systems on the best list but are excluded solely on your risk of a 20% account loss being higher than I would conclude C2’s criteria is doing a good job of highlighting the best systems, because what it’s telling people is that although your stats might make your system look as good as the others it can see that your risk of loss is greater than theirs and that alone makes theirs the more attractive proposition.
somehow, that is par for the course for this vendor…
Jon,
The point is that “Best” systems SHOULD include those with the most advantageous “ratio” of compounded annual average versus maximum drawdawn.
I really don’t consider any systems that can’t meet the 1 year mark with success, so do a screen and see what YOU come up with. I guess since there are not a whole lot of 'em, the < 20% stat is most useful in keeping some systems on the list.
Contrast this list (annual average versus < 20% DD) with another “Best” list and assume there are 10 systems that meet the following criteria: compounded annual average versus maximum drawdown. You just might have some systems that last more than a year with a 250% return and a 50% DD.
These would never meet the requirements of the current “Best” list that may have 10 systems with 3 month track records of 100-150% return versus a 20% DD.
Gilbert
(In retrospect I see I made comparisons using my actual DD and not the “risk to DD” which is less and WOULD make my point even more compelling.)
FYI - the following system has a 20% “risk to DD” and just fell off the list. . .but did not have a recent more severe drawdown??
[LINKSYSTEM_31082140]
I think you’re possibly misunderstanding how the Monte Carlo Simulation is used to determine risk or rather the possibility of loss to an account. I think it can be a very useful tool to weed out riskier systems that otherwise appear statistically sound.
Jon,
I have to admit I am not too familiar with this simulation, however the premise you state I am not adverse to. "Weeding out" systems that are somehow trying to "game" the C2 platform SHOULD be averted.
Gilbert
[LINKSYSTEM_33895077]
I wouldn’t put much weight on the outputs of a Monte Carlo analysis that uses equity curve data or the trades that make up an equity curve.
Monte Carlo analysis assumes the inputs are independent when there really is almost always correlation, either serially, or across the portfolio components. This correlation usually shows up in an equity curve as periods of exceptional out-performance, or under-performance.
When the market is behaving as your model assumes, your trades will contain winner after winner, but when market conditions are counter to your model assumptions, you will see loser after loser.
In the recent commodity run-up the metals, grains, and energies were all highly correlated. Going long in all those groups would have netted tremendous profits, just as short positioning would have yielded crushing losses.
But Monte Carlo analysis takes neither of these two types of correlation into account.
Real trading results is what C2 is about. I look at those metrics.
I’m not a statistician (a statement that would receive hearty assent from my college stats professor), but I think the C2 Monte Carlo simulation might at least partially address your concerns, Keith. The randomly drawn elements that make up the Monte Carlo sim are not trades, or even daily stats, but rather are percentage-based deltas of intra-day equity curve (which themselves are calculated at nearly-random intervals). The idea being: if a system vendor has lots of “incidents” where intra-day equity changes very negatively, then the Monte Carlo sim is more likely to display those kinds of dips than a system where the equity-curve, measured at intervals, undergoes slower changes.
Note that the randomly drawn elements have nothing to do with trades, or even with dates. The time periods between delta measurements may be as small as minutes, while a trade is open. I would guess that, as the frequency of these intra-day equity measurements grows larger, we more and more eliminate the danger you astutely warn about – that wins and losses are not independent of each other, and that sequence matters.
But again, this is all a bit over my head, and I may be very wrong about this. Even so, I think the Monte Carlo simulations do have some value (if limited).
I believe that “homogenizes it” even more.
A proper Monte Carlo analysis is useful in “worst case” analysis: based on limited observations, how bad can the drawdown get. When correlations are ignored you can come up with too optimistic an answer. If a real equity curve shows a 54 percent drawdown, the Monte Carlo analysis should not say there’s a 0 percent chance of a 50 percent drawdown.
"If a real equity curve shows a 54 percent drawdown, the Monte Carlo analysis should not say there’s a 0 percent chance of a 50 percent drawdown."
Yes I can understand how that is seen as being counter-intuitive and a frustration for people. The only benefit for vendors is that it is possible to demonstrate that your system has improved over time as it is possible to improve your MCS risk of x% loss figures over time whereas your max drawdown is there forever (unless you have a worse one).
For example, if you have a system that in it’s infancy had a max DD of 50% and subsequently over time the risk of 20% a/c loss went from 60% to just 5% (no doubt accompanied by an improved equity curve) it would at least help demonstrate the conditions under which the max DD was achieved are perhaps not as likely to repeat themselves either because the market conditions have changed significantly or more likely the vendor has adjusted the system or his own behaviour as a result of the DD.
As an example take a look at DPRussell, he had a drawdown of nearly 50% and has since improved his risk of 20% loss to just 12.5% since he changed his system in September. So the MCS figures subsequently help put his max DD in perspective for those that might just look at the max DD and instantly be turned off.
Jon,
You’re really just highlighting another problem with Monte Carlo analysis. In addition to the assumption on independence (lack of correlation), Monte Carlo analysis assumes the data are stationary.
Your example which can happen to the benefit of a strategy (the developer improves the signals or money management), or the detriment of the strategy (the developer does something stupid and blows it up (sounds familiar to the happenings on C2) ) is what real-world trading is all about and it’s anything but stationary.
Monte Carlo analysis has no way to know about the real-world correlation among the data or it’s nonstationarity. It assumes all are the same and lumps them together.
People can see that the equity curve is getting smoother and make their own assumptions, the Monte Carlo algorithm doesn’t see that and treats every piece of data the same. That’s what independent, stationary data assumptions dictate.
Right, it includes all data but that at least then means as your system improves so the sample of trades from which it does it’s analysis improves. It would be less useful if it were able to cherry pick data to simulate from, all the data should be included. You could argue it almost acts like an EMA giving more weight to the most recent data (it doesn’t of course, but the interpretation of reduced risk percentages as a system improves suggests it does).
If you had a Monte Carlo simulation that correctly modelled the interactions of a real time strategy, that simulation would have to yield a finite probability of occurence for a portion of trading that already occurred in real time. When you model the throw of dice, you will get a finite probabilty for any non-infinite sequence of real throws, for instance 3 10 7 11 1 8.
When the Monte Carlo model we use at C2 doesn’t show a finite probability of a future drawdown exceeding a drawdown that has already occurred in real trading, the model is obviously flawed. Why use a flawed model when you’ve got actual real time results?
You make some excellent points, but then all statistics have their flaws, no indicator or measure is perfect, but I think like most I would rather have it at my disposal to determine it’s usefulness than not.
What may be better is to create another group called "100 Most Profitable" systems or something like that…
