Sam and Frank: I agree with both of you. If you’ve been around C2 for a while, you’ll know that I constantly am trying to penalize systems that swing for the fences in order to “get attention.” Thus, for example, my addition of the “max drawdown stats” - which attempted to penalize traders for averaging down during losing trades.
But the reality is that as much as C2 tries to give traders the tools to make good choices, it is up to traders to actually make those choices. It seems to me C2 provides quite a few tools and metrics that can be used to see if systems are taking large risks.
But trades need to use those tools.
Matthew
Matthew,
When potential subscribers eyes open wide, and their mouths hang open, many don’t see much beyond. And in reality they probably don’t want to. They don’t want to lose their dream come true.
I’m afraid that is a current that runs very deep through human nature, and is not something that a lowly Web site like C2 can alter on its own.
> This would be the solution, but only if the reward is significant.
If the reward is significant relative to risk…than it IS significant.
Then end user can add leverage as he wishes.
Simplify, focus on what counts, a steady equity curve.
This would be easier if the equity curves were realistic for all the systems represented. Take ExtremeOS for example, which has one of the longest track records (18 months) here and an absolutely beautiful equity curve.
Has anyone in the C2 universe traded this system and realized anything close to the 320% gain shown after RF and commission (ie. come close to this same equity curve slope over a few months time trading similar quantities)?
What tools are there on C2 that would allow a potential subscriber to look at this system and realize, without actually trading it, that the results (after considering RF and commissions) can’t come close to being duplicated in a real account of even a few tens of $K in size?
This, IMO, is the biggest problem in trying to find/compare viable trading systems … the equity curves and stats based on the reported trade data are just too far from reality to use for many of the best looking systems.
> Sam and Frank: I agree with both of you. If you’ve been around C2 for a while, you’ll know that I constantly am trying to penalize systems that swing for the fences in order to “get attention.” Thus, for example, my addition of the “max drawdown stats” - which attempted to penalize traders for averaging down during losing trades.
Yes. Thank you. However, if there was a rankable criteria like return/max DD it would be helpful to folks who aren’t trying to blue-sky themselves.
This is one reason I keep bringing up the CTA lists. Big money looks at
risk relative to return every which way. Although C2 looks at it some, all
the ranking lists look at JUST return rather than risk adjusted return. C2
rankings do reward over-leveraged systems. And C2 allows trades that would be busted in real life for over-leveraging (please see my recent post re. Rocket Science Dow’s 50 contracts overnight on 125K in equity).
Sam
I really don’t see how Return/Max Drawdown is going to give you any meaningful statistics.
I assume you mean annualized return? If you use this calculation, then a system making 3% annualized with 1% drawdown will have a better rating than a system making 30% annualized with an 12% max drawdown. Which system would you rather trade?
The same if you have a system making 50% annualized for the last 5 years, but had a max drawdown of 25% once in the last 5 years and all other drawdowns were around 10%. Compare that to a system making 5% annualized each year with a max drawdown of 2%, which will come out on top in your calculation. Again, which system would you rather trade? A system returning 5% annualized, or one returning 50% a year with an occasional 25% drawdown?
I do agree with you that risk adjusted returns is important, but I do not think that Return/Max Drawdown is going to give you that. Maybe Return/Average Drawdown would be more accurate. I think there are probably better statistics for these calculations.
Regards
- Fanus
PS: Your example of Rocket Science Dow holding 50 contracts exceeding the margin is not correct. You assumed $4500 margin. Initial Margin for the e-mini dow is $2438. So if the account was 125K, this was enough. Barely.
Extreme-os is an interesting case. Since the system is using its compounded money to increase its trades sizes, the equity curve should be exponential (i.e. have accelarating growth) if the relative performance (profit / $) was constant. But in fact the equity curve looks almost linear for a long time, with only the last month some signs of accelerated growth. This means that the relative performance has been decreasing for a long time. So even in the best case you should not expect 320% if you start to trade now (or 9 months ago). (If profits would be withdrawn from the account, as Hans suggested this week, then everyone would have noticed that directly).
The largest-DD case of the Monte Carlo simulations and its position to the other simulated curves suggest that it is as well reasonable to expect $150K profit (estimated from the graph), and this is with the best-case fills. If I apply the same slippage percentage, then you would get 84% of this, which is $126K. Or, if I multiply with the RF of 69%, you would expect $103.5 in 17.7 months. This, of course, only if you believe that the MC simulations are a good description of the system.
I started to trade the system with $13,870 and today it is $16,530, which is 19% profit in about 5.5 months. If I would have 19% profit every 5.5 months then after 17.7 months this would be 75%, which is still less than the 103% that I computed above. However, I traded 80% of the time without margin (unlike the system on C2). I also played with the settings, the trading permissions and manual interference - which usually made the outcome worse.
So on the one hand I must agree that my results are not close to the 320% that C2 suggests, but on the other hand my outcome is not really unexpected if I also consider the other properties of the system as they are described by C2 (particularly the MC simulations and the use of margin). So I would say that C2 provided the adequate instruments for me in this case. Of course, an important point is that I do not satisfy your criterion of ‘tens’ of $K. It may very well be that you can’t do this even with $30K, since the RF is limited by volume in many trades.
Nevertheless I am happy with my result so far. I won’t complain about the system as long as I have good profits, even if these profits are much less than I expected / dreamed.
So I wonder how you would react to a system that, according to C2 statistics, should have 300% profit, and that in your account with your scale has 100% profit per year. Consistently. Would you be unhappy with it? What if it is only 50%?
I suppose not. So I think that the main problem is not the quantitative prediction of the profit percentage in your account, but rather the prediction that you will have profits at all. Quantification will be a part of that, but I can imagine that one choses a very different approach in this - e.g. the percentage autotraders that have a profit after x trades. Oops, sorry, that would make C2 too difficult for some of those who don’t autotrade.
Another problem is of course the systems that go south after a short while. No MC, SR or RF will help against that, as you know.
Interesting discussion about Max DD versus Average DD. The Average DD has also a limitation: Suppose you have 99 DDs of 1% and 1 DD of 99% then the Average DD would be 198/100 = 2% which doesn’t really capture the risk.
Both ignore the number of DDs. E.g. in both cases 1 DD of 30% would be seen as having the same risk as 100 DDs of 30%. That doesn’t agree with (my) intuition. On the other hand, a few days ago I said that having many DDs (and recovering from them!) may actually be a good thing.
The standard deviation of the Sharpe ratio (and also the one-sided standard deviation of the Sortino ratio) on the other hand, are insensitive to the order in which losses come. E.g. the profit sequence -1, -1, -1, 2, 2, 2 has the same net profit and the same standard deviation as 2,-1, 2, -1, 2, -1 although the first one has a much larger DD.
So I wonder how you would react to a system that, according to C2 statistics, should have 300% profit, and that in your account with your scale has 100% profit per year. Consistently. Would you be unhappy with it? What if it is only 50%?
I would be very happy with 30%, and a smooth equity curve like that shown for Extreme-OS. I traded this system before some of the autotrading software improvements on both C2 and TB, and the best I could do was near breakeven (small loss not counting commissions, larger counting them). C2 over the same period showed a 6% gain which matched its longer term slope precisely. So my RF would be zero. Maybe I should try it again with the better fill management options that are available.
Another problem is account size. If a system can’t handle the minimum $100K account size, and for at least some number of subscribers, it shouldn’t be presented that way on C2. This is where I agree with your comments on RF vs. account size. I am not interested in a system that can only be traded successfully with a few tens of $K or less, but is useless for larger account sizes. If I have $100 to allocate to the system, but it can only be traded with $10K, my net return is already 1/10 that shown on C2 even if the C2 fill numbers and RF were realistic. I have to find 10 similar systems to trade!
This, of course, only if you believe that the MC simulations are a good description of the system.
I don’t … I never look at these since I don’t think they provide any useful information.
> I really don’t see how Return/Max Drawdown is going to give you any meaningful statistics.
I apologize. I should have gone into greater detail. I’m using old fashion TradeStation software. Their basic format for system evaluation has been
around since 1990 (or earlier with SystemWriter). Literally on the bottom
lines of their Performance Summary are “Max Intraday Drawdown, Profit factor (gross profit / gross lost), Account size required, and Return on account”.
Account size required is Margin + Max intraday drawdown. Return on account is the Net profit / Account size required.
Lets look at some actual C2 systems based on a TS “Return on account”. These numbers are estimates as not all the data is spelled out.
Midas Hedge: Max Intaday Drawdown (MID): $275,000 + Margin Required (MR): $100,000 = Account size required (ASR) $375,000
Cumu $ $298,661 /$375,000 (ASR) =
.79 Return on account (ROA) [27 weeks]
Timac: $400,000 (MID) + $200,000 (MR) = $600,000 (ASR)
Cumu $ $1,065,050 / $600,000 (ASR) =
1.77 (ROA) [54 weeks]
Rocket Science: $14,000 (MID) + $125,000 (MR) = $139,000 (ASR)
Cumu $94,500 / $139,000 (ASR) =
.67 (ROA) [6 weeks]
Qtips: $40,000 (MID) + $40,000 (MR) = $80,000 (ASR)
Cumu $157,325 / $80,000 (ASR) =
1.96 (ROA) [14 weeks]
I am not judging any of these systems, just giving you the bottom
line according to a 16 year old TradeStation standard. Moreover,
these calculations show more precisely the potential return on an actual investment rather than $100,000K. I would like to see systems ranked in some similar way. This would reward real returns and risk control. Right now Midas ranks way ahead of Qtip and AtDow (ROA
about 2.2 or higher). Z1 (shouldn’t even be ranked) has an ROA of .2 or so and is in the top 50 systems and is coming up on the top of the “gainers” lists.
Anyway, any ideas that would level the playing field are welcome in my book. Right now C2 doesn’t rank systems in anyway accounting for
risk adjusted returns. The bigger you swing, the bigger the reward in the rankings. Indeed, not only is there no penalty for striking out, there is
none for being thrown out of the game (see Z, Z1, etc.).
PS: You are right about the YM margins. Once again my apologies. I was looking at Man’s margin list. After looking at IB and CBOT’s I realize I was wrong.
Here is another example. The 9th ranked system in all of C2:
Mosaik Hedge 62 wks 318.6%
The actual ROA would be near 50% (.5)…if it had survived a
margin call when holding 100 @QGN6 and as well as:
9 SBV6
2 HUQ6
7 USD/ISK
12 USD/DKK
14 EUR/USD
38 USD/CAD
9 GBP/INR
56 GBP/DKK
58 EUR/GBP
32 GBP/CAD
over the same period.
Randy,
So would you agree if I say that the main problem is rather the lack of “good” systems that satisfy all your criteria (age, curve, tradable, volume) - than the accuracy of C2 statistics as predictors?
I’ve no idea what the performance of CTAs and hedge funds is, since they I’m not in that league anyway, but it seems that what you want simply doesn’t exist at C2. I wonder whether you could perhaps be less demanding with respect to the curve. That is the only criterion that might be open for relaxing. (It would be foolish to demand less of tradability and volume, and relaxing the age requirement isn’t wise either, since 9 out of 10 systems go south after a short while.)
I mean, why should the curve be like Extreme-os and not like Hawk-fx? (I mean only the curve, not the other features). Well, you didn’t say that, so perhaps I draw that conclusion too fast.
With respect to the MC simulations, I agree that these are often not very usefull, but cases like Extreme-os I think that they carry some information since that system is stable, compounds, and doesn’t hold positions very long.
With respect to tradability, perhaps you would be helped with a simulation where every signal is delayed randomly between 2 minutes and 1 hour, and/ or where you miss the 10% most profitable trades 
Here you make no distinction between day-trading systems (whose intra-day drawdowns are not even recorded at C2) and longer-term trend following systems.
Closed-equity curve of all systems would present a more accurate picture.
There were no margin calls…
I would certainly agree that there is a lack of profitable (in real life trading) systems on C2 that aren’t a wild rollercoaster ride (but that do trade regularly … several times a week at least), but it is impossible to judge many of them from the C2 stats because:
1) These stats are based on the stated fill prices in the P/L table
2) The fill prices in the P/L table cannot be realized in an actual account for many systems that use limit orders, or trade rapidly, or trade huge volumes of penny stocks, or some combination of these as has been discussed ad nauseum.
RF is supposed to account for some of these issues, but it doesn’t penalize nearly enough in my experience and some of the systems listed as most tradable (eg. Starfinder, Big Cat, E-wolf Emini to pick three) have reasonably high RFs. I’d bet that none of these three systems can be duplicated in performance in a real account by a fair margin … far less than the RF would indicate. I’d love to hear from some subscribers to see what they are getting in real accounts.
A new review (8/28) came in for Extreme-OS and that reviewer is flat after autotrading for 3.5 months. That was my experience as well. You have a profit after 5.5 months. So are the C2 stats correct for that system? It obviously depends on what scaling factor is used and how the autotrading parameters are tweaked, neither of which go into the stats.
The roller-coaster effect can be nullified by using options judiciously, in particular options on futures (that is what they are meant for…), but unfortunately they are not available at C2 as in the real-world…
Equity curve can say very little, especially if the system is not available for subscription, but only via managed account. Equity curve is $1,610 since beginning of the month for single contract, real account is +174 for the same time with commissions being $ 600.
Peter
I agree. But a closed equity curve is a good starting (perception) point on which concepts can be built, because it is based on a long-range view of the system. The short-range, viewed long-range (as it is now at C2), is self-destructive. This is the practical point missed here, which is a case of pragmatism, which tells people to judge each choice of a system not by reference to abstract theory, but only by its results after it has been tried; which insists that today’s results need not recur tomorrow; and which urges that each situation be approached “experimentally,” “on its own terms.” Such a philosophy amounts to the decleration: drop your mind, discard your capacity for thought, decide each case perceptually. This is precisely man cannot do; not for long.
http://www.collective2.com/cgi-perl/board.mpl?want=listmsgs
&boardid=14747319&threadhilite=1475&displayblock=&sorttype=
&session=355769119043324101218744082688510
