I’m not adverse to changing the way Sharpe is calculated on C2 (or even, for that matter, jettisoning Sharpe and finding an alternative that doesn’t penalize upside volatility). But regardless of whether we use Sharpe or some alternative, the real problem is that systems with short life-spans will have wacked-out numbers, since the “period” used will be very brief. Even so, I will have another look at how Sharpe is calculated on C2, and will look at various potential changes.
On the other matter raised in this thread, I agree that the way equity charts are drawn here on C2, they do favor systems that close out positions at the end of the day. I will take a look at changing the charting algorithm to factor in daily min/max equity numbers. I know many people have asked about this, many times.
At the risk of sounding a bit defensive: I am trying my best to add the features you want. But everything is a trade-off. Working on one thing delays another. Etc. Etc. That’s the nature of software development, and, I suppose, life in general. So, be patient. I hear you, and will try my best to act on your requests.
Matthew
Maybe the solution for statistics which get out of whack over short life-spans, is to not calculate these statistics until a system is at least 3 months old for example? That way many systems which started of with huge returns the first month, just to blow out the next, will be filtered out and the good ones will still be around after a few months.
And I for one do appreciate all you do on here and there are many more things I like about C2 than I dislike.
Regards
- Fanus
MK … I second Fanus in appreciating all the continuous improvements, and no doubt every regular user does as well. I will toss in a comment on the equity charts if you’re tweaking these in the future. It looks like some of the data in those charts is retained even if corrections are made (after the fact) in the individual trade tables due to errors.
For example, recently Steady-II had a trade error which showed an $11,000 profit on a closed trade due to what appeared to be a contract month mishap at the time. That was corrected a couple of days later in the trades table, but this artificial positive equity spike still shows up in the equity chart near the right hand side and really screws up the actual history of the equity for that system.
I put a lot of weight on these charts when shopping for systems and wish they could be used more interactively (eg. click on the chart and a new window pops up with a bigger chart and lots of detailed info/options). Probably a ton of programming work but it is always easy to ask for the moon on a wish list.
You suggested ignoring the Sharpe ratio (or whatever we wind up using) for young trading systems. Admittedly, at first that sounds like a good idea. But there’s a problem. A lot of C2’s system scoring is at least partially based on the Sharpe. (Hot lists, Best Systems, etc.) So, if we throw out the Sharpe for young systems, then either we decide that young systems can’t appear on these lists, or that we need to come up with a placeholder value for the Sharpe until it can be decently calculated (1.0?). The problem with the first choice is that I think C2 would lose something (and system vendors would be sad) if young systems couldn’t get any recognition. The problem with the second choice is that by setting a placeholder value, you are - in fact - just making up an arbitrary value (perfect 1:1 correspondence between abnormal return and abnormal risk, say) when clearly that is not necessarily true, and in fact there is available data to determine otherwise.
My feeling is: You can look at an equity graph – even for a young system – and come to somewhat useful conclusions about its return versus risk. That is, indeed, what the Sharpe is supposed to capture. It is true that over very short periods the number should not be weighted very highly – but, at least for things like hot lists, best systems, etc – system youth is taken into account elsewhere in the formulas.
My point here is to say simply that – regardless of how I ultimately change the Sharpe or replace it – the number will still be suspect for young systems. That said, I don’t think there’s an easy answer about how to handle this fact. I don’t want to choose an arbitrary number or to insist that young systems should not be evaluated.
So, in the end, we’ll be back where we started: we’ll use some number, and live with the fact that – for young systems – it’s not terribly meaningful. But it’s better than no number at all, and it’s better than saying: "C2 shouldn’t evaluate young systems."
MK
Randy:
Re: "For example, recently Steady-II had a trade error which showed an $11,000 profit on a closed trade due to what appeared to be a contract month mishap at the time. That was corrected a couple of days later in the trades table, but this artificial positive equity spike still shows up in the equity chart near the right hand side and really screws up the actual history of the equity for that system. "
It’s true that the equity graph doesn’t get corrected for errors. I had a bum fill in a pork belly trade that resulted in something like a $18K profit - which should have been shown for a loss. The trade was promptly corrected, but my equity graph shows a big spike where the error occured. This distorts - makes it look worse - the current drawdown that I’m experiencing. Who knows, it might even be calculated into my Sharpe ratio!
Actually I believe that “C2 shouldn’t evaluate young systems” is a perfectly valid statement. Young systems typically don’t have enough data points to evaluate effectively and I see nothing wrong with excluding them from various lists until they reach a certain age.
Matthew,
why not use some logic that does make the calculation of the risk-adjusted return possible but _unfavorable for young systems?
You could estimate the quarterly volatiliy for example, and divide the returns by the quarterly volatility. When a system has less than a quarter of history, just divide the lower returns of the system’s lifspan by the quarterly volatility.
I think that young systems ought to prove themselves anyway. IMHO, penalizing them is the smaller error compared to overestimating them. The latter will only lead to all the lucky-one-shots rising to the top (as they tend to do now).
As a side note, I very much approve your replying here and the continous work!
Best regards
Nico
P.S.: @ Jules: You’re right, I was contradicting myself, because I left out the potentially missing excess return. I was too impatient to write that and thought it was self-explanatory, which it obviously wasn’t. Sorry.
P.P.S. I also like Sortino ratios a lot more than Sharpes, for the same reason that were discussed here.
OK, if we have to live with Sharpe ratio, I’d like to offer some advice for all of you, vendors and subscribers alike:
Vendors: If it looks like you have a tiger by the tail - a real whopper of a trade - CLOSE IT OUT IMMEDIATELY! And then reenter right away. Sound dumb? It’s not. Whatever happens, you don’t want a big trade showing on your record. It’ll ruin your Sharpe. Break the big trade up into bite sized chunks. So what if it will confuse your subscribers and pad the pockets of the brokers with extra commissions.
Traders: I’m sure that you will want to focus on high Sharpe systems. Consider the following list - systems which have some of the highest Sharpes here. Now, please indulge me and look at their equity graphs. The easiest way is to do an “Advanced Search” for Sharpe > .9.
Midas Med-Term (Prudent) - 1.937
YeKai Swing trader - .916
Bender’s S&P Emini crossover - 2.126
Entropia - 1.053
ER2 Cash Cow - 1.596
Goofiz Foliage’s Future ATM - 1.524
Kondor Futures Trading System - 1.036
Under Development - 1.001
I’m in no way trying to knock these systems, as you can probably make money with many/most/all of them. Some of them are quite good, in fact. But I ask you, are they the model of consistency - as represented by their Sharpe? Are they what you are really trying to find when you are searching for a more “tame” system? Please, take a look at the graphs. And, lastly, one of the most highly recommended systems here, Hawk-FX has a Sharpe of .506.
Hans - the anti-Sharpe crusader.
Hans: What you wrote is innacurate. Closing and then opening a trade will have zero effect on the Sharpe ratio. The Sharpe is calculated based on account equity, not individual trade P/Ls.
When you close out a profitable trade, your account equity doesn’t change a bit (well, except for some slippage). You merely exchange open trade equity for cash. The overall account equity stays the same.
Matthew
Hmmm… OK. But that just makes more mysterious the high Sharpe on some systems with wild roller coaster equity graphs. See my examples. For my own entertainment, I’ll have to dig a bit and find examples of low Sharpe systems with tame graphs.
Hans.
OK, here’s the bad Sharpe list. Easist way to view graphs is “Advanced Search”, Sharpe <.5, but be sure to check the 3 boxes for systems more than 3 months old - else you will get too large of a list.
These systems have either a smoother - to the eye - graph than the high Sharpe systems previously listed, or else are profitable - and well above the S&P plot - but have negative Sharpes. Some are extremely steady, but have a more conservative return in relation to the S&P. It appears in the latter cases, Sharpe is evaluating profit more than volatility. In all cases, the Sharpe misrepresents them, IMHO.
ARS - (-.091)
Beck TA Commodity Trader - .176
Blog.fallondpicks.com - (-.091)
CTS CLC System - (-.235)
CTS Crash Control - (-.296)
CTS SnapBack System - .283 (much smoother than you’d expect from the S.R.)
ETF investor - (-.839) - Wow! Look.
Granville’s Nemesis - .09
MB trading - .36 REAL steady!
Momentum Buys - .171
Mutual Fund Trader - .076 A bit volatile, but not like the high Sharpes posted earlier
Peridot Capital Contrarian Strategy - .165
QQQQ-Alpha - (-.376)
QQQQ-crude - (-.134)
QQQQ-Omega - (-.313)
Swing Trader - (-.33) OK, just kidding!
Hans - Riding off on my trusty steed, Sortino!
Ok, no need to get so practical. 
I also have been thinking about something Pal said, that the periods over which the Sharpe gets calculated differ when the system gets older. Weekly, Monthly, Quarterly… Is this true? If so, then I would vote to scrap Sharpe Ratio completely. From what I read and saw by doing a few trial runs in Excel, Sharpe Ratio results differ drastically depending on the period it is calculated on. That would not be fair then to compare a system with a year of trade history and Sharpe Ratio calculated over Quarterly periods, against a new system with Sharpe Ratio calculated over weekly periods. A new system with a good month, will almost always have better sharpe ratio than a decent system with a good year, but a case can be made that the system with longer history is more reliable since it has proven itself already.
On the same token, this will also not be fair to compare all systems with Sharpe Ratio calculated over weekly periods when weekly period calculation will favor more active and short term systems over long term systems.
I agree with you that you can look at the equity graph to get a feeling for risk/return. But the problem with that is, that since the Sharpe Ratio is used to filter systems out of the Best System lists and to calculate rankings, very few people are aware of good systems with low Sharpe Ratio as they doesn’t get featured anywhere and would not look at the equity graph.
I also agree that any calculation will be suspect for young systems. But right now the calculation is also suspect for established systems with good statistics. By including young systems, there will also be a few which get off to an astronimical start and those kind of systems seems to always overshadow the established ones. Even though many of these young systems blow up, there are always another young system to replace them and in the process the established systems get lost in the picture.
I think Sharpe Ratio is more valid if you compare similar like systems and do calculations the same way, for example mutual fund returns. But this is not really a good way to compare different type of systems with different type of Sharpe Ratio calculations.
Regards
- Fanus
Hans
I wonder why you did not find Russel Triple Swing, using those criteria. More than 3 months old, Sharpe 0.312, over 70% annualized, just the graph is a little erratic, well, at least it can be traded that way with 15 minutes per day…, because it’s EOD.
Peter
I agree. You cannot compare systems on Sharpe Ratio if it is based on different time periods.
However, I would not vote to scrap the Sharpe Ratio, since it is a quite conventional measure and it gives new people at least one thing that they recognize and by which they can compare C2 systems with other investments. I would welcome the addition of other measures though.
One way to look at the influence of age is that ‘being young’ is itself a dimension of risk. A suggestion done by Sharpe is that you may look at the T-statistic,
T = (Sharpe Ratio) SQRT(N)
where N is number of time periods. This can be interpreted as the amount of evidence that the system outperforms a risk-free investment. The multiplication with sqrt(N) will favor old systems. Even then the standard deviation of T will be large for small N (i.e. a young system can easily get an extreme value of T) and to compensate this you may compute the p-value from a Student distribution with N-1 degrees of freedom. A p < .05 would say that you can be fairly sure that the system outperforms a risk free investment. It will be difficult but not mathematically impossible for a young system to achieve this.
I must warn you that this test assumes normality and independence and that these are likely to be violated. Nevertheless, it might be a nice heuristic. Note that it is based on the Sharpe ratio and therefore it is also subject to the limitations that others pointed out. But the effect of age would be different in that older systems will get a proper advantage.
Jules
This time period issue in Sharpe calculation has been bugging me. (OK, the whole thing has been bugging me. Or didn’t you notice?) I was just able to put my finger on it.
There are some systems with absolutely horrible canyons in their equity graph, yet they sport good Sharpes. I think I know why, and, if I’m right, it relates to the time period of measurement.
Suppose you do the Sharpe calculation on the 1st of each month. But system “Master Gunslinger” has had for the last several months a 30% DD midmonth. Don’t panic, he makes it up plus some by month end. So the equity graph looks like the neighbor’s dog took some big bites out of it, but since the beginning-of-month numbers are stable and growing, he gets a good Sharpe. If it were calculated weekly, you’d discover some of those bones that Fido buried. Is this thinking flawed? And if my thinking ISN’T flawed, then Sharpe Ratio is!
Hans - Hi-ho Sortino, and awaaaay…
I don’t even understand why this is being debated.
Only 2 things are relevant:
1) Sortino Ratio is clearly a superior measurement tool as compared to Sharpe Ratio. Sharpe is just more well-known. In short, it is a newbie’s tool. Any serious trader knows that, and all of the competing sites whether forums or system review sites etc all use Sortino, and few use Sharpe.
2) It is all academic unless we can convince MK to use Sortino, its his site, his rules.
FWIW MK, I endorse either a move to Sortino Ratio from Sharpe, or at the very least the addition of Sortino. The discussion on young vs mature systems will continue I’m sure, but the bottom line is that vendors will always want exposure for their young systems too, and subscribers will want to see them.
W
Well, you forgot to mention my system, and I have the highest Sharpe ratio among the top 50 systems. My system started out with a bang, with a few big trades and it didn’t seem to effect the overall Sharpe ratio, which is still 2.65…So your advice doesn’t seem to be true.
Since the initial few big winners, the system has stalled and right now going only slightly up, but still haven’t had a losing month and I am up for March too…
So in my case the Sharpe ratio seem to be working.
This is a good and valuable thread. Please don’t turn this into a marketing thread for systems.
Chris
Well, did I have a point or didn’t?
Hans said, if you are a vendor, you shouldn’t make big wins because the Sharpe ratio punishes such systems. I provided a counterexample. Oh, I am sorry, this happened to be my system…
Here is another example, no affiliation: Goofiz Foliage has a Sharpe of 1.5 but it has huge wins. On the other hand Bender’s Sharpe is much higher, 2.1 but it has much smaller wins, compared to Goofiz…
Conclusion: Hans was wrong with his advice to cut up big trades into small ones…
Your “point” and conclusion was already made by Matthew, 6 hours before your post. There was no reason to repeat it, unless to try and push your system.
Chris
