Leaderboard Requirements

C2
1

I am curious as to what the requirements are to appear on the “Leaderboard” on C2. A review of the systems displayed there will reveal some systems that would seem not to be “Leaderboard” quality. Below are the statistics of a system that is currently displayed on the Leaderboard.

Cum. return 3.5%
Age 46 DAYS
MDD 4.6%
Profit Factor 0.9
Winning Trades 48.5%
Winning Months 50%
Sharpe 2.785
Calmar 14.146
Popularity 793
C2 Score 51.8

Has outperformed the S&P 500 only 5 of the 46 days of it’s existence (10.8% of the time).

While the Sharpe and Calmar scores are certainly good (most likely distorted by the short life of the system), everything thing else about this system would lead one to believe that it would not qualify for the “Leaderboard”. Anyone that has any idea of the requirements to be listed on the Leaderboard please share here. Thanks!

2

Here is another example of a questionable system appearing on the Leaderboard.

Cum. Return 4.5%
MDD 7.2%
Age 12 Months
Profit Factor 1.2
Winning Trades 54.3%
Winning Months 83.3%
Sharpe 1.110
Calmar 2.113
Popularity 840
C2 Score 65.9

2016 Return +5.4% (S&P 500 +11.96%)
2017 Return -0.8% (S&P 500 +9.34%)

Currently underperforming the S&P 500 by a wide margin.

And this system is on the leaderboard because?!

3

It has probably something to do with the “Heart-Attack Index” parameter, perhaps.

4

Noticed this as well. May be that you first have to “pay your dues” by Featuring Your Startegy before you are given the preferential treatment of being on the Leaderboard, the Log In page, etc.

5

It certainly could be. I decided to look at some of the systems posted on the Leaderboard sorted by “Heart Attack Index (HAI)” (which is not available on the Grid).Below are the stats for one such system with a high HAI.

System Statistics

CAGR 28.8%
MDD 43.4%
HAI 92 (100 is the Worst)
Age 32 Months
Profit Factor 2.2
Risk of Ruin (ROR-Monte Carlo) of 30% of portfolio - 19.5%
Sharpe 0.921
Calmar 1.007
Popularity 819
C2 Score 66.1
Winning Trades 81.7%
Winning Months 81.2%

Yearly Returns
2014 (1 Month Only) +6.8%
2015 +57.7% (S&P 500 +1.38%)
2016 +4.4% (S&P 500 +11.96%)
2017 +10.8% (S&P 500 +9.34%)

While the CAGR is good, as is the winning percentages, the MDD, ROR (30%), Calmar, C2 Score and outperformance compared to the S&P 500 is poor to average. A cursory review will reveal NUMEROUS systems with stats well in excess of what is presented above that are not listed on the Leaderboard.

On another note; how is the HAI calculated anyway? I don’t remember seeing an explanation of how it is done.

6

Very good question,but I don´t think that Collective2 will answer this - Looking at other threads they seem to avoid answering stuff like this.

1 Like
7

There is an old thread showing a popup window with at least a vague explanation of HAI:

Not sure if that link will come across, but if it does clicking on the image enlarges it enough to read the description (and I expect this popup is still around somewhere). No specifics, but at least something.

8

I suppose C2 has to keep some of its formulas secret, to prevent developers from gaming the system and getting an unfair advantage over other C2 developers.

9

On April 4, I posted this:

I don’t know what C2 uses to set the order on the Leader Board, but C2 score and popularity are probably involved.

I just ran a regression on Grid data I downloaded a few weeks ago (when I joined C2 [in Feb. 2017]). The C2 score can be predicted quite well (r=.804) with just three variables:

  1. Max DD;
  2. log of Annual Return (w trading costs);
  3. log of strategy age in days.

Popularity was not automatically codable, so it was not used.

I was afraid that perhaps C2 maximized % winning trades or ignored the age of strategies, but neither turned out to be true.

So I wouldn’t say that C2 favors new strategies, except to the extent that many new strategies have unrealistically small drawdowns and unrealistically high annual returns. But system age is very important, too. And taking the log of annual return reduces the excessively high return systems somewhat.

The criteria that C2 uses are a priori sensible to use, though it might be slightly better to use variables that had been empirically proven to predict good future returns or low future system failure.

++++++++++++++++++++++++++++++++++++++++++

So there is an easy answer to your question. On 2 of the 3 criteria for the C2 score, the strategies you cite would score very well:

  1. LOG of ANNUALIZED return (w trading costs), and
  2. MAX Drawdown.

And C2 Score is almost all of the ranking on the Leader Board.

Personally, I’d like to see the development of a C2 rating model based on past data that in the past predicted future performance.

Failing that, I’d favor a model for C2 scores (and the Leader Board) that was comprised of:

  1. Total Return since autotrading began, probably logged (after adding a constant) or standardized,
  2. 90 or 180 day return, probably logged (after adding a constant) or standardized (there must be something for strategies that have no autotraders),
  3. Max DD (or an average of max DD and max DD divided by years of age). and
  4. Strategy Age (either raw or logged).

BTW, part of my PhD training was in social science scale construction. But this is just a rough guess about what might work well.

1 Like
10

Thank you for your work.

2 Likes