Leaderboard enhancement proposal

I have raised this before, I find the leaderboard not so useful and very misleading. There is absolutely no comparison possible between a system that existed for 4-5 years to a system that existed for 6 months, no matter what kind of ranking system you come up with. It is disheartening to see leaderboard being occupied by the recent systems, pushing better systems that has long track record down the ranking.

My proposal - make the leaderboard a level playing ground. If comparing systems, make sure all systems are compared in a defined time period. say by default is last 6 months(and make it modifiable by user). Then compare and rank systems only with last 6 months data i.e consider only trades from last 6 months. This will make sure long lasting systems are compared on the same grounds as with a recent 6 month system. If a user wants to compare systems with 2 years of data, do not take into account systems that lasted less than 2 years for the ranking.

Would love to hear your opinion

8 Likes

I am biased, but couldn’t agree more. The odds of a system being successful for years on this site are very low. Search the grid for systems greater than 3 years, annual return of 10% and a maximum drawdown of 25% (generous measures of success). There are 12 systems listed. If you go out 4 years, there are only 3. And depending on the number of trades a system makes, 3-4 years is not a lot of time to evaluate a system and see how it performs over various market cycles. 6 months is pretty much useless unless you are high frequency trading.

6 Likes

In my opinion, the number of trades is much more important than the age of a system.

For instance, a 3 month old system with 200 trades has more statistical value than a 3 year old system with 6 trades.

5 Likes

number of trades is important, it should be high enough for statistical significance, but that does not say anything about longevity. As an investor, I want to see system survived different market conditions - number of trades will not tell that. Even more important, I want to see the trader has the right mindset, if he/see can survive time. It is very easy to see 1 month old wonders that disappear in a jiffy.

2 Likes

Historically the IRA leaderboard seems to have performed the best on average. I don’t have data for it going back crazy far, but the analysis I have seen has the IRA leaderboard and Old-Timers doing the best and the main leaderboard being one of the worst.

To use your own system as an example, It took more than two years for NQ Quickie to complete 131 trades.

An active day trader can place the same number of trades or more in a single month. That’s more than enough trades to make a sound judgment about the skills of the trade leader.

That means that after just one month interested investors will be able to answer some basic questions, like:
How does the trade leader handle losses?
What percentage of capital is he/she risking per trade everyday?
Is he/she adding to losing positions and/or using a martingale? etc…

With “slow” systems (strategies that only place a couple of trades a month or so), it could take years to get that kind of information. So if the system costs say $200 a month, you could spend $5 000 just to find out that the system you subscribed to was a losing system!

You want to see how the system performs in “different market conditions”, you said?
That’s not even an issue because the day trader must deal with them every day: uptrends, downtrends, sideways, you name it, and that’s why I think that the number of trades is a more valuable metric than the system’s age.

After 100 or 200 trades you can quickly determine if the trader has any skills, even if his trade record is “short”.

2 Likes

I totally agree with you @TheSystem , being a daytrader my self and with my teams portfolio “Falang”. However i must say even if you do 1000 trades one year with good risk/return ratio that does not mean the market will move in the same ways and that you will be profitable next year. So its hard to compare anyway. I think many investors have been burned here on C2 and are more carful now. Best combination is to have enough trades for statistics but also some history, maybe a year or two. I as a trade leader of course would not enjoy to wait that much, neither the investor. But just some perspective. Regarding the leader board i agree that it could be divided between different length of existing. For example under 3 months,6 months,1 year,2 years and so on unto old timers. But if mixing everything its hard for any algo to do ranking. Currently i see that leader board algo adds more weight to recent performance and max DD and leverage. Dont have any proof just a feeling. We dont use the score workbench but i guess its a tool if you are interested in that. Best regards to everyone.

3 Likes

Good evening c2125358467,

There is already an old timers section on the Leader board: Leading Trading Strategies
(Notice that some of these systems have a sizable drawdown and very few or no subscribers, despite their age, which shows that this metric does not seem to influence potential investors that much)

Your Falang system is doing quite well by the way, it returned almost 10 times the drawdown after 431 trades, really impressive!

1 Like

Sure, it may tell something about the trading skill of the trader, but not really whether the system will survive different market conditions as @c2125358467 mentioned, nor whether he/she would go crazy and lever up and blow up. You chose nq quickie system as an example and its 231 trades… Let me give my another system as example Mischmasch (collective2.com). Look 2021 in isolation, 315% with may be 15% drawdown and may be 250 trades. Does that tell something about what 2020 and 2022 looked like? Nope. There is no substitute for longevity, is my opinion. Surviving a long time is definitely better indicator than number of trades. But I do agree number of trades is important to make it statistically significant.

@MatthewKlein , would love to hear your thoughts about the proposed enhancement:)

2 Likes

When you click on the Leaderboard, the default window is this one:
https://collective2.com/leader-board
This page simply shows the most POPULAR systems, as of today. Popularity has nothing to do with longevity, even though the age of the system is also used as a criteria.
The Leaderboard has also an old timers section, here:
https://collective2.com/selector/old_timers

Of course, all things being equal a long track record is better than a short trade record, if we keep in mind that longevity is not a guarantee of futures returns.

I think the minimum is 6 months to qualify for Leaderboard position, among other things.

I am also frustrated and somewhat stymied by why C2 puts more emphasis on the shorter term systems in their scoring. Seems very counter-productive to me.

In my case, I was a follower here on C2 over a decade ago. I followed a system which, foolish me, had great returns, but not too long a track record. It ended up tanking and I got burned. My bad. I left and it took me over 8 years to venture back onto C2 - this time as a system provider. I suspect that more emphasis on longevity would lead to newcomers staying longer - which benefits all parties. It may also lead those developers who are designing only for high-risk, short-term gain to go elsewhere. This would also benefit the subscriber community - especially newbies.

My main system Insights W1 has been out for over 2.5 years so it is too young to make the Old Timers List. The only place it appears is on the Leader Board if you select the:

  • Trades Own System - presently in 4th spot. Top % performer in the last 6 months.
  • IRA-friendly - presently in 4th spot. Top % performer all-time. Only one in top 5 that Trades Own System.

This system has made it into the top 5 by C2 Score a few times, and usually stays in the top 50. That is not usually enough to get it noticed, hence a low popularity. With scoring algorithms that put less weight on longevity making short-term strategies more popular we see that the 7 of the top 10 Most Popular strategies are less than 12 months old.

I don’t see the TOS attribute in the Score Workbench. I think C2 should introduce a metric that does get it included in the Score - they could use the #days of TOS similar to Strategy Age.

BTW, I just noticed that the TOS badge appears superimposed on the performance charts that are displayed on the Leaderboard - but only when a user chooses “Trades Own Strategy”. This badge is not displayed if a user selects “IRA-Friendly”. I would call this a bug - the badge should appear on all the lists for those strategies that are TOS.

2 Likes