I think collective2 is not evaluating the systems correctly. They are allocating C2 Score for every systems but that is not reflecting the accurate picture of the systems.
I saw many poor systems, which are in loosing streak for long times and having high draw downs, are given 900+ scores.
When a new visitor comes and check it they will be confused by this inaccurate scoring.
The score must indicate the current strength of the system, not the past.
They must take into account every trades, their risk,historic draw downs,slippage and popularity.
So the score will fluctuate trade to trade, so it won’t give false picture.
Doesn’t matter how a particular system had performed last week or last month. the important thing is how it is moving now.
Other thing is collective2 is biased towards some systems. they are giving highest score to some systems which they are calling themselves “TOP SYSTEMS”. For Example: Topaz. To this system they have given 1000 score.
How can they give 1000 score to a average system, which are continuously loosing money and having higher draw down.
So i think collective2 must be independent and unbiased. then only clients will trust them and subscribe.
I definitely agree with you on this one. Every system has their own goals which makes it hard to compare them. Some look for low risk, others offer a very high return. It seems a system that has one great month has a perfect score forever because its P/L line will be way above the benchmark. But as you said, that doesn’t mean the future of the system will be good. That’s why I always zoom in on the P/L so that I can ignore some of the lucky trades.
I don’t want to seem like I’m just self-promoting my system but this score has been a little frustrating for me. Topaz lost almost 50% recently and my system lost (max DD) about 4.5%. Then it recovered to have a gain of 3.0%. [LINKSYSTEM_62867893]
I agree on the Score yet that is only one tiny very tiny stat that one should look at in my opinion.
One would hope you look at Draw Down first- This will tell you how the system has done in the past- ( yes I know past performance is no indication of future performance) however if a system had a Mdd of 40% in the past- you can rest assure that it will have one close to that again in the future- So why would one be surprised when the system hits a 30% draw down?
Compare Mdd and return- if the Mdd is higher than the return why trade it?
C2 shows the slippage percentage too- this is important too- if you are going to see less than 90% of the actual price C2 is showing then the return can be off dramatically
I am sure that many people look at win/loss ratios. I have said this for years and will again here- This is a farce- a joke- any real trader looks at this and then determines if the system holds the fund longer than it should to bring up the win/loss ratio- OR does the system sell out the losses quickly so the capital can be used in a better position? This would being the win/loss ratio down- and would increase annret-
One stat that should be and I can not find it is how long in a loosing trade and how long in a winning trade- This would help one determine the above
[LINKSYSTEM_55644026]
I agree with the gist of this thread but some of the substance is barking up the wrong tree.
The C2 score measures a vendor, not each system. And yes, on the whole it is not a very good indicator - most people who have been on C2 for a while ignore it. This topic has been discussed many times on these forums.
"Doesn’t matter how a particular system had performed last week or last month. the important thing is how it is moving now."
What is ‘now’ if not last week or last month? Today? Last minute? There is no ‘now’ in this sense. What we ideally want is to know how something will do next minute or day or week and that’s impossible. The best - though imperfect - clue we have is the past in some form or another!
"Other thing is collective2 is biased towards some systems…"
In a way yes - C2 score is biased towards older systems and with good reason. It is not the same as being biased towards a particular system or vendor. Also, no matter what the criteria for any score, it will be biased towards something or other. That is the point of a score: to be biased towards systems that have a higher probability of success.
"For Example: Topaz. To this system they have given 1000 score.
How can they give 1000 score to a average system, which are continuously loosing money and having higher draw down."
First of all let me say you are surely right that the score should have come down a bit, and surely nobody can have the maximum of 1000… except maybe god!
But I will also go on to say that Topaz is an above average system by far and is not continuously losing money - once a system has been around for almost 5 years without blowing up then a few months’ drawdown is not a long enough time to decide that the system is no longer any good.
How many systems or vendors are there which have made it for longer than two years delivering decent returns without blowing up a real account to the point of no return… precious few! Those are quite rightly the ones that deserve a high score, recent drawdown notwithstanding.
If you continue your excellent performance to date I’m sure you will also eventually have a score pushing 1000. But it takes time. And one day you too will have a sizeable drawdown, hopefully well managed though, and I will then also say that such an episode should not damage your score by much, at least at first.
Yes. My ranking got killed, yet am really trying to make a comeback and even though I am up almost 80% for the month with almost 40% profitable trades comparable to the top systems, do not come up in any results.
I guess I will keep working my system and maybe if my returns are 20,000% I may show up in the results. Why offer a service then?
Entrants into my system now may get even better upside since I can only seem to go up from here.
My system makes for a fine example of a need for possible changes to evaluation methods.
I took some huge drawdowns. I was new, testing ideas, and then went big and it worked as expected. read my system details for the specifics.
The C2 system evaluation method may be a bit lacking if now is really what matters. up 75% month to date on a $1 million account? 42% profit trades. Some consistent winning streaks, risk control.
I used to evaluate hedge funds and maybe could offer some input. http://www.hedgefundresearch.com/ does a good job of ranking managers.
Maybe c2 could expand the ranking system and include more attributes to search on.
C2 maybe good on the technology end, perhaps they bring in some marketing managers. As long as they keep bringing in the traders and prospects keep it up. I imagine the last thing C2 wants is traders getting blown up, so steering traders to SAFETY will save them from losing all their work in the meantime they collect revenue from most of the underperformers.
How about as our performance improves, our subscription fee gets reduced?
You have done well lately but you have to put the whole history into perspective, as any research company would do.
What ranking would you give a fund with -74% drawdown and -0.89 Sharpe ratio, and an annual return of -61%?
A one hit wonder does not make for a wonderful system- Even Kool and the Gang had more than one hit-
you have one great month and the others are not good
The annret is about -61%
your Mdd is so high that people will run for the hills
Nothing about the system you believe is wonderful shows it has any staying power or that it is consistent
The only thing most will see is that your system got lucky this month
Now if you have a few months like this one then I am sure your rating will increase- however your sharpe index is not even positive either at this time
[LINKSYSTEM_55644026]
it was not luck. certainly results will speak for themselves, statistics are a funny way of determining results. 1000% in a month 5000% in a month, makes no difference to me. downside risk is my new obsession and concern, and upside will take care of itself. nevertheless, the market over the next several months is a challenge I am looking forward to.
Your system is a little too volatile and the fees are also pretty high. Just wondering, why did you make your price for the system a Fibonacci extension ratio?
[LINKSYSTEM_62867893]
I posted this thread because few of my subscribers got confused and asked me about C2 Score. Infact I too got messed up then i did some research and found that C2 Score is not indicative of present and not reliable.
Collective2 has some excellent statistics to evaluate a system. such as: They rank every trade’s risk exposure in seperate column (Low,Normal,High,Extreme). That is very useful in assesing risk associated with a particular trade.
One of the other most useful stat is Average Win and Average Loss. It is quite useful in assessing how a trader is maximizing profits and minimizing losses.
Along with it, You can check profit factor ratio as well.
Other Important Risk management Indicator is the Max.Peek-to Valley Drawdown.
It is reflecting past but if a system manages its drawdown below 30%, I think it will continue to trade safely in near future too.
Along with it, You can cheack the probabilities of account losses in future. that will give you how risky a particular system in future.
You can get idea of a system when you deeply analyse the avobe mentioned factors but unfortunately, most novice investors dont have knowledge to do that.
They will mostly look at C2 score and which systems are categorised under Most popular systems or hot systems.
I am sure these C2 score and classifiaction of Hot systems or most popular systems are misleading new investors.
In fact, when you click hot systems or most popular systems tab, you will get mostly under performing systems are being listed.
Therefore,Collective2 must fine tune those criterias and must be more proactive rather than being reactive.
Some people said, If a system stays here for years then that should be a better one in future too. that is not true because past performance is not indicative of future. so the scores can not be given merely for its historical records.
Most mechanical systems will perform well for particular period of time. when conditions change they will start to lose money. For example: ETF TIMER. It was very popular a few months back but now it has gone.
This system was around here for couple of years but still it failed.
So past records and C2 score did not stop it from failing.
I wondered the above mentioned system was ranked among top systems and awarded highest C2 score even though it was in losing streak for more than 5 or 6 months.
Therefore, those sort of things must be corrected. I am not trying to criticize a particualr system or Collective2 but I express my suggestions.
So that, Collective2,subscribers and signal providers all can benefit by improving the above mentioned factors.
Regards,
Moderate Trader
Yes, I’m sure you’re right that C2 score and Hot systems misleads many new C2 members. But unfortunately that seems to be what many people want: "Just tell me what’s the best one, I don’t want to bother evaluating for myself"
All stats of whatever type are necessarily based on the past. There is no way to get away from that. When you say “proactive rather than reactive” what do you mean by that? I can envisage a situation where an algorithm detects whether a system’s trading style alters suddenly which could for example help early detection of a “going on tilt” situation. Do you mean something along those lines?
Even that would end up producing false alerts in some situations however it’s done.
In the absence of a thorough theoretical understanding of a system (which as third party observers we don’t have access to) past track record is the only thing we have. An older system that has done well in the past is of course not guaranteed to do well in the future; it is simply more likely to do so than a very young one for which we have very little track record to go on.
I think we are basically saying the same thing though: for a proper analysis of a system one must use a variety of stats and interpret them with care. Condensing all that into a single ‘score’ is very difficult.
BTW, that individual trade risk statistic (Low,Normal,High,Extreme) is also misleading in certain circumstances, such as with option spread trades where two positions are intimately linked. In that case looking at the risk of each in isolation is meaningless.
The fee is a fib ratio because the use of these ratios is part of my method.
The signals are real and the majority of the developers assets are behind these trades, so both subscribers and the developer are on the same side.
I was heading up a hedge fund investment search to deploy capital on behalf of an experienced investor and it was late August 2001.
I had narrowed the search to select candidates, all had very strong REAL, audited and verified performance and risk metrics for 12 months or longer and size relative their strategy was at least $20 million.
No down months exceeding three months in a row, and no significant draw downs in any given DAY.
I personally had met and knew each manager down to their children’s favorite color.
No investment decision had been made and I was awaiting performance reporting for the week ending mid September 2001.
The events of Sept 11, 2001 revealed the reality of my manager’s operations. It was a shock to their system which was able to show me what no performance and risk metrics ever could.
The one manager who had looked the least attractive relative the stats, showed the best performance that September and continues to do so. That investment continues to perform.
This manager had some of the following key successful attributes.
95% of the money was held in cash and only fully deployed into huge winning opportunities.
The manager had specific setups and only traded those setups, removing impulse trading.
The manager had significant personal net worth in the operation.
The manager had lost over 80% of his trading capital and learned what it means to fail, so he knew what it takes to win.
The manager used this failure as an opportunity to improve his method and soon realized significant outperformance.
Managers who have done well in the past and considered safe, I have found are more risky. They lack the required endurance, and drive to overcome setbacks because they seek perfection, have an illusion of control and overconfidence. They do not remember or fail to realize what it takes to win until once again they experience some kind of major setback which is only becomes worse when the market chaos adds insult to injury. This usually happens at the worst time and too quickly to prepare for.
There are managers that cannot perform and mangers that look unsafe and are actually the best investments overall.
There are manager that look safe and can perform while most safe managers are actually the higher risk relative return because the safety hides their major weaknesses.
I would propose C2 create an evaluation system that takes the expertise requirement away from the subscriber.
That C2 create a performance attribution system that
measures SKILL related to RISK in the low, med, and high categories. and measures skill ranked from 1 to 10.
It also is recommended to measure the ability for a manager to SHORT SELL except for long only methods.
Strong performance that is not correlated to market direction is also important and should be given more weighting.
The greatest of these is skill through humility and the superior ability to sell short.
