Since we cannot comment on things like “MY ANALYST” postings, I decided to comment on one vendor’s personal rating system of Collective2 systems: Murray Nickel (TrendSensor). His personal rating of top C2 systems is on his website:
http://www.trendsensor.com/Best_Trading_Systems.htm
Unfortunately, I have strong disagreement with most systems on his list, as follows
1) Have Fun - No problem including this in top 10, but not as #1. This system has gone nowhere since March, and it may have peaked/stopped working. A nice equity curve followed by randomness is a hallmark of many C2 systems.
2) EXLENCE does not belong on the list. Soared for 3-4 months and died the last 6. This system no longer appears to work.
3) Gold Survivor Energy Portfolio. You cannot ignore a vendor’s other systems. He has 26 other systems. He keeps creating systems to keep the revenue stream going. When you have almost 30 systems, SOME will look good. Some of his crashed. some are going nowhere. Some of the popular ones recently have reversed and are dropping quickly or becoming random in behavior. This particular system might come in 9th or 10th in my list at a lower price, but for $700 a month I would not trade it unless the other systems get much more reliable.
4) Trending Futures - I will not argue with this rating.
5) Turning Point - - I will not argue with this rating.
6) Corporate Investments - I suspect this system has become random. Not a top 10 at this point.
7) FX.WAVE - - I will not argue with this rating.
His watchlist:
# PIPPROFIT.COM - Had 3 weeks up and now doing nothing. Not worth a watch list. If it does well for another 1-2 months, THEN it might make my watch list. Systems with such a short history are very unreliable.
# TRENDSETTER - THIS system is a better top candidate for the top 7 than the others I dissed above.
# Swing - Had 3 weeks up and now doing nothing. Not worth a watch list. If it does well for another 1-2 months, THEN it might make my watch list. Systems with such a short history are very unreliable.
# ST TREND - This system has a number of trades that average down. These systems tend to blow up. -$1080 DD to bank an $18 profit (started with $10K capital)?? Take the loss!!!
Hi Index,
As you say, it’s my “personal rating of top C2 systems”. As such, it is based entirely on objective performance measures and criteria. Most of your comments appear to be subjective (based on your experience or observations about what tends to happen to systems on C2).
I’m not saying my criteria are perfect, and I’m constantly working to improve them. However, I’m not prepared to overlay a subjective assessment as generalized observations may not apply to a specific instance, even if they are correct over 50% of the time. On the other hand, a system either meets, or doesn’t meet, specific criteria.
No doubt my personal biases influence the selection and cut-off levels for those criteria, which is why I emphasize it is my personal view of the best systems on Collective2.
Some members on the list do warrant additional comments/alerts (in my view) and I have added these at My Analyst Page:
http://www.collective2.com/cgi-perl/favorites.mpl?want=view&seewho=15051131
Some specific comments:
1) "Have Fun - No problem including this in top 10, but not as #1."
Actually, it ranked as #3 (see the Muzza Rank in the green spreadsheet rows).
"This system has gone nowhere since March, and it may have peaked/stopped working."
Actually, May and June were close enough to “normal”, while April was -0.8%. July is not going well, but is still only -2.6% to-date. I am unable to conclude this means the system has “stopped working”, but perhaps I can’t predict the future as well as you (if I could my systems would be on the list! … then again, so would yours …)
Have Fun has proven itself one of the most reliable systems on C2, and I think it premature to write it off after one negative month. I think too many subscribers have unrealistic expectations and move on (to get their fingers burnt again) at the first hint of trouble. Part of my goal with the objective criteria is to encourage a longer-term view.
2) "EXLENCE does not belong on the list. Soared for 3-4 months and died the last 6. This system no longer appears to work."
I’m not sure “died” is accurate, but it has certainly slumped into a trading range and gone nowhere - I allude to this in my analyst page (see above).
It meets all the criteria, so remains on the list. I agree it is doubtful it should be on the list, but I think the solution is to revise my criteria, not to add a layer of subjectivity.
3) "Gold Survivor Energy Portfolio. You cannot ignore a vendor’s other systems. He has 26 other systems. He keeps creating systems to keep the revenue stream going. When you have almost 30 systems, SOME will look good."
This comment is an objective one, but I have no way of knowing whether the conclusion, or inference that this system will eventually crash, is correct. Some of the most successful traders in the world started out with a series of dismal failures. Maybe this vendor has finally developed a winning system. I don’t know. I can only evaluate it on it’s performance to-date.
Certainly it’s wise to check a vendors other systems and ask some probing questions, but lots of other losing systems doesn’t guarantee this will become a losing one (it may increase the odds, but I’m trying to evaluate actual performance not speculate about possible future performance).
6) “Corporate Investments - I suspect this system has become random. Not a top 10 at this point.”
I also have reservations about this system, and have notes on both my website and my analyst page about the leverage the system uses - I think it will be its undoing. Certainly, July is not looking good, but as at July 16, it still met all the criteria.
I’d love to add a portfolio leverage criteria, but unfortunately C2 doesn’t report on this, despite it being requested in these forums on numerous occasions. It’s difficult to calculate accurately without downloading all the trading data, which I’m unwilling to venture into at present due to the time the work would involve.
Re the watchlist:
As a general comment, I’d say the purpose of this list is to identify young systems that may be worth following as they age and “prove themselves”, NOT to identify systems worthy of investing in now. Note also that in the next update I will be extending the minimum age for those on the watchlist from 60 days to 120 days.
A specific response:
PIPPROFIT.COM - I’m not sure how over 6% up in July qualifies as “now doing nothing”.
Finally, feel free to create your own “best systems” list. I began mine because I felt those on C2 lacked some key measures and were generally focused too much on recent performance and too little on performance over the long-haul.
I don’t claim its perfect, but I’m committed to keeping it objective and reviewing and improving it over time. Thanks for your feedback - it helps my thought process around criteria improvement.
Murray Nickel
M
Some of your comments, you are looking at the performance monthly (May, June, July, etc…). I am not. I am looking at the equity curve. Your analysis basically ignores this aspect, but many potential subscribers scrutinize this.
A vendor’s other systems are a very critical part of an analysis. For example, Gold Survivor had some 5-6 systems a little while back that looked very good, Then most except the Energy One started to go into a strong drawdown. There are other vendors who continue to generate systems. Your thought is not correct - vendors who continue to generate new systems and kill off or ignore failures tend to keep failing in the future. If your analysis ignores this, then your analysis is not being objective, it is incomplete - blind to a strong predictor of a system’s future behavior.
Index
"I think too many subscribers have unrealistic expectations and move on (to get their fingers burnt again) at the first hint of trouble. Part of my goal with the objective criteria is to encourage a longer-term view. "
Murray / Ross,
I agree entirely with this point. Most people are just looking at an equity curve and as soon as it stops stair-stepping ever higher they conclude it must be broken. Of course there will be instances where this becomes true but I think in the case of systems like Have Fun which has a 15 month history with a max drawdown of just 3.5%, to conclude it may have stopped working just because it goes sideways for 2 months is slightly premature and a little harsh to say the least.
Good systems that truly stand the test of time will always have periods where they don’t perform, to not expect this is as unrealistic as not expecting to have losing trades. It WILL happen. As long as the vendor hasn’t changed anything and the system is still statistically sound it should be viewed as a good entry point for subscribers. How often have we heard people complaining because they subscribed to something after an improbable run only to then abandon it and write a poor review when it comes back to a more realistic level and yet all the time the system has performed within it’s parameters?
I would say that unless a system specifically stated it’s objective is purely to have a perfectly uniform equity curve then it would be wrong to conclude it’s broken when it’s curve briefly deviates or plateaus. Surely to write off a system based on it’s curve alone is no less foolish than subscribing to one based on it’s curve alone.
Murray,
Specifically on Corporate Investments I’m not sure why you think it had severe intraday drawdowns, the intraday chart looks very similar to the daily average to me, I’m not aware of having had any specific day that wasn’t within expectations. I’m wondering if when you captured the data it still had some faulty stats in there as recently I had to send a lot of ‘fix’ requests to MK as Crude and Bond risk/drawdown stats showed errors with some prices down at single figures or zero. That may explain it.
I thank you for your concern on leverage, it is always a dangerous area but I believe the trend following futures trades are fairly conservative trading just 1 or 2 contracts, possibly on some of the commodity/forex swing trades there may be a case to answer and I will certainly look into that.
Ross,
I’m reasonably confident the system isn’t random, rather that with the recent sharp reversals and volatility the market has not been kind to long-term trend following systems but again, this is to be expected from time to time as markets do not always trend. Trend following systems often tend to have low win % (often just 40%) so it is always tempting to conclude they are no better than random (50%) but the key is rather avg win being higher than avg loss so a trend following system is able to be right a lot less but ideally still have a profit factor of >1.5:1 or in the case of Corporate Investments 2:1.
As a subscriber who does not have a system to defend, I have to say you’ve lost focus of why people will subscribe to a system here on C2.
Unlike for the Vendor who created a system, for us Subs (and I speak for those with similar objectives) a system is NOT an interesting idea to explore, or an intellectual challenge, or a sickly puppy to be pampered back to health, or an investment of weeks, months, or even years of my life that I’m emotionally attached to.
For Subs like me, a trading system is simply a way to win more trades and increase our bank account, nothing else. It is a simply a TOOL to reach this objective, and that here in C2 we Subs can rent on a monthly/weekly basis to use for that. It has no intrinsic value to us, only the value derived from its continued performance.
And if it stops working, not only I am losing the rent money, but this broken tool is now also depleting my bank account - the opposite of my goal in renting it.
So should I keep paying for it, to see it it will eventually stop bleeding my account, or should I return this broken tool and get one that actually works?
As any investor knows, “hope and pray” is not a good investment strategy, even when the underlying asset has an intrinsic value. But systems are no assets to a sub. There is no reason to stay and hope for the best. Once a system malfunctions, swift action is required by the Sub, for a system can go down in a single day - and take with it real chunks of money from the Subs account!
This is an understanding that every Vendor should have: your system is just as good as its last trades. Lose more than a few in sequence, and you performance of weeks ago just doesn’t matter anymore - it’s simply history.
If the Vendor believes that his system still has merit (not always the case- look at how many systems are abandoned after a steep drop) then it’s clearly and solely the Vendor’s burden to prove it to Subs in a convincing manner. And not with words, but with PROFITS.
And until that’s very well established by the Vendor, no Sub should stick around losing money in an broken system, unless he’s is a fool. And a fool and his money won’t be together for too long here at C2.
Index is correct,
As a Sub I ALWAYS look at other/abandoned systems for hints of future performance issues.
Unfortunately, it is easy in C2 for a failed Vendor to simply create a new identity and star life fresh, while still displaying the same traits/skills that led to the previous failures.
I can assure I understand why people subscribe to systems and I assure you we have the same aims, but I guess we differ on how you define when something isn’t working, but you cannot know that unless you know the vendors reasoning and parameters in devising the system. Personally I think if you abandon every system at the first downturn and yet it is statistically sound you could end up selling at the end of every correction in a long uptrend.
Respectfully, what you’ve described that your looking for is called the Holy Grail and I think I can say without fear of contradiction you won’t find it.
Jon.
Maybe you and I have the same aims, but I can’t say the same of most C2 system Vendors.
For some Vendors a system is just to test an idea or concept, and they would never trade their own money with it since they know how fallible the system can be. But this is never disclosed to Subs, who instead are fed the typical “make lots of money with low risk for only $199/month”.
There are scores of examples here on C2 of “stable” systems that failed spectacularly in short order and with no warning. How can a Sub protect his own account from such seemingly well-behaved systems, with previously sound results?
By having a quick mouse, that’s how. Click first, then ask question later!
Once a system misbehaves, there is NOTHING that says it will “revert to a mean” of sorts. Look at C2 history for proof. It’s more illustrative than any scholarly statistical study. Once a system fails, chances are it will go down. And many vendors acknowledge that by simply abandoning those systems behind.
Should a Sub stick around and go down with it?.. Not this Sub! What I am advocating instead is a quick response by Subs in cutting loose failing systems. That should at least protect their capital from further erosion. And by doing exactly that this week I saved myself over $3,400 from additional losses in systems that had great records so far but started faltering in the last few days. So I KNOW that works.
As for chasing the Holy Grail, I know it won’t be found, at least not in one single system. Here is my strategy, in place of it.
We all know that the sum of all systems, all markets, everything will EVENTUALLY revert to an historic mean. Look at Mutual Funds. We all know that loading on a winner of recent years can lead to disappointment in future years, as it goes on to underperform the market.
HOWEVER, what several studies have shown is that momentum doesn’t stop overnight. So a strategy of moving to a mutual fund with a recent winning streak has a great probability of profit for AT LEAST THE NEXT 3 MONTHS. Like a train, even it is going back, it will take time to stop and reverse course.
I follow the same in C2 systems: momentum.
Of course, C2 systems behave more like racing horses than trains, and it takes only a second for one to trip and fall. So jumping from a racing horse to another to another is trick, and requires attention, time, and skill.
But in the absence of a single system that can outperform forever (what I agree is the HOLY GRAIL), being in different systems during their individual prime periods can be profitable. As long as you have a quick mouse!..
It’s all a matter of personal opinion. People have different risk tolerance, return expectation, available capital, confidence trading different instruments, available time, and so on. Otherwise, everyone on C2 would be subscribed to the same system, trading with the same broker, etc.
I’ve seen systems that I would describe as mediocre at best, yet they have real-life autotrading data available, so at least someone is trading them. Murray doesn’t even rank the only system that I’m trading. I personally celebrate that not everyone thinks alike.
Mind sharing what system that is?
mvp-3
I just started trading it a few days ago, so I can’t speak for my real, personal results yet. But as far as statistics go, which is how Murray ranks systems, it is definitely on my (very) short list.
I’ve been subscribing to it for a while with impressive results. Highly recommend. [LINKSYSTEM_29261068]
"HOWEVER, what several studies have shown is that momentum doesn’t stop overnight. So a strategy of moving to a mutual fund with a recent winning streak has a great probability of profit for AT LEAST THE NEXT 3 MONTHS. Like a train, even it is going back, it will take time to stop and reverse course."
If you can do this and make money then you should be able to apply the same strategy to trading the markets. What you are doing is trading with systems rather then a market. Equity curves and price movement are one and the same. If you can read price movement you can read equity curves and vice versa. Both are numbers and vibration.
Hi Ross,
1) "I am looking at the equity curve. Your analysis basically ignores this aspect, but many potential subscribers scrutinize this."
I assume you mean the equity curve as charted on C2. I think this is both subjective and misleading. Have you ever zoomed in on a chart and noticed how the equity balance on the LH scale declines as you zoom in (often by 40% or more!) Probably just a scaling issue, but it hardly encourages confidence that what you’re viewing is accurate. This happens with both IE and Firefox.
Secondly, there’s no fixed scale, on the left. That’s why a system that does over +6% in July (PIPPROFIT.COM - now over 7%) can look like it’s “doing nothing” - because July follows a +30.3% in June, so the scaling makes July look relatively flat on the chart, even though it continues to perform fine. Meanwhile, a different system, that gains 2% every month will look better on the charts as it will have different scaling and show no roll-off in performance. Yet its actual performance for July is less than a 1/3 that of PIPPROFIT.
Yes, subs look at the charts. Despite being a statistician, I’m a person who likes pictures more than numbers - it’s just important to be aware of their limitations.
2) "vendors who continue to generate new systems and kill off or ignore failures tend to keep failing in the future. If your analysis ignores this, then your analysis is not being objective, it is incomplete - blind to a strong predictor of a system’s future behavior."
Yes, any vendor who ignores failures and fails to learn from both failures and successes is doomed to fail again. I strongly agree that subs should check a vendors other systems and ask probing questions.
Anyone who has read The Market Wizards series will know that failure precedes success as reliably as it precedes further failure, so as a predictor in itself, it is of little value. The key is what is learned from the failure, and what is changed in future attempts at success. The best way to determine this is to take the effort to ask some probing questions.
In the case of Gold Survivor Energy one could also ask:
a) If this system is recognized by the vendor as their best and one with long-term merit, why haven’t they closed all other systems to focus on this one.
b) Do the recently opened systems take the Energy strategy and apply it to other markets that backtest well? If not, why not?
So yes, I think the alarm bells should be ringing for this vendor (I’ll add some comments to my analyst page), but I think your conclusion re the predictive nature of failed systems is flawed, and one needs to dig a little deeper to find out more about the vendor.
Hi Jon,
If the stats are OK now, there is still the issue of a $13.7k intra-trade drawdown on 04/22/08 (short 60 EURUSD). At the time the system balance appears to be around $120k, so this DD represents 10%+ of balance, which rates as “severe” in my book. A few others exist as well.
Re: use of leverage. My reference was to portfolio leverage (also sometimes referred to as “gearing”) as opposed to leverage on an individual trade. Let’s take an example: at 4:00am on 06/13/08 balance was roughly $180k. Open trades were EURUSD, GBPUSD, @TYU8, @WN8, RIMM, plus the potential for others that still remain open (as I don’t subscribe I’m can’t tell whether there are more). Total value of these is at least $2.1m, giving a PF leverage of over 20:1, a level I view as excessive and dangerous. Yes, it seems to be the forex trades that are massively overweight while the rest of your risk-taking seems reasonably appropriate.
Hope this helps,
Murray
""Anyone who has read The Market Wizards series will know that failure precedes success as reliably as it precedes further failure, so as a predictor in itself, it is of little value. The key is what is learned from the failure, and what is changed in future attempts at success.""
I totally agree. If the attitude displayed here towards vendors with failed systems were replicated in the real world then most if not all of the greatest traders who ever lived would never have made it. Early failure (and learning from it) is often what ultimately made them successful.
Yes that makes sense, Forex is obviously much higher leverage than other asset classes, I treat it the same in terms of the actual capital at risk but depending on position sizing the leverage used can vary enormously, definitely something for me to consider, thank you.
Re mvp-3:
Yes, I agree: I mention on my website that in the next update I’ll make some changes that should enable systems like mvp-3 to be included.
Currently, it is excluded because of a 25% drawdown early on in its life, but it has performed very well since then.
I plan to increase the max DD allowed to 30 or 35%, and place more weight on the Annual Return:Max DD ratio. I’ll also add weight to an adjusted Market Experience factor (one that takes into account the quality of the decision making - using Sharpe - as well as the number of decisions made).
"I assume you mean the equity curve as charted on C2. I think this is both subjective and misleading"
I have been here more than 2 years. You are thinking like an academician. To most subscribers, the equity curve is the PRIMARY THING that draws them to a system. Everything you said about scaling and the equity curve is not particularly interesting to a newer subscriber with a brokerage account and a desire to make funds. Wladimir seems to understand this. You seem not to.
When you understand this, you will probably realize the gaping hole in your ranking methods. Until you do, your ranking system leaves much to be desired.
Frankly, most of the “good” systems here on C2 are simply the result of sheer luck. If you have 500-600 systems monitored, then 5-6 of them will be in the top 1%. They will look good. And that is usually due to PURE CHANCE. HAVEN’T YOU HEARD HOW MANY PEOPLE SAY “I SUBSCRIBED AND IT STOPPED WORKING?” Murray, that is because most systems never worked, they were lucky. Once they attract interest, like most other nonworking systems, they randomize. THAT is why I said systems that have not worked for 2-5 months, or had a brief spurt and were flat otherwise, ARE PROBABLY WORTHLESS.
I have 4-5 systems ahead of most of yours (and no, I do not plan to share that).
As long as XLENCE stays on your list or a system with 13 trades/couple of up weeks is on your watchlist, you do not understand what subscribers are looking for and your list is of questionable value. You share academic ideas and objective rankings, and are basically leaving out what matters to a person with money.
People vote with their feet.
Just going to the “most popular” list is probably a better list than yours.
How should one REALLY judge systems?
1) Your method is one version of step 1 – one makes a list based on their criteria
2) Then one follows that ORIGINAL list for a month + to see if their few favorites CONTINUE TO OUTPERFORM. If they don’t, they are culled. That is step 2. .
3) Step 3 is where you subscribe and follow. And MAYBE you have a system that works…
