Folks you have to be very cautious with this system.
According to the vendors feedback the system has had a max. Drawdown of about 62 points ( 3/05) and other big ones DD (30, 47, 38 etc.). Please dont forget that this is only one of thousand other possible outcomes. The system has been tested only from 2002-2005. During that time the market has been in a nice slow swing modus in favor of this kind of systems like blackdog. Should the market come back to large and totally unexpected movements like during the years 1999-2002 you may face you with cumulated DD of hunderts of points. As I asked the vendor for more statistics and trade data especially for the years 1999 to 2002 the only answer I got has been why all these questions?. The vendor either doesnt have the information or he doesnt like it (maybe too bad).
My advice to all of you (especially to Michael): Unless the vendor releases more historic data for all possible market situations the risks may be extreme high should the market leave his current slow swing modus.
Peter
Thanks for the report, Peter. We need more of this reporting so subscribers can look out for each other.
FWIW, I actually think this system has promise, but it is still very young.
W
Walter
Thank you for the reply. Every system has chances and risks and it’s the relation between these two componenets that has to be declared and controled by each vendor. As long as a vendor doesn’t declare these two components his systems are nothing more than gambling machines.
Peter
Peter,
Thanks for the feedback. Although The Black Dog doesn’t use stops, I manually enter them at levels to fit my risk tolerance. This has cost me a few times, as I’ve been stopped out on trades that have turned around for The Black Dog. I’m not willing to hold overnight without stops, whether that be long or short. For those that don’t set stops, they should get identical, or better, results than is posted on C2. I have tried quite a few systems here on C2, and each of them have their quirks. I have vowed to never trade a limit system on C2, or anywhere else again because the record is always skewed towards a best case scenario. I like a system where “what you see is what you get”, so that I can guage what my results probably will be. Regarding The Black Dog doing well because the market has been in a nice slow swing mode, if there is a system there that performs exceptionally in every market condition (and returns more than 10% a year), and is not a limit system, let me know about it, and I’ll check it out.
Michael
It’s very wise from you to use stops. This is the only way to limit the risks. Of course there is no system that performs exceptionally in every market condition but a good system has to know the market conditions that go against his conception and limit or reduce these losses as much as possible (risk management). As long as the provider has no concept to go through these difficult phases and/or he is not willing to show the results (because maybe too bad) his system is worthless.
Peter
Michael
I forgot to mention that for an EndOfDay (or almost EOD) system 3 years of performance results is nothing. If you had a 10 or 20 years performance data set then you would have the chance to evaluate the system’s reaction to different market conditions. Otherwise the so called “market structural risk” is too high.
Peter
Peter
But if the contract you are trading has only been in existence since 10/26/2001, then what do you backtest 20 years back? Of course, you could use the corresponding index as proxy, but then, what good does backtesting do back into the dark ages, when everything was different. Even 13 years ago, when I started trading, you trailed a stop three times a day by phoning it into your broker, and they complained, because some guy on the floor had to go through his gymnastics to change that stop.
Emini Russel, which I am trading, came into existence 10/26/01 with only a few dozen contracts traded, it grew to a few hundred the next year, and probably a few 10.000 nowadays. All that influences the dynamics of a market.
Therefore in my opinion, something that is trading well today, does not need the proof of having traded as well 20 years ago. Which, of course is no guarantee for the future, but then, your next breath of air has no guarantee as well.
Peter_HI
Peter
1) The market indexes for S&P and Russell have been here for decades. Only the electronic markets came later. EOD systems can be easily tested by using the Index and not the specific future contract
2) Noone is asking for guarantees. But what is expected from a vendor is the relation between chances (profits) and risks (drawdowns) so that every investor can judge whether he can afforde it and how to plan his exposures.
Peter
Peter
I absolutely agree with Peter Pritzl. It is in fact absolutely impossible to backtest an intraday system that was created for a futures-contract with data of an index.
I have a system that generates wonderful, absolutely unbelieveable gains and very little drawdowns with index-data, but when I take the signals and trade the futures-contract, the system fails.
The reasons are: Volatilty and time-gaps (while the index-data is calculated, the futures-contract has already made the move).
That is my opinion.
Andreas
Andreas
Thanks for agreeing with me, but that’s not really what I meant.
I meant it’s silly to backtest something that was not really there at the time it’s being backtested for… like a futures contract.
It’s like backtesting the accident rate of high speed trains for a time when there were only horse buggies. Who the h… cares?
Times are changing, and so is the dynamics of the market.
I have something that works now, and I don’t care if it did not work 20 years ago…
I have been presenting my system to a private group of traders for over a year now, and their only obsession is to backtest it this way and that way and with Rydex funds and with index stocks and what not…meanwhile it made a few 100 %…
That was the original reason I put it on C2, to prove to myself that it works FORWARD.
I live in the present…
Peter
Andreas
For intraday systems your remarks are absolutely correct but the system we are talking about is an EOD (or almost EOD) system and not an intraday system.
Peter
Hi Peter Huber…
sounds just like home…
I have not even found that Black Dog.
Sometimes the search engine here is pretty dysfunctional…
Peter_HI
Peter H
I know that you think this is an intelligent request, but there is zero value in getting back tested results for a current system. Zero. Zero. Zero value. You can keep asking, but it is still an invalid question.
1) How would you know those are the REAL results? Almost every web based vendor uses hypothetical results, usually meaning they keep running the numbers until they get one that works
2) The market this year is vastly different than any other year. Sometimes it is flat, like the early 90s. Sometimes it crashes, like the Depression days. Sometimes it skyrockets like the internet bubble days. Sometimes it wanders. Few systems work through more than one of these behaviors. Trying to figure how much leverage, or what kind of stoplosses or what kind of profit targets are best does not work over different types of market geometry.
3) Most vendors combine indicators or forecast signals based on what they see now, and that probably has no bearing on the market environment 8 years ago.
It doesn’t matter how many decades the S&P has been around. It doesn’t matter whether it is EOD. Collective2 allows you to see whether a system has promise. But most of these systems will disappear within a few months.
It only matters whether a system works NOW.
How would one know a system has merit if you don’t do backtesting to get some kind of idea what work and what not? If you say backtesting count for zero, should someone then just dream up some system and start trading it without any backtesting to validate the idea?
If you do backtesting over a long period over different types of markets, at least this give you some kind of idea how the system peform in different markets. What if you look at a system over the last three months and it make 50% with 3% drawdown and you know during backtesting at some stage it had a 80% drawdown. Will you still say, backtesting is not valid and all you care about is the last 3 months and subscribe to such a system?
Everyone now that backtesting is not real results. This is why it is called backtesting.
Don’t take this the wrong way; I am asking because I am really curious. If you don’t believe backtesting is valid and you don’t believe systems on C2 will still be around in a few months, what are you using C2 for and what do you expect to gain from being here?
Regards
- Fanus
Long-term back-testing reveals robust parameters, for the indicators, chart patterns, trading system and for money-management if it is performed with an objective to maximize expectancy. It also reveals which is the best time to enter the market, i.e., Market on Open or EOD etc., Of course, this is optimization, but the moment one has decided to enter at the open rather than at EOD or any other time, one has optimized their system. If a trader is ignorant of the best parameters for his system, what is he trying to prove? That his system is not optimized? To lose money because a trader is ignorant of the best parameters of his system is foolish.
If given the choice of which is best at revealing robust parameters, back-testing or walk-forward testing, I would choose back-testing over walk-forward testing anytime. The results of the long-term back-testing are validated by short-term walk-forward testing as at C2, and as such, cannot fly with short-term walk forward testing alone, as some seem to think so.
ps: seems to me that Ross is naturally pessimistic about other peoples intentions and loves to put cold water on other peoples grand designs, however perfect or flawed it might be, and in the same way he seems to be too pessimistic about the future also. Life is way too short to worry all the time and it might actually become much shorter with too much pessimism around us.
Ross
I totally agree with Fanus in that you really need backtest results and here some arguments:
1) Even if the vendor gives you wrong numbers by asking for backtest results you force him to commit him to something. Should his numbers be wrong then it will take not long to see that reality does not follow the backtest results. For example if he mentions that his Win%=75% and after some hundert trades the Win% is only about 50% then you really know that the system is out of plan and you can react to that.
2) If he gives you an AvrgProfit/avrgLoss ratio of lets say 1.8 and after some hundert trades you find the ratio to be lets say 0.9 then you know that the system is again out of plan. All in all you have to follow the actual trades expectancy and compare it to that of backtested results. The good systems are the ones with stable W:L ratio or expectancy over the time. The higher the volatility of these ratios over the time the higher the risk that the system is only working under certain market conditions and whenever these conditions change the system gets into big troubles.
3) When you have the backtest number you can find out the range between the worst and te best case scenario. That allows you to decide how much risk you want to take and plan your exposures according to that.
4) Once a system gives you as backtested Avrgprofit/AvrgLosss ratio < 1 the you very well know that this system has no chance to survive in the long run. This is because those systems react extremly sensitiv to market conditions changes. With a little bit experience you can find out that a drop of their Win% lets say from 75% to 70% is enough to wipe out any profits and get into the losses zone.
There are many other arguments but it would take long to describe them.
Peter
