Bravo Matthew. A step in the right direction for screening trading systems we can actually use and obtain realistic results. When do you plan on adding a column for the Realism Factor to the 1-Click Search results?
I agree - would be useful addition to search criteria
Column-header-click for ranking by Realism Factor would be just perfect 
ChrisQ
One of the factors that go into the calculation of Realism Factor is:
(2) Likelihood of just-touched orders. If a limit order is filled in a hypothetical trading record on a “just-touched” basis, C2 determines how likely it would be for a real-life trader to receive the same fill. Further, if a real-life fill is unlikely (or impossible), then other, subsequent orders may be affected. This “cascading” effect is a major advance in trading system analysis.
I wonder, under the circumstances whether the Realism Factor (RF) (which measures how much the “hypothetical” performance of a system is deemed to differ from its “real” performance reflecting the true productiveness of the trader) of a system truly reflects the tradeability of a system, because for systems that use an excessive amount of limit orders, the RF seems to be still very high.
Only Market orders are likely to be filled. Limit orders that are filled in a hypothetical trading record on a “just-touched” basis, should be deemed as unlikely (or improbable), because it does not pass a specified minimum confidence level (say 95%) to be deemed likely (probable) in the real-world. Statistics deals with probabilities, not possibilities. The question one should ask is what is the probability (not possibility) that the limit order fill can occur? Dealing with possibilities instead of probabilities is a mockery in practice, of the theory of statistics.
If a man attempting at a compromise asks “Who am I to know?” or “Who is C2?” he can hardly be loyal to the knowledge he denies having. Nor can a pragmatist be loyal to his mind, claiming as he does that “truth” is a name for whatever satisfies men’s desires. Nor can an agnostic, who boasts that his mind remains “open” no matter what the state of evidence. Nor can a relativist, who declares that there are no absolutes, forgetting that he is uttering an absolute. All such individuals, whatever their pose, dispense with convictions as a matter of conviction and are, therefore malleable. In their view, loyalty to principle - whether the principle be self-interest, independence, justice, or any other - is dogmatism; it is “rigidity” and disruption of the social fabric.
Since skeptics stand for nothing in theory, they can stand by nothing in practice.
There actually should be no need for a realism factor. If all limit orders are required to WAIT until price PASSES them by at least the bid/ask spread; then the price point is valid and realisable by anyone. As to volume, Matthew is well aware that there needs to be some sort of reality check on the volume a system trades. IMHO unrealistic position sized orders should be rejected out of hand. Those that are realistic, but potentially oversized for a slipless execution, should be slipped by some reasonable amount, say half the 5 minute average range. That will bring a lot more realism to the results we see here. It isn’t a comprehensive solution, because it is very hard to deal with volume issues on things like options, and some pit traded contracts.
If all the results are more realistic, then we won’t need to have a realism index - the results will speak for themselves.
But even without that - where else are you going to find a place like this?!! NOWHERE, so I say BRAVO Matthew - and we can all be patient while you continue to improve this site!
Realism index is a very important and usefull tool for the systems trading options, I agree. I’m convinced that most of the option system are indeed untradable in reality.
Considering stocks, e-mini, futures, the realism index still makes sense, because of the volume issue and the variable spreads.
As C2 doesn’t take brokerage fees into account, the validity of the results displayed for most of the systems is anyhow very doubtfull. I even think 90% of the systems are totally useless, even if they apparently get some good returns.
On the other hand, this new realism index is not usefull for forex system.I even think it’s actually misleading. Why ? Here are the reasons:
1) volume isn’t an issue on forex market.
2) C2 computes results with larger spreads than the real life spreads.
3) There is no brokerage fee on forex. These fees can virtually turn almost any winner system on C2 into a loser.
I claim that if the least realistic forex system deserves an index of 80 points, then the most realistic option or e-mini system doesn’t deserve more than 60 points.
Brokerage fees are included in system performance statistics on C2. Recheck your stats page.
As I said above, the volume issue and limit fill issues should be dealt with in the engine.
I will disagree with you on several counts though. Volume IS an issue on the FX markets - as soon as you start trading 1000+ lots you will start moving the market with some of your orders. The only effect a spread has on trading is on the actual execution of limit orders. In order for a limit buy 03 order to fill on a 3 pip spread, the bid has to drop to 00. C2 does not do this - C2 will fill a limit buy when the bid hits its price, not the ask.
The only system that can go from winner to loser on the addition of commissions, is a system whose average trade is less than the commission for that trade. While there are SOME systems in C2 that would suffer this fate, they are by far a minority.
I currently trade my systems for a hedge fund and a private money manager (not C2 subscribers) and while there is always going to be some slippage, its effect is minimal on overall performance, as are commissions.
Sorry for the mistake concerning fees, you’re right and you know what you speak about, thanks.
I use to trade EUR/GBP pair rather often and I never had an impossible fill. C2spread is 5 or 6 pips on that pair, my own broker quotes 3 pips spread. I never had a sell order executed at ask price or a buy order executed at bid price. So I don’t understand your post very well (?).
I guess that nobody uses C2 signals to trade 1000+ lots on forex. If the market moves when big orders are executed, it moves 1-2 pips, no problem. On some stocks it would easily move more than 25% and that’s the big issue.
The fact is that some systems have very low realism indexes (some under 60). That’s a very good point for C2 credibility. Thank you C2!
The realism factor can be deceiving since if we buy market price , it takes too long for c2 IM to appear thus can be a problem , therefore I suggest you send out signals on a slow market at market price it seems to be better the trader since high volatile market will not be able to acheive a proper grading system base on market price. Thus the level of realism should take that into account and I think the best way to design a working system is to do market to maket somehow the IM should work faster so that it would let the subscriber know faster rate of trading
But AutoTrading fills are nearly instantaneous. (It is true that ITM signals can take up to 10 seconds on average to appear.)
MK
Ahh, but let us not forget - lag isn’t just from C2 to the client, it exists also between the signal generator and C2.
Vendor’s system --> Internet --> C2 --> TB --> order entry app --> Internet --> exchanges. All I can do is to pray.
There are even more steps:
Vendor’s system --> C2 --> TB --> order entry app -->broker server–>possible risk management server–>gateway–> exchange.
You probably don’t want to know how many hops the packets will travel through on the internet.
What matters is once TB gets the order it’s at the exchange within 0.1 second with IB and even less with other brokers, assuming a reasonably good internet connection.
Francis
I’d like to congratulate Matthew as well. Unfortunately, because I don’t trust “words”, even protected by first addendum, I spent my three evenings comparing RF from C2 and my own RF. I was surprised with insignificant difference in the two RFs that can be explained by unavailable info for me (real auto trades via TB).
Very respectful
I checked only system that I’m a little bit understand (stocks & futures) and with significant amount of trades from my point of view.
Still be respectful
Of course. RF must be added to search to filter out paper boys. it’ll save a lot of time for subscribers when they compare/check C2’s systems.
Also it’ll be nice to have RF for every trade, not just average.
Matthew,
If you’re not tired from raw ideas. RF is really big step from paper to real word. And it gives you opportunity to offer forward testing. So far, forward testing was trading one lot to see how a system performs in real world. Subscription to C2 is dirty cheap lol if we compare it with forward test expenses. I think, it might be good idea to split C2 systems to published/unpublished. Published is as it is now. Unpublished systems are for forward testing that aren’t for public. If creator of unpublished system wants to offer the system for public he must start form the beginning with published system.
Eu
Eu,
Here in the USA, it’s protected by the first AMENDMENT.
Anyway, you have a good idea as far as published vs. unplublished systems. But why must you start over if you want to make a system published? Start unpublished, at a reduced fee, then go published when you get a good track record established, at a higher fee. But be able to keep the trade history. The fee for a published system can then be set higher than it is currently for all systems. This would be a good way to separate the wheat from the chaf. Of course, all history would go toward your rating. That way maybe you avoid some of the vendors playing systems until they get a good (lucky) one going.
Brian
BTS Bottle Rockets
BTS Small Cap Value
Here in the USA, it’s protected by the first AMENDMENT.
Bingo! Anyway, I’m not American and I would not like to discuss American culture.
>But why must you start over if you want to make a system published?
It was just a raw idea. Most of business models will work.
Eu
If you are using C2 as a forward testing tool, then go ahead! There is no need for a published/unpublished categorization. If you don’t want subscribers to see your signals, set your billing to “no trial period” and “$10,000 per trade.” Trust me, no one will be buying your signals!
What C2 effectively does is provide a third party audited report on the performance of your signals against the real market. Published or not changes nothing.
Why don’t we let subscribers have some input in the Realism Factor? Part of the factor can be based on grades/ratings submitted by actual subscribers (how well were they actually able to follow through on trade signals?). Without this, the Realism Factor has little merit.
Brian Schumacher
