Caveat emptor : Quant Models Volatility false track record

And I would add, that X% return achieved over any short period of several days or a few weeks is so irrelevant, that it’s very strange anyone who wishes to be taken seriously repeats explaining again, and again, and again how much that would amount to IF projected over a year.

2 Likes

Hi C2 -

We’re looking at these fills now to make sure they were correctly recorded by C2, and that C2’s Hypothetical Fill Engine functioned correctly. C2’s HFE uses bid/ask spreads and actual trade prints to determine if an order would have been filled.

I’ll post more information after we do some analysis. For now, to avoid a lot of potential posts with high noise-to-value ratio, I’ll temporarily close this thread, and re-open it when I have more information.

2 Likes

Okay, I have reviewed some of the hypothetical fills and agree several did not meet the criteria we try to adhere to. I have adjusted the trading record.

Let me say first that the strategy developer did nothing wrong here. There was no intent to deceive or mislead. Very simply, our software didn’t handle some of the hypothetical fills correctly. The fault was solely mine. I have fixed the incorrect data.

Now, some general caveats about hypothetical track records in general:

As many people on these forums have pointed out many times before (including me!) often they are unrealistic. Even though “market data” can and does validate that a trade could have been made in theory, no allowance is made for actual market liquidity. So it’s possible to create a track record that looks great, but, once more than a few people try to trade it, there is not enough volume to go around for all subscribers, and so fill prices crapify.

This problem is especially acute in thinly traded instruments such as options. Of course, even stocks can be thinly traded, so if you see a strategy that seems super-profitable but trades micro-caps, you should probably discount those hypothetical results substantially.

For these reasons, and others, I strongly encourage C2 members to look for strategies with hypothetical track records that are based on actual AutoTrade data. The more AutoTraders, the better. (Keep in mind, however, that even these track records must still be considered hypothetical.) Sure, it’s a chicken-and-egg problem: is it reasonable to be the first AutoTrader to jump on board if there are no other AutoTraders? It depends on the nature of the strategy, and your ability to discern how realistic the hypothetical fills are.

If you are a strategy developer, this is yet another argument that should lead you to consider becoming Trades-Own-System Certified. While TOS Certification doesn’t alleviate all problems (after all, maybe the strategy owner will trade only a small quantity for himself, which does not eliminate the concern about lack of liquidity), even so it’s a good start (and maybe the first chicken in the chicken and egg dilemma above).

5 Likes