There is a great need for a statistic that grades each system as if it always traded a constant dollar amount.

I do see the point that presumably bigger bets are placed when a vendor is more confident of a trade BUT the problem is this:

Say a system trades minor amounts for a long time and accumulates a mediocre record. Then one day decides to bet big and the trade goes well. All of a sudden now ALL the statistics of the system are great because of that one last trade that dwarfed all the others in size. Now the vendor can resume trading minor amounts in which case it becomes very hard to compromise the system’s stellar stats.

Currently the only indicator of a system looking good just because of one (or a few) huge bets gone well is the Chart. But the chart of such a system can easily be mistaken for the chart of an honest system.

Any of several indicators currently used could be used to uncover a one-trade-wonder if calculated using a constant trade size. For example Sharpe, Profit Factor or APD.

Please consider adding something like that because it will give a perspective into the systems that is currently not revealed unless one goes through the trades one by one.

Dimitri

Dimitri - I realize you are fairly new here. You can go to the Grid and filter/sort on Sharpe Ratio or Profit Factor and there is also an indicator of hidden risk.

I agree completely with Steve, here. These statistics will come out in the Sharpe.

Steve,

I am familiar with the grid and how to sort and filter it on the different metrics.

I was asking for a new statistic that will reveal information about the non-sustainability (or dishonesty) of certain systems that is now completely concealed by all other statistics including: Sharpe, Profit Factor, APD or Hidden Risk. I know this may have sounded uninformed or newbie-ish but it wasn’t, please try to see my point.

I’ll give you a made up example of the kind of system that I am referring to that I want to uncover by using the statistic I am asking for:

System started trading stocks one year ago with 100k.

During the year, about twice a week it traded one stock worth about 25k and closed the position within a week.

In one year of trading it made 99 trades. 66 winners and 33 losers.

Avg win $2000, Avg loss -$4000. Overall broke even. not good, not bad, no subscribers.

Now for the 100th trade system decides to go all in using 3:1 margin. and buys $300k worth of a high flying stock. Trade goes straight up.No drawdown. A week later the position is worth 450k and it gets closed. System Profit for the year: 150%

Now that is a system with really good looking stats. Would anyone consider this system to be as good as it’s numbers indicate? Not if they are thinking straight! And yet here it is looking really good at least to anyone using Sharpe, Profit Factor, APD, ProfitableTrades or any other stat for measures.

Monte Carlo will not even catch this system because Monte Carlo will take all past trades and produce a series of histories randomly generated by sampling from those past trades. Having no big losses and no big drawdowns among the sampled trades Monte Carlo will tell you with high confidence that the distribution of the random paths the system can take is not risky at all.

Same goes for “Hidden risk” at the Grid. Hidden Risk is simply made up of "1) the average intratrade drawdown as a percentage of account equity, and 2) the “risk of 20% ruin” as calculated in C2’s Monte Carlo simulations."

So if all the stats tell us this system is not risky and most stats tell us this system produces high returns how will we ever know we don’t want to trade this system?

Well, I see 3 ways:

1) Look at the trades 1 by 1. Disadnvatage: Time consuming and it defeats the purpose of having stats to summarize them for us.

2) Look at the chart. Disadvantage: It’s subjective and not very convenient in comparing many systems. Not always obvious (although in my example it is)

3 )Have a statistic that will put equal weight to all the trades that have been taken by a system. This can be done by using the equal weight Sharpe, the equal weight Profit Factor or the equal weight APD or perhaps something else that uses a constant dollar amount or a Volatility adjusted constant dollar amount. Disadvantage: Not talking into consideration the fact that bigger trades may be taken when a vendor is more confident (but that’s reflected in all other stats)

Overall, I think such a stat is seriously missing.

Dimitri

Dimitri - Sorry, I misunderstood your point in your first post. I don’t know anything about equal weight indicators so I can’t comment on that. But one bit of information I would like to see is the peak margin used over the life of the system. So if at some point in time, even if only once, the system vendor was close to max’ing out on margin then it would be readily apparent. This would be a good warning bell.

Steve

there are outliers in advanced stats. You can have first warning signs from the stats.

Eu

The outlier statistics basically answer this question:

"How have the best/worst few trades fared in relation to the rest of the trades and how many of them have there been so far?"

What’s missing is the answer to these questions:

“How good would the system have been if each trade was equally weighed?”

“Would the system be better or worse if it traded the same $ amount every time?” (or same volatility adjusted $ amount)

In other words: Would those outliers (and also the trades that are not currently outliers but would have been had they traded the usual amount) make a big difference or not? (in terms of the Sharpe Ratio for example)

If a system fared a lot better by risking a constant amount then maybe the vendor wants to reconsider or even stop trading bigger amounts on his AAA trades.

If a system fared a lot worse then someone following it should consider whether they would be comfortable when those few extra-large trades that are now propping up the system happen to later start going the other way.

I’m telling you having such a statistic would shine a different light to the systems and expose quite a few of them.

Dimitri

It is a very interesting idea and something I should be able to implement.

Now we’re getting somewhere. Word got to the right person… Matthew thanks for looking into this!

Some thoughts on implementation issues in case they are helpful:

Compound Annual % (or Sharpe) may not be the right indicators to use as constant $ indicators due to the fact they would not be directly comparable to the regular Compound Annual % (or Sharpe) of a system because there is no compounding effect when using constant $ trades. Of course that would still be acceptable if we settled for comparing only against other systems and not against a system’s own Regular stats.

On the other hand a constant dollar APD (or even Profit factor) would be directly comparable to a system’s own regular APD (and Profit Factor) in addition to other systems’ and therefore in my opinion preferable.

If the more accurate volatility adjusted constant dollar amount was to be used it would be sufficient to simply set the amount of each trade so that this ratio: (trade amount)/(historical standard deviation of the stock or future on trade day) is always equal. The look back period for calculating standard deviation is obviously arbitrary. I would favor a longer period to minimize the influence of one time large jumps but just using what’s already being calculated would also work. Also to state the obvious: look back period ends before trade starts to avoid any look ahead bias.

There may be computational intensity or other issues to consider so if this was implemented by some other equivalent way would still be a very welcome useful addition.

Dimitri

In the Grid, there is a “Assuming each buy/sell signal is traded at exactly 1 lot” section. Won’t it help you?

This feature is only visible for forex systems. But I can extend the functionality to non-forex systems.

If 1 lot of a stock is 100 shares it wouldn’t work.

It has to be for example $1000 worth of stock OR if the volatility adjusted version was going to be implemented then it’d have to be:

$1000 worth of stock when Standard deviation is 1,

$500 worth of stock when Standard deviation is 2

$250 worth of stock when Standard deviation is 4

$2000 worth of stock when Standard deviation is 0.5 etc

Same goes for futures.

Preferably the constant amount would be the same for stocks and for futures so we can compare between systems that trade either. Using fractional shares or contracts would be fine. It’s the $ amount that’s exposed that matters (or hopefully the volatility adjusted $ amount.)

I do not think we’ll be able to do volatility-adjusted constant sizing in the first iteration of this, though perhaps in a later version. But constant-dollar value is simple enough. Let’s see how the first update works out and then we can add additional functionality.

*The outlier statistics basically answer this question:
“How have the best/worst few trades fared in relation to the rest of the trades and how many of them have there been so far?” *

Exactly. And mean of the trades. Your initial question was how you can recognize suspicious system without digging in to all trades.

*In other words: Would those outliers (and also the trades that are not currently outliers but would have been had they traded the usual amount) make a big difference or not?*

Equally weighed trades will show you a system without MM. We assume that MM is part of a system and system vendor already has the stats and uses MM for some reason. I’m not sure what you can have from the additional stats. Comparison of system stats with MM and without MM? How are you going to practically use it?

*(There is strong assumption, that system vendor made a home work and already knows why he/she uses particular MM, from my side. In the case the stats is nearly useless for you )*

Routine task for outliers is more simple and understandable. All

**positive**outliers might be removed from calculation of system stats, because they aren’t repeatable and lay out of the system expectation. You will have “no luck” statistics for a system. It’s very simple.

Eu

This isn’t really addressed to you, EU, but it’s very obvious people who don’t have wealth lab would not know that this is a very simple backtest to create. One position size should be able to be shown by the vendor on demand.

Matthew,

Thats fair enough, thank you for your work!

Eu,

The constant amount indicator will tell us things that the outlier statistics won’t:

In just one number (eg constant dollar APD) we have a way to measure a system that by itself will tell us the system’s total worth in a way that is different from the equivalent standard indicator or any other stat currently available.

We can use that one number to call a system better or worse than another!

We are also not throwing away data points just because they are outliers. What we are throwing away is indeed the vendor’s judgment as to which trades to bet bigger. Instead we are treating each trade as an equal and trying to see how good a system those equally weighed trades make. I consider it obvious that it’s a lot easier to find a positive expectation trade than it is to know what that expectation is and so bet accordingly. So I would like to have the option to give each trade a system recommends the same influence on my judgment for that system.

Additionally who is to say that systems under consideration are betting higher because of higher expectation and not because of other reasons including (but not limited to) trying to make the chart look smoother or using some form of a limited (or not) martingale strategy and which system is which. There is also the possibility a vendor is good at picking good trades but not good at deciding how much to bet each time or he could just have a different utility function than the person trading the system. A constant dollar APD would not penalize such vendor but would just highlight the fact that the size of a trade recommendation is not mandatory.

My concern when asking for this is not the outliers. My main concern is how to uncover systems with long histories that have gotten lucky in their big bets regardless of the reason those bigger bets were placed. By looking at the outlier stats you can’t tell if one is there because the particular return distribution of the system justifies it (in which case it should be there) or because it was a lucky large bet.

Also, having many or few or large outliers will by itself not tell us a thing about a system’s total worth not to mention how it compares to other systems. Not unless examined further and most likely going into the individual outlier data.

The outliers are what they are, the constant dollar statistic is calculating something else.

Assuming your “MM” stands for money management: Each person is responsible for managing their own risk. I may or may not use a system as intended. I may trade multiple systems and just be concerned about my overall portfolio risk. I may only trade a system when going short (or long) to hedge my risk in the rest of my portfolio. I may just be looking for trade ideas. It’s good to know how good a vendor’s trades are without getting mixed up with the vendor’s position sizing.

Dimitri

Dimitri, you’re ignoring the fact that the vendor’s I know who are the very best on the site scale up as their equity curves grow. Your statistic would not allow for it, and certainly not with standard deviation scaling talked about here.

"the vendor’s I know who are the very best on the site scale up as their equity curves grow"

Beau,

Those vendors are great examples of why a constant $ indicator (and especially a Standard Deviation adjusted one) is useful. It will take their systems’ records and treating all their trades equally across time will tell you if their systems still qualify as “best.” You got to be able to see that is useful.

Someone making $1000 trades in March and $2000 trades in April and $4000 in May as their equity goes up will in fact have totally different current stats depending on whether their best trades percent-wise came in March in April or in May.

A constant $ indicator will show you those systems under a different light: There won’t be any skewing by favoring trades in May (or in March if equity went down).

It’s a free gift when you can use the same data to get a different picture, take it!

Probably true.