How to evaluate a trading strategy in the right way. A lot of experience in one topic

[Daniil]

I intend this post to share a more reasonable view on the investment process.
I’m a trader of Wait4Trade team, and I’m going to write for the whole team.
We’ve got rich experience in trading strategies development, and we would like to share it.
We want to tell about the way strategies generate profit, what indicators you should pay attention to, how to evaluate the future perspective, etc.

The need in this post arose after reading the strategies discussion on the C2 forum, as well as facing the situation with subscriptions to various trade leaders.
The objects of attention of investors are not the strategies that deserve it.
Often, investors face strategies that are not worth any attention.

How to evaluate the strategy.

To evaluate the strategy, you’ll need the following data:

1. Strategy description from the trade leader and communication capability (a good trader is not a “hikikomori” or a mad isolated scientist).
2. The maximum and minimum lot for each instrument.
3. Strategy Age and Number of trades.
4. Winning trade percentage.
5. Trades duration (opening and closing time).
6. Maximum drawdown.
7. Rate of return.

1. Strategy description from the trade leader and communication capability.

It is important to have instruments to compare the actual trading process with expected. You should make sure that there is a system and a trader adheres to it.
Don’t you ever think that for investor there are strategies too difficult to understand, and details which are better not to get into.
Quite the contrary: strategies are primitive in their simplicity, and investor must have the understanding of what a trader is doing and whether he isn’t gone mad.

2. The maximum and minimum lot for each instrument.

Everyone raises the lot. All Ladies Do It. Those who do not - deserve the closest attention. They are either geniuses and have pure Insight on the market, or they fell into fatal luck.

We need this max and min lot value in order to determine the step size and the number of steps to increase the lot.

Having 3-5 steps in the order: 1, 2, 3, 4, 5 lots; or 5-10-15-20-25 lots; i. e. +1 starting lot, is a normal, reasonable approach to profit taking from a strategy that gives 0 ticks (this is a good strategy).
But when you see that starting lot is 1, and the highest is 50-300, then this is just an uncontrolled martingale which will lead to one losing trade for the entire deposit.

Of course, everyone heard about the martingale, but in fact nobody uses the classic martingale. Instead, averaging into a position martingale is in use.

To get over 90% of profitable trades, you just need to stop placing a stop-loss order and set a short target. But there will definitely be a situation when the price goes against the position, and in order to quickly bring it into profit, averaging is used with a simultaneous increase in the lot. For example:
Lot 1 of crude oil was bought at the price of 60, the target at 61, i.e. $1000 or 100 ticks per lot.
Price dropped to 55:
The first position in lot of 1 has -500 ticks, or -$5000. In order to hold a target of 100 ticks and make a profit of $1000 (to get a profit of $1000 at a price of 56), another 5 lots of crude oil are bought, and that is 6 in total.
Price dropped to 50:
The first position in lot of 1 has -1000 ticks, or -$10000
The second position in lot of 5 has -500 ticks, or -$25000
Total: -$35000
In order to hold a target of 100 ticks and make a profit of $1000 (to get a profit of $1000 at a price of 51) up to 36 lots are needed, which means buy 30 more.
And so forth.
As a result, the profit chart will look like this:

3. Strategy Age and Number of trades.

The minimum number of trades is no less than 100 for a period of at least 2 months.
Basic statistics require a minimum 100 trades. In our outcome evaluation of algorithmic strategies, we started with 1000 trades, and the more experiments are made the more accurate data and more unbiased evaluation obtained.
Without a description of the strategy essence, 100 trades are still not enough, since there is no way to compare the data obtained with the main concept. (see Section 1)

4. Winning trade percentage.

This is one of the most important indicators; It reflects the trading methodology.
When selecting strategies, we recommend looking for <= 60% instead of >=.
The fact of profitable trades indicator is less than 60% shows that the trade leader uses stop-losses, and he probably has an understanding of where his forecast stops working (but that is not necessarily right).

Most long-term strategies (10 years and more than 1000 trades) bring a profit of 0 before commissions, the best bring a profit of 0 after commission, regardless of the percentage of profitable trades.
The percentage of profitable trades primarily indicates risk management.

To get a high percentage of profitable trades (>80%) is just needed to place the stop-loss to target ratio as 10 to 1.

Of course, it is better to remove the stop-loss at all

5. Trades duration (opening and closing time).

This parameter helps to understand the stop-loss limit, if any.
It will require manual work with Excel: finding the longest trades and determine how much the market has gone against this position using data on entry and DD Worst prices.
Compare that with profitable trades and get an idea on the stop-loss and target ratio.

Stops? No - I have not heard.

6. Maximum drawdown.

This is just a historical fact, and in isolation from understanding the risk and lot management means absolutely nothing.
For example, strategies with martingale averaging can show exceptionally good results, but with drawdowns, your loss will grow exponentially.

It is worthwhile to avoid strategies with a high indicator of maximum drawdown, but don’t expect that your maximum risk is limited only to this indicator.
Even for automatic strategies, real drawdowns are always higher than historical ones. “PAST RESULTS ARE NOT NECESSARILY INDICATIVE OF FUTURE RESULTS.” - you remember, yeah?

7. Rate of return.

Rate of return of the strategy can be estimated in two ways:

• Money Profit
• Tick Profit

It may seem all the same. If there is a profit in money, then there is a profit in ticks too. But this is not so.
It’s a great thing when the strategy shows a positive number of ticks, but our backtest experience on the history of 10 years shows that all strategies tend to 0, the only difference is how this happens: evenly or as a result of a large unprofitable cycle.

When the tick profit looks like a short sinusoid around 0, then you’ve got a good strategy, that, with a reasonable increase in the lot (+1 starting lot, up to 5 steps of increase), will give a stable 1-3% profit per month.
But when the sinusoid is extended, long cycles take place, and that in turn means that months of profit turn into months of losses and such a strategy can’t be considered as stable.

Money/percent return, as well as the maximum drawdown, in isolation from understanding the principle of the trading approach means nothing.
Only after selecting strategies using lot management model and risk management, you should pay attention to rate of return as the final choice factor.

Examples of bad, good, and excellent strategies via sinusoid of tick profit.

Bad - expanded sinusoid.

Good - short sinusoid around 0.

Excellent - rising short sinusoid.

We want to contribute to the development of Collective2 service in particular and trading in general, and encourage good strategies creation and their fair evaluation by the investor community.

If you’re interested, in this topic we all together can gather and analyze any Collective2 strategies.

[PhilD1]

That’s an interesting way to look at trading. The way I understand it, it sounds like you believe that all strategies have zero edge and $ profitability comes from position sizing? Is that the gist of your post?

In section 2 (lot sizing), are you using that trade example as a positive or a negative example? Taking -$35k of drawdown in order to realize $1k of profit is really poor trading.

[AlgoSystems]

For me statistical analysis of a trading system should not be taken too seriously. We always take a historical sample to give us these numbers but this does not tell us what will happen going forwards.

For example the sample could change dramatically going forwards and we did not take into consideration many factors such as volatility , trader psychology, liquidity, etc.

In the end, trading is difficult and requires complete understanding of the strategy weakness and strengths so as to avoid the times where the strategy is weakest and not trade during those times.

I would think buy on value and other income strategies would have more longer term success. However, trading can offer short term rewards that far exceed buy on value.

The only consistent winner is the brokerage houses that make money from our commissions…lol and the institutions that trade with huge volume.

However, kudos to your analysis and makes good reading.

[SmartTraderMS]

Indeed interesting read Daniil and great piece of knowledge in one place.
Are you trading manually or with algos?

Truth is that I have developed multiple algo strategies with excellent 5-10 years track record on multiple instruments, however only 15+ years of diligent backtesting in different market conditions with 1000+ trades can separate the grain from the chaff.

On top of what you said we are applying one additional level of analysis - sensitivity of key statistics (rate of return, max DD, sharpe ratio) to changes of key strategy parameters.

I have described it shortly here in Multi Timeframe Strategy thread - perhaps you will find it useful in your analysis.
https://forums.collective2.com/t/multi-timeframe-forex-strategy-details-and-results/13167/11

MS

[Daniil]

The gist of my post is to present a trading strategy evaluation method based on 10 years algos experience.
All examples show how easy it is to get good results for popular indicators. Therefore, we can say that they are negative.
Section 2 shows the strategy results with random entries. I wanted to demonstrate that it doesn’t matter where to open trade if you use averaging with a martingale.
For the remaining examples, I used a strategy with two simple averages; entry when they cross.

About zero profit.
This is not a matter of faith; It is an experience confirmed somewhere 500 times by various strategies.
Collective2 is just that rare place where you can put aside speculation and faith and analyze the actual results of strategies.
Let’s analyze any futures or forex trading strategies together, with at least 500 trades and confirm or refute sections of my post.

[Daniil]

You’re right.
PAST RESULTS ARE NOT NECESSARILY INDICATIVE OF FUTURE RESULTS.
But all investors want to get a little confidence.
I have long wanted to share my experience, but I could not find a suitable audience. I think it’s a right place :).
I don’t have any experience in stock trading, all I posted concern only Futures and Forex trading strategies. 
Of course trading is difficult and the most difficult in trading is to find helpful information and evade deliberate and unintentional delusions.

[Daniil]

My trading is manually and it’s based on five basic principles:

1. Improving trader using pre-processed market data, not replacing him.
2. Apply 4 step position lot raising to shorten DD period and increase profit.
3. Portfolio of instruments to raise good trades probability.
4. Intraday trading to get low volatility equity results.
5. Using stop orders to eliminate market surprises.

I looked your post, good work.
But you do not operate tick profit in your presentation.
When I say about zero profit, I mean zero profit in ticks.
Evaluation entry trading point performance requires becktest with one lot entries, one entry to one side.

[PhilD1]

Can you give a little more detail and walk us through an example of how a system with a good 0 tick profit might be managed so as to create a positive $ profit over time (you mentioned 1-3% per month, as an example).

Does it come down to increasing position sizing as cumulative tick profit grows and decreasing it as cumulative tick profit falls?

[Daniil]

Does it come down to increasing position sizing as cumulative tick profit grows and decreasing it as cumulative tick profit falls?

Opposite, need to increase lot while equity is in drawdown and reset to starting lot when high peak equity updated (strategy made a new high in $).

You can download this example here.

I apply this method to my strategy (x-axis is Days):

Wait4Trade Day Trading

[PhilD1]

I looked at the example in the XLS. To confirm, every time cumulative tick profit drops, you add a lot on the next day’s trade, up to a maximum of 5 lots. Conversely, every time cumulative tick profit rises, you reset back to 1 lot on the following day. edit: I re-read your post and see it’s reset to 1 at every highwater mark.

It looks like you generated the tick profits using a random sequence of some sort?

This looks like a type of Martingaling, but instead of applying it to individual trades you apply it to the strategy as a whole. The key to success here is knowing that the strategy will have roughly stable cycles of gains and drawdowns. If the up and down tick profit cycles vary too much in duration then eventually you could have an extended $ drawdown that ruins the account.

[Daniil]

Philip, it’s nice to discuss with you, thanks.

Everything is so: random data in example; key of success are stable cycles of gains and drawdowns.
But drawdown cycles with too much in duration can ruin account without lot raising at all. It’s a question of leverage level.
I know three ways to get stable tick profit cycles:

1. Portfolio of instruments - raise good trades probability.
2. Good entry strategy - raises percentage of profitable trades on 2-5% compared with random entries.
3. Short stops - exclude dramatic losses and afford stable drawdown cycle.

I do not defend my trading strategy in this thread, I only want to say that the profit in the systems is taken according to a certain logic, and the quality of trades entering is only a small part of it.
I am convinced that we can take any strategy from collective2 and analyze it according to the criteria described in this thread and understand how it works.

[PhilD1]

Ok, here’s one I’d like you to analyze:

The Spirit of Nicolas Darvas.

He’s had 6 straight positive years. What’s your opinion of this one?

edit: he uses Turtle Trading position sizing, which means he adds volume to trades with open profits, which is the opposite of your method.

[Daniil]

I have more competencies in Forex and Futures, all my examples based on these markets, but I’ll try to analyze this strategy.

The stock market differs from others by rising trending.
So first at all we need to check a balance for entry side.

This strategy has 78.89% of long trades, so we can argue that it is based for stock rising trending.
Is it good or not? When the crisis comes, it can ruin your account. In the meantime, everything is fine.

1. Strategy description from the trade leader and communication capability.

Description is present. Let select data to be verified.

The systems buys strength, short sells weakness and cuts losses very quickly.

Do you use leverage?
Rarely, but yes during strongly trending markets I do to a limited extent.

Do you use stops?
No, but positions are sold if they close below a pre-determined level the next day.

2. The maximum and minimum lot for each instrument.

Stock prices vary widely.
Therefore, I lead all trades to a common denominator as position volume in $. [Qty Open] x [Avg Price Open ]
It helps us to get something like Lot.

This table contains all trades in the form of a histogram in position volume ranges.
As we see, the strategy uses lot rising.
Only 30.69% trades have lot increased, but with ratio up to 1 to 15 from a starting position volume.

3. Strategy Age and Number of trades.

Excellent! We do not need more data to make a conclusion.

4. Winning trade percentage.

Winning trade percentage is 36.2%.
Author told us that stops do not use.
Do you use stops?
No, but positions are sold if they close below a pre-determined level the next day

This graph confirms it

When the stops not using, the low percentage of profitable trades tells us that the trader actively manages the position, manually closing losing trades.

The systems buys strength, short sells weakness and cuts losses very quickly.
Confirmed.

5. Trades duration (opening and closing time).

This section calculates the risk per position, we already know that it is 100% for long trades and infinitely large for short trades.

6. Maximum drawdown.

Consider Long position orientation and non-stops trading, there is a significant risk of a complete loss of account during a crisis or a very strong correction.

7. Rate of return.

Zero tick profit with two dramatic trades.
Profit gains with lot raising and its normal.
Sinusoid is expanded, so so.
The average annual return is 30.9% and I think it is too small for these risks (we can lose all).

All calculations are here

The strategy states that Suggested Minimum Capital $35,000.
Look at position volume. It can be easy more then 100k.

[VG2]

Thank you for doing this, Daniil, it is so interesting to see your methodology applied to a stock trading system.

[PhilD1]

Interesting analysis. My only point of criticism is on section 7, Rate of Return.

You use the cumulative tick value, but I don’t think this is appropriate for stocks. The reason is that a 200 tick move ($2/share) on a $300 stock is very different qualitatively and quantitatively from a 200 tick move on a $5 stock. I think a better way to do this is to use cumulative percentage return, so as to normalize for the differing price levels of different stocks.

When I do this, this is what I get:

which looks like a rising expanded sinusoid, according to the terminology you use.

By the way, the two dramatic trades in your last graph are a loss on UVXY in 2013 and a gain on GBTC in 2017. The UVXY trade shows entry and exit prices in the thousands of dollars, which then gives a huge tick gain/loss value. It isn’t real - the price data has been back adjusted to account for reverse splits, of which UVXY has had 9 since 2012. Another reason to be wary of using cumulative tick value with stocks.

[Daniil]

I also did not really like the 7th section.
I am not aware of stock trading nuances; It was my first stock strategy analysis.
I was thinking about your comment… You’re right, tick calculation do not appropriate for stocks because shares volume can differ by an order with the same equity volume due to different cost.
About UVXY very interesting! How does it work?

Thank you all for feedback!

[PhilD1]

About UVXY very interesting! How does it work?

UVXY Split History

@Daniil

I am not aware of stock trading nuances; It was my first stock strategy analysis.

I think the key thing to keep in mind is that as soon as a system trades more than one instrument or stock in its portfolio, you can’t use cumulative tick for the portfolio any more because ‘tick’ has differing values for various instruments and stocks. My suggestion is to use cumulative return %, because this normalizes for the overall base price level.

[Daniil]

I agree, now in the analysis of section 7. Rate of return
I will use cumulative percentage return; It is universal and will allow us to evaluate the return of any portfolio.

Thanks a lot, Philip!

[AlgoSystems]

@Daniil, what does your backtested results show for your system? Is it so far matching the backtested results? I’m not being critical but just curious if the backtesting matches the forward testing.

[Daniil]

I don’t quite understand what system are you asking.
In this thread, I review the analysis method and apply it to strategies from collective2.
In my own strategy I use intraday, 4 step lot rising and a portfolio of twelve instruments, now it’ll be 24 (current drawdown shows me that strategy do not stable yet).
The entry strategy is discretional based on pre-processed market data with ML technology.
So I can’t backtest it yet, to do this right way I need to collect not less than 3k profitable trades and after that our team could train a neural network.