The opinions expressed in these forums do not represent those of C2, and any discussion of profit/loss is not indicative of future performance or success. There is a substantial risk of loss in trading. You should therefore carefully consider whether such trading is suitable for you in light of your financial condition. You should read, understand, and consider the Risk Disclosure Statement that is provided by your broker before you consider trading. Most people who trade lose money.
I think the random sampling gives a view of the possibilities, because the conditions that would trigger the system to make certain trades could happen in a different order and frequency in the future. So while the system would make the same trades for the same data, the order of trades could be completely different and possibly lead to a higher max drawdown.
Still for now, the return / risk ratio is very good.
Thanks,
However, the monte carlo method still doesnât take into account particularities of a system. For example, a system may be âtrainedâ to give more weight to forecasted volatile days vs. forecasted less volatile days. A simulation would not be able to take this level of detail into account when it runs its millions of trades. The simulation would assign trades at random where they would not apply in real life.
Such a simulation should also then, in theory, be able to tell you a systems best future performance, etc. etc. (Who needs backtestsâŚjust run a monte carlo on it. 
)
These simulations, imo, are not accurate for every type of system. And Iâm not sure how accurate they are period; as Iâve never seen a study done on them.
Given enough simulations (aka: curve fitting/optimizing), and tweaking, you can find/generate most any statistic youâre looking for.
Regardless, one ought not rely on it anyway, imo. Protective measures would still be used in any case. So I donât see much use in themâŚbut thatâs just me.
If trades have a correlation between them then Monte Carlo wonât be as accurate, but if trades are uncorrelated then simulations will be quite accurate presuming you have an accurate distribution. The distribution already contains the effects of system volatility scaling (for example) as well as other âparticularitiesâ of the system. Simulations on the distribution will contain these particularities implicitly because they are represented in the real results that generated the distribution. This is similar to how all market news is reflected in the price action of the market and a system can work simply by watching price rather than having to follow and understand news.
And you are correct, these simulations can tell you the likely range of system performance, likely drawdown ranges, expected âlullâ periods without new highs, etc. I even prefer using MC simulated numbers over backtest numbers because they are based on live results and donât suffer from the potential overfitting problems of backtested numbers. Backtests give you results for specific historical market periodsâthey are useful in many ways but have significant problems (e.g. overfitting, limited data points). MC simulations give you statistical ranges for what to expect independent of specific past markets (presuming the system holds to the distribution), have infinite data points, donât suffer from the âoptimismâ of overfit backtests, and are useful in many ways as an additional tool despite their limitations. You generally want to see an agreement between MC simulations and your backtestsâif you donât, something is off (e.g. your backtests might be overly curve fitted). The backtests of my system Drunk Uncle and the MC simulations on my systemâs live data from C2 generally match for both max DD and return, and that gives me confidence in my estimations for these numbers.
Your final comment about curve fitting/optimization on MC simulations makes no sense. There is nothing that gets curve fit, optimized, or âtweakedâ in a MC simulation. MC simulations are objectiveâI can run them on any system in C2 and get objective, repeatable answers (with limitations of course). Itâs backtesting that has problems with curvefitting/optimization.
Perhaps this can explain it better than I can:
Excerpts for the underlying article:
The problem is that the typical assumption set used in Monte Carlo simulation assumes normal distributions and correlation coefficients of zero, neither of which are typical in the world of financial markets.
The problem is the confusion of risk with uncertainty. Risk assumes knowledge of the distribution of future outcomes (i.e., the input to the Monte Carlo simulation). Uncertainty or ambiguity describes a world (our world) in which the shape and location of the distribution is open to question. Contrary to academic orthodoxy, the distribution of U.S. stock market returns is far from normal.
The probability results from Monte Carlo simulation may look impressive to a client. However, if that number is derived from assumptions that are not realistic, there is no value to the number. It does provide a good excuse: âWell, the Monte Carlo model did tell us there was a 15 percent chance of this happening.â
In the end, Monte Carlo simulation seems to clash with the continuing development of a holistic approach that allows changes in the clientâs investment allocation over time, with the corresponding changes of anticipated rate of return over the same time periods. Modern software can and should incorporate greater flexibility within the financial planning model, making random rate-of-return simulations even less relevant.
While the profession quietly questions Monte Carlo simulation, the benefits are being loudly proclaimed by the software industry as the hottest new innovation in financial planning in decades. Marketing hype touts this new information as one more item of value in a clientâs financial plan, while opponents say that when we extend client expectations out 20 or 30 years, identifying factors such as lifestyle expenditures, tax rates, inflation and investment preference based on risk may be more important. Chaos theory teaches us how small errors in the early years of a financial plan can make dramatic consequences when compounded over a long period of time, so letâs not make our real problems any bigger than they are. Some âwhat ifâ scenarios are necessary, but letâs do it right and do it often. The real answer is to make the plan as representative as possible under the circumstances and to update it regularly. The benefit that Monte Carlo simulation promises to provide might be better achieved by using common sense in the financial planning process.
Real world results are worse than monte carlo because of emotional trading during drawdowns .
A couple repliesâŚ
Most importantly Iâm well aware of the limitations of MC but I find it useful despite these limitations. Having a supporting (or contrasting) estimate on return/DD as an additional indicator to backtesting or C2 stats is a lot better than not having it. And I observe the value of MC as my MC estimates usually correlate quite well with C2 stats on mature systems that have a lot of data. Accuracy drops with younger systems as there is not as much live data to build a representative distribution, however itâs still better than âflying blind.â
As for the excerpt you posted, notice Iâm not using MC to create a general financial plan, Iâm evaluating trading system performance (again, getting that 2nd opinion over backtest data or C2 stats alone). Iâm also not presuming the market is normally distributed. Itâs not the market returns that are being simulated, itâs trading system returns. Trading system returns might not be normally distributed either (most are skewed or otherwise NOT normal), but Iâm not presuming normal distribution, Iâm using a distribution created from actual live trading results. Of course a system could be changed by its developer or otherwise not have returns that make a representative distribution (especially true for young systems without much data), but again I find it more useful than not having it. There also doesnât need to be a presumption about the correlation of trading system returns as itâs trivial to actually check their auto-correlation with a simple spreadsheet call.
I get that youâre not convinced of the value of MC simulation but is it just because it reports something you donât agree with? What is your estimation of the DD for your system over the longer term based on backtests or whatever youâre using? I have my estimation and I have my basis for that estimation (and it will improve as your system generates more data). I trust my live-data-based estimations more than developer backtestsâI know how easily backtests are optimistic and over-fit.
I agree that emotions may adversely affect certain discretionary traders with regard to their trading.
However, those using a âsystemâ are likely less affected.
And those using an algo are even less affected.
Seasoned traders should have realized that drawdowns happen. No one trader or system will be correct everyday, of course.
There is no way to verify a backtest; yet, everyone wants to see one. There is also no way to verify a MC simulation. Both must really just âwait and see.â
Whether one has backtests, MC simulations, 20 years of data; the bottom line is that a smart trader will useâor should useâtheir own methods of protection. A trader doesnât have to follow every signal from an unknown, but profitable-so-far system. (I could speak more on that.)
Iâm happy if someone can rely upon what they see so far, combined with their personal proper money management, and trade using a so-far well-performing system.
And Iâm also happy if someone wants to watch me perform well for 20 years (to get more data to analyze) before even considering following my system.
I appreciate your confidence and I donât like being âthat guy,â but there is no way that system will limit to 28% DD. That is a near mathematical certainty. The instruments you are trading are extremely leveraged, and your trades have variation even as they make moneyâjust from the normal volatility of the trades you are making you will eventually double that DD even presuming your profitability remains the same. These are mathematical realities. You have built a great loaded dice and youâre rolling mostly big wins with it, but regardless there are mathematically predictable drawdowns you will eventually suffer simply due to chance. These are what MC simulations express.
The market will be a better teacher than I however. Perhaps after you hit a 40% or 50% DD youâll be more open to furthering discussion. Iâd wish you luck, but I think this goes beyond luck. If you roll a pair of dice enough times youâll eventually hit snake eyes.
Ok, weâre back to where we started. You say you know what my drawdown will be going forward based of your random simulations or âmath.â
I say you donât know my system.
Iâve already indicated I now use stop-losses and profit targets where I havenât in the past. I also donât use 100% allocation like I did in the past. You seem to admit I can forecast with âloaded diceâ like accuracy. Yet, none of the above has swayed your guess.
Nevertheless, you insist my dd will be closer to 50% than 28%. I say it will be closer to 28% than 50%. Weâll just have to wait and see and agree to disagree, no?
Iâll preface this question by deferring to the knowledge & expertise of DavidStephens & MachineLearningTradr, and to say I do not mean this to be antagonisticâŚ
Why have you changed your methods to include these two parameters?
Good question.
Well, âchangedâ methods is a strong phrase. I wouldnât call it that; but I know what you mean.
I had a few followers prior to taking subscribers here on C2. Since day one, one particular follower suggested stops would help performance. I indicated maybe, and that Iâll get around to looking into it. I wanted algo controlled or derived stops, so it would require some coding and time to implement. So, the priority was improving the algo first; stops, etc. later.
So, with the algo improved, and other tests and experiments out of the way, I put the stops in. I put the targets in since I was under the hood anyway, it wouldnât be that much more effort to add them as well. Also, I would curious to see how both would affect performance. The algo could choose to not use them if it determines they donât help performance. But in fact, they do, and it does.
I am currently still doing experiments with a dynamic position sizing feature. It should be ready tonight or tomorrow.
Admittedly, one or two of my subscribers, and just being responsible for the well-being of othersâ money has torqued my brain more towards the importance of risk counter-measures than it was when I only traded my own money.
Just as a note, changes to a system will usually modify the systemâs return distribution and invalidate data from prevoius MC simulations based on a now non-representative distribution.
Iâm not a fan of changing a live system, but thatâs another topic.
You shouldnât change a system because of subscribers ⌠it implies that the system wasnt working in the first place or you are not sure it will work well .
Right. Thatâs why I thought it was strange that you stood by your simulations even after I first mentioned that I added stop losses, etc.
I understand some feel that way. But, Iâve gotten this far by constantly improving my system. If the change doesnât improve it; itâs reverted back.
Or, as in this case, the change is merely an ability for the algo to change something on its own. It wonât make the change unless it determines an improvement will take place.
I donât believe I said that.
What if the subscriber had meaningful recommendations for improvement of the system?
Edit: And/or, had suggestions that may make the system more appealing to other potential subscribers?
You take notes and feedback yes but you dont change your system because of a subscriber , doesnât make sense , what if you had 100 subscribers or maybe the subscriber should start a system himself .
IMO the problem with changes to a system is the changes can take a functioning system and introduce subtle curve-fitting issues. To make a change (even a minor change) there is usually optimization done against historical data and thatâs a difficult process to get right. Just getting a system to function correctly going forward a single time is incredibly challengingâeach further change is a risk to the forward functioning of the system. There are exceptions but generally my view is that after a change to a system the forward functioning of the system needs to be proven again with months of live results. A good way to do it is to group several changes and then start a new âversion 2â of the system with the changes. Users can decide to switch to the new system or stick with the old one until the new one proves itself out.
When you look at how few systems last more than a few years it makes you wonder. I have no doubt âchangesâ are at least partially responsible for quite a number of systems that stopped working.
I donât think a general statement like that can responsibly be made without evidence that it applies so broadly.
Even worse than MC simulations; it brushes every developer that ever improves the system that they built from nothing with a broad, unsubstantiated, opinion.
I respect that this may have been your result; but I donât think you have grounds to make such a general statement. It certainly hasnât been my result.
But once again, like beforeâŚweâll see.
