You wrote:
I’m not sure that I follow your statistical logic. If I studied only strategies that failed or had major problems, then I would have no meaningful variation in the dependent (outcome) variable: collapse. One needs to compare the win rates of strategies that collapsed with the win rates of those that didn’t. That’s how stats are done.
And I found that there was a relationship:
As for my selection of strategies, I selected the top 41 strategies according to C2 score at the time because they were a great selection of the most popular and most profitable strategies on C2 at the time. Any people who have been around at least a year can look at the list I used (click on the link in my earlier post) and see that these were a good representation of the most popular and profitable strategies. Back in Jan. & Feb. the C2 score was computed very differently than now and almost all of the favored strategies had very high c2 scores. And in early Feb. C2 scores were in effect frozen for a while so these 41 were the strategies that were highest rated BEFORE the mini-crash. So it was a good sample of the top strategies here.
You write:
Again, I don’t follow. Very few statistical methods require identical sample sizes (some “matching” techniques do).
I think that the best way to analyze these data is to use a Fisher’s Exact test (which is ideal for small sample sizes and does not require normally distributed data). Using Fisher’s Exact test, the significance of my main analysis is .003. In other words, a relationship as strong as the one I reported would happen randomly only 3 times in a thousand, way below the 5% threshold (50 times in a thousand). Contrary to your claim, my results are statistically significant–indeed, highly significant for such small sample sizes.
BTW, my B.A. (Yale) and my Ph.D. (U. of Chicago) are both in the quantitative social sciences, and I have taught or lectured on statistical research at Harvard, Yale, Columbia, Stanford, Berkeley, Northwestern, Chicago, Duke, Texas, Virginia, and most other top US universities. Of course, that doesn’t make me necessarily right on this.
And remember, I was only reporting the probability of serious problems or failure during one particular market downdraft, not the certainty of that happening. If the next market correction is very different, then my results would not apply or would not apply as strongly. Further, in my study there was one strategy that was a big exception, JUST FOREX TRADES. That strategy did not suffer in Feb, despite having a win % and a WIN / LOSS RATIO similar to Bread and Butter today.
As I wrote, past performance is no guarantee of future performance. Since you discount my analysis, and predicting the future is difficult (and is usually impossible without simply assuming that the future is much like the past), go ahead and invest with Bread and Butter. The results so far are indeed terrific.
Maybe if you invest in Bread & Butter, you’ll get lucky.
