Matthew,
Congratulations on the new look and functionality of the site! However, as I reported a while ago on this forum as a reaction to your request to look at the beta site, using the ‘personalized’ settings gives non-sensical results for my system Kauai. I think this functionality needs to be disabled until it has been fixed. If a potential subscriber applies his or her personal settings, it looks like my system gives horrendous results (i.e. a 90% negative return), which is confusing and misleading.
The “personalized” chart, which allows a user to see how a system would perform under various scenarios (for example, with user-defined commission costs included, or with user-defined starting capital amounts, or with subscription costs included) is an advanced feature. It involves a lot of on-the-fly estimates and dynamic calculations, and clearly it’s not perfect. In some small number of cases, it’s obviously grossly wrong, and of course we want to fix those cases as soon as possible.
While it isn’t realistic to turn off the feature entirely (for the majority of systems, it works fine), I can assure you that we’ll work on fixing the incorrect cases as soon as we can.
I understand why this is important, and agree with you that it needs to be high on the priority list.
Matthew
Matthew,
I find it already quite a challenge to convince the C2 crowd to subscribe to Kauai, but with this new bizarre (and wrong) "personalised" equity curve I am giving up all hope 
However, I am happy to hear you agree on the problem and are planning to fix the incorrect cases as soon as you can. I hope this is more a matter of days than weeks.
I agree with ST. Even if I know that the personalize curve might be wrong, it is very discouraging. One reason for this is that I am afraid that it may actually be correct. IMHO this is the first thing that should be improved on the new site. See also my post in the thread about the beta site.
Matthew,
It seems you haven’t had time yet to fix this problem. May I kindly bring it to your attention once more?
Kind regards,
ST
One odd thing I noticed with the personalized results option was that, with at least one strategy, it seemed to shift the results in time on the grid that shows percentage profit by month.
By that I mean that the strategy looked as if it started a month later, and months that originally had a big move (positive or negative) would often turn into small moves, while the next month along would go from a small to big move.
Dear Matthew,
Three months ago I brought a problem to your attention regarding the personalized chart. You agreed that the results obtained through this feature were grossly wrong and that a fix needs to be high on your priority list. Since this has not yet been undertaken, I want to re-iterature my view that features causing misleading comparisons between systems need to be disabled, until they make a fair comparison between systems possible.
In the ToS you mention you "try [y]our best to make ratings, scores, and reviews as accurate and useful as possible". I think everybody here welcomes new features to the site, as long as you stick to that promise. In my view disabling the personalized charts feature (at the very least for those systems where the results are known to be wrong) leads to more accurate comparative information.
Kind regards,
Science Trader
Hi, Science:
The way the personalized charts work is that the personal “what-would-things-look-like-if…” chart adjustments are made on a closed-trade by closed-trade basis. So systems that open and close trades frequently will have very accurate personalized chart adjustments. Systems that buy and hold for longer periods will see unintuitive and weird-seeming step-like adjustments to the chart.
Thus the feature isn’t appropriate to use for all systems. That doesn’t mean it is useless, or that it should be removed from the site, or that it is full of bugs. It’s a mathematical approximation that works well for some cases, and not others. For many systems, it provides a very useful tool.
Perhaps the best solution is that we can display a warning or even remove access to the chart personalization for systems that have long multi-week trade-holding periods – that is, systems where the personalization approximations will produce less-accurate results.
As far as when we can get to this development effort: One of the great things about working at a small company with close customer contact is that customers tell you exactly what they want to see, and how soon they want to see it. And one of the difficult things about working at a small company with close customer contact is that customers tell you exactly what they want to see, and how soon they want to see it.
I totally understand why this development effort is really important to developers of systems with longer holding periods (my personal favorite type of system, incidentally), and we will try to get to it as soon as we can. But there is a lot to do, and we’re working as hard as we can. Please hang in there.
Matthew
Hi Matthew,
Thanks for your quick reply! I don’t think it’s a problem if the personalized charts for some longer-term systems lead to a less precise equity curve (i.e. larger steps), as long as the final (last day) equity is accurate, i.e. that the total profit over the entire trading history is correct (after factoring in the personalized settings).
But this is where it goes wrong for Kauai: The personalized chart shows an equity curve going from $5000 to less than $1000 (with zero transaction cost), while the standard curve goes from $5000 to $7500. Thus, my concern is really about the accuracy of the final equity, rather than the granularity of the curve.
Interestingly, getting (at least) the final equity right (on the personalized chart) seems to be possible for some other longer-term systems. For example “Rebound Select” has an average trade duration of 76 days, and indeed a “personalized” equity curve that has more ‘steps’ than the standard curve. However, for this system the final equity on the personalized chart does match the final equity on the standard chart reasonably well.
My preferences in terms of a solution would be (starting with most preferred):
1. fix the problem entirely
2. make sure at least the final equity is accurate on the personalized chart (ignoring the intermittent ‘steps’ for now)
3. disabling the personalized feature for all systems where results are wrong
4. leaving everything as is but adding disclaimer/warning that personalized charts might show erroneous curves for certain systems with longer holding periods
Many thanks,
ST
