"Realism Factor" vs "Adjusted by Realism Factor"

I would like to know how the "Realism Factor" (the first number appearing under Trading System Results) relates to the Equity curve when "Adjusted for Realism Factor" is checked.



The Realism Factor of my system seems to be in bad shape (57.4), but there is no difference between the standard (unadjusted) equity curve and the equity curve adjusted for realism factor. I would expect otherwise.



Thanks in advance for your support.

I haven’t received a response yet, so I’m politely asking my question again:



why isn’t there a difference between the standard (unadjusted) equity curve and the equity curve adjusted for realism factor, while the realism factor of my system appears to be in bad shape?

I saw what you mentioned the other day, and that was strange…

OK. It took me a while to figure this out.



Let me summarize the problem that Sciene Trader asked about. His system (Portfolio Picker) showed a Realism Factor (RF) of approximately 50%, but the equity graph that is adjusted by RF looked almost identical to the non-adjusted graph.



The reason for this was the following: Portfolio Picker has four closed positions, but about a dozen open positions. Those open positions have been open for a while.



The RF number (~50%) is based on both open and closed trades. But, up until this morning, the chart-generation software, which is responsible for generating RF-adjusted charts, used the RFs from only closed trades. In most systems, this discrepancy was hardly noticeable, but in cases like Science Trader’s, where the number of open positions is far larger than the closed trades, and those positions have been open for a long time, the effect is noticeable.



I have now modified the chart-generation code to factor in both open and closed positions. Portfolio Picker’s chart has been updated. The other 2,000 system charts will get updated over the next 24 hours or so.



Thanks for pointing this out. It was a rare problem, and tricky to find.

You might want to take another look. Realism Factor, with commish is now

better than "best case" w/o commish.

Nope, it’s correct.



If you have a losing track record, and a low RF, that means it’s not likely that you would have been able to enter all the trades C2 says you entered in a “best-case” world (think about just-touched limit orders). In these cases, your realism-factor-adjusted graph would look better than the non-RF-adjusted graph, where you got into 100% of all trades… including all the losers.

> In these cases, your realism-factor-adjusted graph would look better than the non-RF-adjusted graph, where you got into 100% of all trades… including all the losers.



So all winning systems have negative slippage and all losing systems have

positive slippage? That’s not my real world experience.

I assume that you multiply the daily changes in the graph with the RF/100 (or something like that). That would explain the effect that you describe. It would correspond to missing trades randomly. I doubt whether that is realistic.



My experience with extreme-os is more like: if the system if flat, I lose. If the system loses, I lose even more. In my case, it can’t be due to missing trades, because I didn’t miss trades lately. So it must be caused by slippage and commission. But shouldn’t that be part of the RF too?



So I would rather like a formula in which the RF affects losses in the opposite direction, i.e. makes them worse. A simple example of this would be to multiply positive changes in the equity by RF/100, and negative changes by 100/RF. Whether that’s good formula depends on the definition of the RF. This formula would leave flat curves flat, which still isn’t realistic imho.



Perhaps this would be an improvement: I assume that the RF is supposed to adjust the cumulative profit, in that Adjusted Cumulative Profit = (RF / 100) * (Displayed Cumu $ profit). The end point of the corrected curve should be $100K + Adjusted Cumulative Profit. The start point should be $100K. Now compute the adjusted curve as



Adjusted Equity(day i) = Equity(day i) - (1-RF/100)C(i/ndays)



where i starts counting on the first day of the system, C is the displayed (unadjusted) Cumu $ profit, and ndays is the number of days that the system is active. This adjusted curve goes through the start point and end point that I just described, but it is based on subtracting a constant from the daily changes rather than multiplying the daily changes by a constant. If the unadjusted equity curve is flat, then adjusted equity curve can be decreasing.



However, it would still have the (in my eyes inappropriate) property that, in a losing system, the adjusted curve lies above the unadjusted curve. You can change that if you replace C by te maximum of C in the formula; but what would justify that? It would be better to adjust the curve on a tradewise basis, where the effect of slippage, commission and the probability to miss the trade are computed for each trade separately and the aggregated into an adjusted curve.

"maximum of C"



I’m quite sure that I wrote absolute value signs, but they are deleted. I meant: maximum of the absolute value of C.

Hmm, thinking a little longer about it, I think you cannot have one formula that deals properly with all cases of a flat curve. A flat curve can be the result of ineffective trades, in which case the adjusted curve should decrease; or it can be caused by having no trades and no positions in that period, in which case the adjusted curve should be flat too. Even more reason to adjust the curve on a tradewise basis.

I should probably clarify how the "realism-adjusted" graphs are drawn.



(A) If the system is AutoTraded on C2, and we have collected a statistically significant set of real-life trade execution data, we adjust the graph solely by slippage data. That is to say, if C2 shows an entry price of 100, but most people trading the system actually execute the trade at 110 (unfavorable), we will adjust the equity graph accordingly for that particular trade.



(B) If the system is not autotraded, or we do not have enough slippage data, then we use the per-trade RF to adjust the graph.



Let me describe part (B) in more detail. While C2 does indeed assign an overall Realism Factor (RF) to a system, that RF score is actually a weighted average of per-trade Realism Factors. Each trade has its own RF. The software adjusts the equity graph by the probability of each component trade being made, at each point in time.



Does that make more sense?



> It would correspond to missing trades randomly. I doubt whether that is realistic.



Right. And it doesn’t account for slippage which in the long run is negative win, lose or draw. It’s been reported here that limit orders are always filled on losers and only sometimes filled on winners. This effect will apply to losing and winning systems as a negative.



Not to pick on Portfolio Picker, but it uses LMT and market orders. The

50% RF should NOT have a positive effect.



FWIW, 15 year old TradeStation technology 1.0 assumes slippage is negative and simply subtracts it from net. I’m not saying it should be as simple as that, but saying the ONLY significant impact of a low RF is missed trades doesn’t seem right.

Yes, that makes sense. It’s better than I assumed and I withdraw my comments. So you’re implying that in this system where the adjusted curve is above the unadjusted curve, some people actually missed some of losing trades. That would indeed justify the adjusted curve being above the unadjusted curve.



Jules

I would actually be interested to get some guidance on how to improve my realism factor. Is it possible to enter an order such that the realism factor stays at its maximum, while all “irrealistic” aspects of the order show up in the slippage? I mean, in practice you’ll always get filled with a market order but it may come at excessive slippage. Does this work the same way on C2?





All of this is complicated further by the use of settings in the order execution software (I use Tradebullet, TB) which are made to try and minimize the number of missed orders by either offsetting limit orders, converting limit orders to market or MIT orders, etc. To fold these into the RF calculation would require C2 to know the Tradebullet (or whatever is being used) settings for fill management, and I doubt if these are being considered at the moment since everyone likely has different configurations.



My actual “RF” on limit order systems has been about 0.7 for both winning and losing systems compared to the C2 P/L tables, over a dozen or more systems I have subscribed to in the past 10-11 months. That is, my profits are about 70% of what C2 shows and my losses are about 130% of what C2 shows, roughly, averaged across all the systems (including Extreme-OS). But I try to make sure I get every trade by adjusting the TB fill management options, so I do worse on both the winners and the losers because of slippage.



An example of the effect of these different execution software settings is in the brief discussion Jules and I had earlier about the limit order offset factor. He uses a negative offset for Extreme-OS to force better fills than the system uses (at the expense of only a 50% fill precentage prior to order adjustment), along with a TB setting to convert limit orders to market orders after X seconds to guarantee fills on all orders.



If I was trading this system now I would use a positive offset on the limit orders to try and achieve 100% fills with no further fill management settings in TB. This would give me worse fills some of the time (50% using Jules’ numbers) but maybe a better average overall as I would have fewer market order fills than I would with a negative offset.



Only a real trading account test over time would answer the question as to which settings are best. But the net effect on the C2 “best case” equity curve would likely be different for these two cases and my guess would be that fill management settings in autotrading accounts are not part of the RF calculations at the moment.

> Let me describe part (B) in more detail. While C2 does indeed assign an overall Realism Factor (RF) to a system, that RF score is actually a weighted average of per-trade Realism Factors. Each trade has its own RF. The software adjusts the equity graph by the probability of each component trade being made, at each point in time.



> Does that make more sense?



No. In this case Port. Picker is a long only stock system. It uses only GTC limit orders to go long. The ONLY trades that might not be filled are the long winners (so, IOW, LESS real profit). The losers would ALL be filled. How could the RF improve this system? FWIW, it improves it a lot.







Since I clearly can’t explain C2’s RF, I surely don’t know about that, but in real life using odd (very odd!) lots in thin markets will increase slippage.



You used: 217 in UUU, 132 in MIKR, 330 in EXM and 2037 in TGA…



Why? Why not 200, 100, 300, 2000?

Randy,



Interesting comments.



Does this mean you’re getting worse fills on limit orders if they are converted to market once the price is touched than if you set a positive offset? (which should get you a worse fill than the system vendor)



Francis

> My actual “RF” on limit order systems has been about 0.7 for both winning and losing systems compared to the C2 P/L tables, over a dozen or more systems I have subscribed to in the past 10-11 months. That is, my profits are about 70% of what C2 shows and my losses are about 130% of what C2 shows, roughly, averaged across all the systems.



Thank you…but I suspect some will prefer to leave real trading out of the “Realism Factor”.



Let’s not cloud the issue with facts.

TB reports back to C2 how each order was filled. For example, in the log file of friday June 23 2006, my log file contains among others these lines:



2006-06-23 21:04:12:984 (2006-06-23 19:04 UTC) Processing new C2 order: TB Order TB910534 72484488 PLACE BUY 26 ZION LIMIT 77.2 DAY, System: extreme-os , Signal: 21713555



2006-06-23 21:05:37:906 (2006-06-23 19:05 UTC) Order was reported filled by Collective2, starting timer to convert into Market in 10 seconds (id =‘0ez3801:054k’ state=Working symbol=‘ZION’ action=Buy limit=77.1 stop=0 quantity=26 type=Limit tif=Day oca=’’ filled=0 price=0 token=‘TB910534’)

2006-06-23 21:05:47:750 (2006-06-23 19:05 UTC) Converting Limit order to a Market order: (id =‘0ez3801:054k’ state=Working symbol=‘ZION’ action=Buy limit=77.1 stop=0 quantity=26 type=Limit tif=Day oca=’’ filled=0 price=0 token=‘TB910534’)’



2006-06-23 21:05:58:796 (2006-06-23 19:05 UTC) Confirming 1 executions to C2: 21713555sep26sepTM_20060623_0009sep77.15

2006-06-23 21:05:59:734 (2006-06-23 19:05 UTC) Got the C2 fill acknowledgement for a total of 26 units for Signal # 21713555



So what you see is that C2 ordered a BTO @ limit 77.2, this was sent to MBT as BTO @ limit 77.1, and after 85 secs C2 reported a fill, my order was converted to a market order, and filled at 77.15. Then this fill was reported back to C2 as a fill with the orginal order id. It seems that my fill management is not reported, at least there no sign of that in the log file.



I suppose that C2 uses these reported fills to compute slippage and the RF. That means that it is a kind of average across the different settings that autotraders use.



From discussions on the forum it is my impression that market orders will always get a RF of 100. So that would be one way to get a high RF, but as you say, at the expense of slippage. Another possibility seems to be to send limit orders and urge your subscribers to autotrade with this conversion (convert limit to market if C2 reports a fill) on in Tradebullet. You would still have slippage, but perhaps a little less than with direct market orders. I don’t know how that would work in the other autotrade programs.



Jules