This morning's slow server

This morning, C2’s Hypothetical Fill Engine (HFE) and quote feeds slowed down to a crawl under heavy loads. Quite a few systems received very slow (and thus innaccurate) fills.



I want to let you know that I am aware of the problem, have identified the cause, and have fixed it. If you are a system vendor and you received a lousy fill in your hypothetical trading record, please fill out a Trouble Ticket and I will be happy to investigate and (if warranted) fix. Instructions for filling out a Trouble Ticket can be found at: www.collective2.com/how_trouble



I do apologize for the inconvenience this may have caused. The good news (if there is any) is that the lag was caused by a spike in C2 users, which benefits everyone on the site, particularly system vendors. As you may know, we are spending a lot of resources upgarding the site infrastructure to support our growing user base. I suspect that this morning’s problem won’t recur anytime soon.



Matthew

I have to tell you that this was a very frustrating experience. It isn’t about the fills. It is about the many trades that I did not send, or canceled, because there was a 2-3 minute delay in getting the HTML code to C2 and everything backed up. As I buy weakness as it recovers, this was the strongest day for my system in the past 6 months. Thankfully I was able to enjoy it in my trading account.



Everyone here is supportive of your efforts and appreciates the concept of C2. But all that is useless crap if the system keeps bogging down during busy market environments and trades can’t be tracked or entered.

My goal is to insure that this doesn’t happen again.



I appreciate your frustration. For what it’s worth, I’m pretty certain this morning’s problem has been fixed and won’t recur.



MK

I am thinking about setup an account in Open E Cry to do auto trading futures with C2 systems. Will slow response time on C2 server impact C2 system signals and/or auto trading fills? Can someone explain it to me what’s the possible impact factors for auto trading? Where are C2 systems? Are they on vendor’s server or C2 server? What will happen to auto trading system subscribers if a system vendor decides to withdraw his system from C2?



Thanks.



The actual hardware architecture is complicated. There are separate web servers, database servers, quote feed servers, autotrade servers, and broker interface servers. Some of these components reside on the same hardware, but not all. Further, each pieve of hardware is replicated in at least two locations, so that we will be able to try to cope in case of geographically-local catastrophe.



The problem yesterday was a slow quote feed, which cascaded to affect the Web server. Yesterday morning’s slowness didn’t affect AutoTraders directly, although if a system vendor was running a system and trying to type in an order into the C2 web interface, it would have been slow going for some time during the slow period.



The fact is that no technology is perfect. Some of the largest brokers in the world – with billions of dollars of market capitalization – have recently had Web outages preventing order placement. At C2, we are spending a lot of money to make our systems as reliable as possible. Yesterday morning’s event was unfortunate, but I urge readers of this message not to lose sight of the context: we haven’t had a major outage in quite some time; yesterday’s sluggish Web server was annoying, but fixed fairly quickly – by 11:00 am ET the problem was completely gone. I don’t mean to minimize yesterday’s events; no problem is acceptable. But in the general scheme of things, C2 has been getting more reliable, and generally faster – even as our user base and AutoTrading volume has increased substantially.



No guarantees are possible in the technology and trading world. But a trader needs to evaluate the potential risks and rewards of relying on any particular platform – whether it is C2’s, or some other firm’s.



Matthew