What happened this morning?

Oh, you’re asking me?



I saw this great thread title lying around - “What happened this morning?” - and so I thought someone would tell me exactly what happened this morning. You know, why C2 freaked out for the past 16 hours or so.



Okay, well, since you insist on my answering you, I’ll tell you what I know. Last night at around 4:30 pm Eastern, I installed new C2 upgrades. Everything seemed fine. Except, this morning, it wasn’t. Some order-processing processes got hung up in endless loops, tanked the database, hogged CPU cycles… yuch. It was a mess.



I de-installed the upgrades, rebooted the web servers (to get the gremlins out), and things seem okay now.



So now it’s damage control time.



Let me start with:



Whoops. Sorry.



And then: please email me if you notice any further problems. There may be some legacy issues caused by last night’s mess, so let me know.



Believe it or not, last night’s software upgrades were meant to solve some sporadic problems, and they may have actually succeeded in doing just that. C2 is a giant piece of web software - a living thing, sort of - constantly being upgraded, prodded, poked, etc. Most of the time people don’t notice the fixes - they just see new features, or faster page loads, or whatever. But every now and then, something like the events of this morning happen, humbling me, and making me ask for your forgiveness.



Sorry for the trouble. I think it’s behind us now, but - as usual - I can be wrong, so let me know if you have any order problems this morning. Email me and I’ll take a look. (matthew@collective2.com)

-Matthew



MK, maybe C2 has reached the point in its lifecycle that you need to follow some standard policies about when the software is upgraded, what kind of testing is done before the production system is changed, how your customers are notified of changes, etc.



I know its easier to get stuff done when you have no one to answer to (except when something breaks) and can just change whatever you want whenever you want on the fly but since there is real money on the line it appears to me that some discpline is in order.

Well that probably explains the sht load of anomalies that I have encountered recently. Unauthorised execution, stops not been triggered and the like. Mistakes happpen…right? It’s not what happened it what you do about it…right?



Can you correct the errors to my account please? I have reported the problems numerous times but to no avail. I don’t want to get in a position where I have to keep track of a backlog of errors, which is rapidly becoming the case.

Max: There are 2 separate things happening.



First, due to yesterday morning’s glitch, there were in fact trades that were not executed in a timely manner. I do believe that all of those issues have been solved and filled correctly. If I am wrong, just email privately and I’ll take care of it.



The other set of issues is the following. According to the C2 contract database, the LFH6 contract expires today. C2 automatically exits a position in an expiring contract if you don’t take care of it yourself. (Not so weird: Interactive Brokers does the same thing. There’s no “taking delivery” of the contract, like at full-service brokers.)



That’s why you see “unauthorized execution” – your contracts expired and you didn’t roll them. Please email me privately with any questions about this.



Sorry for all the confusion yesterday morning. This LFH6 issue is unrelated, though.

I apologise. I didn’t realise that the contract expired today. A rookie mistake!