Posts

Showing posts from April, 2020

The most expensive bug in history...

  The development team gets a call by CEO that they need to implement a new feature in SMARS (their system) to take trades from dark pools as well in addition to the regulated markets.   The development team had 30 days to comply with the new rule. Although the CEO himself was not in favour of this new feature he had to anyway go for it as it meant retaining existing business.   The release date for the new feature in the system was set to 9:30 AM EST on August 1, 2012   The plan was to deploy the new system behind a feature flag the week before the deadline; when the market opened on August 1, they'd simply turn it on.   At 9:30 AM EST on August 1, the Knight developers did just that: they enabled the feature flag, and SMARS began to route orders through to the RLP—they were live!   But something was wrong. Their charts showed anomalous spikes in trading activity on the open markets. At 9:34 AM, the NYSE called  Knight was executing a lot of trades—so many, in fact, that trading v