The True Cost of a Software Bug: Part One

Wondering what the cost of a software bug is? It depends on how late you find it.

The Systems Sciences Institute at IBM has reported that “the cost to fix an error found after product release was four to five times as much as one uncovered during design, and up to 100 times more than one identified in the maintenance phase.”

The cost of a bug goes up based on how far down the SDLC (Software Development Life Cycle) the bug is found. When a bug is found in production the code needs to go back to the beginning of the SDLC so the agile development cycle can restart.

Then there’s the domino effect to think about. The software development approach often need to change to accommodate the code fix, which can in turn bump back other code changes. So not only is the bug going to cost more to fix as it moves through a second round of SDLC, but a different code change could be delayed, which adds cost to that code change as well.

To illustrate: if a bug is found in the requirements-gathering phase, the cost could be $100. If the product owner doesn’t find that bug until the QA testing phase, then the cost could be $1,500. If it’s not found until production, the cost could be $10,000. And if the bug is never found, it could be secretly costing the company money and no one could be the wiser. A 2003 study commissioned by the Department of Commerce’s National Institute of Standards and Technology found that software bugs cost the US economy $59.5 billion annually.

Cost_of_A_Software_Bug.jpg

Here are some examples of costly software bugs:

Example 1: NASA – Mariner 1

When the Mariner 1 spacecraft was launched in 1962, it was the first attempt by the US to send a spacecraft to Venus. The Mariner 1 just made it past Cape Canaveral when a software bug caused the rocket to veer off course. A NASA range safety officer on the ground issued a self-destruct command, which destroyed the spacecraft 290 seconds after takeoff. One of the official reports was that the omission of a hyphen in coded computer instructions in the data editing program, resulted in incorrect guidance signals being sent to the spacecraft.

The cost for the rocket was reportedly more than $18 million at the time.

Example 2: Toyota recall

There have been three separate recalls by Toyota since 2009, and all the recalls have been related to the accelerator sticking. In August 2009 a Lexus ES350 suddenly accelerated out of control at speeds estimated to exceed 100 mph. One of the passengers called 911 and reported that the car had “no brakes.” All four passengers were killed when the car crashed. In November 2009, Toyota dealers were instructed to remove and shorten the gas pedals and to update the onboard computers with a new program that would override the electronic gas pedal when the brake pedal was pressed. Toyota ended up recalling more than 9 million cars worldwide in 2010, but it wasn’t because of a mechanical issue. The cars had a software bug that caused a lag in the anti-lock-brake system.

Due to increased incentive campaigns, legal liabilities, and marketing efforts, the recalls were estimated to cost Toyota as much as $3 billion.

Example 3: Knight Capital Group’s trading violations

In August 2012, Knight Capital Group Inc., one of America’s largest trading firms, mistakenly sent out more than four million stock orders in less than an hour. These orders should have been spread out over a period of days—and reversing the trades cost almost half a billion dollars. Knight Capital would have been sent into bankruptcy had it not been for a group of investors that came up with $400 million. The problem was that when a code change was not deployed to all the servers, one server was missed, and this caused the server to use old code to create millions of orders. The firm’s shares lost 75 percent in two days after the faulty software flooded the market with unintended trades, sending dozens of stocks into spasms.

The software bug caused over $440 million in losses, which is almost four times what the company made in 2011.

As you can see, finding a bug later in the life cycle of the software costs exponentially more. Collaborating with the customer through Agile software testing will greatly help reduce development against buggy requirements.

In my next post, I’ll break down of the cost implications of software bugs in each phase of the SDLC. Stay tuned!