We just shipped Vista Beta 2 and we are now on the march toward shipping what we call RCs or Release Candidates. During this time, we are forced to make many hard choices about what bugs to fix and what bugs we can live with. There are those who believe that software should ship without bugs but those people have probably never tried to ship a product of any significant size. There will be bugs in software, just like there are bugs in a new house or a new car. The important thing is to make sure there are no important bugs.
Eric Sink is a software developer at SourceGear and I recently ran across an article by him about shipping with bugs. Eric makes a few points I think worth repeating. He says there are 4 questions that must be asked:
1) How bad is its impact? (Severity)
2) How often does it happen? (Frequency)
3) How much effort is required to fix it? (Cost)
4) What is the risk of fixing it? (Risk)
The first two determine whether the bug is worth fixing at all. If the bug doesn't pass muster there, you don't even look at the 3rd and 4th. A bug being easy to fix is never a reason to take the fix late in a product cycle. Just because it is a single line doesn't mean we should do it. There is risk with all fixes. I've seen the easiest ones go wrong.
The second two can be reasons to reject even righteous bugs. If the bug is bad and happens often but is very hard or very risky, it still might not be worth fixing. This is a hard balancing act. Sometimes the bug is bad enough that you delay the product. Other times it is possible to live with it. Making that choice is never a fun one. As owners of a product, we want all bugs to be easy to fix and have low risk. Unfortunately, that isn't always the case and when it isn't, hard choices have to be made.
I recommend you read the whole article. Apparently the one I found is based on a longer article which can be found here.
Risk levels are not static through the development cycle.
ReplyDeleteFor example, the closer the product gets to being released, the risk for fixing any bug increases.
You are correct. Risk levels are not static. Early in a project, all fixes are on the table. Everything that is found gets fixed or at least gets on the list of things to be fixed. If something goes wrong, there is plenty of time to correct it later. As the project nears completion, however, the risk rises and so does the bar for fixing bugs. A "one line fix" might be rejected if it is late enough in the project. While it might seem innocuous, there could be consequences. If everything goes right, it is a no-brainer but it might somehow change the program flow or it could inadvertendly break the build costing everyone critical time.
ReplyDelete