How many defects are likely to be delivered in a software product?

Short answser

How many defects did you find in final test? You will probably release about that many to the field. Absent better information, a 50% yield in test is a reasonable assumption.

 

Longer answer

Consistency is the key. Follow a consistent process and you will get reasonably consistent results.

If you measure the defects throughout the process, you will find some process phases tend to inject defects (e.g. design and code) Others tend to remove (inspection and test). You will find some removal processes are more constent than others. It’s very surprising how consistent injections are. (I’ll update this post at some point with some plots).

The phase removal Yield is a ratio. Yield is the number of defects removed divided by the defects present at that time. Using historical defect density profiles and yields one can compare a product to benchmark data from the past and make some educated projections.

If

  • you have a similar product
  • spent a representative amount of time developing test cases
  • have similar defect profiles and yields

Then you can, with high confidence, estimate the defects remaining in test. How many defects were found? What is my historic yield? How consistent was that yield?

If you have this information, you have the keys to the kingdom.

Advertisements

About Bill Nichols

PhD in Physics from Carnegie Mellon University I'm a software team coach and instructor with the TSP Team at the Software Engineering Institute
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s