The effectiveness of a development team to capture defects before product

release significantly reduces the total cost of ownership of an application.  It is far cheaper to correct a defect before product release.

One simple measurement of this effectiveness is TCE (Total Containment Effectiveness).  This is the percentage of defects contained or discovered before a product is released, divided by the total defects both pre and post release.  The higher the percentage the more effective a development team is.

                   preRelease Defects
TCE = ------------------------------------------- * 100%
            preRelease + postRelease Defects

To get the postRelease Defect count the application must be used for 6 to 12 months.  If you track the version a defect was injected into then you can further extend this analysis while other versions of the application are release.  This is helpful in an agile environment where TCE would be more difficult to track because of the number of releases required per cycle.

If you then track the development phase the defect was injected then analysis will lead you to improving the effectiveness of a struggling phase.

YES!!!

I have had many discussions about requirements, if they are needed, and then

if they are really needed.  By this I mean, some people say they are needed but they don’t really believe it or at least they don’t know that they don’t believe it.  Then you have the rare few who not only know in their mind, requirements are needed, but they know it in their hearts.  IE: They actually do it.

You can easily distinguish the two.  One will demand requirements are written down and are the soul foundation of the project, the other will think they should be written down but they would rather get coding than push for it.

The relative average cost of fixing a defect based on when it is found is as follows:

Requirements  Architecture  Construction  System Test  Post-Release

     1             3          5-10          10          10-100

Clearly, discovering requirement defects, (IE: incorrectness, omissions, conflicting), is cost prohibitive.

Quote from Code Complete:

“Iterative approaches tend to reduce the impact of inadequate upstream work, but they don’t eliminate it.”

“One common rule of thumb is to plan to specify about 80 percent of the requirements up front, allocate time for additional requirements to be specified later, and then practice systematic change control to accept only the most valuable new requirements as the project progresses.”

“Some projects spend too little time on prerequisites, which exposes construction to an unnecessarily high rate of destabilizing changes and prevents the project from making consistent progress.  Some projects do too much up front; they doggedly adhere to requirements and plans that have been invalidated by downstream discoveries and that can also impede progress during construction.”

“If the requirements are explicit, the user can review them and agree to them.  If they’re not, the programmer usually ends up making requirements decisions during programming.”

“A plan to follow the requirements rigidly is actually a plan not to respond to your customer.”

“How much change is typical? Studies at IBM and other companies have found that the average project experiences about a 25 percent change in requirements during development (Boehm 1981, Jones 1994, Jones 2000), which accounts for 70 to 85 percent of the rework on a typical project (Leffingwell 1997, Wiegers 2003).”

“Generally, a well-run project devotes about 10 to 20 percent of its effort and about 20 to 30 percent of its schedule to requirements, architecture, and up-front planning (McConnell 1998, Kruchten 2000).  These figures don’t include time for detailed design – that’s part of construction.”

Software development as a craft?

As with any profession we should continue to perfect our craft. I'll begin this blog with a discussion about requirements and see where this leads. While I have my opinion on certain matters, I honestly want to know what others think. I remain open to change myself as I know reads of this blog are. On with the journey...

Software requirements are the foundation of any software development effort. Regardless of whether you collect the requirements up front or through-out development, all requirements will be known. If documented they will remain known else they will be known in the form of tribal knowledge.

Why do some resist following process which includes requirement discovery while others embrace it? Is there a cowboy mentality in our community or is it simply more efficient to code a little, gather a few requirements, then code a little more? For those who embrace process, is it possible that a certain comfort is found in a predictable process oriented environment?

So is it worth the effort to develop our craft as software programmers? Should we mature beyond the technical skill set and venture into the unknown?

Requirement Statistics:

What is the unknown? In fact much of this process driven world provides imperial proof of its claims. Here are some facts.

Concerning defects injected into requirements:

Incorrect Facts: 40 percent
Omissions: 31 percent
Inconsistency: 13 percent
Ambiguity: 5 percent
Misplaced: 2 percent

Over 70 percent of defects are due to incorrect and omitted facts.

How much more misunderstanding do we have when no value is placed on gathering requirements?

This is a real problem in our industry and in my mind is part of why software development is not the respected craft it should be. We are often viewed as an expense and in many cases rightly so.

Source of Errors and Cost:

According to the "Software Project Management Course Workbook (Pittsburgh, PA.:CMU/SEI/September, 1992). the following facts are true.

The following are where defects are injected into applications:

Requirements definition: 50%
Software design: 35%
Coding: 15%

So if requirements analysis is not held as critical then you can expect your defect rate to be at least doubled.

The following is the cost of fixing a defect at each stage, according to the same study:

Requirements definition: $150 per defect
Software design: $500 per defect
Coding: $1,100 per defect
Testing: $1,200 per defect
Deployment: $7,500 per defect

Conclusion:

Nothing else we do seems to matter if requirements aren't collected, scrutinized, and documented. Yes, it can take time but can you afford to not take the time. You will never be the first to market with a quality product without a well defined process and a culture that can deliver it. A process alone cannot deliver on the promise of that process. Software developers must perfect their craft of which software process is a part and a culture of quality must be promoted.

I have been on both sides of this issue but admittedly I am much more comfortable in a predictable environment. So my tendency will always be to promote process and as a human I will seek proof that process is a necessary part of the craft. Having fully disclosed my tendency and bias, I welcome all comments and opinions.