YES!!!

I have had many discussions about requirements, if they are needed, and then

if they are really needed.  By this I mean, some people say they are needed but they don’t really believe it or at least they don’t know that they don’t believe it.  Then you have the rare few who not only know in their mind, requirements are needed, but they know it in their hearts.  IE: They actually do it.

You can easily distinguish the two.  One will demand requirements are written down and are the soul foundation of the project, the other will think they should be written down but they would rather get coding than push for it.

The relative average cost of fixing a defect based on when it is found is as follows:

Requirements  Architecture  Construction  System Test  Post-Release

     1             3          5-10          10          10-100

Clearly, discovering requirement defects, (IE: incorrectness, omissions, conflicting), is cost prohibitive.

Quote from Code Complete:

“Iterative approaches tend to reduce the impact of inadequate upstream work, but they don’t eliminate it.”

“One common rule of thumb is to plan to specify about 80 percent of the requirements up front, allocate time for additional requirements to be specified later, and then practice systematic change control to accept only the most valuable new requirements as the project progresses.”

“Some projects spend too little time on prerequisites, which exposes construction to an unnecessarily high rate of destabilizing changes and prevents the project from making consistent progress.  Some projects do too much up front; they doggedly adhere to requirements and plans that have been invalidated by downstream discoveries and that can also impede progress during construction.”

“If the requirements are explicit, the user can review them and agree to them.  If they’re not, the programmer usually ends up making requirements decisions during programming.”

“A plan to follow the requirements rigidly is actually a plan not to respond to your customer.”

“How much change is typical? Studies at IBM and other companies have found that the average project experiences about a 25 percent change in requirements during development (Boehm 1981, Jones 1994, Jones 2000), which accounts for 70 to 85 percent of the rework on a typical project (Leffingwell 1997, Wiegers 2003).”

“Generally, a well-run project devotes about 10 to 20 percent of its effort and about 20 to 30 percent of its schedule to requirements, architecture, and up-front planning (McConnell 1998, Kruchten 2000).  These figures don’t include time for detailed design – that’s part of construction.”

Some Sad IT Facts

 

These are fairly known but often dismissed facts about the IT industry. If you see these facts presented you will find IT managers and developers nodding their heads in agreement.
That is as far as it goes.

I'm always amazed to see an IT manager hesitate to purchase a piece of software because he is afraid of wasting 300 dollars but there is no hesitation to waste 10's of thousands or even 100's of thousands through his/her indifference to good software management practices. 

I found a company that specialized in process. I was very excited to be a part of a team dedicated to making quality software at an affordable cost. One week into this project, I realized that very basic steps were missed. They were not oversights. It was simply thought that gathering quality requirements was not worth the time and effort. I was shocked. Don't these people realize that while not sexy, this is still a critical step in the process?  All quality is related to requirements.

As I predicted, we went over budget around 200 percent. We missed our deadlines on a fix priced project and pissed off the customer. All because gathering quality requirements was not sexy enough. Seems it was more important to measure employee productivity and create schedules. Of course, it's impossible to measure productivity if you don't have a clear definition of what you are supposed to produce and let's not get into the impossibility of predicting a schedule without requirements.

SIDE NOTE: I'll cover this in another blog, after further thought. NEVER measure your employees for the purpose of criticism. When you do this, you will never get accurate numbers from your employees ever again. It's just a fact of life.

The following are the sad truths of IT:

From the Hugh W. Ryan article of Outlook Journal

• Only 8 percent of application projects costing between $6 million and $10 million succeed.
• Among all IT development projects, only 16% delivered to acceptable cost, time and quality.
• Cost overruns from 100 to 200 percent common.
• Cost overruns for IT projects have been estimated at $59 billion in the United States alone.
• IT workers spend more than 34% of their time just fixing software bugs.

Other reports make the following claims:

• Only 28% of projects are completed.
• In the UK, over $1 billion a year is wasted on poor software quality.
• 70% CRM project strategies fail.
• 90% companies cannot show a positive ROI for CRM.

Here is an interesting fact that I 100% believe is true but it is still shocking to comprehend the incompetence that must allow this.

"According to the Cranfield School of Management, the more ambitious the return on investment for the project cited in the business case, the more lacking the project plan is likely to be."

Software development as a craft?

As with any profession we should continue to perfect our craft. I'll begin this blog with a discussion about requirements and see where this leads. While I have my opinion on certain matters, I honestly want to know what others think. I remain open to change myself as I know reads of this blog are. On with the journey...

Software requirements are the foundation of any software development effort. Regardless of whether you collect the requirements up front or through-out development, all requirements will be known. If documented they will remain known else they will be known in the form of tribal knowledge.

Why do some resist following process which includes requirement discovery while others embrace it? Is there a cowboy mentality in our community or is it simply more efficient to code a little, gather a few requirements, then code a little more? For those who embrace process, is it possible that a certain comfort is found in a predictable process oriented environment?

So is it worth the effort to develop our craft as software programmers? Should we mature beyond the technical skill set and venture into the unknown?

Requirement Statistics:

What is the unknown? In fact much of this process driven world provides imperial proof of its claims. Here are some facts.

Concerning defects injected into requirements:

Incorrect Facts: 40 percent
Omissions: 31 percent
Inconsistency: 13 percent
Ambiguity: 5 percent
Misplaced: 2 percent

Over 70 percent of defects are due to incorrect and omitted facts.

How much more misunderstanding do we have when no value is placed on gathering requirements?

This is a real problem in our industry and in my mind is part of why software development is not the respected craft it should be. We are often viewed as an expense and in many cases rightly so.

Source of Errors and Cost:

According to the "Software Project Management Course Workbook (Pittsburgh, PA.:CMU/SEI/September, 1992). the following facts are true.

The following are where defects are injected into applications:

Requirements definition: 50%
Software design: 35%
Coding: 15%

So if requirements analysis is not held as critical then you can expect your defect rate to be at least doubled.

The following is the cost of fixing a defect at each stage, according to the same study:

Requirements definition: $150 per defect
Software design: $500 per defect
Coding: $1,100 per defect
Testing: $1,200 per defect
Deployment: $7,500 per defect

Conclusion:

Nothing else we do seems to matter if requirements aren't collected, scrutinized, and documented. Yes, it can take time but can you afford to not take the time. You will never be the first to market with a quality product without a well defined process and a culture that can deliver it. A process alone cannot deliver on the promise of that process. Software developers must perfect their craft of which software process is a part and a culture of quality must be promoted.

I have been on both sides of this issue but admittedly I am much more comfortable in a predictable environment. So my tendency will always be to promote process and as a human I will seek proof that process is a necessary part of the craft. Having fully disclosed my tendency and bias, I welcome all comments and opinions.