Why Software Size and Effort Estimation Continues to Haunt Software Projects?

The profit margin one is expected to make in a software project is a function of the productivity one expects to operate at and the cost that would be incurred at that level of productivity.

Accurate and precise estimation of software size and effort estimation has remained a key barrier in estimating the productivity of software projects.

In some sense, software size and effort estimation continues to haunt software projects.

Finding the "silver bullet" method for estimating size and effort is akin to finding a drop of water one can drink while one is stranded in the middle of an ocean.

Almost impossible.

Looks like that.

However, that's not true.

Size and effort are simple but powerful concepts.

Size indicates the amount of work or the volume of work that needs to be performed.

Effort indicates the amount of work hours spent to generate a certain size.

And by definition productivity is the size generated for a unit of effort spent.

Very simple.

The key consideration however is that effort should use the size information.

That makes the whole game of estimation interesting and hard.

So how to estimate size?

There are many ways like function points (FP), kilo source lines of code (KSLOC), situation-specific complexity points (CP), number of requirements, number of pages, number of test cases, etc.

It is better to use FP since there is lot of literature available on it in addition to the support system available.

Once size information is available how to estimate effort?

Effort is based upon size as well as additional factors like complexity of work, competency of the person performing the work, effectiveness of project management processes and controls.

All the above factors do somehow get captured in the concept of productivity.

So effort can be estimated from size simply using productivity figures.

Many organizations never get started with the above methods due to several reasons like poor understanding and incorrect notions regarding the industry methods, lack of will, lack of ownership.

In fact, in some of the organizations, there are some "dummy experts" on estimation who think they know more than those who are recognized in the industry as experts on estimation.

The dummy experts end up creating an estimation framework that is neither usable (the method is over-engineered and with too many layers of details after details) nor useful (the estimates that get computed by using the framework are not realistic and hence of no use in real).

On top of it, the experts do not own the future maintenance and enhancement of this framework.

Lack of any effort on sustaining the framework is like slow poison that kills that framework eventually.

The whole thing may happen with lot of fanfare but meets its natural death at some point in time.

So what to do if nothing works out?

If nothing works out, the best bet is to create a very very granular WBS (work breakdown structure) or PBS (product breakdown structure) or a combination of WBS and PBS.

Effort for the WBS elements can then be estimated based on the volume of work at the WBS element level.

That brings the flavor of size being estimated followed by size information being used for deriving effort estimates.

All said and done, estimation is a hard nut to crack.

And while you are at it, trying to crack it, it may make you go nuts!