Audits and Project Success

Project success depends on several mechanisms which form part of the overall governance framework.

The mechanisms include management reviews like proposal reviews, project status reviews, program status reviews, top management reviews and process audits.

In addition to the above management reviews, there needs to be rigorous technical reviews of requirements, design, implementation, verification & validation, delivery and installation.

Audits are but one of the several mechanisms for an organization to make sure things go right. Some organizations think that audits can solve all their problems.

Its like thinking that more testing will help improve product quality.

However, the focus should be on improving how things get done right the first time rather than how to better find whether they got done right.

In the case of software, quality of the software depends on the quality of the code and not how well it got tested.

Testing cannot inject quality it can only enable quality by identifying issues and defects before it gets shipped to the customer.

It would, in fact, add to the eventual cost of the software and someone has to pay for it.

CMMI Next Gen

CMMI for development, version 1.3 was released in the year 2010, towards the end of that year to be more precise (sometime in October/November 2010, to be even more precise).

Like the ISO standards which are revised every 8-10 years, CMMI models also undergo revision periodically and the process for the latest revision has already been initiated. It is being referred to as CMMI Next Gen or CMMI Next Generation.

As a side information, it is useful to note that the revision to 2008 version of ISO 9001 standard is due and is interestingly enough slated for release sometime this month - September 2015!

As is the convention it will be called as ISO 9001:2015. The technical committee entrusted with the 2015 revision has claimed many significant changes.

However, many people believe that these are mere paper changes and will not lead to significant changes in the implementation on the ground.

And that is how it should be. The standard is based on fundamental principles of quality management as propounded by Shewhart, Deming and Juran which remain unchanged.

These principles apply to CMMI as well.

Coming back to CMMI, the CMMI models (development, service and acquistion) and the appraisal method (called with a complex sounding term SCAMPI A) can undoubtedly be improved.

Following aspects related to CMMI need a closer and deeper investigation as CMMI Next Gen gets crystallized:

• There is no area that focuses on customer management. There are areas related to internal management and partner/supplier management, however, the most important stakeholder, the customer, for which work is performed doesn't get the kind of focus it should.
• The appraisal duration is on the higher side which leads to excessive strain on the organization. First, getting appraisal team members for a long duration is not easy. Second, logistics challenges tend to be huge. And third, the cost of appraisal becomes high.
• Appraisal team involves internal employees which is not efficient. The internal members have a narrow view and understanding of the practices and it is not clear how significant their contributions are given the lead appraiser would eventually have his way. The reason for this is that the lead appraiser has a better understanding or at least this can be assumed to be so in most cases.
• Use of the term "high maturity" in CMMI model is not appropriate. Use of control charts, regression, design of experiments, etc. are common practices in a manufacturing context and there is really nothing "high maturity" about it.
• In fact, applicability of statistical techniques especially control charts and modeling in CMMI needs to be validated much more rigorously. Software processes owing to their high dependence on human factor tend to have high outcome volatility. An example to illustrate this is that the quality of same code written by a developer would be different on different days as also the quality of same code written by two developers would be different.
• CMMI level 5 is really nothing but six sigma and it would be appropriate to label it so. Use of the term OPM makes it sound more complicated than it actually ought to be.
• Explicit referencing of existing tools and methods like six sigma and TQM is the right way to go. Explicit linkages to these should be part of CMMI model. For example CAR is nothing but RCA and Fishbone/Ishikawa diagram which is a TQM method known much before CMMI came into existence. Similarly, QPM is nothing but statistical monitoring and control, again a decades old concept.
• Lead appraisers for CMMI need to have formal education/training in decades old concepts from TQM, lean six sigma, etc. Many lead appraisers don't have clear understanding of statistical theory and hence use approach for level 4 and level 5 practices that are not scientifically validated.

CMMI Next Gen provides a good opportunity to further improve CMMI framework. The key is to recognize and articulate how it can fit into the overall business excellence framework of an organization.

Another aspect worth pondering about is how it can be promoted as a framework that can help organizations that are not just doing software. Legacy has ordained CMMI to remain limited to software companies and CMMI Next Gen should focus perhaps on breaking this barrier more than anything else.

CMM/CMMI Development Timeline

1979 - Quality Management Maturity Grid Philip Crosby, "Quality is Free"

1985 - IBM Maturity Grid
R.A. Radice, J.T. Harding, et al, "A Programming Process Study", IBM Systems Journal, Vol. 24, No.2, 1985.

1987 - Maturity Questionnaire
W.S. Humphrey, "Characterizing the Software Process: A Maturity Framework", Software Engineering Institute, CMU/SEI-87-TR-11, DTIC Number ADA182895, June 1987.
W.S. Humphrey and W.L. Sweet, "A Method for Assessing the Software Engineering Capability of Contractors", Software Engineering Institute, CMU/SEI-87-TR-23, DTIC Number ADA187320, September 1987.

1989 - Capability Maturity Model
Watts Humphrey, "Managing the Software Process", Addison-Wesley, Reading, MA, 1989.

1991 - SW-CMM, v1.0

1993 - SW-CMM, v1.1

2002 - CMMI, v1.1

2006 - CMMI, v1.2

2010 - CMMI, v1.3

20XX - CMMI Next Gen

Six Sigma Development Timeline

1733 - normal curve developed as an approximation to binomial distribution by Abraham de Moivre

1783 - normal curve used to describe the distribution of errors by Pierre-Simon Laplace

1809 - normal curve used to analyze astronomical data by Carl Friedrich Gauss

1924 - statistical process control launched for quality improvement by Walter A. Shewhart

1984 - internal quality research report in Motorola on the correlation between how well a product did in its field life and how much rework had been required during the manufacturing process by Bill Smith

1985 - development of a four-stage problem-solving approach: Measure, Analyze, Improve, Control (MAIC) by Mikel Harry

1987 - six sigma launched in Motorola by Bob Galvin

1991 - six sigma launched in Allied Signal by Lawrence Bossidy

1995 - six sigma launched in General Electric by Jack Welch

ISO 9001 Development Timeline

WWII -  Inspectors and “quality system requirements” for suppliers (Ministry of Defense, UK)The Government placed inspectors in weapons factories in an attempt to solve the problem of bombs going off in the factories. To become a supplier to the Government a company had to document the procedure for making the product, have the procedure inspected by the inspector and ensure the procedures were followed.

1959 - MIL-Q-9858a (US Military)

1968 - AvP92 (British Ministry of Technology which handled defense materiel procurement)

1969 - AQAP (NATO quality management specifications)

1973 - DEF STAN 05/21 (Ministry of Defence, US)

1979 - BS5750 (British Standards Institution)

1987 - ISO 9001:1987 (ISO)

1994 - ISO 9001:1994 (ISO)

2000 - ISO 9001:2000 (ISO)

2008 - ISO 9001:2008 (ISO)

2015 - ISO 9001:2015 (ISO)

Agile Development Timeline

1986 - "approach similar to rugby" for commercial new product development Takeuchi, H. and I. Nonaka, "The New Product Development Game", Harvard Business Review, 1986 (January-February).

1991 - use of the term Scrum (term of rugby) for the "approach" DeGrace and Stahl, “Wicked Problems , Righteous Solutions”

1993 - Scrum used at Easel Corporation by Jeff Sutherland

1995 - Scrum paper presented by Jeff Sutherland and Ken Schawaber in OOPSLA'1995

1996 - XP used at Chrysler by Kent Beck

2001 - Agile manifesto signed by 17 original signatories