1. 0 Introduction
This paper describes software process maturity and some of the
various models that are available to software practitioners. The
importance of improving software process to small software development
companies is also discussed, as are the difficulties faced by
these companies when implementing such improvements. The author
presents a model, based on Quality Function Deployment, which
can be used as a tool to aid the implementation of a software
process improvement action plan, and discusses the use of this
model in a small software development company in Ireland.
2.0 Software Process
Research into the improvement of software products during the
1970's and early 1980's focused on software development methods.
However, since the late 1980's software process improvement has
been gaining popularity as an alternative approach to improving
software products. This interest has it's roots in manufacturing,
where at one time, it was important to test the product, but more
modern manufacturing techniques focus on the process so that the
final product had fewer faults.
Paulk et al. (1993) define a software process as "a set of
activities, methods, practices and transformations that people
use to develop and maintain software and the associated products".
The process is often documented in a set of guidelines and/or
procedures which are followed during software development. A number
of models have been developed which endeavour to measure the process
maturity of an organisation. These include the Capability Maturity
Model and BootStrap. Currently, there is an international software
process group developing the Software Process Improvement and
Capability dEtermination model (SPICE) within which these models
fit. Also, of interest to software development companies is ISO9000-3,
which is the International Standards Organisation software standard.
2.1 Capability Maturity Model
One of the widely-publicised process models is the Capability
Maturity Model. This was developed at the Software Engineering
Institute, Carnegie Mellon University, in the United States of
America and funded by the U.S. Ministry of Defence. This model
evaluates the software process within an organisation by measuring
the level of process maturity and indicating its key weaknesses.
Subsequently this allows the software development group to develop
a strategy which should improve the organisation's software process.
Within this model, there are five levels of software process maturity: (Paulk et al., 1993)
Level 1, Initial Level - the environment in which software is developed and maintained is not stable;
Level 2, Repeatable - the organisation has established policies for managing a software project along with procedures to implement those policies;
Level 3, Defined - the organisation has documented a standard process, including software engineering and management process, for development and maintenance of software;
Level 4, Managed - the organisation sets quantitative quality goals for both software products and processes;
Level 5, Optimizing - the entire organisation is focused on continuous
process improvement.
As of December, 1994, although numbers at the higher levels have
increased since measurements began in 1988, 73% of the assessed
industries were at level 1 (initial), while only 0.6% and 0.3%
were at levels 4 and 5 respectively. (Neupauer, 1996).
2.2 Problems with Software Process Models
One major weakness of the Capability Maturity Model which has
been recognised and quoted, Peterson (1995a), Draper et al. (1995),
is that is does not present a process improvement strategy. This
is also the case for other process models. Consequently, an organisation
must develop an improvement strategy for themselves. More recently,
the Software Engineering Institute has proposed a strategy for
organisations, with the IDEAL model has been presented as a possible
solution. (Peterson, 1995b)
Another problem faced is the amount of resources expended when
assessing a software development organisation. Buchman and Bramble,
1995, describe software process assessments as "expensive,
time consuming, exhaustive and disruptive to the organisation
being assessed". Also, assessors need to be specifically
trained in assessment, which is costly. The argument is made that
companies will make significant cost savings in the future, so
they must be prepared to make that initial investment.
3.0 Consequences for Small Software Development Companies
Increased interest in software quality, particularly in the United
States of America, has caused many software companies to strive
for higher levels of process maturity. This is particularly the
case for software development companies who sub-contract from
larger organisations. This trend is likely to continue and grow.
This problem is now being faced by the small software development
companies which are prevelant in Ireland.
While it is possible for a large and profitable multi-national
company to absorb the overheads associated with software process
assessment and improvement projects into their organisation or
to inject cash to a special project up-front, this is much more
difficult when the company is small. Also, it is not enough for
the organisation to know how they are doing, they need to be able
to develop an improvement strategy which will work for them within
their organisation.
Other software development companies are attempting to tackle
software process due to their pursuance of ISO9000 certification.
This may be happening because of a growing quality culture and
customer requirements, but whatever the reason, companies are
prepared to invest time, money and effort in obtaining ISO9000
certification. For software development purposes, the ISO9000-3
guidelines are of interest, those concerned with the development,
supply and maintenance of software.
4.0 Quality Function Deployment as a solution
In the light of the problems discussed in the previous section,
this research is examining whether Quality Function Deployment
can be used to aid software process particularly within small
software development companies, thus reducing the resources expended
at the start of the process.
The proposal is that Quality Function Deployment (QFD) can be
useful. Using QFD the software process model is treated as the
customer. Software process model requirements are treated as the
voice of the customer, as these are the requirements of an improved
software process. However, in this case, instead of going to the
gemba as would be normal with a QFD exercise the software process
requirement is the same in every organisation (as they are going
to be assessed on the same criteria). Therefore, it is more useful
to look at the software process assessment as the gemba, and extract
the information required from this source. Once this is complete,
all practices that affect each process must be identified as the
HOWs in a QFD matrix.
Following this procedure, a House of Quality can be built, containing
WHATs and HOWs, and their correlations.
This generic QFD matrix allows organisations to assess how effective
their current process is, how they can improve it, and to what
levels they can improve. They also need an action plan - how do
they go about improving it. QFD allows the practitioner to map
the requirements of the software process - the key process areas
- onto the practices of the process. They can then weight the
effect of each practice on each key process area. From this, they
can establish the priorities which need to be included in any
software process improvement action plan.
5.0 Overview of Model Development
Using the QFD matrix just described (denoted as SPI/HoQ), it is
also possible to create a QFD model which includes Business Processes.
This involves the development of a generic model in three stages
as outlined in Figure 1.
5.1 Stage 1: Voice of Software Development Company
This stage involves identifying, with the software development company their important Business Processes. From customer discussion and use of Affinity Diagrams, some Business Processes which can be identified are:
5.2 Stage 2: Business Process / Key Process Areas
Within Software Process, Key Process Areas are the means by which we should be able to solve our Business Process problems. Based on the BootStrap model developed by the European Software Institute the Key Process Areas can be subdivided into:
Some examples of Project Management Key Process Areas are to:
Building a House of Quality, the Business Processes identified
in Stage 1 are treated as WHATs, and are correlated with the Key
Process Areas (HOWs) from software process improvement. Using
the BootStrap model as a basis, 55 Key Processes have been identified.
Focus groups consisting of Software Process practitioners are
correlating the Business Processes and Key Process Areas.
The outcome from this stage is a matrix containing correlations
between Business Processes and Key Process Areas. This matrix
will be available for use by software development companies as
described in the following section.
5.3 Stage 3: Key Process Areas / Key Practices
When Quality Function Deployment is used within manufacturing,
WHATs are identified as those attributes which the customer requires
in a product. For the purposes of software process, we are no
longer looking at a product, but at a process. Therefore, to identify
WHATs, consideration must be given to the required output - which
is that the company implementing the strategy must follow a particular
standard. This poses the difficulty - which standard should be
used? For the purposes of this research it has been decided to
use the BootStrap standard, but allowing the effects of ISO9000-3
to be included, as this is what many Irish companies are working
towards.
Now, considering BootStrap as the product, the attributes are the Key Process Areas. Therefore in SPI/HoQ, Key Process Areas become WHATs - these are the processes upon which the organisation should focus to improve their Business Processes. However, to do this, they must follow various practices. Many practices can be identified from the Software Process literature. Examples of practices are:
In developing the Software Process model, a total of 155 practices
have been identified.
A crucial part of the House of Quality is to identify the WHAT
vs HOW correlations. These are usually identified as Strong, Medium
and Weak, and for visual purposes these correlations are marked
on the matrix as , , and . They are given values 9, 3, and 1 respectively.
Yoji Akao, the devisor of the QFD technique, states that this
correlation is one of the more troublesome tasks in the QFD process,
since the correlation is often based on experience, intuition,
and determination. (Akao, 1990).
From the Software Process literature, it is also possible to identify practices which have effects on particular processes. An example:
Practice: To ensure software's traceability to requirements
has a strong effect on
Process: The systematic development of detailed design.
Correlations which are explicitly mentioned in the literature
can be identified. The problem which remains is to correlate processes
and practices which are not explicitly mentioned.
5.3.1 Completing the House of Quality
For the purpose of this project, the correct development of the
Software Process Improvement House of Quality (SPI/HoQ) is vital.
The SPI/HoQ is extremely large (55 rows by 155 columns) but it
is imperative that all cells are examined to ensure that every
correlation existing between practices and key process areas are
identified. It is impractical to gather a group of practitioners
and organise a focus group to correlate such a large matrix. Therefore,
alternatives were sought.
Initially, we abstracted the matrix to a higher level, giving
a matrix measuring 6 WHATs by 23 HOWs. Software Process experts
were asked to correlate this matrix, giving only two options -
some level of correlation exists and no correlation exists. Those
cells where no correlation exists were eliminated from the correlation
process at the more detailed level.
Once the high level matrix was complete, the correlations in the
detailed House of Quality was completed by the author. Using various
statistical techniques and expert opinion the model was then validated.
6.0 Using the Quality Function Deployment Software Process
Model
The generic Quality Function Deployment Software Process model
consists of two matrices, the Business Process matrix and the
Software Process Improvement House of Quality. This generic model
can be used by software development companies to provide an action
plan for use by the organisation. The development of the action
plan is represented in Figure 2.
The basis for the model is that organisations will be able to carry out a self-assessment of their own business, both for business processes and key software process areas.
6.1 Stage 1: Measurement of Business Processes
This stage of the model is optional. Businesses will be asked
to focus on their business requirements, and to provide three
measurements: current performance, planned performance, and the
importance of this particular requirement on their business. The
outcome from this section of the model would then be used in implementation
stage 2.
6.2 Stage 2: Measurement of Key Process Areas
Using a self-assessment questionnaire devised by the European
Software Institute, companies can indicate how they currently
perform in each Key Process Areas (KPA). They are also required
to give measurements for their planned performance and the importance
of that key process area to their business.
For those organisations that are pursuing ISO9000-3 certification,
a Market Leverage Factor can be included putting a greater emphasis
on those KPAs which affect ISO-9000-3 requirements. Another feature
of Stage 2 is that a cost / time effective scale can be included.
As an initial step an organisation may be interested in pursuing
those items which would have an immediate effect or items which
are not too costly. The Quality Function Deployment Software Process
Improvement model allows these to be taken into account.
6.3 Stage 3: Developing an Action Plan
Once self-assessment has been input to SPI/HoQ in Stage 2, the
priority practices to be pursued are now identified. These are
then incorporated within a software process improvement action
plan for implementation within the organisation.
7.0 Case Study
The Quality Function Deployment Software Process Improvement model
is currently being used as a tool to aid software process improvement
efforts within some small software development companies in Ireland.
The company discussed in this paper develops software for use
in business, targeting companies of 200 employees or more. They
employ 22 people and have two separate development groups. Like
many software companies, customers expect them to reach standards
within their development process. To develop the action plan,
they used the KPA matrix only, and did not look explicitly at
the Business Processes. As the final matrix is so large (55 rows
by 155 columns) only results using a sub-section of the matrix
are shown. In fact, the organisation use the full matrix to identify
practice priorities.
7.1 Self-Assessment
Initially, a self-assessment of current performance was carried
out by eight software engineers and results were averaged. At
the same time, a software group manager was asked to complete
two other questionnaires, assessing planned performance in two
year's time and the importance of each key process area to the
organisation. Results for 4 Key Process Areas are shown in Table
1.
| KEY PROCESS AREA |
|
|
|
|
| (a) Development of software requirement | ||||
| (b) Development of architectural design | ||||
| (c) Development of detailed design | ||||
| (d) Unit testing & software integration |
Improvement factor was calculated as:
This company are also interested in ISO9000, and had not performed
well on the Inspection and Testing requirement. Therefore, they
have included a Market Leverage for those Key Process Areas focusing
on testing. Total Importance was calculated as:
as shown in Table 2.
| KEY PROCESS AREA |
|
| ||
| (a) Development of software requirement | ||||
| (b) Development of architectural design | ||||
| (c) Development of detailed design | ||||
| (d) Unit testing & software integration |
7.2 Key Process Area Correlation Matrix
These figures are input to the SPI/HoQ matrix. Examining the four
Key Process Areas shown, it can be seen that Unit Testing and
Software Integration is most important. If we had prioritised
based on current performance within the organisation (which would
be acceptable in most organisations), then Development of Detailed
Design would get top priority.
However, from a Software Process viewpoint, we need more than
to know the importance's of each Key Process Areas. Using SPI/HoQ
it is possible to identify those practices which have the greatest
effect on the Key Process Areas. We also require that the company
should be able to implement an action plan. Therefore, they are
interested in ensuring that the practices are prioritised. A subset
of the Software Process Improvement House of Quality for the organisation
is given in Figure 4.
Priorities of the practices are calculated using the overall Importance
of the Key Process Areas and the correlations. Taking for example,
Define the Order of Testing of Units (number 12),
which has one strong correlation, two moderate correlations and
one weak correlations, the calculation is:
1*10.37 + 9*17.28 + 3*13.83 + 3*12.57 = 245.09
The three most important practices for the company to action identified using only this subset of the correlation matrix are:
These three actions account for over 40% of the Importance of
the HOWs.
Using the full SPI/HoQ gives the organisation a full list of priorities
to follow so that they can cause the greatest improvement in their
software process.
7.3 Software Process Improvement without Quality Function Deployment
The matrix in Figure 5 shows the SPI/HoQ containing only those
correlations which were explicitly identified in this literature.
This is how the matrix looked before author input and expert validation.
It can be seen that this method of building the correlation matrix
gives strong correlations along a diagonal, and that the Importance
of HOWs (practices) is therefore based solely on how important
each WHAT (Process) is. The sub-matrix gives a different set of
priorities to be worked on, as it does not reflect that each practices
will often have varying effects on more that one process.
8.0 Conclusion
Measuring whether the difference between the outcomes from the each of the sub-matrices in Figures 4 and 5 is significant or not is a problem faced by the researcher and can only be done through a longitudinal assessment which is currently underway. However, the development of the Quality Function Deployment Software Process model shall provide small software development companies with the ability to do a self-assessment based on the BootStrap model and consequently list the practices which should provide the greatest improvement within their organisation. These prioritised practices are then used to develop an action plan.
8.1 Future Use of Matrix
A number of small software companies are carrying out self assessments
which will allow them to implement action plans established by
using the Quality Function Deployment Software Process model.
At the end of a fixed time period, they will be re-assessed. If
the model is successful in effecting software process improvement
the overall software process maturity of the company should have
improved not only within the company itself, but in comparison
to other small software development companies in Ireland.
Akao, Yoji [Editor-in-Chief] (1990), "Quality Function Deployment
: Integrating Customer Requirement into Product Design",
Productivity Press, Cambridge, Mass.
Buchman, Caroline D. and Bramble, Larry K. (1995), 'Three-Tiered
Software Process Assessment Hierarchy', Software Process - Improvement
and Practice, Vol 1, Issue 2, December.
Draper, Lee, Kromer, Dana, Moglilensky, Judah, Pandelios, George,
Pettengill, Nate, Sigmund, Gary, Quinn, David, (1995), "Use
of the Software Engineering Institute Capability Maturity Model
in Software Process Appraisals", output from CMM v2 Workshop,
February, Pittsburgh, Pennsylvania.
Neupauer, Steve, (1996), "Improving the Software Process",
Presentation to CEEDA 96, 3rd International Conference on Concurrent
Engineering and Electronic Design Automation, 18-19th January,
Poole, U.K.
Paulk, Mark C., Weber, Charles V., Garcia, Suzanne M., Chrissis,
Mary Beth, Bush, Marilyn (1993), "Key Practices of the Capability
Maturity Model for Software, Version 1.1", Technical Report
SEI-93-TR-25, Software Engineering Institute, Carnegie Mellon
University, U.S.A.
Peterson, Bill, (1995a), "Transitioning the CMM into Practice",
Proceedings of SPI 95 - The European Conference on Software Process
Improvement, The European Experience in a World Context, 30th
Nov-1st Dec, Barcelona, Spain, pp. 103-123.
Peterson, Bill (1995b), "Software Engineering Institute",
Software Process - Improvement and Practice, Pilot Issue, August.