Software Engineering-Establishing Software Metrics Program

The Software Engineering Institute has developed a comprehensive guidebook for establishing a “goal-driven” software metrics program. The g...

The Software Engineering Institute has developed a comprehensive guidebook for establishing a “goal-driven” software metrics program. The guidebook suggests the following steps:

1. Identify your business goals.
2. Identify what you want to know or learn.
3. Identify your subgoals.
4. Identify the entities and attributes related to your subgoals.
5. Formalize your measurement goals.
6. Identify quantifiable questions and the related indicators that you will use to help you achieve your measurement goals.
7. Identify the data elements that you will collect to construct the indicators that help answer your questions.
8. Define the measures to be used, and make these definitions operational.

9. Identify the actions that you will take to implement the measures.
10. Prepare a plan for implementing the measures.

A detailed discussion of these steps is best left to the SEI’s guidebook. However, a brief overview of key points is worthwhile. Because software supports business functions, differentiates computer-based systems or products, or acts as a product in itself, goals defined for the business can almost always be traced downward to specific goals at the software engineering level.

For example, consider a company that makes advanced home security systems which have substantial software content. Working as a team, software engineering and business managers can develop a list of prioritized business goals:

1. Improve our customers’ satisfaction with our products.
2. Make our products easier to use.
3. Reduce the time it takes us to get a new product to market.
4. Make support for our products easier.
5. Improve our overall profitability.

The software organization examines each business goal and asks: “What activities do we manage or execute and what do we want to improve within these activities?” To answer these questions the SEI recommends the creation of an “entity-question list” in which all things (entities) within the software process that are managed or influenced by the software organization are noted. Examples of entities include development resources, work products, source code, test cases, change requests, software engineering tasks, and schedules. For each entity listed, software people develop a set of questions that assess quantitative characteristics of the entity (e.g., size, cost, time to develop). The questions derived as a consequence of the creation of an entity-question list lead to the derivation of a set of subgoals that relate directly to the entities created and the activities performed as part of the software process.

Consider the fourth goal: “Make support for our products easier.” The following list of questions might be derived for this goal

• Do customer change requests contain the information we require to adequately evaluate the change and then implement it in a timely manner?
• How large is the change request backlog?
• Is our response time for fixing bugs acceptable based on customer need?
• Is our change control process  followed?
• Are high-priority changes implemented in a timely manner?

Based on these questions, the software organization can derive the following subgoal: Improve the performance of the change management process. The software process entities and attributes that are relevant to the subgoal are identified and measurement goals associated with them are delineated.

The SEI provides detailed guidance for steps 6 through 10 of its goaldriven measurement approach. In essence, a process of stepwise refinement is applied in which goals are refined into questions that are further refined into entities and attributes that are then refined into metrics.