Program Management 7

Published in PM World Today - July 2007 (Vol. IX, Issue VII)

Managing Programs to Success: Key Program Management Metrics (Part 7 of a Series) By: Russ Martinelli and Jim Waddell Parts 1 - 6 in this series available at: http://www.pmworldtoday.net/featured_papers/2007/july.htm#2 Introduction Humans know that measures such as their blood pressure, cholesterol, blood sugar, and white blood cells levels reflect the state of their health. Similarly, program managers know that metrics such as time-to-money, development cost, gross profit margin and profitability index are measures that reflect the health of their program. It is commonly understood that one of the key rationales for using metrics is that what gets measured warrants attention and gets improved. In particular, using program performance metrics will help program managers, their sponsors, and other stakeholders understand how well a program is performing, where and why a program has problems and tailor actions to eliminate the problems. This will, in turn, improve the program and keep it on track toward achievement of the objectives. Program metrics not only measure the health of individual programs but also show the effectiveness of program management-related processes, such as strategic management and portfolio management. In this manner, program management metrics are an effective means to integrate and synchronize strategy, planning, and execution activities. In the final paper of this series, we’ll look at how to design the right set of metrics for a program, describe how strategy drives the metrics selection process, and give some guidance as to how many metrics are needed to fully comprehend the performance of a program. An extensive set of program management metrics can be found in our book titled, Program Management for Improved Business Results (ISBN: 0-471-78354-4). Designing a Metrics Solution Recent studies cite the use of a comprehensive set of metrics as a key factor in program and project success1. Programs using comprehensive metrics to measure and monitor performance will have fewer problems, hence higher success in accomplishment of program goals. Another study found that companies using consistent, tiered, balanced, and mutually aligned metrics outperformed companies who used sporadic, non-tiered, scheduleoriented, and non-aligned metrics2. The underperforming companies still favored schedule-only metrics that are sporadically utilized, normally when a program is in trouble. The use of program metrics is an institutionalized practice of leading companies. They focus on regular, periodic measurements of program and business performance. Figure 1 demonstrates the key differences between companies that methodically design their program metrics set (which we refer to as best-in-class companies), and those that take an ad hoc approach (which we refer to as rest-in-class companies). Best-in-class companies experience both a higher rate of program goal accomplishment and produce a higher quality metric set – with quality being qualified by usefulness3. A detailed comparison between the best-in-class and rest-in-class companies revealed that the best companies engineered and installed a system of metrics that were balanced and mutually aligned. Balanced metrics are those that cover all dimensions of a program including schedule-oriented metrics, as well as those for financial, customer, process, and human resource utilization. Balanced metrics also include both leading and lagging PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net

Page 1

Published in PM World Today - July 2007 (Vol. IX, Issue VII)

metrics. Leading metrics are forward-looking, such as a projected program finish date based upon the rate of milestone completion. Lagging metrics, such as percentage of deliverables completed, are most valuable for a retrospective view of program performance. Both leading and lagging metrics are important and useful. Mutually-aligned means that metrics are compatible, using the same baseline information. For example, performance-to-planned schedule, probability of completing the program by a certain date, and the cumulative percentage of milestones accomplished are all based on the same baseline – the program schedule.

Source: Program Management for Improved Business Results

Figure 1: Best- versus rest-in-class metrics design To reflect how metrics vary across firms and industries, successful companies create metric tiers that differentiate between various types of metrics. Tiering allows the companies to categorize a diverse continuum of programs and projects, and define specific, as well as common, metrics for each category. For example, one medical equipment manufacturing company uses three tiers for its research and development programs: basic research programs (tier 1), technology development programs (tier 2), and product development programs (tier 3). Metric Selection Each company will need to define a distinct set of metrics, depending on its business strategy. Therefore, the set of metrics will vary by the type of industry the company is in and by its particular business strategy - e.g., differentiation, cost leadership, or best-cost strategy4. When a business unit determines its own distinct set of metrics, it should do so in collaboration with company stakeholders. Each stakeholder will emphasize different metrics according to their needs. PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net

Page 2

Published in PM World Today - July 2007 (Vol. IX, Issue VII)

•

Business management stakeholders will show a strong interest in metrics that assess the strategic health of the business unit, the alignment of development programs with business strategy, and the balance of the program portfolio relative to the business unit objectives and needs.

•

Program management directors will be concerned with metrics assessing the portfolio of programs and the management of individual programs.

•

Members of program core team and project teams will be most keenly interested in metrics addressing their specific project management and team execution goals.

The best way to demonstrate strategy-based metric selection is through example. Let’s look at three companies with three different business strategies. In Table 1, Company A utilizes a differentiation strategy to offer its customers something different from their competitors. In particular, Company A focuses on technology innovation and fast time-to-market to achieve differentiation and gain higher profits and market share. Secondarily, Company A strives to achieve high quality and doesn’t allow runaway cost. To measure performance to their strategy, Company A selects metrics for measuring time-to-market, percentage of new technologies developed, profitability index, and market share. Additionally they choose customer fallout and development cost metrics to measure their secondary strategy of providing high quality and cost containment. Company A

Company B

Company C

Company Business

High-tech electronics

Conventional manufacturing

Automobile company IT

Business Strategy

Differentiation

Cost leadership

Best-cost

Type of programs

New product development

New product development

Information technology

Time-to-market

Development cost

Defect rate assessment

Percentage of new technologies developed

Manufacturing cost

Customer fallout

Profitability index

Market share

Development cost

Market share

Staffing level

Manufacturing cost

r

Sales volume

M

e

Gross profit margin

t

i

c

s

First Priority

Second Priority Customer fallout

Development time

Development time

Manufacturing cost

Defect rate assessment

% of new technologies to company

Development cost Source: Program Management for Improved Business Results

Table 1: Strategy drives metric selection

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net

Page 3

Published in PM World Today - July 2007 (Vol. IX, Issue VII)

Company B focuses on a low cost strategy aimed at establishing a sustainable cost advantage over their rivals. The intent is to use the low cost advantage as a way of under pricing rivals, and capturing market share. This strategy will leave company B with small profit gross margin per product, but profit can be achieved through a large volume of sales and lean staffing. Metrics to support this business strategy should measure development and manufacturing cost, market share, program staffing level, gross profit margin, and sales volume. Secondary objectives are to bring the product to the market in an average industry standard time and with an average quality. This is complemented with the second priority metrics of development time and defect assessment rate. The best-cost strategy pursued by Company C combines upscale features with low cost. Company C aims to become a low-cost provider of information technology products that have high quality features. A secondary strategy is to develop innovative information technologies within reasonable time frames. Primary metrics to measure this strategy should include defect rate assessment, customer fallout, development cost, and manufacturing cost. Secondary metrics could include percent of new technologies developed and development time. These examples illustrate how business strategy drives the choice of metrics. We showed only three example companies with three different strategies and their corresponding sets of metrics chosen. Every company will have its own unique business strategy, and each needs to have its own unique set of program management metrics to measure the effectiveness and achievement of that business strategy. How Many Metrics Do You Need? The answer to this question of course depends on the nature of the program. The strategy, culture and values of the company, as well as how big, complex, technically and commercially risky a program is all play a major role in deciding how many metrics you need. Additionally, accelerated time-to-money objectives for the program will require special attention to how the program is managed and what metrics will best assist management in keeping it on track. As a general rule, one needs a sufficient number of metrics to maintain good control of a program, and as few as possible to still stay nimble. How many is that for an “average” program? Some examples may provide guidance. In one successful high tech company, program teams have eight standard metrics agreed upon with senior management, in another company programs use between 10 and 14 metrics. In a leading semiconductor company such as Intel, very complex fab building programs evaluate about 20 metrics each month. A study by BusinessWeek and the Boston Group showed that the sweet spot is somewhere between 8 and 12 metrics5. Our personal experience supports this finding, we find that about ten metrics are sufficient if an effective metrics system is constructed. Conclusion As we have consistently stated, program management is about achieving business success. Therefore, the purpose of program management metrics is to help improve the business results of the organization. To serve this purpose, metrics need to be carefully developed to ensure they are compatible (or mutually aligned), balanced, consistently utilized, tiered, and are driven by the business strategy of the company. This completes our seven-part series on program management. Throughout this series, we explored various aspects of program management that we feel are most important in managing programs to success. We thank David Pells for the opportunity to create this series of papers, and we hope you have enjoyed reading this series as much as we have enjoyed writing it! Feel free to contact us with your questions and comments – we’d like to hear from you. References

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net

Page 4

Published in PM World Today - July 2007 (Vol. IX, Issue VII)

1. Hauser, J. and F. Zettelmeyer, “Metrics to evaluate R&D,” Sloan Working Paper #3934, MIT, (October 1996). 2. Milosevic, Dragan Z., R.J. Martinelli, J.M. Waddell, Program Management for Improved Business Results, Hoboken, NJ: John Wiley & Sons, 2007. 3. Tipping, J.W., E. Zeffren, et al., “Assessing the Value of Your Technology,” Research Technology Management, Vol. 38, No. 5 (1995): 22-39.

4. Porter, M. E., Competitive Strategy: Techniques for Analyzing Industries and Competitors, 1st edition. New York, NY: Free Press Publishers, 1998. 5. McGregor, Jena. “The World’s Most Innovative Companies,” Business Week Online (April 24, 2006).

©

Copyright 2007 by the Program Management Academy (www.programmanagement-academy.com)

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net

Page 5

Published in PM World Today - July 2007 (Vol. IX, Issue VII)

Russ Martinelli Author

Russ Martinelli is the Manager of Program Management Methods within the Corporate Platform Office at Intel Corporation, where he focuses on the definition and implementation of program management practices across Intel. Additionally, Russ is the chairman of Intel’s global Program Management Community of Practice, an adjunct professor at the University of Phoenix, and co-founder of the Program Management Academy. Russ has held a variety of positions at Intel and Lockheed Martin in the areas of systems engineering, general management, operations management, and project and program management. Contact Russ at: mailto:[email protected]

Jim Waddell Author

Jim Waddell is an independent consultant specializing in program management and mergers and acquisitions. He is the former Director of Program Management for Tektronix Inc. where he established and led the Tektronix’s first worldwide Program Management Office (PMO). Additionally, Jim is an adjunct professor at the Oregon Graduate Institute, a founding member of the Program Management Forum in Portland, and the cofounder of the Program Management Academy. Jim has held a wide range of managerial and operational roles ranging across engineering, marketing, systems and manufacturing in the high tech and energy industries. Contact Jim at: [email protected]

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net

Page 6