The usage of metrics to measure all aspects of business has gained ground in this age of Big Data and Business Analytics. This trend has been incorporated into the management system for development in many companies. Regular operational reviews of organizations include a deep dive into the development process where measures are taken of things like bugs. These metrics are reviewed on a quarterly or even monthly basis and used by the highest levels of a company to drive decisions on strategy, funding, and resources. Given the ability to gather more and more information, such an operational posture seems to make the most sense for companies today.
Metrics are gathered from a wide variety of sources and posted on a regular basis on dashboards which are then used to manage functions. In the development environment, these metrics do provide a strong view to how the business is performing in the short term. However, if these metrics are not completely aligned with the goals of the business, they drive less than optimal results.
Product development is an area which has been challenging to manage effectively with metrics. Remember, in most organizations, a development organization’s primary goals are to provide the business with products or solutions that generate profits and growth over the long haul. This involves managing products through a long term process where measurements of operations will vary depending on the stage of development of each product. If you want to truly want to come out with products that are differentiated (and therefor, able to drive more profits) you need to innovate. And innovation is a notoriously difficult thing to measure accurately or in a timely manner.
Often development metrics used by enterprises often have the following limitations:
Metrics are gathered from a wide variety of sources and posted on a regular basis on dashboards which are then used to manage functions. In the development environment, these metrics do provide a strong view to how the business is performing in the short term. However, if these metrics are not completely aligned with the goals of the business, they drive less than optimal results.
Product development is an area which has been challenging to manage effectively with metrics. Remember, in most organizations, a development organization’s primary goals are to provide the business with products or solutions that generate profits and growth over the long haul. This involves managing products through a long term process where measurements of operations will vary depending on the stage of development of each product. If you want to truly want to come out with products that are differentiated (and therefor, able to drive more profits) you need to innovate. And innovation is a notoriously difficult thing to measure accurately or in a timely manner.
Often development metrics used by enterprises often have the following limitations:
- Short Term - Metrics are generally short term. Metrics gathered and monitored on a short term basis tend to drive short term actions. Typical metrics include development costs, productivity, error rates, and schedules. All of these are key things to track, but they don’t take into account the entire product development process or provide means assess long term performance. Remember that the key to long term development is how well the products meet the market / customer needs and how well they sell. Since product development cycles are often measured in years, they are not captured adequately each month or quarter.
- Easy - Metrics are ‘easy’ to gather. Often companies identify metrics that can be extracted from IT systems related to the Product Development Process. This means that any parts of the process that aren’t captured in the IT systems, are hard to measure. Metrics related to innovation, new product areas, requirements gathering, and collaboration are notoriously difficult to measure accurately and generally don’t come directly from IT systems. Metrics related to costs, productivity, and on time delivery are easier to capture and often become the basis for any development metrics. The net result is that metrics related to reducing costs overwhelm measures of developing products that generate the most growth.
- Multiple Sources - Development metrics often come from multiple sources. These include systems to track parts or code, quality systems, HR systems, financial systems, and customer satisfaction systems. Leveraging multiple sources of data without normalizing does not present an accurate view of performance, but the process of normalization is not always easy or even possible.
- Impede Innovation – As shown above, the use of metrics that are often gathered for development monitoring do not have the longer term horizon or provide for measures of growth in a way that foster innovation. In fact, they tend to favor strategies that will reduce your business’ innovation and therefor reduce competitive advantages that are needed to grow profitably.