Performance Measurement for Real Agility

You need to measure performance within your business. At the organization level, we have many standard financial metrics that help us to do that. But it’s harder to do at the team or project level, especially when you’re also trying to become more Agile. Here are some tips and ideas about performance measurement for Scrum projects and other delivery-level activities.

Business Performance

Hopefully, you have a Product Owner or similar type person who is accountable for business performance in your delivery-level work. This is a business person making business decisions to ensure top-notch business results. But how do they know?

At the most fundamental level, good business results are measured with return on investment. So, if you have a Scrum team doing Sprints, you measure how much it costs to run that team for a Sprint, and then measure how much value that team delivers.

Depending on the team this can be easy… or very very hard.

Leading Metrics

Ideally, we would find some sort of reliable leading metric to be able to predict the total future value of the work delivered in a Sprint. Ideally, this metric would be measured as a Net Present Value (NPV) so that it is directly comparable to the immediate cost of the Sprint. Here are some examples of metrics that you could use to predict the value of the work:

• Estimated value of the work items delivered (e.g. Product Backlog Items completed) as determined by business stakeholders (e.g. using the Bucket System estimation technique or the Buy a Feature technique).
• Lead time to complete the work items delivered where shorter lead times indicate higher likely future value. Here, the time would have to be converted to dollars, likely using some sort of heuristic.
• Customer review feedback scores such as Net Promoter Score (NPS) taken immediately at the time the work is delivered to customers. Again, a conversion from NPS to NPV is needed.
• Pre-paid or contracted customer orders for work items delivered.
• Estimated or calculated cost of failure or cost of delay for not delivering the work items of the Sprint. You can convert to value by calculating the NPV of the incremental costs of delay over the usable lifetime of your work.

Not all of those will apply to every situation. Ideally, you want to measure actual business value delivered. Your business will probably struggle to do so at the granularity you need for performance measurement at the team or project or Sprint level. In order to measure actual value delivered, you must trace revenue or cost savings backwards in time and allocate that value to appropriate Sprints. In other words, at best, you can measure actual value only for work that is long-finished.

Efficiency and Performance

Good teams work at 80% or better efficiency. Normal teams work at 30 or 40% efficiency. Bad (common) teams work at 20% efficiency or worse. So, even if you can’t directly measure business value, if you can measure efficiency and improve it, then you can theoretically improve business performance. But how do we measure efficiency?

First, you must clearly understand the distinction between value-added activities and non-value-added activities. A value-added activity is any activity that meets two important conditions:

1. The activity changes the form, the fit or the function of your product, service or system. In other words, users and customers can see the results of the activity.
2. The customer (who pays for the product or service) would accept the cost of the activity as part of the pricing of the product or service, if that cost was known to the customer. This does not mean that you have to show the customer a detailed breakdown of costs… it’s more a hypothetical.

We can summarize these two conditions in the phrase “customer recognizable value”.

Measuring Efficiency

Now that we can identify value-added activities, we can look at the way that work is done to deliver results to customers and track value-added vs. non-value-added activities. In order to calculate an efficiency, we want to find the proportion of time spent on the two types of activities. Here are some things to consider for non-value-added activities:

• fixing quality problems
• waiting (work waiting in queues, people waiting for dependencies)
• documentation and bureaucratic meetings
• rework
• interruptions (even if that interruption is valuable on its own)
• task-switching and hand-off times
• discarded work (overproduction, gold-plating)

One technique for doing this assessment is called value-stream mapping.

If your organization improves efficiency from 40% to 80%, then it should be delivering twice as much value per unit of time as before the efficiency improvements. Reaching this level of efficiency is hard, but possible. (That’s a big aspect of our Real Agility™ consulting and coaching services!)

Culture and Happiness

Your organization depends on culture. The overall sustainability of your organization’s agility and the employee’s happiness depends on culture. Measuring culture is hard. There are dozens of well-known cultural assessment tools, and hundreds of assessment tools that can show different aspects of culture. Unfortunately, there is no direct culture metric to show the performance of your organization’s culture or how well a team is aligned to that culture.

Nevertheless, we can get some clues and use some techniques to help us see into culture. (PS. I recommend the book “The Corporate Culture Survival Guide” for great background on this topic.) One of the main clues into culture comes from an organization’s leadership. We can use this idea to either:

• assess the mindset and behaviour of leaders directly and use this information to figure out culture, or
• assess the gap between stated values and behaviour and use this to understand culture.

The second approach is generally more effective since asking people about culture directly often leads to unusable information; like the proverbial fish in water, we aren’t aware of culture and struggle to describe it even when asked.

Dimensions of Culture

There are five dimensions of culture that when improved, will lead to greater levels of agility in practice. These dimensions are:

• Improvement culture indicates how easily and frequently people within the organization take the initiative to work on systematic improvement initiatives, vs. temporary problem-fixing and bandaid solutions.
• Focus culture refers to the ability of people within the organization to work in an uninterrupted fashion towards a single clear objective vs. the level of multi-tasking, interruption and distraction.
• Urgency culture indicates the positive feeling of urgency that means that people in the organization care about the timeliness of delivering solutions (sooner rather than later) and have a strong emotional connection to the need to deliver vs. urgency related to technical or business emergencies coming from individual agendas.
• Vision culture is the degree to which people in the organization are working towards “transcendent” goals (related to long-term, organization-level, service-oriented aspects of work) vs. individual and possibly self- serving goals such as career development, financial success or even supporting a family.
• Teamwork culture refers to the degree to which the people in the organization work in well- established teams with strong mutual commitment, collaboration and communication vs. working as individuals with isolated goals, management directives, or a team-disruptive environment.

The BERTEIG Real Agility(™) Assessment tool is a survey and analysis tool that helps organizations to learn how they measure on these five dimensions. Organizations also receive advice on how to improve their level of cultural agility. Agile teams can use this tool to self-assess as long as they are able to involve stakeholders and the team’s immediate leadership (e.g. functional managers) in the survey portion of the assessment.

Generally speaking, improving the scores on these five dimensions will lead to greater levels of team and organizational agility. When you have great scores, you will have better business performance. Most importantly, when your culture is aligned with Real Agility™, your staff, managers and leaders will all be happier.

Conclusion

Many organizations struggle to understand the performance of their Agile implementations. Measuring performance is not easy – particularly since many traditional metrics and KPIs are either irrelevant or even inimical to a good Agile environment. The key areas to consider are business value, efficiency and culture.