Great Metrics Don’t Make Great Teams

As an Agile Coach, a common management request I receive is to provide metrics on how teams are performing. This is usually followed by my being asked why we need dedicated Scrum Masters and Agile Coaches?   If you are a sports fan like myself, it seems obvious – great teams have great coaches and managers. How many championship teams do you know of that don’t have any coaches?  Why then do we expect to create world class Software Development teams that magically coach themselves?

Organizations that are early Agile adopters tend to cling to a legacy “command and control team metrics” mindset.  Though well intentioned, it is usually perceived by teams as lacking trust and micro-managing.  Metrics are very important, particularly when generated from within the team and leveraged to contemplate what they can do to improve.  Focusing on individual metrics however, especially those imposed on the team rather than generated within by a well coached team can risk missing the forest for the trees.  The danger of focusing myopically on individual metrics (particularly early in the Agile Adoption journey) are:

1. Most metrics can be easily gamed.

I worked in an organization that was focused relentlessly on utilization.  It was amazing how each week team members would estimate 40 hours of work and every week their actuals were 40 hours.  Either they were the world’s best estimators or….   I call it the “Darwin Metric,” i.e. if you are unable to figure out how to game this metric then you should be let go.  Instead of focusing on utilization, focus on creating a safe and trusting environment where team members share they have additional bandwidth and volunteer to help other team members.  When you create an environment where people are fearful of sharing that they have no work, they are more likely to game the system vs. offering to help and support the team  and thus deliver more value to your organization.

2. Choose what you measure wisely – what you choose will seem to improve

Ask yourself if improving this particular metric is what you need to focus on to move the needle or if there is a deeper root cause issue that needs to be addressed.

3. Metrics can be used to micro-manage teams

Metrics can force teams to be defensive versus to leverage them as information radiators to spark conversations around how to learn, improve and adjust.

So, how do we get from metrics that leave a bad taste in teams mouths to metrics that taste great and empower teams to get better and deliver more value and customer delight ?

The Secret Sauce of Team Metrics

Where I grew up, the sauce my Italian friend’s Mom made was referred to as gravy.  Our gravy can be boiled down to three ingredients –  objectives, goals and outcomes.

1. Team Objectives – If you do not have team objectives, I highly suggest creating them.  Objectives will differ by team type –  “run the business team”, new product team, start up team or operations team each will require different metrics depending on their objectives.  Objectives also help teams in prioritizing work and deciding what work falls outside their purview.  When prioritizing stories for an upcoming sprint, having objectives helps the decision making process on which stories make the cut.

2.  Metric Goals:
   • Predictability – Defining and consistently accomplishing sprint goals can be leveraged to show team consistency.  Define 3 – 5 key goals for each sprint on which the team can base creation/prioritization of stories for the upcoming sprint.  It is also a metric that is less likely to be gamed. Team Velocity Trends and Team Project Burnups can also demonstrate consistency over time.
   • Productivity – I like to use (total story points / capacity).  This metric helps to normalize teams that fluctuate in size and is particularly useful during holiday seasons in preventing teams from over committing when capacity tends to be lower.
   • Value Delivery –  Scaled Agile has great examples for sharing value delivery – Metrics Abstract, Objective Metrics,  and a spot on outcome summary.
   • Quality – Production / Test environment defects opened/closed by severity and age, unit test and test automation coverage.
   • Cycle Time –  A great information radiator that can help teams break work down into smaller chunks, discuss outlier stories that took significant time to complete and actions that can be taken in the future to improve.
   • Innovation – Check out Why Measuring Innovation Matters by Brian Quinn for insightful ideas on innovation metrics.  A simple example can be how many “A/B tests” did the team run that quarter and their impact.
   • Happiness  – Very simple to do.  Check out Scrum Inc. thoughts on Henrik Kniberg’s Crisp Happiness metric for simple yet powerfully effective example.  This can be very scary to do for fear of what the team will say.  However, I encourage taking the risk to make feedback transparent so that the team can improve and address challenges sooner rather than later when it is too late.

Depending on your team objectives a combination of the above and others can be used.  The challenge that I see organizations facing is that they want to choose a couple of silver bullet metrics and apply them to all of their teams generically.   Without understanding team objectives and outcomes this is a recipe for chasing after metrics that in the best case, add little value and in the worst case, impact value delivery.

3. Outcomes

In an organization’s Agile journey, having teams coached from within by experienced ScrumMasters/Coaches to create clear goals and outcomes empowers the selection of meaningful team metrics that support achieving greater organizational objectives.  An  example outcome for an organization focused on predictability could be to empower teams to plan, coordinate and deliver enough predictably to make release level commitments.

Metrics can be leveraged to support teams in their journey to agility, but they are lagging indicators that need to be chosen carefully with clearly defined team objectives, team outcomes and team buy-in.  I encourage organizations instead to put their focus and attention on nurturing great teams built on a climate characterized by interpersonal trust and mutual respect.  Stay tuned as our next post will be on “What Makes Great Teams”.   Until then I highly recommend a recently published NY Times article – “What Google Learned From Its Quest to Build the Perfect Team” by Charles Duhigg.

Now, I’d love to hear what metrics, objectives, outcomes you have used to support your teams to be great. Please share in the comments below.

Links to posts on metrics:

1. Metrics for Scrum Teams by David Koontz
2. Do you want Crappy Agile by Ron Jeffries
3. Scrum Metrics by Robert Boyd