Skip to content

New pattern: Misusing Metrics #501

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

New pattern: Misusing Metrics #501

Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
641dd89
Create misusing metrics.md
Trueblueaddie Dec 4, 2022
fcc537b
Fix linter issues
spier Dec 6, 2022
043a710
Formatting as list
spier Dec 6, 2022
b9b8509
Formatting as list
spier Dec 6, 2022
7830b7c
Formatting as list
spier Dec 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions patterns/2-structured/misusing metrics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
## Title

Misusing Metrics

## Patlet

Metrics can be misused causing friction across teams. In these situations, create a sense of ownership within teams on how to use the data, avoiding comparisons, and begin measuring progress with deltas.

## Problem

The existence of metrics may lead to undesired situations and frictions across teams. Gamifications, list of rock stars or any other way of having people or teams “better” than others usually brings into the discussion defensive behaviors where the metric needs to be deprecated.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "brings into the discussion defensive behaviors where the metric needs to be deprecated" sounds strange to me.

If you would split this up in multiple sentences, how would you phrase it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existence of metrics may lead to undesired situations and frictions across teams. For example, implementing a gamifications approach which elevates some contributors or teams as "better" than others has a negative impact to the internal culture. Utilizing metrics as a weapon to indicate performance results in defensive behaviors or disengagement. These delay and inhibit successful and comprehensive understanding of the value of metrics. Ultimately, teams and individuals become weary of metrics all together.

How is that? In retrospect, "metrics being deprecated" needs to be added to the solutions below.

Copy link

@ranton256 ranton256 Dec 15, 2022 •
edited by spier
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Negative consequences of metrics is a great thing to tackle. Thanks for writing this up. Gamification of metrics definitely leads to some strange (and sometimes unwanted) behaviors in organizations sometimes, i.e. "promotion driven development" as described in https://twitter.com/gergelyorosz/status/1442162670753431559?lang=en

Another lens to view this through, is that incentives, intentional or otherwise, almost always have second order effects that may be desirable or undesirable besides the intended consequences. I think Kent Beck's blog has some good writing about this, but I can't find a good reference online to share other than https://geekincentives.substack.com/p/the-geek-incentives-manifesto which is very brief.

I think it's important to note is that these unintended consequences can lead to negative outcomes in an organization even when the incentives had good intentions and the ones affected by them did as well.

Using the "weapon" phrasing may imply that bad outcomes are intended by bad actors, whereas I think in reality, bad actions are often the unintended consequences of well meaning people, or at least people with an average amount of healthy-self interest in advancing their career or project, but in an organization there are always competing projects and other peers seeking similar outcomes that need the same resources (money, time, attention, budget, etc.)

That's not to say that there are never bad actors, just that things actually can go wrong without requiring it as a precondition.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the references and calling out the word choice. I'll rephase.


## Context

There is often a competitive spirit within development teams. In these situations, metrics tend to be used as evidence that supports a “better than” culture. This culture is a barrier to the healthy use of metrics to inform the successful progress in development velocity.

## Forces

* Individuals or teams may not understand the purpose of the metrics being used
* Individuals or teams may feel threatened by how the metrics are being used
* Leadership doesn’t understand how to utilize the metrics in a healthy way

## Solutions

Get a team internal agreement about the metrics that make sense in that specific team. Build a sense of ownership so everyone has contributed to the definition of the metric. Educate people about the right use of metrics and avoid comparisons. Measure deltas instead of net numbers, and look for outliers and how far projects are from the average instead of stating some behavior is “good” or “bad”.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Get a team internal agreement about the metrics that make sense in that specific team.

I am confused here. Are we talking about a metric that is only measured in a single team?
Or a metric that applies to a whole org, and is measured for each team belonging into that org?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adoption of metrics should start on the team level. First educate people about the best use of metrics i.g. avoiding comparisons, measure deltas instead of net numbers, look for outliers, estimate progress from the average or beginning, do not use "good" or "bad" as descriptions for individual or team performance. By building a common understand, everyone develops a sense of ownership because they contributed to the definition of the metric. Understand and utilization of a common metrics language starts with the individual then builds out to teams and finally entire organizations.


## Resulting Context

Everyone being tracked and ongoing tracking are aligned into the existing and needed metrics. These metrics are publicly documented and anyone can learn from them. An existing set of metrics are part of the usual work of the teams with the goal of improving efficiency.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds more like it belongs into the Solutions section.

Quoting from the pattern template, this section should focus on:

What is the situation after the problem has been solved? The original context is changed indirectly by way of the solution.

So e.g. if in the Problem is that the unhealthy competition between teams was toxic, what is the behavior of the teams after the Solution is in place?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Then this "By building a common understand, everyone develops a sense of ownership because they contributed to the definition of the metric. Understand and utilization of a common metrics language starts with the individual then built out to teams and finally the entire organization. This strengthened the internal culture around metrics as a tool for best development practices. " would be best for #resulting context.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, probably.


## Known Instances

Santander Bank

## Status

Structured
Published internally in Santander Bank; drafted via pull-request in December of 2022

## Authors

Alberto Pérez García-Plaza
Daniel Izquierdo Cortázar
Addie Girouard

## Acknowledgements

Igor Zubiaurre