New pattern: Misusing Metrics #501
Conversation
Trueblueaddie
requested review from
lenucksi,
NewMexicoKid,
cewilliams,
spier and
robtuley
as code owners
spier
added
the
📖 Type - Content Work
There was a problem hiding this comment.
Thank you for sharing this pattern.
My main question here is how specifically the pattern relates to InnerSource?
In its current form it reads more like it applies to any form of metric, and not specifically metrics related to InnerSource.
Is there a Story that you can share about a problem that occurred at this org, which was then improved upon by establishing this pattern/practice?
I would further suggest to move this pattern to folder patterns/1-initial. That way we can give this pattern some time to be vetted by the community before it goes live in our online book.
I left some inline suggestions as well. I will resolve the minor ones that are just about formatting myself, so that you can focus on the rest.
|
|
||
| ## Problem | ||
|
|
||
| The existence of metrics may lead to undesired situations and frictions across teams. Gamifications, list of rock stars or any other way of having people or teams “better” than others usually brings into the discussion defensive behaviors where the metric needs to be deprecated. |
There was a problem hiding this comment.
The "brings into the discussion defensive behaviors where the metric needs to be deprecated" sounds strange to me.
If you would split this up in multiple sentences, how would you phrase it?
There was a problem hiding this comment.
The existence of metrics may lead to undesired situations and frictions across teams. For example, implementing a gamifications approach which elevates some contributors or teams as "better" than others has a negative impact to the internal culture. Utilizing metrics as a weapon to indicate performance results in defensive behaviors or disengagement. These delay and inhibit successful and comprehensive understanding of the value of metrics. Ultimately, teams and individuals become weary of metrics all together.
How is that? In retrospect, "metrics being deprecated" needs to be added to the solutions below.
There was a problem hiding this comment.
Negative consequences of metrics is a great thing to tackle. Thanks for writing this up. Gamification of metrics definitely leads to some strange (and sometimes unwanted) behaviors in organizations sometimes, i.e. "promotion driven development" as described in https://twitter.com/gergelyorosz/status/1442162670753431559?lang=en
Another lens to view this through, is that incentives, intentional or otherwise, almost always have second order effects that may be desirable or undesirable besides the intended consequences. I think Kent Beck's blog has some good writing about this, but I can't find a good reference online to share other than https://geekincentives.substack.com/p/the-geek-incentives-manifesto which is very brief.
I think it's important to note is that these unintended consequences can lead to negative outcomes in an organization even when the incentives had good intentions and the ones affected by them did as well.
Using the "weapon" phrasing may imply that bad outcomes are intended by bad actors, whereas I think in reality, bad actions are often the unintended consequences of well meaning people, or at least people with an average amount of healthy-self interest in advancing their career or project, but in an organization there are always competing projects and other peers seeking similar outcomes that need the same resources (money, time, attention, budget, etc.)
That's not to say that there are never bad actors, just that things actually can go wrong without requiring it as a precondition.
There was a problem hiding this comment.
Thanks for the references and calling out the word choice. I'll rephase.
|
|
||
| ## Solutions | ||
|
|
||
| Get a team internal agreement about the metrics that make sense in that specific team. Build a sense of ownership so everyone has contributed to the definition of the metric. Educate people about the right use of metrics and avoid comparisons. Measure deltas instead of net numbers, and look for outliers and how far projects are from the average instead of stating some behavior is “good” or “bad”. |
There was a problem hiding this comment.
Get a team internal agreement about the metrics that make sense in that specific team.
I am confused here. Are we talking about a metric that is only measured in a single team?
Or a metric that applies to a whole org, and is measured for each team belonging into that org?
There was a problem hiding this comment.
Adoption of metrics should start on the team level. First educate people about the best use of metrics i.g. avoiding comparisons, measure deltas instead of net numbers, look for outliers, estimate progress from the average or beginning, do not use "good" or "bad" as descriptions for individual or team performance. By building a common understand, everyone develops a sense of ownership because they contributed to the definition of the metric. Understand and utilization of a common metrics language starts with the individual then builds out to teams and finally entire organizations.
|
|
||
| ## Resulting Context | ||
|
|
||
| Everyone being tracked and ongoing tracking are aligned into the existing and needed metrics. These metrics are publicly documented and anyone can learn from them. An existing set of metrics are part of the usual work of the teams with the goal of improving efficiency. |
There was a problem hiding this comment.
This sounds more like it belongs into the Solutions section.
Quoting from the pattern template, this section should focus on:
What is the situation after the problem has been solved? The original context is changed indirectly by way of the solution.
So e.g. if in the Problem is that the unhealthy competition between teams was toxic, what is the behavior of the teams after the Solution is in place?
There was a problem hiding this comment.
Good point. Then this "By building a common understand, everyone develops a sense of ownership because they contributed to the definition of the metric. Understand and utilization of a common metrics language starts with the individual then built out to teams and finally the entire organization. This strengthened the internal culture around metrics as a tool for best development practices. " would be best for #resulting context.
There was a problem hiding this comment.
I feel there would be greater value here in adding some of this content to the existing pattern covering metric usage that is also in initial. For example this pattern has some similarities with these bits of that pattern...
People do not like to be tracked or measured.
Metrics can be misunderstood ...
i.e. the "Forces" section of that existing pattern has quite a few references to the dangers of metrics.
|
Sounds good. Will do. |
No description provided.