Measuring Agile maturity
Quite some commotion lately about measuring agile maturity. Agile is all about outcome over output. So how come the maturity models I have seen focus much of their measurements on output? Complaints about resistance to change are also often heard. So let’s try to get two wrongs to make a right and view Agile maturity from a change perspective.
Measure what matters
To focus our measurements on things like release rate, automation levels or velocity could actually be quite wasteful. Metrics that are output driven are an invitation to game the system, potentially resulting in more harm than good. We could have a high release rate but not add any actual value to our product. An app could have 100% test coverage but still not be adequately tested. We could have a high level of CI/CD automation and have gateways approval procedures in place that could slow the whole flow down to a standstill.
Outcome
Now don’t get me wrong; most of these agile maturity metrics are good indicators of things that are beneficial for agility. But to assess if we are moving in the right Agile direction, it would be better if we would use outcome-based metrics.
What things should we measure that are related to agility? Well, a core aim of Agile is adaptiveness. How could we measure this? Let’s say we are potentially incredibly adaptive, but we never adapt. Would we be agile? We would probably be able to be agile, but agility would only truly manifests itself if we would really adapt.
No Change, No Agile
So might adaption not be a good thing to measure? Adaptation consists of several measurable activities, but we would at least need some sort of change; without change, we can not adapt. What if we would try to measure the rate of change? However, change for change’s sake is not enough; it needs to lead to improvement. Although we can not predict success in our complex systems, we should be able to identify problems. The probability of improvement is a lot higher if we would start with a problem to solve. So let’s also measure the rate of problems found. Finally, all this changing should lead to something, namely solutions. So for good measure, let’s keep track of those as well.
Balance
Solving problems we don’t have or randomly changing stuff is probably not how we would effectively continuous improve. So we’d better introduce a balancing relationship between problems and solutions. If we would have a large number of problems without solutions, stop looking for problems and initiate change to get to solutions. On the other hand, if we are solving problems at a higher rate then we are finding them, we might want to shift some effort to the problem detection department.
“Too much change!”
As local optimisation is wasteful, maximising the rate of change would not benefit the system as a whole. Changing our way of working every day would probably hinder us greatly from delivering valuable software. What could balance out the rate of change to just enough? What could be negative feedback in our change optimisation loop? Well, I don’t know what things are like in your Agile organisations, but I have come across quite some resistance to change. No judgment, we are system thinking, but let’s be honest, change is often like pulling teeth.
Safe change experiments
What if we would actively address the level of resistance to change? Not to suppress it, but as a natural moderator in our change orientated model. Not as an agile coach’s excuse metric, but as an intrinsic part of our system, to make change safe and explicit. This indicator could, for instance, be a number on a scale from 0 to 10 or some colours like red, yellow or green. Anything that would allow us to know if we should up the change rate or ease off a bit. So let’s experiment and see what works better in our particular system.
Do you have change?
We learned that it is helpful to have an information radiator on this subject giving us a view on where we are. Starting our journey with a classic improvement board, we found that it was beneficial to be more explicit about our changes and renamed the board accordingly. When reviewing how we were feeling about the changes we had gone through we made a distinction between systemic changes and transient alterations. If it would change our way of working structurally, we considered it a systemic change. But minor adjustments or temporary trials that did not significantly alter our way of working as such were excluded. For example, changing our seating arrangement was considered a change, were experimenting with a new technical implementation was not.
Change the change
We did a 0 to 10 blind vote on the changes we had in progress. Team members were asked to focus on the type and impact of the change not so much on whether they liked the current state of the change. The basic question we asked was: “Would you like more changes like this or less?” The results were quite promising, not only did we have a healthy discussion about the rate of change but also for the need for change. We also agreed that the barometer could be used for individuals to indicate that they experienced some impact by current change whether good or bad. A certain change in seating arrangements resulted in the sudden arrival of a low-pressure area for instance.
Systemic Change
Not all change is created equal. Responding to changing circumstances is not the change we mean here. Special cause problems should not necessarily lead to a system change; a temporary adjustment might suffice. However, a common cause problem is something we should try to solve by changing the system. If someone is late for the daily standup once, we should not directly alter the start time. But if people are regularly absent due to other obligations, we might consider rescheduling the Daily.
Feedback in the loop
To make this explicit, we should add the distinction between real changes and minor adjustments at the problem state in our model. Also, if our solution did not solve our problem, we should reiterate and change some more. Finally, if we find that the resistance to change is too high, we should keep evaluating the solution state and first get accustomed to the last change before we move on to the next problem.
Just a model
It’s a model, and as all models are wrong, this one is no exception. Also, it’s far from “complete”, but it might enable us to safely incorporate some intrinsically Agile metrics in our agile maturity models concerning the core of being adaptive: Change
Metrics:
- Change rate: number of changes/time frame
- Problems found: number of problems found/time frame
- Solution Problem fit: number of solutions found for problems/time frame
- Resistance to change level: Some sort of gateway indicator
Now let’s start experimenting and iterating. Let’s see if we can safely change our way out of some common problems. Let me know what you think and find! I’ll do the same in a next post.
