I am often asked what to measure in Agile delivery. The common measure appears to be velocity, which I concede is useful to track and is also readily available (assuming scrum) but, as is well discussed in articles and blogs, it can be dangerous to measure this publically as it morphs into something that is judged. It can end up be held it as a target for the team and then Goodhart’s law kicks in https://en.wikipedia.org/wiki/Goodhart%27s_law and what was useful information is now a manipulated artificial construct to give the desired answer. (Using it internally to help project delivery forecasts is very sensible – just don’t assess the team against it).
So given that, what would be a sensible answer? I would advise digging a little deeper into the question rather than giving a snap answer. Information is only as good as the decision made off the back of it (and by extension information provision when no decisions will be taken is waste). Why do you want to measure the Agile delivery? And there are a few honest answers to this:
- I want to see how much value the team are delivering
- I want to ensure the team aren’t slacking off
- I want to understand how the team are performing / struggling
Each of these really justifies a different approach.
For Value you need a discussion as to where the organisation perceives value – and this usually causes some uncomfortable moments when comparing existing process to fundamental Agile values – Working Software over Comprehensive Documentation and Working software is the primary measure of progress. The real measures should be associated to the benefits the organisation reaps as a consequence of the software but that can be a little unfair on the team (as they are not responsible for the requirements) – but critical for the organisation as a whole. Considering just the software delivery then Cycle Time is probably the best measure. How long it takes from idea to delivered software. Measuring, and by extension managing, this will also encourage the organisation to break their deliverables into smaller units. The benefits of the delivered work should still be tracked to avoid a situation where a highly efficient software delivery outfit rapidly and consistently delivers a stream of valueless changes.
The “I want to avoid slacking off” is a difficult one, and is probably always just under the surface, even if they say value – they may really mean – I need something to beat the team with. Despite what they say, what we really have here is fear of loss of control, someone who maybe accountable but has little influence over delivery. This suggests a structural issue with people put in management positions abstracted from delivery, and it also suggests a culture where trust is lacking. The Product Manager or Project Owners on the engagement should be close enough to the teams to have an opinion on current activities and levels of engagement. They will have a first hand understanding as to the reasons behind delivery ups and downs (reflected in velocity) associated to complications or setbacks identified in development (perfectly normal and expected in a complex industry). I usually suggest these people provide sprint activity information to the programme, which can decrease overtime as the management structure adapts.
The third request is a little refreshing and implies a maturity often lacking. This says we are concerned about value – trust the team and want to know when to try to help to support them. To understand what to measure here requires a little “Genba”, real world observation of how the team copes with adversity. Systems are inefficient when operating at 100% capacity, any change to an input variable that worsens the situation will cause failure, so the more dynamic the system the lower the optimum operating capacity. The difference between the optimum and the maximum could be considered contingency – if you want to give support to a team, measure the use of that contingency. This will give an indication of when and by how much the team is struggling and therefore when to act. This could be overtime, or compromising the Definition of Done, items added mid sprint etc.
As a metaphor consider a racing yacht, if you just record the speed then you may miss the unsustainable efforts the team are making to achieve it. Instead note just how many people are hanging off the side of the boat – and how long they have been there, they can’t hold on forever – and if they are made to, then the yacht will speed along until they drop off, at which point the yacht won’t slow down, it will capsize.