Few days back , while assessing Agile project, I observed an interesting point – there was significant difference between the story points computed using story point estimation technique and final story points reported to the management. The reason given by team and Scrum Master was whatever story points they get using story point estimation technique is just the size of development work. It does not give size of testing work. So they just doubled the story points to get overall size for a story.
Probing further I realized that reason for this adjustment was the team reporting lowest velocity in their unit (if they use estimated story points). So initially when they reported velocity, they got lots of questions and push from management to increase the velocity as management perceived that as compared to other teams, their productivity was very low. Now after making this adjustment management is happy as velocity (Productivity) improved drastically.
The fundamental problem here is Management using Velocity as a performance metric to judge and compare the productivity of different teams. This does not make sense because most of the times teams are not sizing product backlog items using common baseline. (Unless they are from same ART and following normalized estimation process prescribed in SAFe).
Velocity is planning tool and team diagnostic metric. If one starts using it as a performance metric to judge and compare the productivity of teams, velocity can motivate wasteful and dangerous behavior. We just saw one example of it!
Another example of wasteful behavior is to just change the scale the team uses to estimate PBIs. So team now sizes same item as 8 or 13 which was originally sized as 3 or 5. Even worse and dangerous behavior is team starts compromising quality to achieve higher velocity. This will eventually increase technical debt and finally decrease the overall productivity.
So velocity should not be used to compare performance of two teams. Even if you want to use it to improve the performance of the same team I would recommend you to be careful. Go to the next level of detail and check how the velocity is really improved. One can do this quickly by going through the improvement actions identified in retrospective and then by checking how implementation of those items helped in achieving better velocity.
But if one really wants to compare the performance of the teams in organizations, how it should be done? Well, one can use either Predictability Measure from SAFe or use Process Efficiency metric originated from Lean.
In SAFe, predictability measure is used at Program or solution level by comparing the actual business value achieved at the end of PI with planned business value, assigned by Business owners, to PI objectives identified during PI planning. If one is not following SAFe, then some additional work needs to be done to implement this metric at iteration/sprint level or release level (which typically contains multiple sprints).
Another Metric is Process efficiency which is ratio of Value Added Time / Total Lead Time. This metric can be implemented in any team and independent of technology or domain. Process efficiency metric can be an eye opener for all stakeholders as sometimes process efficiency can be even in single digit. (For example total lead time is 80 hours but actual value added time can be 5-6 hours.).
Last year Jeff Sutherland and few others published paper on process efficiency where they defined process efficiency as (Cycle time – Interruption time) / Cycle time. Some of the examples of interruptions are Meetings, Manual testing and Manual deployment.
Organizations need to perform some additional work for implementing above two metrics. But in my view, these are certainly better metrics for comparing teams than velocity. And even if you do not want to compare teams, implementation of these metrics will certainly benefit to all stakeholders.