Improvement MetricsPosted: January 12, 2015
Back in November, I spoke at a couple of conferences about some improvement metrics that I’ve been experimenting with. I share them below in the interest of getting feedback from the community on how others might be using them.
- What it measures: The efficiency of your workflow
- How to measure it: Divide the average time spent working on stories by the average work time plus wait time and express as a percentage.
- How to use it: Very carefully! Because of necessary dependencies and common-cause variation in work, aiming for 100% flow efficiency shouldn’t necessarily be your goal. Instead, use flow efficiency to ask questions about why your rate is what it is, and see if you can reduce the unnecessary waste in your workflow, such as unnecessary wait queues and avoidable dependencies. (David Anderson claims that having a flow efficiency above 15% is normal, and above 40% is good.) Zsolt Fabók explains it well.
- What you might replace with it: staff utilization (“resource” efficiency)
Percent accurate and complete
- What it measures: Quality of your process
- How to measure it: Divide the number of work items that go through the process without defect, blockage or rework by the total number of work items and express as a percentage.
- How to use it: Identify the work items that do not go through the process “complete and accurate” and conduct root-cause analysis. The trend should go up over time. Kevin Kriner, who introduced it to me, is the go-to guy for this metric.
- What you might replace with it: Defect counts
- What it measures: Average time that it takes for a particular kind of work item to go from the commitment point to delivered.
- How to measure it: Track the date when each work item crosses the commitment point (i.e., the decision to start working on it) and the date when the work item is considered done (e.g., in production).
- How to use it: One of the goals of kanban is to deliver a predictable delivery time (a function of WIP). Use average delivery time to create a target delivery time per class of service (and possibly further characterizing stories by type). You can also use it to calculate delivery rate (average work in progress divided by average delivery time) and forecast/plan with it. Like velocity, delivery time should be seen as a means rather than an ends and a gauge rather than a dial.
- What you might replace with it: Velocity and estimates
Return on investment
- What it measures: The bang you’re getting for the buck you’re spending.
- How to measure it: Divide Value (either notional/nebulous relative units or absolute value) by Investment, such as a weekly or monthly team run rate.
- How to use it: For planning and scheduling work (use in conjunction with economic planning models, like cost of delay/opportunity cost), within a product as well as across products or a program. In the big picture, it helps validate “Is this thing worth it?” and “Should we be spending our money on something else?”
- What you might replace with it: Burn rate