The delivery-time scatterplot is one of the highest return-on-investment charts I recommend. Where else can you find such an elegant visualization of a team’s or organization’s delivery speed and predictability?
The scatterplot is amazingly simple to build and yet highly informative about a system’s behavior.
What you would use it for:
- Forecasting when future individual work items might complete
- Identifying delivery-speed trends
- Quantifying predictability
What it shows: the elapsed time for each work item in a system to complete over time:
- The x-axis is the completion date.
- The y-axis is the delivery time (usually in days).
- Each plot is a work item.
A few things to note on the above example:
- A: This item took 38 days to complete, so it appears to be an outlier.
- B: Two items finished with same delivery time on each of these two dates (with delivery time of four days completed on 12-30 and two days on 12-31).
- C: Five items completed on 12-31: one that took 23 days, one that took 15, one that took five and two that took two days.
- This chart shows 10 work items that have completed.
The real value is when we overlay percentile lines onto the chart. Percentiles are basically showing where a certain percentage of our data points are falling.
- E: In this example, we have exactly 10 data points, so we can simply count the data points from the bottom up to five, and that will yield the 50th percentile line (that is, exactly 50% of our total data is below that line).
Same thing with the 70th percentile and 85th (D).
How to read it: You can read it in a number of ways, observing:
- The percentiles: The speed of delivery — how long we might expect any particular future work item to take to complete. You take a percentile and observe that, for instance, “We have an 85% confidence that any particular work item will finish in 17 days or less.”
- The trendline at a percentile: Whether our delivery speed is improving
- The vertical tightness (or lack thereof) of the plots: Whether we’re becoming more or less predictable
A few key things to note about the scatterplot chart:
- It is different from a control chart. Since delivery data is not normally distributed, we use percentiles rather than standard deviations.
- You can of course chart anything on a scatterplot. I give it the qualifier “delivery-time” here as a generalized way to refer to elapsed time to deliver a work item (so as to avoid the semantic issues with “cycle time” and “user story,” etc.).
- Resist the urge to remove “outliers.” They tell an important story about your system. Moreover, an outlier won’t necessarily change your percentiles (whereas it would definitely change the average).
- The 50th percentile is not the same thing as the average of the data.
Sources and Resources
- Actionable Agile Metrics for Predictability, Chapter 10 – Introduction to Cycle Time Scatterplots (by Dan Vacanti)