What We’re Learning from the NoEstimates Game


NoEstimates workshop at the 2018 LeanAgileUS conference

Having facilitated the NoEstimates game for more than a year, in many places around the world with differing groups — most recently at the outstanding LeanAgileUS conference — I’ve observed some patterns for success. Though these “winning strategies” may at first appear to be useful only if you want to play the boardgame, I believe that they likely translate into real-world success in intangible-goods (e.g,. software) delivery processes.

(Spoiler alert: If you haven’t played the game yet but plan to, you may not want to read the following — unless, of course, you want to cheat your way to victory!)

To remind you of some context: The game is a simulation of a group of interdependent work teams, each with an identical backlog of 25 work items. The teams play in simulated days, and, depending on how long the session is, usually play between 15 and 30 days. Teams earn “money” based on how much value they deliver, according to the following table:

Delivery Time (Days) Value ($)
1-2 days $700
3-5 days $400
6+ days $300
Urgent -$100 per day

Using data that I’ve collected from the teams over several sessions, I’m seeing that the teams who earn the most money per day are also the ones that are most predictable. That is, while they can’t do anything about some of the variation (e.g., the essential effort required to do the work), they either consciously or unconsciously follow common policies that reduce other kinds of variation. This appears to support Dan Vacanti’s idea that “doing predictability” is a rewarding business strategy.

Teams typically earn the most value per day and deliver most predictably by following these policies:

  • Limit work in progress: We generally know that this is a helpful policy. The learning for me with the game is that the optimal work-in-progress levels are even lower than one might expect, typically half (or fewer than) the number of people on the team. Even four or five-person teams who follow a single-piece flow policy don’t trade off much, if any, throughput. For small teams, the difference between having three-to-four WIP and one-to-two WIP can yield twice as much revenue per day in the game!
  • First-in, first-out: It’s easier to do this when you’ve got low WIP levels, of course. And single-piece flow is the natural extension of this policy. The game includes a few random “urgent” work items, which cost the team $100 each day they’re in progress, so they’re highly incentivized to “jump the queue” with these cards. Even so, the teams that have low WIP (a conWIP of one or two) are able to continue to honor their FIFO policy, which creates better predictability, throughput and value delivered. (Dan Vacanti has written about this.)
  • Cross-functional collaboration: Probably because the game makes both the scarcity of effort available and the work highly visible, players almost naturally “focus on the work, not the worker.” Rather than optimize in their specialty areas, players on successful teams instead work outside their specialties, where they get only half credit for their effort. (This appears to support the research that Dimitar Bakardzhiev has done.)
  • Flow over utilization: Winning teams generally don’t mind not fully utilizing all of their capacity, preferring to leave some effort on the table (literally, in the form of effort cubes) rather than pulling in enough cards for everyone to “stay busy.” One of the event cards attempts to entice teams to improve utilization, but nearly every team chooses not to.
RPS_Image-289 cropped

This team executes a strategy of limiting WIP to fewer than half the number of team members at the 2018 LeanAgileUS conference.

Although these lessons are from simulations, I think that, to the extent that the game emulates real work, the lessons can be extended into our actual work environments. In general, these gameplay experiences — because they are rooted in the incentive to optimize value — tend to manifest the mantra “Value trumps flow, flow trumps waste reduction.” So why to teams playing the game seem to know these lessons almost intuitively? The reasons aren’t necessarily anything that can’t also be done in real life: Connect more directly to the value feedback loop (John Yorke’s recent post on verifying value of user stories helps with this) and use flow metrics (e.g., delivery time depicted on a scatter plot) to make your process more predictable. “Keeping score” — of things that matter, anyway — doesn’t need to be limited to games, after all.


Self-Management Practices, Part 2: Information Intake

[Note: Following is the second part of a series on personal productivity. The first post is Email Control.]

Now that you know how to control your email, how to you deal with everything else? Throughout the course of your day, you gain ideas, discover tasks and learn of pressing needs that you absolutely can’t forget to do. How can you keep track of it all?


The key for me is streamlining my information-intake process. I used to be like my colleague who cobbles together a combination of post-it notes, scraps of paper, notecards and whiteboard scribblings, not to mention email in her inbox — all a recipe for an excessive cognitive burden! For me, the way to relieve stress — and get things done — is to limit the intake methods and funnel them into one master.

Intake Methods


fieldnotes-coverIn my typical day, I’m not behind a computer screen 100% of the time. I’m walking around, sometimes by myself, sometimes having chats with people, sometimes in meetings. So I need a way to capture ideas and to-do items while being respectful of other people and being able to be in the moment (and not distracted). Therefore, in most of my human interactions, I unplug. No laptop (unless I’m pairing), no phone. I find that a screen between someone else and me creates a barrier, however slight, to our interactions, and of course, increases the risk of being interrupted or distracted (“Oh, hold on, just got an IM that I need to reply to…”). So I use an old-fashioned tool: the paper notebook. My notebook of choice is the pocket-size Field Notes notebook. It fits in my pocket, so I can come to a meeting carrying nothing but my coffee cup. I can go for a walk, hands-free. I can have an ad-hoc conversation by the water cooler and jot down an idea or task (starting from the back of the notebook) without having the awkward moment of pulling out my phone, which unintentionally brings a whole external world to intercede between my fellow human and me, when all I need is something to record a few words with that I’ll know where to find later. And that last part is the key: Rather than a random assortment of post-it notes, notecards and scraps of paper, I can relax knowing that all of the ideas and tasks that I’ve accumulated during the day are in one handy place.


[On a side note, I once lost one of my notebooks while walking in San Francisco. A kind-hearted person took the time to mail it to me, realizing how important it was — and perhaps appreciating what a humanizing tool it is! Thank you again, Sue Ann!]


Evernote-Logo-1200Of course, sometimes it’s easier to record stuff digitally. (I’m not a caveman, after all!) Just as I limit myself to one analog tool, I allow myself only one digital intake method: Evernote. If I’m using my computer, it’s natural  and fast to CMD-Tab over to Evernote, where I keep one (and only one!) to-do list. If I’m reading/researching something online and either my brain’s focused mode thinks of something or its diffuse mode presents something I wasn’t thinking of, I quickly jot it down in that Evernote note and continue. Particularly during my daily inbox-zero quest, I offload those discovered tasks into the to-do list, which is essentially what allows me to not use email as my to-do list. (Though it seems like I’m adding an unnecessary step here, I have a reason — keep reading.)

Funneling to One Work-Management System

Limiting the intake methods and recording info is only half the battle, though. At the end of the day, I have two sources of information that I need to reconcile: my Field Notes to-do list and Evernote to-do list. It won’t surprise return readers of this blog to find out that my preferred solution for managing work at the personal level is the same as my preferred solution for team and organizational work: kanban.

My personal kaban in Kanbanize. (Yes, I know that I’ve exceeded my WIP limit!)

I’ll go a bit deeper into personal kanban in my next post in this series. But for now, I’ll explain the basics. Essentially, I keep the columns (a.k.a. work stages) fairly simple:
  • Options (not usually displayed)
  • To Do: Stuff I’m wanting to work on next
  • Doing: Stuff I’m working on now
  • Done: Stuff I’ve recently completed

My Field Notes to-do list after discharging everything into my personal kanban.

The really effective bit is the rows (a.k.a. swim lanes). I use cost-of-delay profiles (more on those later, too) for the lanes:
  • Expedite: Stuff that needs to be done immediately
  • Fixed Date (and Intangible Fixed Date): Stuff that truly has to be done by a certain date
  • Standard Urgency: Stuff that needs to be done
Each morning, I spend a few minutes moving my to-do items from my two intake methods into my personal kanban. I use the cost of delay profiles to sort those tasks according to both “value” (or importance) and time. This is why using a simple tool like an email box or checklist is insufficient: It can’t distinguish between the time consideration of each item. Once I have created a card for each entry in my Field Notes to-do list or Evernote to-do list, I mark them as complete (Field Notes) or delete them (Evernote).

My Evernote to-do list each morning.

Knowing that I have captured and properly discharged every to-do item, I can relax, because:
    • I don’t have to worry about anything slipping through the cracks.
    • I don’t have to carry any of these things “in memory.”
    • I have one place (my personal kanban) where my tasks live (and I don’t have to keep track of various disparate papers, post-its, etc.), which is perfectly fit for the purpose of managing work.
    • I have decoupled the capture from the prioritization, so I don’t have to worry about when to do things.


Kanban’s First Change-Management Principle and Chesterton’s Gate

At a recent Lean Kanban St. Louis meetup, I shared that, while the Manifesto for Agile Software Development has been amazingly enduring, it was silent on the issue of change management, which, in my experience, is the area that most commonly inhibits the ability for the Manifesto’s values and principles from taking root.

This is why I appreciate that the Kanban Method explicitly addresses change-management, and in particular, sets the tone with its first change-management principle:

Start with what you do now, understanding current processes, as actually practiced, and respecting existing roles, responsibilities and job titles.

In their book Essential Kanban Condensed, David Anderson and Andy Carmichael explain it this way:

…the current processes, along with their obvious deficiencies, contain wisdom and resilience that even those working with them may not fully appreciate.

This is the challenge that people brought into organizations as “change agents” or “agile coaches” face — I know, because I’ve been one and, to my and my client’s disservice, not heeded this advice. And in fairness, it’s difficult: The ambit — get results, immediately — is often at cross-purposes with this principle.

It reminds me of an earlier bit of wisdom from the writer G.K. Chesterton:

In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.

This metaphor has become known as “Chesterton’s Fence.” To be sure, it takes time to understand the reason for the fence, and, to be sure, in many organizations, time has obviated the need for the fence. But lest we strip away something that provides resilience, or simply create ill will by disrespecting another’s work, we’ll do well to first understand the reason for the fences — “the more intelligent type of reformer.”

Self-Management Practices, Part 1: Email Control

[Note: Following is the first part of a series on personal productivity. Whether you work remotely, in-person, in a team or as solo contributor, most professional people in today’s knowledge economy need to develop the competency of self-management. I aim to share some of my techniques for self-management, learned through discovering my own weaknesses and using my strengths and applying the ideas of others to create structures and practices that militate against those weaknesses and help me focus on getting things done. Topics I plan to cover include personal kanban, pomodoro technique, planning by cost-of-delay and sustainable pace.]

Email Control

I’ll start with perhaps the most ubiquitous and nefarious enemies of personal productivity: Email. Do you control your email, or does your email control you?
Email used to control me, making me respond to it on its terms, overwhelming me, creating unnecessary cognitive load. Now, I control this silent killer of productivity. Here are a few of my practices.

Inbox Zero


Why am I smiling in this picture? Because I have no email!

I can’t tell you how much of a difference having a policy of inbox zero — that is, at some point during my day, having no email in my inbox— has made. Before Merlin Mann’s thorough talk inspired me to set and live by a inbox-zero policy, I used to worry about those unresolved messages: Had I forgotten something? Was someone waiting for a response? Sometimes I had indeed forgotten, or someone was waiting — in which case I caused dissatisfaction around me. However, many times, no one was waiting, but still my brain, not being certain of that, couldn’t let it go, and I had to deal with unresolved threads spinning in my mind, causing unnecessary cognitive load.

Today, I have a recurring daily card in my personal kanban (more on that in another post) to get to inbox zero — actually two: one for my work email, and one for my personal. Because I devotedly follow my personal kanban, this ensures that I actually do it. (If you’re interested in combining your inbox with personal kanban, you may want to try Flow-E, a new tool from the makers of Kanbanize.)
This leads to other virtuous practices: First, I’m able to give people a 24-hour response expectancy 90% of the time. (Don’t you just hate it when you have to nag someone with a followup email?) Also, I don’t overreact to email requests*, which frees me to manage my work on my own terms, not simply the random arrival rate of the message; one study found that 70% of work emails were attended to within six seconds of their arrival — that’s what I consider overkill in terms of service-delivery expectation! And because of those two things, I’m able to actually switch off email throughout the day.
*The technique that supports this is planning by cost-of-delay, which is another post in this series.

Turn Off Email

Given that it takes people on average about 25 minutes to reorient back to a task when they get interrupted, it’s a wonder that people who keep their email open ever get anything done (other than incessantly replying to email!).
Yes, it’s possible to not have your email tab (or email app) open all day. I check my email roughly three times per day (morning, lunchtime, late afternoon). And I work in a global team, in a global organization, so I’m getting email around the clock. If people need me to respond to a request faster than 24 hours (or during my core work hours, four hours), I make sure that that know they can reach me other ways (e.g., text, Google hangout).

Just say no to enabling desktop notifications.

What percentage of your day is email open? Why is that? Perhaps you have a good reason, but you might try removing desktop and phone notifications and checking to see if you’re more productive.

Email should scale with your role/position/level

If you’ve ever gotten promoted, did you notice an increase in the amount of email you received? Regardless of my level and responsibility, I’ve tried to keep my incoming email manageable — that is, I can usually get to inbox zero in 30 minutes. If you reach a place in your career in which you have to hire someone to manage your email for you, you’re doing it wrong. Instead, look for different ways to interact with your stakeholders — better feedback loops, more face-to-face time, delegating decisions to others, working in a team. By responding to people’s email — or having your admin do it — you’re simply exacerbating the problem.

Five sentences (or less)

Given that you will always have some email to deal with — and that can be an appropriate communication medium — the last tip relates to knowing how to communicate effectively with email. My main practice here is the “Five Sentences or Less” response (http://five.sentenc.es/), which forces me first to think about whether email is the right medium and second to clarify and condense my thoughts in my reply. This allows me to communicate assertively and clearly, while “paying forward” to my recipients less email cruft on their end. Be honest: When you receive an email that requires scrolling, how many of you actually bother reading it in its entirety?
How do you know whether and how to reply? Some smells:
  • You’re attaching something: Before you drag that doc into the email body, consider putting it into a shared team space (e.g., Jive, wiki, Google Docs) and simply including the link to it. In addition to simple hygiene of facilitating versioning and collaboration,  you’re also helping people to avoid using email as a document repository. Don’t force people to keep your email.
  • You’re writing more than five sentences: If you can’t say it in five sentences or less, chances are that you need better fidelity communication; try a video or phone call. If you need something to serve as documentation that needs to live beyond a phone call, write the document and share the link.
  • You’re writing only one sentence (or word): Simple yes/no responses can be better handled in other ways, such as instant messaging.
  • You’re writing something that will require group discussion and/or decisions: We have other, better ways of group discussions, namely video calls or quick in-person meetings. If it’s a simple decision or set of preferences, create a survey and send the link. And please — whatever you do — don’t use email to try to find out “what time works” for each person in the group. It’s not 1999 anymore!


How Many Runs Would Man City’s Seven Goals Have Been?

After Manchester City scored seven goals in their Oct. 14 match against Stoke City, my first reaction was: Wow, they’re playing some beautiful, unselfish soccer. Being also a baseball fan, my second reaction was: That’s a load of goals — how many runs would that equate to in baseball?

To find out, I used the same technique that we can use for understanding the performance and predictability of our knowledge-work systems, such as software delivery.

First, let’s look at the distribution of goals per team in soccer. Since the new English Premier League season has only just begun, I’ll use the data from 2016-17, the most recent complete season of play:

From this we can then start to understand the likelihood of a seven-goal outburst by a single team. For instance, with 246 occurrences in a total of 760 total outcomes, the goal total of one is the most likely, at 32.4% Seven goals happened only once last year, making it 0.1% likely.

We can do the same for baseball. Let’s look at the runs scored per team for the entire 2017 regular season, which recently concluded:
Source: baseball-reference.com

(That 23-run game was when the Washington Nationals beat the Mets by a landslide on Apr. 30.)

To compare these outliers, we could use something like an average with standard deviations away from that. But the data from both the EPL and MLB are not normally distributed, which renders that approach inappropriate. Instead, we’ll use percentiles. Why? As Dan Vacanti writes in When Will It Be Done?:

Percentiles are not skewed by outliers. One of the great disadvantages of a mean and standard deviation approach (other than the false assumption of normally distributed data) is that both of those statistics are heavily influenced by outliers.

A percentile is simply a level that contains a certain percentage of data points. For instance, if I looked at the Premier League data at, say, the 61st percentile — the “one goal” column, that would mean that 60% of our outcomes were teams who scored one goal or fewer (the total percentages for zero goals (28.2%) and one goal (32.4%). We could even draw a curve that shows those numbers:
From the Premier League data, we see that the seven-goal outcome doesn’t happen until the 100th percentile, which makes sense because it was the highest-scoring outcome! We have to go all the way to the 100% percentile in terms of likelihood of possibilities to arrive at seven goals.
So where is the 100th percentile for baseball? Naturally, it will be the highest-scoring run total of the season:
Now we have our answer! Seven goals, at least from recent data from the English Premier League, is equivalent to 23 runs in Major League Baseball.
Okay, so maybe that wasn’t all that interesting, since all we did was take the top outcome from each league. But using the same approach, we could develop a reference table for all of the scoring outcomes.
0% 60% 80% 90% 98% 99% 100%
MLB runs 0-4 5-6 7-8 9-11 12-14 15-22 23
EPL goals 0 1 2 3 4 5-6 7
Reading the table, you can make statements like:
  • In 60% of MLB and EPL games, a team scores six or fewer runs and one or fewer goals, respectively.
  • Seven or eight runs (or fewer) in baseball occurs at about the frequency as two (or fewer) goals in soccer.
We can apply this same approach to our delivery-time data in software delivery, because, like these professional sports, the data is not normally distributed. In fact the distribution of both leagues probably looks a lot like your team’s (graph it and see!). In knowledge work, as in this little exercise, we’re also trying to determine the probability of a single outcome happening, as in when we ask the question: “When might I expect this user story to be finished?” We can answer that question, and then plan, using percentiles, just like we did with the sports scores, like: “We have a 90% confidence that we’ll complete any given next user story in 11 days or fewer.” And like the sports scores, the longer the range in the “tail” the farther it pushes out our highest confidence intervals.

So the next time someone asks you about the likelihood of your favorite sports team — whatever the sport — scoring a certain number, you’ll know what to do — just as you will in your own team when someone asks when to expect a single piece of work to be finished.

Special thanks to Dan Vacanti for the insights from his recent book, When Will It Be Done?

Profiles in Cost-of-Delay: “Intangible Fixed-Date”

  • The “Big Four” cost of delay profiles (a.k.a. archetypes) — Expedite, Fixed-Date, Standard Urgency, Intangible — are usually sufficient.
  • In my personal kanban, I have seen a new archetype emerge, one that has aspects of both Intangible and Fixed-Date, which I’m calling “Intangible-Fixed-Date.”
  • This profile is less for the purpose of selection and more for scheduling.
  • So far, the main application of “Intangible-Fixed-Date” for me is for buying airfare, whose cost-of-delay curve fits none of the “Big Four” curves.
  • I use a couple of features in Kanbanize to deal with this new profile slightly differently from the other profiles, namely by creating a separate swim lane and using two dates (rather than one).
  • Takeaway: “Listen” to your data, patterns and behavior of work items and be flexible enough to adapt and create new profiles when they emerge.

I routinely use cost-of-delay to assist in scheduling, selecting and sequencing work, both in professional settings and in my own personal life. The “Big Four” cost-of-delay profiles (aka archetypes) promoted in the Kanban community — Expedite, Fixed-Date, Standard Urgency, Intangible — are usually sufficient for the work that organizations, teams and I personally need to handle. However, lately in my personal kanban, I have seen a new archetype emerge, one that has aspects of both Intangible and Fixed-Date, which I’m calling “Intangible Fixed-Date.”

Basically, the new profile has arisen from the need to handle airfare purchasing, something that I do somewhat often. Since I’m not independently wealthy and I like to do right by my employer when I’m traveling on business, I care about the cost of airfare. Anyone who travels knows that there is some optimal time to buy airfare (even if he or she doesn’t know exactly when that is). I want to be able to research fares just-in-time to get the best deal. However, if I model the task as a simple fixed-date item, I either have to use the true date by which the cost-of-delay is unacceptable (that is, the day I need to be somewhere) or use a fake deadline. I typically added the task to my Fixed-Date lane with its true deadline, then would occasionally check it to see if it “felt” like I should act, but it was too easy to forget about it until past a responsible moment, even if it wasn’t the last responsible moment. Which brings us to the crux of the problem: This type of item has two last responsible moments, the financially last responsible and absolute last.
Let’s review the cost-of-delay curves of the existing profiles:

COD expedite


COD standard

Standard Urgency

COD intangible


COD fixed date

Fixed Date

According to the reliable Andy Carmichael and David Anderson in their Essential Kanban Condensed, Fixed-Date items “have high impact but only if you miss the deadline. The scheduling imperative here is to make sure you start before the last responsible moment and deliver before the deadline.” Also, you don’t get any economic benefit from completing the work before the deadline. In software delivery, this usually means understanding how long it can take you to complete the work (see your handy delivery-time histogram), then backing up from the deadline from there based on risk tolerance. However, booking airfare doesn’t take much time (I can do it in a few minutes) — so that’s not a concern. But the deadline only represents the absolute last responsible moment, so this curve is insufficient.

If airfare price increases were linear, I could use a standard urgency profile blended with the fixed-date. But some basic research shows that price increases have an inflection point somewhere around the 30-day mark, although the lowest prices may occur earlier:
Other factors matter, too, of course (seasonality, specific locations). But I don’t need to get too deep into modeling that yet — I just need a better solution. That means incorporating the Intangible curve, whose items “have an apparently low urgency,” but that indicates “a rise in urgency – possibly a steep rise – will happen in the future.”
My airfare purchases exhibit aspects of both curves, though neither is sufficient. Unlike a typical intangible item, I actually do know when that future date will be, along with a rough idea of when the “rise in urgency” will happen. And unlike a typical fixed-date item, there is some economic benefit from doing it sooner than the deadline.
COD intangible fixed-date

Intangible Fixed Date

Now that I understand the cost-of-delay curve, I need to be able to handle it in my kanban system. One approach that I’m trialling is to create a separate swim lane — “Intangible Fixed-Date” — and use a second date to signal readiness (the second date being my optimal time to commit).
Here’s how it works: Say I find out on Sep. 6 that have to be in Chicago on Nov. 6. That’s two months away. Given that airfare cost won’t typically rise until about 30 days out, I don’t want to worry about this yet, which is to say, I don’t want this card on my Requested column yet. So I set up a rule in Kanbanize (my work-visualization tool of choice) to keep it in the backlog until 35 days out (the five-day buffer allows me some flexibility), at which time it moves the card from the backlog to Requested.
It has the ultimate deadline — Nov. 6 — displayed on the card. But since it’s in its own swim lane, I have an explicit policy that I begin work on these items as soon as they appear in Requested.

Screen Shot 2017-10-05 at 12.52.01 PM

If I want to really be disciplined, I can then set a service-delivery expectation that sets the bar for how well I handle these (e.g., 90% of Intangible Fixed-Date items will be completed within five days), and analyze my performance at my personal service-delivery review. But now I fear I’m exposing just how geeky I am (if that wasn’t clear already)!

So what’s the takeaway? Well, you might find value in this “new” cost-of-delay profile (if you need to book airfare, or to plan birthdays or anniversaries, which follow a similar curve). But abstracting out a bit, the idea is that it’s helpful to pay attention to — “listen” to — your data, patterns and behavior of work items and be flexible enough to adapt and create new profiles when they emerge. Pursuing incremental, evolutionary change is one of the underlying principles of kanban method; improve using models and experiments is one of its core practices.

Special thanks to Prateek Singh, Josh Arnold and Mike Burrows for their early feedback in the Lean Agile and Beyond Slack community.


Service-Delivery Review: The Missing Agile Feedback Loop?

I’ve been working for many years with software-delivery teams and organizations, most of which use the standard agile feedback loops. Though the product demo, team retrospective and automated tests provide valuable awareness of health and fitness, I have seen teams and their stakeholders struggle to find a reliable construct for an important area of feedback: the fitness of their service delivery. I’m increasingly seeing that the service-delivery review provides the forum for this feedback.

What’s the problem?

Software delivery (and knowledge work in general) consists of two components, one obvious — product — and one not so obvious — service delivery.  I’ve often used the restaurant metaphor to describe this: When you dine out, you as the customer care about the food and drink (product) but also how the meal is delivered to you (service delivery). That “customer” standpoint is one dimension of the quality of these components — we might call it an external view. The other is the internal view — that of the restaurant staff. They, too, care about the product and service delivery, but from a different view: Is the food fresh, kept in proper containers, cooked at the right temperatures, and do the staff work well together, complement each other’s skills, treat each other respectfully (allowing for perhaps the occasional angry outburst from the chef, excusable on account of “artist’s temperament”!). So we have essentially two pairs of dimensions: Component (Product and Service Delivery) and Viewpoint (External and Internal).
In software delivery, we have a few feedback loops to answer three of four of these questions and have more-colloquial terminology for that internal-external dimension (“build the thing right” and “build the right thing”):
The problem is that we typically don’t have a dedicated feedback loop for properly understanding how fit for purpose our service-delivery is. And that’s often equally the most vital concern for our customers — sometimes even more important than the fitness of the product, depending on whether that’s the concern of a delivery team or someone else. (One executive sponsor that I worked with noted that he would rather attend a service-delivery review than a demo.) We may touch on things like the team’s velocity in the course of a demo, but we lack a lightweight structure for having a constructive conversation about this customer concern with the customer. (The team may discuss in a retrospective ways to go faster, but without the customer, they can’t have a collaborative discussion about speed and tradeoffs, nor about the customer’s true expectations and needs.)

A Possible Solution

The kanban cadences include something called a Service-Delivery Review. I’ve been incorporating this to help answer teams’ inability to have the conversation around their service-delivery fitness, and it appears to be providing what they need in some contexts.
David Anderson, writing in 2014, described the review as:
Usually a weekly (but not always) focused discussion between a superior and a subordinate about demand, observed system capability and fitness for purpose Comparison of capability against fitness criteria metrics and target conditions, such as lead time SLA with 60 day, 85% on-time target Discussion & agreement on actions to be taken to improve capability
The way that I define it is based on that definition with minor tweaks:
A regular (usually weekly) quantitatively-oriented discussion between a customer and delivery team about the fitness for purpose of its service delivery.
In the review, teams discuss any and all of the following (sometimes using a service-delivery review canvas):
  • Delivery times (aka Cycle/Lead/Time-In-Process) of recently completed work and tail length in delivery-time distribution
  • Blocker-clustering results and possible remediations
  • Risks and mitigations
  • Aging of work-in-progress
  • Work-type mix/distribution (e.g., % allocation to work types)
  • Service-level expectations of each work item type
  • Value demand ratio (ratio of value-added work to failure-demand work)
  • Flow efficiency trend
These are not performance areas that teams typically discuss in existing feedback loops, like retrospectives and demos, but they’re quite powerful and important to having a common understanding of what’s important to most customers — and, in my experience, some of the most unnecessarily painful misunderstandings. Moreover, because they are both quantitative and generally fitness-oriented, they help teams and customers build trust together and proactively manage toward greater fitness.

Service-delivery reviews are relatively easy to do, and in my experience provide a high return on time invested. The prerequisites to having them are to:

  1. Know your services
  2. Discover or establish service-delivery expectations

Janice Linden-Reed very helpfully outlined in her Kanban Cadences presentation the practical aspects of the meeting, including participants, questions to ask and inputs and outputs, which is a fine place to start with the practice.

Afterward #1: In some places I’ve been, so-called “metrics-based retrospectives” have been a sort of precursor to the service-delivery review, as they include a more data-driven approach to team management. Those are a good start but ultimately don’t provide the same benefit as a service-delivery review because they typically don’t include the stakeholder who can properly close the feedback loop — the customer.

Afterward #2: Andy Carmichael encourages organizations to measure agility by fitness for purpose, among other things, rather than practice adoption. The service-delivery review is a feedback loop that explicitly looks at this, and one that I’ve found is filling a gap in what teams and their customers need.

Afterward #3: I should note that you don’t have to be in the business of software delivery to use a service-delivery review. If you, your team, your group or your organization provides a service of any kind (see Kanban Lens and Service-Orientation), you probably want a way to learn about how well you’re delivering that service. I find that the Service-Delivery Review is a useful feedback loop for that purpose.

[Edited June 12, 2017] Afterward #4 (!):  Mike Burrows helpfully and kindly shared his take on the service-delivery review, which he details in his new book, Agendashift: clean conversations, coherent collaboration, continuous transformation:

Service Delivery Review: This meeting provides regular opportunities to step back from the delivery process and evaluate it thoroughly from multiple perspectives, typically:
• The customer – directly, via user research, customer support, and so on
• The organisation – via a departmental manager, say
• The product – from the product manager, for example
• The technical platform – eg from technical support
• The delivery process – eg from the technical lead and/or delivery manager
• The delivery pipeline – eg from the product manager and/or delivery manager

I include more qualitative stuff than you seem to do, reporting on conversations with the helpdesk, summarising user research, etc