Theory of Constraints in the Wild: Queuing for the Tube

Tube queuing after the Tottenham match at Wembley

Tube queuing after the Tottenham match at Wembley

When a WWT colleague invited me to the Tottenham – CSKA Moscow Champions League match Wednesday, he treated me to more than simply world-class soccer — he unwittingly gave me a chance to see Theory of Constraints in action: after the match, while we waited for a Tube ride.

Theory of Constraints tells us that every system has a single constraint — which is anything that limits a system’s performance relative to its goal — that governs its throughput. A particularly effective way of dealing with a constraint (aka bottleneck) is Eli Goldratt’s Five Focusing Steps, which I talk about this way:
  1. Identify the constraint
  2. Optimize the constraint
  3. Subordinate everything to the constraint
  4. Add supply to the constraint
  5. Goto 1
If you’ve ever been in London during rush hour, Step 1 is easy: The constraint is the supply of train cars. At busy times, London simply has more people needing to ride than it has trains to take them. Step 2 is fairly straightforward: We optimize the constraint by ensuring that the constraint is
  • always busy (Londoners are particularly adept at squeezing as many people as can fit into the car, so this is never a problem!), and
  • is only ever doing “value-adding” work, which in this case is moving passengers forward (as opposed to failure demand, which doesn’t typically happen in the Tube in the form of redoing work — going backward — but does often take the form of people getting their bags or themselves stuck in the doors).
That’s where Step 3 comes in. Only after we’ve properly satisfied the first two steps, we want to subordinate — which is simply to adjust the speed of things arriving to the constraint and departing from it to match the cycle time of the constraint. For the Tube, that means making sure that when a train stops, the platform has enough room for the passengers to spill out and leave the station. But the more vital subordinating occurs upstream — such as after the 62,034 fans emptied out of Wembley Stadium and headed for the nearest Tube station. That’s what happened the other night, as the photo shows: Tube security personnel created multiple upstream “batches” of people by holding up signs with either a red Stop or green Go (an indicator of capacity, a kanban). Only when the batch of people saw the green could they advance to the next “gated” area and finally onto the platform. This is exactly the “pull” behavior that an explicit  work-in-progress limit encourages in knowledge systems: Rather than pushing as much work forward toward the constraint, we wait to advance new work until we get the signal, which indicates we are now under our WIP limit and therefore have capacity.
The result was a very orderly movement of people through the station and into the trains, avoiding the evil of overburdening and its demon spawn of waste, unpredictability and delay (and in this case, possible physical harm). This was good enough for the problem at hand; Step 4, add supply, is likely too expensive because it means adding trains to the line for what is essentially an exceptional case (a mid-week soccer match at Europe’s second-largest stadium). In any case, the result is testable, in that we can review the three TOC measures to see if any improved:
  • Throughput (should go up)
  • Operational Expense (should go down)
  • Inventory/WIP (should go down)
Presumably, the London Tube officials have indeed tested whether the practice of subordinating has improved the situation. My guess is that the practice has increased throughput, while holding steady their operational expense and WIP (which is irrelevant in this case). It has probably also improved human safety.
Seeing flow in the physical world — especially when we’re personally involved! — is usually obvious. Knowledge work, on the other hand, is by its nature invisible, so we need special lenses to see bottlenecks in order to manage and improve flow. Thankfully, we have aids such as kanban boards and cumulative-flow diagrams that make visible the flow of work — work in progress, throughput and average approximate delivery time. It’s also helpful to be aware of physical manifestations — such as post-match queuing for the Tube — to train ourselves to look for flow, inhibitors to it and ways to manage it.

From New Hire to Team Contributor, ASAP: How Asynchrony Prepares People to Deliver

inception.jpgHow do you help a new hire start contributing to an agile delivery team in as little time as possible? At Asynchrony — an agile software-delivery firm expanding and hiring rapidly — we embarked on an experiment exactly a year ago to bridge the gap for our new employees between orientation and the high-expectations of a real, billable customer-facing delivery team. What we tried we called simply “The Learning Team,” an ongoing internal team where new hires and veterans alike can safely learn new skills like TDD, pairing, continuous delivery and data-driven forecasting so that they can hit the ground running with external-facing teams.

 

Our current organizational context necessitated that we find a way to more effectively onboard people. We’ve been growing at a pace that has meant aggressive hiring goals, which increasingly means that we’re adding people who have the aptitude and attitude for but sometimes not the experience of agile delivery. That means that some new hires may not have experienced pairing, TDD or other vital practices for how we deliver quality software for our customers. We’ve found that, although pairing dramatically shortens the learning curve for new team members, new hires were not always able to go straight into that environment.

 

In the past, we had occasionally used ad-hoc and sometimes loosely run “bench projects” as places where new hires spent a few days before jumping onto a real project. But those were not designed to provide much in the way of mentoring or professional development; they were more a place to bide time. We had also recently successfully experimented with research teams, but those were more oriented toward R&D than product delivery.

 

Enter the Learning Team.

 

With the leadership of our CTO, Nate McKie, we envisioned it to be a permanent, first-class team, staffed by a full-time tech lead, a full-time product owner and a part-time agile coach. Following the first week of a new hire’s tenure at Asynchrony, which includes the same three-day agile overview workshop that we offer to our customers, he or she would join the Learning Team. Depending on the individual’s learning goals, the person might be on the team from a few days to a month or longer. It would also be a safe place for existing Asynchronites to acquire new skills or transition to a new role. (For instance, one colleague who has been with the company for a few years wanted to learn the skills of product ownership, so we made her the team’s product owner.)

 

Since the project would be entirely internal, we would have few, if any, dependencies outside of our control, so we could run the project in an exemplar way, focusing on learning through delivery. As an advisor to the team, I occasionally encourage the new hires to immediately apply the learning that they get in the agile overview workshop to what they’re doing in the Learning Team. The pattern should then be:
  • Implement in the Learning Team the good practices you hear about in Agile Overview, then
  • Implement in your future customer-facing team the good practices you experience in the Learning Team.
Our Crossing-the-Chasm-style elevator pitch looks like this:
For new hires and veteran Asynchronites
Who need to learn the Asynchrony delivery “way,” brush up on skills or learn new ones
The Learning Team is an internal delivery team
That provides a safe place to learn proven practices and experiment with new ones
Unlike being thrown straight into a team without knowing what great delivery practices look like​
The Learning Team is an exemplar team where people can learn healthy practices that they can take to other teams.
We kicked off the team with a proper inception, complete with “customers” and internal stakeholders. We added a twist to the typical visioning, chartering and release planning of inception: personal learning goals. Each person on the initial incarnation of the team took some time to write and share his or her goals with the rest of the team, which they would hold each other accountable for and encourage each other in.
learning goals.jpg

The Results

Obviously, the biggest risk was whether the return would be worth the investment. After all, it’s not chump change to intentionally run a bench project, staffed with a coach and dev mentors. Nate had gotten buy-in from the rest of exec management to carry out the Learning Team experiment for a few months. Had it failed, we would’ve pulled the plug and tried something else. But we’re now to the team’s first anniversary, and we’re continuing because we’ve seen how much it has bridged the gap for new hires — less friction when they join new teams, and the added bit of confidence that they know something about delivering in the “Asynchrony way.” Moreover, we now have a couple of employees who have gained valuable experience as product owner and tech lead who are potentially able to do those roles in other teams. It’s true that we incur opportunity cost by “benching” our people to work on the Learning Team. But the tradeoff in reduced risk is worth it. People who are new to Asynchrony and our way of working — in which transparency and honesty are precursors to learning — aren’t always trained or made to feel safe to expose what they don’t know. The Learning Team helps them develop that mindset so that, when they start pairing in earnest under the delivery pressures of customers, they don’t feel they need to hide problems or take shortcuts, which can lead to messy delivery problems. With so many new people coming onboard, we need to invest in preparing them for success. It’s one of the necessities of fast growth.

Challenges

Because the team composition is  always in a state of flux, we have to always be prepared for someone to leave. We’ve mitigated that somewhat by having three of the roles — product owner, tech lead and coach — be staffed by the same people, which provides some continuity. Also, because members don’t generally know when they’re going to be “promoted,” it can be disconcerting and inhibiting to team morale.  Along with that, we’re still trying to figure out the answer to the question “When is a person ready to move out of the Learning Team?”
We also learned that we need to choose the product and tech stack wisely. The product that the Learning Team has been developing over the last year was built on a legacy code base with a not-insignificant amount of a not-altogether easy language to pick up (Erlang). With dependencies on some business-facing concerns about the legacy code base (some real customers use it) and the steep learning curve of Erlang, we have decided to use a simpler, more straightforward product so that the team members can focus on Asynchrony agile delivery rather than some of the unnecessary distractions from our previous product choice.

In Summary

We’ve seen benefits for our new hires being able to more smoothly integrate into their delivery teams and for our veteran employees learning new skills and roles. One of unexpected benefits that I’m looking forward to in the new year as we emphasize exemplar practices in the Learning Team will be to the delivery teams themselves, getting infused with people who not only avoid a drag on their throughput, but provide fresh, innovative practices to invigorate the teams. My goal is to hear more things from Learning Team graduates such as “We were doing continuous delivery on the Learning Team; why aren’t we doing that on this team?”
 learning PO.jpg

Recent recommendations

I’ve recently attended a couple of thought-expanding conferences and met many inspiring people (including a few back at the Asynchrony St. Louis office), so I’m sharing some of the recommendations that I’ve picked up from keynotes, sessions, hallway chats and pub discussions. Here are a few things that people have been talking about, some of which have been around for a few years:


Make risks visible with “RAID bingo”

IMAG5949IMAG5950To help teams mitigate risks and other concerns — and make them explicit and visible — I like to do a RAID brainstorm, either at a kickoff/inception and/or at points after a project has started. (RAID is an acronym for Risks, Assumptions, Issues and Dependencies.) As with any sort of group brainstorming, the key is to facilitate cognitive diversity by allowing individuals to come up with as many ideas without being biased or inhibited by groupthink.
One technique I use to draw out as many unique ideas — and avoid the boredom and tedium that usually come with talking about risks — is RAID Bingo. Here’s how you do it:
  1. Divide the group into multiple small teams of 3-5 people.
  2. Have each team draw on a wall/large-poster sheet a 4×4 grid* with column headings of R, A, I and D.
  3. Instruct the teams to write risks, assumptions, issues and dependencies on post-it notes and place them in the appropriate columns as fast as possible. The goal is to be the first team to have three items in any column or four in any row. The first team to do so should shout “Bingo!”
  4. When a team shouts Bingo, tell all teams to pause. Invite the winning team to announce and briefly describe each of their items.
  5. Resume play! Have the teams keep their post-it notes on the boards and continue to try to get a(nother) full column or row.
  6. The next team to shout Bingo must have all unique items (they cannot have the same items that the first winning team used).
  7. Repeat until the teams have covered most of their boards (usually by the third “Bingo”). Then do the final Bingo “coverall” — first team to cover all squares wins.
Unified, final RAID list

Unified, final RAID list

After you’ve generated many unique project concerns, unify all teams’ contributions into one board. At this point, you can discuss them in more depth or simply defer the discussion and mitigation strategy until later (but hopefully not too much later!).

*Depending on time available and the number of people in the group, and therefore the number of small Bingo teams, you may choose to make the columns bigger (four blank squares for a smaller group) or shorter (three blank squares for three or more teams or less time).

Kanban Iceberg Poster

Kanban Iceberg poster

Kanban Iceberg poster


Improvement Metrics

image (1)Back in November, I spoke at a couple of conferences about some improvement metrics that I’ve been experimenting with. I share them below in the interest of getting feedback from the community on how others might be using them.

Flow efficiency

  • What it measures: The efficiency of your workflow
  • How to measure it: Divide the average time spent working on stories by the average work time plus wait time and express as a percentage.
  • How to use it: Very carefully! Because of necessary dependencies and common-cause variation in work, aiming for 100% flow efficiency shouldn’t necessarily be your goal. Instead, use flow efficiency to ask questions about why your rate is what it is, and see if you can reduce the unnecessary waste in your workflow, such as unnecessary wait queues and avoidable dependencies. (David Anderson claims that having a flow efficiency above 15% is normal, and above 40% is good.) Zsolt Fabók explains it well.
  • What you might replace with it: staff utilization (“resource” efficiency)

Percent accurate and complete

  • What it measures: Quality of your process
  • How to measure it: Divide the number of work items that go through the process without defect, blockage or rework by the total number of work items and express as a percentage.
  • How to use it: Identify the work items that do not go through the process “complete and accurate” and conduct root-cause analysis. The trend should go up over time. Kevin Kriner, who introduced it to me, is the go-to guy for this metric.
  • What you might replace with it: Defect counts

Delivery time

  • What it measures: Average time that it takes for a particular kind of work item to go from the commitment point to delivered.
  • How to measure it: Track the date when each work item crosses the commitment point (i.e., the decision to start working on it) and the date when the work item is considered done (e.g., in production).
  • How to use it: One of the goals of kanban is to deliver a predictable delivery time (a function of WIP). Use average delivery time to create a target delivery time per class of service (and possibly further characterizing stories by type). You can also use it to calculate delivery rate (average work in progress divided by average delivery time) and forecast/plan with it. Like velocity, delivery time should be seen as a means rather than an ends and a gauge rather than a dial.
  • What you might replace with it: Velocity and estimates

Return on investment

  • What it measures: The bang you’re getting for the buck you’re spending.
  • How to measure it: Divide Value (either notional/nebulous relative units or absolute value) by Investment, such as a weekly or monthly team run rate.
  • How to use it: For planning and scheduling work (use in conjunction with economic planning models, like cost of delay/opportunity cost), within a product as well as across products or a program. In the big picture, it helps validate “Is this thing worth it?” and “Should we be spending our money on something else?”
  • What you might replace with it: Burn rate

Face-to-face Friday!

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

— from Agile Manifesto Principles

[Treat] collaboration as an end as well as a means, creating greater opportunity for interactions of higher quality.

— from Kanban from the Inside

I’m embarking on a personal quest to improve communication around my organization, starting with me. In particular, I’m pledging to intentionally interact with people as close to face-to-face as possible every Friday. I’m calling it Face-to-Face Fridays.

It’s not going to be easy. The lazybones in me is happy to zip off an email (five sentences or less, of course) or even post a rejoinder on Slack. And it’s unrealistic to meet face-to-face with a customer or colleague who’s physically not in the office today. But I’m using the following hierarchy of communication to guide my decisions, in descending order of preference:

  1. Face-to-face in person: The person is working today and physically in the office, even if it’s a different floor or building.
  2. Live video chat: The person is working today but not physically in the office and the communication is more than a quick answer or request.
  3. Live phone call: The person is working today but not physically in the office and the communication is more than a quick answer or request, and the person can’t use video.
  4. Slack/instant-message: The person is working today but not physically in the office and the communication is a quick answer or request.
  5. Comment in visual-management tool: If the person is unavailable now, can it wait until the next standup meeting or when the person returns?
  6. Email: Only if the person on vacation or otherwise out of the office today.

Who wants to join me?