Sticking to the Plan is Not the Solution

[Note: To commemorate the agile manifesto‘s 20th anniversary, this is the first of 12 posts in no particular order on the manifesto’s principles.]

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

Manifesto for Agile Software Development

Changes to the plan mid-sprint. Constantly-changing priorities. Those words strike fear into the hearts of many team members. One popular agile-health assessment even downgrades as immature those teams whose “plan changes frequently.”

The agile manifesto addresses the idea of change, but it’s not to fear it or to view it as a sign of immaturity — rather the opposite: We are to welcome change and value responding to it over following a plan. So what’s going on?

I think this is a case of confusing the symptom with the problem.

It’s true that constantly changing priorities are often a smell of disorganized product management or unclear strategy. But those dysfunctions need not create additional ones. And it’s quite possible that those changing requirements actually, you know, create competitive advantage. Far too many teams and organizations blame “changing priorities” for their own inability to deliver quality product. I often see diagnoses of teams having difficulty delivering with prescriptions like “Stick to the sprint plan.” Changing requirements is not the problem, so sticking to the plan is not the solution.

The problem is not that the plan changes but that the team or organization lacks the capability to work in such a way as to accommodate change easily.

If changes to the  plan result in any of the following things, you’re doing it wrong — and sticking to the plan isn’t going to help you improve:

  • Taking technical shortcuts
  • Working overtime
  • Getting stressed out
  • More defects (leading to the anti-pattern of defect sprints)

I understand why teams and consultants instinctively reach for the “stable plan” panacea: Teams commit to a slate of work, whether it’s a sprint, iteration or program increment, which they’ve often spent time estimating. Stakeholders (including well-meaning scrum masters) exert some amount of pressure on them to deliver what they’ve committed to, heedless of the nature of knowledge work (i.e., complex). Even when external pressure is low, the very design of a sprint or PI plan implies that success correlates to following the plan; I have encountered many teams who suffer low morale simply because they delivered something different from what they planned, whether it was because they weren’t able to estimate complex work or because a stakeholder changed midway through the plan.

And of course, teams are often held accountable for others’ — product management’s or HiPPOs’ — decisions. Sometimes, it’s because stakeholders behave badly: They are aware that their demand creates stress and pain on the team and refuse to acknowledge and respect a team’s finite capacity — they pile on tasks but refuse to remove any. But often, stakeholders do it out of benign ignorance, lacking a feedback mechanism to inform their choices (or, as Don Reinertsen calls it, “an economic framework”) and assuming that the team will figure out how to handle it. Therefore, it is incumbent upon the team to work in ways so as to accommodate change more comfortably.

However, the solution often put forward is to “stick to the sprint plan.” This flies in the face of the manifesto value to prefer Responding to Change to Following a Plan. Clearly, when a stakeholder makes a mid-plan reprioritization decision, it’s because of a real or perceived business need (a.k.a. “customer’s competitive advantage”). The question of whether it actually does provide a competitive advantage is secondary here. The team has an obligation to “welcome this changing requirement” whenever it happens. In what universe it is a good business decision to follow through on building a feature that we determine customers don’t want simply because we need to follow a plan?

People don’t really have a problem with changing requirements; we have a problem with being prevented from doing quality work. So the problem isn’t that we don’t want new information but rather how we accommodate it.

What makes it difficult to do that? Many things contribute, but I’ve seen two in particular:

  • Too much work in progress
  • Not enough investment in technical practices

The good news is that both of these things are typically within the team’s control.

Too much work in progress

Regardless of whether you work in a timebox (sprint, iteration, PI), your team — as well as you as an individual — should have control over how many things you work on at once. Just because you’ve “committed” to a plan of delivering 20 work items doesn’t mean you need to start all of them at once. That means stopping the inane practice of having individual team members sign up for work items (or worse, having someone assign them) before they start. Limit the number of in-progress work items to something less than the total number of people in the team. (Reinertsen again: “We will always conclude that operating a product development process near full utilization is an economic disaster.”)

This pattern replicates itself at the enterprise level. Organizations are unable to respond to big or urgent demands without significant disruption because they are saturated with “projects.” Most executives that I work with don’t even know how much organizational WIP they have, so for them, making it visible is the first step. But the reality is that when plans become promises and success is measured by conformance to them, we effectively lock in the largest possible batch size, which only exacerbates the problem of dealing with change (and of course is one of the worst things an organization can do).

Not enough investment in technical practices

Talking about XP practices seems so … 2000s. And yet I think that our industry has gotten worse, if anything, with respect to building quality in through things like TDD, pairing, loose coupling, emergent design and continuous integration. Here’s the thing about engineering practices and those stakeholders who are making those changing requirements: They probably don’t have a clue about the practices, and that’s okay. Once you pick up a story, take the time to do it right. The team’s definition of done should include things like proper testing, integration (maybe even deployment!). If it means that the story takes more than two weeks, that’s okay, too. Teams that don’t do these things are reasonably afraid to change the plan because they can’t safely undo (see long-lived branches) or share work (see “not my code”). But here’s the reality: If you avoid doing these things, delivery will never get easier. And, of course, the investment in technical practices will only happen if you limit work in progress.

Other options for better accommodating change:

  • Stop planning in artificial time boxes (whether at the sprint or quarterly level) and instead plan just-in-time using pull (e.g., replenish the Next column based on capacity signals).
  • Stop punishing teams and individuals for not completing their sprint or release “commitments,” including the tracking of the ridiculous sprint-completion rate and other “success” metrics based on conformance to a plan.
  • Spend less time estimating work, as it represents a sunk cost that inhibits change accommodation (and likely has very little information value anyway).
  • Show the cost of interruption. If we’re concerned about undisciplined product owners making careless decisions, make the cost of the decision visible. We can do this with simple flow metrics and post-commit discard rates. If we’re tracking delivery times, we can easily see the “flow debt” that we incur by pausing or dropping work to tend to expedite requests. (Typically this is apparent on a delivery-time frequency chart, manifesting as a bimodal distribution.)
  • Use and make highly visible policies that show how you treat work, such as the selection criteria the team uses to pull work items. For instance, some executives, if they knew that their requests were being treated as “drop everything and work on this now,” would think twice about the requests (I’ve met a few).

The bonus is that if we solve the problem rather than the symptom, we’ll actually save time by avoiding daft things like trying to get better at estimating, debating “carryover” stories and coming up with complicated prioritization schemes (just FIFO it). If we’re able to accommodate change gracefully, those remnants of the agile industrial complex go away.

I would even go so far as to aver that being able to accommodate change mid-sprint is actually a fantastic “agile health” metric. After all, if you can’t do that, you won’t be able to do anything very well, whether it’s scaling, product pivoting or increasing throughput, quality or speed. Don’t hide the problem by requiring the team to “stick to the plan.” Fix the underlying issues so that the team can support the organization in what it’s trying to do, which is to exploit variability for economic benefit. Isn’t that the whole idea of business agility?

One thought on “Sticking to the Plan is Not the Solution

  1. Johan Andersson

    Plans are useless, planning is essential is a known quote. I see the underlying theme of the left over right in the manifesto as saying ”ongoing parallell activities” over ”activities hand-offed at specific points in time”. I think it is the backing out of a hand-off that creates the resistance to change.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s