Agile and Changing Requirements or Design

I hear this (or some version) more frequently in recent years than in past:

Agile is all about changing requirements at anytime during a project, even at the very end.

I attribute the increased frequency to the increased popularity of Agile methods and practices.

That the “Responding to change over following a plan” Agile Manifesto value is cherry picked so frequently is probably due to a couple of factors:

  • It’s human nature for a person to resist being cornered into doing something they don’t want to do. So this value gets them out of performing a task.
  • The person doesn’t understand the problem or doesn’t have a solution. So this value buys them time to figure out how to solve the problem. Once they do have a solution, well, it’s time to change the design or the requirements to fit the solution. This reason isn’t necessary bad unless it’s the de facto solution strategy.

The intent behind the “Responding to change” value, and the way successful Agile is practiced, does not allow for constant and unending change. Taken to it’s logical conclusion, nothing would ever be completed and certainly nothing would ever be released to the market.

I’m not going to rehash the importance of the preposition in the value statement. Any need to explain the relativity implied by it’s use has become a useful signal for me to spend my energies elsewhere. But for those who are not challenged by the grammar, I’d like to say a few thing about how to know when change is appropriate and when it’s important to follow a plan.

The key is recognizing and tracking decision points. With traditional project management, decisions are built-in to the project plan. Every possible bit of work is defined and laid out on a Gantt chart, like the steel rails of a train track. Deviation from this path would be actively discouraged, if it were considered at all.

Using an Agile process, decision points that consider possible changes in direction are built into the process – daily scrums, sprint planning, backlog refinement, reviews and demonstrations at the end of sprints and releases, retrospectives, acceptance criteria, definitions of done, continuous integration – these all reflect deliberate opportunities in the process to evaluate progress and determine whether any changes need to be made. These are all activities that represent decisions or agreements to lock in work definitions for short periods of time.

For example, at sprint planning, a decision is made to complete a block of work in a specified period of time – often two weeks. After that, the work is reviewed and decisions are made as to whether or not that work satisfies the sprint goal and, by extension, the product vision. At this point, the product definition is specifically opened up for feedback from the stakeholders and any proposed changes are discussed. Except under unique circumstances, changes are not introduced mid-sprint and the teams stick to the plan.

Undoing decisions or agreements only happens if there is supporting information, such as technical infeasibility or a significant market shift. Undoing decisions and agreements doesn’t happen just because “Agile is all about changing requirements.” Agile supports changing requirements when there is good reason to do so, irrespective of the original plan. With traditional project management, it’s all about following the plan and change at any point is resisted.

This is the difference. With traditional project management, decisions are built-in to the project plan. With Agile they are adapted in.

How does Agile help with long term planning?

I’m often involved in discussions about Agile that question its efficacy in one way or another. This is, in my view, a very good thing and I highly encourage this line of inquiry. It’s important to challenge the assumptions behind Agile so as to counteract any complacency or expectation that it is a panacea to project management ills. Even so, with apologies to Winston Churchill, Agile is the worst form of project management…except for all the others.

Challenges like this also serve to instill a strong understanding of what an Agile mindset is, how it’s distinct from Agile frameworks, tools, and practices, and where it can best be applied. I would be the first to admin that there are projects for which a traditional waterfall approach is best. (For example, maintenance projects for nuclear power reactors. From experience, I can say traditional waterfall project management is clearly the superior approach in this context.)

A frequent challenge the idea that with Agile it is difficult to do any long-term planning.

Consider the notion of vanity vs actionable metrics. In many respects, large or long-term plans represent a vanity leading metric. The more detail added to a plan, the more people tend to believe and behave as if such plans are an accurate reflection of what will actually happen. “Surprised” doesn’t adequately describe the reaction when reality informs managers and leaders of the hard truth. I worked a multi-million dollar project many years ago for a Fortune 500 company that ended up being canceled. Years of very hard work by hundreds of people down the drain because projected revenues based on a software product design over seven years old were never going to materialize. Customers no longer wanted or needed what the product was offering. Our “solution” no longer had a problem to solve.

Agile – particularly more recent thinking around the values and principles in the Manifesto – acknowledges the cognitive biases in play with long-term plans and attempts to put practices in place that compensate for the risks they introduce into project management. One such bias is reflected in the planning fallacy – the further out the planning window extends into the future, the less accurate the plan. An iterative approach to solving problems (some of which just happen to use software) challenges development teams on up through managers and company leaders to reassess their direction and make much smaller course corrections to accommodate what’s being learned. As you can well imagine, we may have worked out how to do this in the highly controlled and somewhat predictable domain of software development, however, the critical areas for growth and Agile applicability are at the management and leadership levels of the business.

Another important aspect the Agile mindset is reflected in the Cone of Uncertainty. It is a deliberate, intentional recognition of the role of uncertainty in project management. Yes, the goal is to squeeze out as much uncertainty (and therefore risk) as possible, but there are limits. With a traditional project management plan, it may look like everything has been accounted for, but the rest of the world isn’t obligated to follow the plan laid out by a team or a company. In essence, an Agile mindset says, “Lift your gaze up off of the plan (the map) and look around for better, newer, more accurate information (the territory.) Then, update the plan and adjust course accordingly.” In Agile-speak, this is what is behind phrases like “delivery dates emerge.”

Final thought: You’ll probably hear me say many times that nothing in the Agile Manifesto can be taken in isolation. It’s a working system and some parts if it are more relevant than others depending on the project and the timing. So consider what I’ve presented here in concert with the Agile practices of developing good product visions and sprint goals. Product vision and sprint goals keep the project moving in the desired direction without holding it on an iron-rails-track that cannot be changed without a great deal of effort, if at all.

So, to answer the question in the post title, Agile helps with long term planning by first recognizing the the risks inherent in such plans and implementing process changes that mitigate or eliminate those risks. Unpacking that sentences would consist of listing all the risks inherent with long-term planning and the mechanics behind and reasons why scrum, XP, SAFe, LeSS, etc., etc., etc. have been developed.

Agile Interns

I had the privilege of presenting to a group of potential interns from the Colorado School of Mines interested in agile project management. Action shot…

The slide shows what we can offer them as interns: Failure, chaos, and confusion. I unpacked that as follows…

Failure

It’s important interns have the opportunity to learn how to fail in small, deliberate and safe increments along with the opportunity to learn how to extract every possible lesson from failures and how those failures lead to eventual success. Much of our business is driven by experimentation and hypothesis testing. Most of those experiments will fail, at least initially.

Chaos

We strive to be anti-fragile. One way to accomplish this is to be good at working under chaotic and ambiguous conditions. When involved with highly evolutionary design sessions, shifting sands can seem like the most solid ground around.

Confusion

One of the values for bringing interns into the organization is the fresh perspective they offer.  Why waste that on having them fetch coffee? However, interns can often be intimidated by working with people who have decades of experience under their belt. So it’s important they know they have the opportunity to work in an environment that expects questions and recognizes no one knows it all. They are in an environment that  seeks alternative points of view. In this organization, everyone gets their own coffee.

That Isn’t What I Expected

Adverse surprises during a team driven project are about as welcome as whooping cough at a glassblowers convention. Minimizing the opportunity for surprises comes down to how well expectations are defined at the very beginning and how well they are managed during the course of the project. Unidentified expectations are like landmines in the project path. When they explode, it’s bad and the course of the project WILL change. Product owners can’t elucidate all the expectations a stakeholder may have, but with experience they can define the major ones. With practice and attention, experienced product owners can tease out all but the minor expectations that are often dependant on discovery within the project’s sprints.

Key to this skill is knowing the questions to ask at the beginning. In my experience, stakeholders rarely deliberately hold back their expectations. They just don’t know what they don’t know and it is the product owner’s responsibility to establish clarity around expectations. Intuitively obvious expectations rarely play out as such.

A few questions for stakeholders that I’ve found helpful:

  • What business problems do you intend to solve with this project?
  • What do you need to see to know the project is progressing?
  • What will you see when the project is done?
  • What is your availability commitment for the duration of the project?
  • How often to you expect to meet to review progress?
  • How long do YOU think it will take to complete the project?
  • To what extent are your functional groups integrated?
  • Describe your process from design to development to implementation?
  • Are there other stakeholders we need to know about and include?
  • What factors have helped and hurt success with past projects?

This is by no means an exhaustive list of questions. And they may even seem obvious. The answers, however, are almost never obvious.

I also find it effective to challenge stakeholders with scenarios.

  • What happens if we discover this project will take two months longer than expected?
  • What happens if we discover a desired solution is technically unfeasible?
  • How will you support us if we encounter significant delays from client deliverables?

Product owners need to keep pursuing clarity around expectations until they are satisfied they have a good understanding of how the people side of the project will unfold. This will go a long way to helping the development team handle the technical side of the project.

While stakeholders answer these questions, product owners need to pay attention not just the words stakeholders use, but how they answer as well. They need to be scanning for underlying assumptions that drive the answers. These often reflect relevant cultural drivers which can signal significant expectations seemingly unrelated to the project at hand.

For example, perhaps the product owner has established the expectation of a three business day turnaround for feedback from the stakeholder when asked to review periodic project deliverables. “We can complete our reviews within three business days and work to get them to you as fast as possible,” says the stakeholder somewhat hesitantly as he looks off into the distance. Where the pain begins is when the inattentive product owner discovers that, while the feedback may be ready, the client organization has a thick layer of compliance and the feedback is hung up in legal for an additional one to two weeks…every time. If the stakeholder’s responses reflect something less than 100% commitment, keep asking questions designed to surface underlying assumptions.

As each sprint concludes, and eventually the project as well, the savvy product owner knows their work with expectations isn’t complete. Retrospectives for each sprint, each release, and the project conclusion should make note of the expectations that were missed and consider questions that could have been asked that would have helped surface the surprise expectations sooner.

This is also an excellent time to consider if any of the existing expectations have changed or if it appears there may be new expectations emerging. Internal forces, such as changes in team composition, and external forces, such as shifting market demands, can significantly impact the set of expectations a product owner is tasked with managing.

If  you expected to read these kinds of things about surfacing stakeholder expectations, then you’re probably an experienced product owner.