The scrum framework is forever tied to the language of sports in general and rugby in particular. We organize our project work around goals, sprints, points, and daily scrums. An unfortunate consequence of organizing projects around a sports metaphor is that the language of gaming ends up driving behavior. For example, people have a natural inclination to associate the idea of story points to a measure of success rather than an indicator of the effort required to complete the story. The more points you have, the more successful you are. This is reflected in an actual quote from a retrospective on things a team did well:
We completed the highest number of points in this sprint than in any other sprint so far.
This was a team that lost sight of the fact they were the only team on the field. They were certain to be the winning team. They were also destine to be he losing team. They were focused on story point acceleration rather than a constant, predictable velocity.
More and more I’m finding less and less value in using story points as an indicator for level of effort estimation. If Atlassian made it easy to change the label on JIRA’s story point field, I’d change it to “Fuzzy Bunnies” just to drive this idea home. You don’t want more and more fuzzy bunnies, you want no more than the number you can commit to taking care of in a certain span of time typically referred to as a “sprint.” A team that decides to take on the care and feeding of 50 fuzzy bunnies over the next two weeks but has demonstrated – sprint after sprint – they can only keep 25 alive is going to lose a lot of fuzzy bunnies over the course of the project.
It is difficult for people new to scrum or Agile to grasp the purpose behind an abstract idea like story points. Consequently, they are unskilled in how to use them as a measure of performance and improvement. Developing this skill can take considerable time and effort. The care and feeding of fuzzy bunnies, however, they get. Particularly with teams that include non-technical domains of expertise, such as content development or learning strategy.
A note here for scrum masters. Unless you want to exchange your scrum master stripes for a saddle and spurs, be wary of your team turning story pointing into an animal farm. Sizing story cards to match the exact size and temperament from all manner of animals would be just as cumbersome as the sporting method of story points. So, watch where you throw your rope, Agile cowboys and cowgirls.
There is a story about a bunch of corporate employees that have been working together for so long they’ve cataloged and numbered all the jokes they’ve told (and re-told) over the years. Eventually, no one need actually tell the joke. Someone simply yells out something like “Number Nine!” and everyone laughs in reply.
As Agile methodologies and practices become ubiquitous in the business world and jump more and more functional domain gaps, I’m seeing this type of cataloging and rote behavior emerge. Frameworks become reinforced structures. Practices become policies. “Stand-up” becomes code for “status meeting.” “Sprint Review” becomes code for “bigger status meeting.” Eventually, everyone is going through the motions and all that was Agile has drained from the project.
When you see this happening on any of your teams, start introducing small bits of randomness and pattern interruptions. In fact, do this anyway as a preventative measure.
One day a week, instead of the usual stand-up drill (Yesterday. Today. In the way.), have each team member answer the question “Why are you working on what today?” Or have each team member talk about what someone else on the team is working on.
Deliberately change the order in which team members “have the mic” during stand-ups.
Hold a sprint prospective. What are the specific things the team will be doing to further their success? What blockers or impediments can they foresee in the next sprint? Who will be dependent on what work to be completed by when?
Set aside story points or time estimates for several sprints. I guarantee the world won’t end. (And if it does, well, we’ve got bigger problems than my failed guarantee.) How did that impact performance? What was the impact on morale?
During a backlog refinement session, run the larger story cards through the 5 Whys. Begin with “Why are we doing this work?” This invariably ends up in smaller cards and additions to the backlog.
There’s no end to the small changes that can be introduced on the spur of the moment to shake things up just a bit without upsetting things a lot. The goal is to keep people in a mindset of fluidity, adaptability, and recalibration to the goal.
It’s more than a little ironic and somewhat funny to see autopilot-type behavior emerge in the name of Agile. But if you really want funny…Number Seven!
C. Northcote Parkinson is best known for, not surprisingly, Parkinson’s Law:
Work expands so as to fill the time available for its completion.
But there are many more gems in “Parkinson’s Law and Other Studies in Administration.” The value of re-reading classics is that what was missed on a prior read becomes apparent given the accumulation of a little more experience and the current context. On a re-read this past week, I discovered this:
It is now known that a perfection of planned layout is achieved only by institutions on the point of collapse. This apparently paradoxical conclusion is based upon a wealth of archaeological and historical research, with the more esoteric details of which we need not concern ourselves. In general principle, however, the method pursued has been to select and date the buildings which appear to have been perfectly designed for their purpose. A study and comparison of these has tended to prove that perfection of planning is a symptom of decay. During a period of exciting discovery or progress there is no time to plan the perfect headquarters. The time for that comes later, when all the important work has been done. Perfection, we know, is finality; and finality is death.
Several years back my focus for the better part of a year was on mapping out software design processes for a group of largely non-technical instructional designers. If managing software developers is akin to herding cats, finding a way to shepherd non-technical creative types such as instructional designers (particularly old school designers) can be likened to herding a flock of canaries – all over the place in three dimensions.
What made this effort successful was framing the design process as a set of guidelines that were easy to track and monitor. The design standards and common practices, for example, consisted of five bullet points. Building just enough fence to keep everyone in the same area while limiting free range behaviors to specific places was important. These were far from perfect, but they allowed for the dynamic vitality suggested by Parkinson. If the design standards and common practices document ever grew past something that could fit on one page, it would suggest the company was moving toward over specialization and providing services to a narrow slice of the potential client pie. In the rapidly changing world of adult eduction, this level of perfection would most certainly suggest decay and risk collapse as client needs change.
In Part 1 of this series, we set the frame for how to use time as a metric for assessing Agile team and project health. In Part 2, we looked at shifts in the cross-over point between burn-down and burn-up charts. In Part 3, we’ll look at other asymmetries and anomalies that can appear in time burn-down/burn-up charts and explore the issues the teams may be struggling with under these circumstances.
Figure 1 shows a burn-up that by the end of the sprint significantly exceeded the starting value for the original estimate.
There isn’t much mystery around a chart like this. The time needed to complete the work was significantly underestimated. The mystery is in the why and what that led to this situation.
Where there unexpected technical challenges?
Were the stories poorly defined?
Were the acceptance criteria unclear?
Were the sprint goals, objectives, or minimum viable product definition unclear?
Depending on the tools used to capture team metrics, it can be helpful to look at individual performances. What’s the differential between story points and estimated time vs actual time for each team member? Hardly every useful as a disciplinary tool, this type of analysis can be invaluable for knowing who needs professional development and in what areas.
In this case, there were several technical challenges related to new elements of the underlying architecture and the team put in extra hours to resolve them. Even so, they were unable to complete all the work they committed to in the sprint. The the scrum master and product owner need to monitor this so it isn’t a recurrent event or they risk team burnout and morale erosion if left unchecked. There are likely some unstated dependencies or skill deficiencies that need to be put on the table for discussion during the retrospective.
Figure 2 shows, among other things, unexpected jumps in the burn-down chart. There is clearly a significant amount of thrashing evident in the burn-down (which stubbornly refuses to actually burn down.)
Questions to explore:
Are cards being brought into the sprint after the sprint has started and why?
Are original time estimates being changed on cards after the sprint has started?
Is there a stakeholder in the grass, meddling with the team’s commitment?
Was a team member added to the team and cards brought into the sprint to accommodate the increased bandwidth?
Whatever is causing the thrashing, is the team (delivery team members, scrum master, and product owner) aware of the changes?
Scope change during a sprint is a very undesirable practice. Not just because it goes against the scrum framework, but more so because it almost always has an adverse effect on team morale and focus. If there is an addition to the team, better to set that person to work helping teammates complete the work already defined in the sprint and assign them cards in the next sprint.
If team members are adjusting original time estimates for “accuracy” or whatever reason they may provide, this is little more than gaming the system. It does more harm than good, assuming management is Agile savvy and not intent on using Agile metrics for punitive purposes. On occasion I’ve had to hide the original time estimate entry field from the view of delivery team members and place it under the control of the product owner – out of sight, out of mind. It’s less a concern to me that time estimates are “wrong,” particularly if time estimate accuracy is showing improvement over time or the delta is a somewhat consistent value. I can work with an delivery team member’s time estimates that are 30% off if they are consistently 30% off.
In the case of Figure 2 it was the team’s second sprint and at the retrospective the elephant was called out from hiding: The design was far from stable. The decision was made to set aside scrum in favor of using Kanban until the numerous design issues could be resolved.
Figure 3 shows a burn-down chart that doesn’t go to zero by the end of the sprint.
The team missed their commit and quite a few cards rolled to the next sprint. Since the issue emerged late in the sprint there was little corrective action that could be taken. The answers were left to discovery during the retrospective. In this case, one of the factors was the failure to move development efforts into QA until late in the sprint. This is an all too common issue in cases where the sprint commitments were not fully satisfied. For this team the QA issue was exacerbated by the team simply taking on more than they thought they could commit to completing. The solution was to reduce the amount of work the team committed to in subsequent sprints until a stable sprint velocity emerged.
For a two week sprint on a project that is 5-6 sprints in, I usually don’t bother looking at time burn-down/burn-up charts for the first 3-4 days. Early trends can be misleading, but by the time a third of the sprint has been completed this metric will usually start to show trends that suggest any emergent problems. For new projects or for newly formed teams I typically don’t look at intra-sprint time metrics until much later in the project life cycle as there are usually plenty of other obvious and more pressing issues to work through.
I’ll conclude by reiterating my caution that these metrics are yard sticks, not micrometers. It is tempting to read too much into pretty graphs that have precise scales. Rather, the expert Agilest will let the metrics, whatever they are, speak for themselves and work to limit the impact of any personal cognitive biases.
In this series we’ve explored several ways to interpret the signals available to us in estimated time burn-down and actual time burn-up charts. There are numerous others scenarios that can reveal important information from such burn-down/burn-up charts and I would be very interested in hearing about your experiences with using this particular metric in Agile environments.
In Part 1 of this series, we set the frame for how to use time as a metric for assessing Agile team and project health. In Part 2, we’ll look at shifts in the cross-over point between burn-down and burn-up charts and explore what issues may be in play for the teams under these circumstances.
Figure 1 shows a cross-over point occurring early in the sprint.
This suggests the following questions:
Is the team working longer hours than needed? If so, what is driving this effort? Are any of the team members struggling with personal problems that have them working longer hours? Are they worried they may have committed to more work than they can complete in the sprint and are therefore trying to stay ahead of the work load? Has someone from outside the team requested additional work outside the awareness of the product owner or scrum master?
Has the team over estimated the level of effort needed to complete the cards committed to the sprint? If so, this suggests an opportunity to coach the team on ways to improve their estimating or the quality of the story cards.
Has the team focused on the easy story cards early in the sprint and work on the more difficult story cards is pending? This isn’t necessarily a bad thing, just something to know and be aware of after confirming this with the team. If accurate, it also points out the importance of using this type of metric for intra-sprint monitoring only and not extrapolate what it shows to a project-level metric.
The answer to these questions may not become apparent until later in the sprint and the point isn’t to try and “correct” the work flow based on relatively little information. In the case of Figure 1, the “easy” cards had been sized as being more difficult than they actually were. The more difficult cards were sized too small and a number of key dependencies were not identified prior to the sprint planning session. This is reflected in the burn-up line that significantly exceeds the initial estimate for the sprint, the jumps in the burn-down line, and subsequent failure to complete a significant portion of the cards in the sprint backlog. All good fodder for the retrospective.
Figure 2 shows a cross-over point occurring late in the sprint.
On the face of it there are two significant stretches of inactivity. Unless you’re dealing with a blatantly apathetic team, there is undoubtedly some sort of activity going on. It’s just not being reflected in the work records. The task is to find out what that activity is and how to mitigate it.
The following questions will help expose the cause for the extended periods of apparent inactivity:
Are one or more members not feeling well or are there other personal issues impacting an individual’s ability to focus?
Have they been poached by another project to work on some pressing issue?
Are they waiting for feedback from stakeholders, clients, or other team members?
Are the story cards unclear? As the saying goes, story cards are an invitation to a conversation. If a story card is confusing, contradictory, or unclear than the team needs to talk about that. What’s unclear? Where’s the contradiction? As my college calculus professor used to ask when teaching us how to solve math problems, “Where’s the source of the agony?”
The actual reasons behind Figure 2 were two fold. There was a significant technical challenge the developers had to resolve that wasn’t sufficiently described by any of the cards in the sprint and later in the sprint several key resources were pulled off the project to deal with issues on a separate project.
Figure 3 shows a similar case of a late sprint cross-over in the burn-down/burn-up chart. The reasons for this occurrence were quite different than those shown in Figure 2.
This was an early sprint and a combination of design and technical challenges were not as well understood as originally thought at the sprint planning session. As these issues emerged, additional cards were created in the product backlog to be address in future sprints. Nonetheless, the current sprint commitment was missed by a significant margin.
In Part 3, we’ll look at other asymmetries and anomalies that can appear in time burn-down/burn-up charts and explore the issues may be in play for the teams under these circumstances.
Some teams choose to use card level estimated and actual time as one of the level of effort or performance markers for project progress and health. For others it’s a requirement of the work environment due to management or business constraints. If your situation resembles one of these cases then you will need to know how to use time metrics responsibly and effectively. This series of articles will establish several common practices you can use to develop your skills for evaluating and leveraging time-based metrics in an Agile environment.
It’s important to keep in mind that time estimates are just one of the level of effort or performance markers that can be used to track team and project health. There can, and probably should be other markers in the overall mix of how team and project performance is evaluated. Story points, business value, quality of information and conversation from stand-up meetings, various product backlog characteristics, cycle time, and cumulative flow are all examples of additional views into the health and progress of a project.
In addition to using multiple views, it’s important to be deeply aware of the strengths and limits presented by each of them. The limits are many while the strengths are few. Their value comes in evaluating them in concert with one another, not in isolation. One view may suggest something that can be confirmed or negated by another view into team performance. We’ll visit and review each of these and other metrics after this series of posts on time.
The examples presented in this series are never as cut and dried as presented. Just as I previously described multiple views based on different metrics, each metric can offer multiple views. My caution is that these views shouldn’t be read like an electrocardiogram, with the expectation of a rigidly repeatable pattern from which a slight deviation could signal a catastrophic event. The examples are extracted from hundreds of sprints and dozens of projects over the course of many years and are more like seismology graphs – they reveal patterns over time that are very much context dependent.
Estimated and actual time metrics allow teams to monitor sprint progress by comparing time remaining to time spent. Respectively, this will be a burn-down and a burn-up chart in reference to the direction of the data plotted on the chart. In Figure 1, the red line represents the estimated time remaining (burn-down) while the green line represents the amount of time logged against the story cards (burn-up) over the course of a two week sprint. (The gray line is a hypothetical ideal for burn-down.)
The principle value of a burn-down/burn-up chart for time is the view it gives to intra-sprint performance. I usually look at this chart just prior to a teams’ daily stand-up to get a sense if there are any questions I need to be asking about emerging trends. In this series of posts we’ll explore several of the things to look for when preparing for a stand-up. At the end of the sprint, the burn-down/burn-up chart can be a good reference to use during the retrospective when looking for ways to improve.
The sprint shown in Figure 1 is about as ideal a picture as one can expect. It shows all the points I look for that tell me, insofar as time is concerned, the sprint performance is in good health.
There is a cross-over point roughly in the middle of the sprint.
At the cross-over point about half of the estimated time has been burned down.
The burn-down time is a close match to the burn-up at both the cross-over point and the end of the sprint.
The burn-down and burn-up lines show daily movement in their respective directions.
In Part 2, we’ll look at several cases where the cross-over point shifts and explore the issues the teams under these circumstances might be struggling with.
In Parkinson’s Law of Triviality and Story Sizing, I touched on the issue of relative expertise among team members during collaborative efforts to size story cards. I’d like to expand on that idea by considering several types of team compositions.
Team 1 is a tight knit band of four software developers represented in Figure 1.
Team 1 represents a near-ideal team composition for a typical software related project. However, the real world isn’t so generous in it’s allocation of near-ideal, let alone ideal, teams. A typical team for a software related project is more likely to resemble Team 2, as represented in Figure 2.
As Agile practices become more ubiquitous in the business world, team composition beings to resemble Team 3, as shown in Figure 3.
The mix now includes non-technical people – content developers and editors, strategists, and designers. Even assuming an equal level of experience in their respective domains, the company, and the business environment, there is very little overlap. Arriving at a consensus during a story sizing exercise now becomes a significant challenge. But again, the real world isn’t even so kind as this. We are increasingly more likely to encounter teams that resemble Team 4 as shown in Figure 4.
As before, the relative circle of expertise among team members can vary quite a bit. When a team resembles the composition of Team 4, the software developers (HTML5/CSS and C#) will have trouble understanding what the Learning Strategist is asking for while the Learning Strategist may not understand why what he wants the software developers to deliver isn’t possible.
When I’ve attempted to facilitate story sizing sessions with teams that resemble Team 4 they either become quite contentious (and therefore time consuming) or team members that don’t have the expertise to understand a particular card simply accept the opinion of the stronger voices. Neither one of these situations is desirable.
To counteract these possibilities, I’ve found it much more effective to have the card assignee determine the card size (points and time estimate) and work to have the other team members ask questions about the work described on the card such that the assignee and the team better understand the context in which the card is positioned. The team members that lack domain expertise, it turns out, are in a good position to help craft good acceptance criteria.
Who will consume the work product that results from the card? (dependencies)
What cards need to be completed before a particular card can be worked on? (dependencies)
Is everything known about what a particular card needs before it can be completed? (dependencies, discovery, exploration)
At the end of a brief conversation where the entire team is working to evaluate the card for anything other than level of effort (time) and complexity (points), it is not uncommon for the assignee to reconsider their sizing, break the card into multiple cards, or determine the card shouldn’t be included in the sprint backlog. In short, it ends up being a much more productive conversation if teammates aren’t haggling over point distinctions or passively accepting what more experienced teammates are advocating. The benefit to the product owner is that they now have additional information that will undoubtedly influence the product backlog prioritization.
Further, the most important person you will ever have to communicate with is yourself. We all think we can remember the design decisions we’ve made, but there will always be the Why did I do that? moments hiding in our future. If we keep the scraps of papers we did our initial diagramming on when we started a design, we’ll eventually find them a useful reference.
That strikes me as a good benchmark for acceptable documentation in Agile. Whether coder, UI/UX designer, data architect, or whatever, if you are keeping a good record of what you decided and why, you’ll probably be able to recreate the rationale for why things got to be the way they are for anybody who needs to know. Especially if that anybody is you. And there is a good chance that someone following in your footsteps will be able to pick up the same rationale even in your absence. All this without putting an unnecessary burden on project progress for the sake of detailed documentation.
Of course, what qualifies as “good” is the tricky part. A suggested threshold would be to specify only as much information as makes sense or for what is known given the current situation. Documentation should be subject to iterative practices just as much as code.
Following from the Agile Manifesto value that is the title of this post, Principle #2 may be the most mis-interpreted and misunderstood principle among the set of twelve. Teams frequently behave as if this principle was prefaced with the word “always.”
Constantly shifting requirements leads to a frustrating and unsatisfying environment in which to work. It feeds burn-out and loss of morale. The satisfaction of a job well done depends on the opportunity to actually finish the job, no matter how small. Consider the effects on a finish carpenter who has just spent several days installing and trimming a full set of kitchen cabinets when the homeowner declares they want to change the kitchen design such that all those new cabinets will need to be ripped out and work begun on a new design. Or a film editor who has just worked 21 days straight to pare down an hour’s worth of video to fit into 7 minutes only to learn the scene has to be re-shot from scratch in order to match a change in the storyline.
Of course, the second principle does not state we should “always welcome changing requirements.” Nor does anyone I know claim that it does. But that doesn’t stop people from behaving as if it did. The rationale offered for agreeing to change requests from the stakeholders may be “We’re an agile shop and agile welcomes changing requirements” when, in fact, the change was agreed to because the product owner didn’t challenge the value of the change or make clear the consequences to the stakeholders. Or the original design was, and remains, needlessly ambiguous. Or the stakeholders have changed without renegotiating the contract or working agreements. Or any number of reasons that are conveniently masked with “welcoming changing requirements.” At some point, welcoming changing requirements is about as attractive as welcoming a rabid dog into the house. This won’t end well.
So, what kind of change is the Agile Manifesto referring to? There are several key scenarios that embody the need for flexibility around requirements.
The change that results from periods of deliberate design, such as during design sprints.
The change that is driven by the lessons learned from exploration and prototyping. If it is understood that the work being “completed” is for the purposes of testing a hypothesis and the expectation is that the work will most likely be thrown away, there can still be a great deal of satisfaction derived from the effort as the actual deliverable wasn’t working software, but the lessons from the experiment (usually in the form of a wireframe or prototype.)
So what is it that locks out the option for additional change? It’s a simple event, really. A decision is made.
Each of these scenarios where adapting to lessons and discovery is essential nonetheless end in a decision, a leverage point from which progress can be made toward a final deliverable. Each of these decisions can themselves form the basis of a series of experiments which, depending on the eventual outcome, may change. Often, a single decision point may look good but when several decisions are evaluated together they may suggest a new direction and therefore impact the requirements. If the cumulative insight from a series of decisions results in the need to change direction, that shift is usually more substantial and on the scale of a project plan pivot rather than a simple response to a single change in a single requirement. The need to pivot cannot reliably be revealed if the underlying decisions do not coalesce into some sort of stable understanding of the emerging design.
Changing requirements cannot go on indefinitely or a final product will never be delivered. Accepting change for the sake of change is what gets teams into trouble.
Much like the forces on evolution, there will always be some external force that seeks to change the project requirements so that the delivered product can be stronger, faster, better, taller, smarter, etc. This must be countered by clear definitions of “minimum viable” and “good enough” relative to what the customer is expecting.
In addition, product owners would serve their teams well by vigorously challenging any proposed changes to the requirements.
What is the source of the change?
Is it random change or triggered by some agent that does not announce its arrival ahead of time?
Was the change in requirements a surprise? If so, why was it a surprise?
Will this (or something like it) happen again? With what frequency? At what probability?
I had the privilege of presenting to a group of potential interns from the Colorado School of Mines interested in agile project management. Action shot…
The slide shows what we can offer them as interns: Failure, chaos, and confusion. I unpacked that as follows…
It’s important interns have the opportunity to learn how to fail in small, deliberate and safe increments along with the opportunity to learn how to extract every possible lesson from failures and how those failures lead to eventual success. Much of our business is driven by experimentation and hypothesis testing. Most of those experiments will fail, at least initially.
We strive to be anti-fragile. One way to accomplish this is to be good at working under chaotic and ambiguous conditions. When involved with highly evolutionary design sessions, shifting sands can seem like the most solid ground around.
One of the values for bringing interns into the organization is the fresh perspective they offer. Why waste that on having them fetch coffee? However, interns can often be intimidated by working with people who have decades of experience under their belt. So it’s important they know they have the opportunity to work in an environment that expects questions and recognizes no one knows it all. They are in an environment that seeks alternative points of view. In this organization, everyone gets their own coffee.