Creating Cycle Time Charts from TFS

Unfortunately, TFS doesn’t have ready-made or easily accessible metrics charts like I’m used to with Jira. There is a lot of clicking and waiting, in my experience. Queries have to be created then assigned to widgets then added to dashboards. Worse, sometimes things I need just aren’t available. For example, there is no cycle time chart built into TFS. There is a plug-in available for preview for VSTS, but as of this posting that plugin won’t be available for TFS. With Jira, cycle time and other charts are available and easily accessible out of the box.

Sigh. Nonetheless, we go to work with the tools we have. And if need be, we build the tools we need. In this post, I’ll give a high level description for how to create Cycle Time charts from TFS data using Python. I’ll leave it as an exercise for the reader to learn the value of cycle time charts and how to best use them.

To make life a little easier, I leveraged the open source (MIT License) TFS API Python client from the Open DevOps Community. To make it work for generating Cycle Time charts, I suggested a change to the code for accessing revisions. This was added in a subsequent version. For the remainder of this post I’m going to assume the reader has made themselves familiar with how this library works and resolved any dependencies.

OK, then. Time to roll up our sleeves and get to work.

First, let’s import a few things.

With that, we’re ready to make a call to TFS’s REST API. Again, to make things simple, I create a query in TFS to return the story cards I wish to look at. There are two slices I’m interested in, the past three and the past eight sprints worth of data. In the next block, I call the three sprint query.

With the result set in hand, the script then cycles through each of the returned work items (I’m interested in stories and bugs only) and analyzes the revision events. In my case, I’m interested when items first go into “In Progress,” when they move to “In Testing,” when they move to “Product Owner Approval,” and finally when they move to “Done.” Your status values may differ. The script is capable of displaying cycle time charts for each phase as well as the overall cycle time from when items first enter “In Progress” to when they reach “Done.” An example of overall cycle time is included at the end of this post.

When collected, I send that data to a function for creating the chart using matplotlib. I also save aside the chart data in a CSV file to help validate the chart’s accuracy based on the available data.

It’s a combination scatter plot (plt.scatter), rolling average (plt.plot), average (plt.plot), and standard deviation (plt.fill_between) of the data points. An example output (click for larger image):

The size variation in data points reflects occurrences of multiple items moving into a status of “Done” on the same date, the shaded area is the standard deviation (sample), the blue line is the rolling average, and the orange line is the overall average.

If you are interested in learning more about how to impliment this script in your organization, please contact me directly.

Mindfulness? There’s an app for that!

It appears mindfulness is…well…on a lot of people’s minds lately. I’ve seen this wave come and go twice before. This go around, however, will be propelled and amplified be the Internet. Will it come and go faster? Will there be a lasting and deeper revelation around mindfulness? I predict the former.

Mindfulness is simple and it’s hard. As the saying goes, mindfulness is not what you think.  It was difficult when I first began practicing Rinzai Zen meditation and Aikido many years ago. It’s even more difficult in today’s instant information, instant gratification, and short attention span culture. The uninitiated are ill equipped for the journey.

With this latest mindfulness resurgence expect an amplified parasite wave of meditation teachers and mindfulness coaches. A Japanese Zen Master (Roshi, or “teacher”) I studied with years ago called them “popcorn roshis” – they pop up everywhere and have little substance. No surprise that this wave includes a plethora of mindfulness “popcorn apps.”

Spoiler alert: There are no apps for mindfulness. Attempting to develop mindfulness by using an app on a device that is arguably the single greatest disruptor of mindfulness is much like taking a pill to counteract the side effects of another pill in your quest for health. At a certain point, the pills are the problem. They’ve become the barrier to health.

The “mindfulness” apps that can be found look to be no different than thousands of other non-mindfulness apps offering timers, journaling, topical text, and progress tracking. What they all have in common is that they place your mindfulness practice in the same space as all the other mindfulness killing apps competing for your attention – email, phone, texts, social media, meeting reminders, battery low alarms, and all the other widgets that beep, ring, and buzz.

The way to practicing mindfulness is by the deliberate subtraction of distractions, not the addition of another collection of e-pills. The “killer app” for mindfulness is to kill the app. The act of powering off your smart phone for 30 minutes a day is in itself a powerful practice toward mindfulness. No timer needed. No reminder required. Let it be a random act. Be free! At least for 30 minutes or so.

Mental states like mindfulness, focus, and awareness are choices and don’t arise out of some serendipitous environmental convergence of whatever. They are uniquely human states. Relying on a device or machine to develop mindfulness is decidedly antithetical to the very state of mindfulness. Choosing to develop such mental states requires high quality mentors (I’ve had many) and deliberate practice – a practice that involves subtracting the things from your daily life that work against them.

“For if a person shifts their caution to their own reasoned choices and the acts of those choices, they will at the same time gain the will to avoid, but if they shift their caution away from their own reasoned choices to things not under their control, seeking to avoid what is controlled by others, they will then be agitated, fearful, and unstable.” – Epictetus, Discourses, 2.1.12

 

OnAgile 2017

Attended the Agile Alliance‘s OnAgile 2017 Conference yesterday. This is always an excellent conference and you can’t beat the price!

I created a mind map throughout the course of the conference as a way to organize my thoughts and key points. This doesn’t capture all the points by any means. Just those that stood out as important or new for me. I missed most of the first session due to connectivity issues.

Clouds and Windmills

I recently resigned from the company I had been employed by for over 5 years. The reason? It was time.

During my tenure1 I had the opportunity to re-define my career several times within the organization in a way that added value and kept life productive, challenging, and rewarding. Each re-definition involved a rather extensive mind mapping exercises with hundreds of nodes to described what was working, what wasn’t working, what needed fixing, and where I believed I could add the highest value.

This past spring events prompted another iteration of this process. It began with the question “What wouldn’t happen if I didn’t go to work today?”2 This is the flip of asking “What do I do at work?” The latter is a little self-serving. We all want to believe we are adding value and are earning our pay. The answer is highly filtered through biases, justifications, excuses, and rationalizations. But if in the midsts of a meeting you ask yourself, “What would be different if I were not present or otherwise not participating?”, the answer can be a little unsettling.

This time around, in addition to mind mapping skills, I was equipped with the truly inspiring work of Tanmay Vora and his sketchnote project. Buy me a beer some day and I’ll let you in on a few of my discoveries. Suffice it to say, the overall picture wasn’t good. I was getting the feeling this re-definition cycle was going to include a new employer.

A cascade of follow-on questions flowed from this iteration’s initial question. At the top:

  1. Why am I staying?
  2. Is this work aligned with my purpose?
  3. Have my purpose and life goals changed?

The answers:

  1. The paycheck
  2. No.
  3. A little.

Of course, it wasn’t this simple. The organization changed, as did I, in a myriad of ways. While exploring these questions, I was reminded of a story my Aikido teacher, Gaku Homma, would tell when describing his school. He said it was like a rope. In the beginning, it had just a few threads that joined with him to form a simple string. Not very strong. Not very obvious. But very flexible. Over time, more and more students joined his school and wove their practice into Nippon Kan’s history. Each new thread subtlety changed the character of the emerging rope. More threads, more strength, and more visibility. Eventually, an equilibrium emerges. Some of those threads stop after a few short weeks of classes, other’s (like mine) are 25 years long before they stop, and for a few their thread ends in a much more significant way.

Homma Sensi has achieved something very difficult. The threads that form Nippon Kan’s history are very strong, very obvious, and yet remain very flexible. Even so, there came a time when the right decision for me was to leave, taking with me a powerful set of skills, many good memories, and friendships. The same was true for my previous employer. Their rope is bending in a way that is misaligned with my purpose and goals. Neither good nor bad. Just different. Better to leave with many friendships intact and a strong sense of having added value to the organization during my tenure.

The world is full of opportunities. And sometimes you have to deliberately and intentionally clear all the collected clutter from your mental workspace so those opportunities have a place to land. Be attentive to moments like this before your career is remembered only as someone who yells at clouds and tilts at windmills.


1 By the numbers…

1,788 Stand-ups
1,441 Wiki/Knowledgebase Contributions
311 Sprint/Release Planning Sessions
279 Reviews
189 Retrospectives
101 Projects
31 Internal Meet-ups
22 Agile Cafés
10 Newsletters
5.5 Years
3 Distinct Job Titles
1 Wild Ride

2 My thanks to colleague Lennie Noiles and his presentation on Powerful Questions. While Lennie didn’t ask me this particular question, it was inspired by his presentation.

The Path to Mastery: Begin with the Fundamentals

Somewhere along the path of studying Aikido for 25  years I found a useful perspective on the art that applies to a lot of skills in life.  Aikido is easy to understand. It’s a way of living that leaves behind it a trail of techniques. What’s hard is overcoming the unending stream of little frustrations and often self-imposed limitations. What’s hard is learning how to make getting up part of falling down. What’s hard is healing after getting hurt. What’s hard is learning the importance of recognizing when a white belt is more of a master than you are. In short, what’s hard is mastering the art.

The same can be said about practicing Agile. Agile is easy to understand. It is four fundamental values and twelve principles. The rest is just a trail of techniques and supporting tools – rapid application development, XP, scrum, Kanban, Lean, SAFe, TDD, BDD, stories, sprints, stand-ups – all just variations from a very simple foundation and adapted to meet the prevailing circumstances. Learning how to apply the best technique for a given situation is learned by walking the path toward mastery – working through the endless stream of frustrations and limitations, learning how to make failing part of succeeding, recognizing when you’re not the smartest person in the room, and learning how to heal after getting hurt.

If an Aikidoka is attempting to apply a particular technique to an opponent  and it isn’t working, their choices are to change how they’re performing the technique, change the technique, or invent a new technique based on the fundamentals. Expecting the world to adapt to how you think it should go is a fool’s path. Opponents in life – whether real people, ideas, or situations – are notoriously uncompromising in this regard.  The laws of physics, as they say, don’t much care about what’s going on inside your skull. They stubbornly refuse to accommodate your beliefs about how things “should” go.

The same applies to Agile practices. If something doesn’t seem to be working, it’s time to step in front of the Agile mirror and ask yourself a few questions. What is it about the fundamentals you’re not paying attention to? Which of the values are out of balance? What technique is being misapplied? What different technique will better serve? If your team or organization needs to practice Lean ScrumXPban SAFe-ly than do that. Be bold in your quest to find what works best for your team. The hue and cry you hear won’t be from the gods, only those who think they are – mere mortals more intent on ossifying Agile as policy, preserving their status, or preventing the perceived corruption of their legacy.

But I’m getting ahead of things. Before you can competently discern which practices a situation needs and how to best structure them you must know the fundamentals.

There are no shortcuts.

In this series of posts I hope to open a dialog about mastering Agile practices. We’ll begin by studying several maps that have been created over time that describe the path toward mastery, discuss the benefits and shortcomings of each of these maps, and explore the reasons why many people have a difficult time following these maps. From there we’ll move into the fundamentals of Agile practices and see how a solid understanding of these fundamentals can be used to respond to a wide variety of situations and contexts. Along the way we’ll discover how to develop an Agile mindset.

Agile Team Composition: Generalists versus Specialists

Estimating levels of effort for a set of tasks by a group of individuals well qualified to complete those tasks can efficiently and reliable be determined with a collaborative estimation process like planning poker. Such teams have a good measure of skill overlap. In the context of the problem set, each of the team members are generalist in the sense  it’s possible for any one team member to work on a variety of cross functional tasks during a sprint. Differences in preferred coding language among team members, for example, is less an issue when everyone understands advanced coding practices and the underlying architecture for the solution.

With a set of complimentary technical skills it’s is easier agree on work estimates. There are other benefits that flow from well-matched teams. A stable sprint velocity emerges much sooner. There is greater cross functional participation. And re-balancing the work load when “disruptors” occur – like vacations, illness, uncommon feature requests, etc. – is easier to coordinate.

Once the set of tasks starts to include items that fall outside the expertise of the group and the group begins to include cross functional team members, a process like planning poker becomes increasingly less reliable. The issue is the mismatch between relative scales of expertise. A content editor is likely to have very little insight into the effort required to modify a production database schema. Their estimation may be little more than a guess based on what they think it “should” be. Similarly for a coder faced with estimating the effort needed to translate 5,000 words of text from English to Latvian. Unless, of course, you have an English speaking coder on your team who speaks fluent Latvian.

These distinctions are easy to spot in project work. When knowledge and solution domains have a great deal of overlap, generalization allows for a lot of high quality collaboration. However, when an Agile team is formed to solve problems that do not have a purely technical solution, specialization rather than generalization has a greater influence on overall success. The risk is that with very little overlap specialized team expertise can result in either shallow solutions or wasteful speculation – waste that isn’t discovered until much later. Moreover, re-balancing the team becomes problematic and most often results in delays and missed commitments due to the limited ability for cross functional participation among team mates.

The challenge for teams where knowledge and solution domains have minimal overlap is to manage the specialized expertise domains in a way that is optimally useful, That is, reliable, predictable, and actionable. Success becomes increasingly dependent on how good an organization is at estimating levels of effort when the team is composed of specialists.

One approach I experimented with was to add a second dimension to the estimation: a weight factor to the estimator’s level of expertise relative to the nature of the card being considered. The idea is that with a weighted expertise factor calibrated to the problem and solution contexts, a more reliable velocity emerges over time. In practice, was difficult to implement. Teams spent valuable time challenging what the weighted factor should be and less experienced team members felt their opinion had been, quite literally, discounted.

The approach I’ve had the most success with on teams with diverse expertise is to have story cards sized by the individual assigned to complete the work. This still happens in a collaborative refinement or planning session so that other team members can contribute information that is often outside the perspective of the work assignee. Dependencies, past experience with similar work on other projects, missing acceptance criteria, or a refinement to the story card’s minimum viable product (MVP) definition are all examples of the kind of information team members have contributed. This invariably results in an adjustment to the overall level of effort estimate on the story card. It also has made details about the story card more explicit to the team in a way that a conversation focused on story point values doesn’t seem to achieve. The conversation shifts from “What are the points?” to “What’s the work needed to complete this story card?”

I’ve also observed that by focusing ownership of the estimate on the work assignee, accountability and transparency tend to increase. Potential blockers are surfaced sooner and team members communicate issues and dependencies more freely with each other. Of course, this isn’t always the case and in a future post we’ll explore aspects of team composition and dynamics that facilitate or prevent quality collaboration.

Story Points and Fuzzy Bunnies

The scrum framework is forever tied to the language of sports in general and rugby in particular. We organize our project work around goals, sprints, points, and daily scrums. An unfortunate consequence of organizing projects around a sports metaphor is that the language of gaming ends up driving behavior. For example, people have a natural inclination to associate the idea of story points to a measure of success rather than an indicator of the effort required to complete the story. The more points you have, the more successful you are. This is reflected in an actual quote from a retrospective on things a team did well:

We completed the highest number of points in this sprint than in any other sprint so far.

This was a team that lost sight of the fact they were the only team on the field. They were certain to be the winning team. They were also destine to be he losing team. They were focused on story point acceleration rather than a constant, predictable velocity.

More and more I’m finding less and less value in using story points as an indicator for level of effort estimation. If Atlassian made it easy to change the label on JIRA’s story point field, I’d change it to “Fuzzy Bunnies” just to drive this idea home. You don’t want more and more fuzzy bunnies, you want no more than the number you can commit to taking care of in a certain span of time typically referred to as a “sprint.” A team that decides to take on the care and feeding of 50 fuzzy bunnies over the next two weeks but has demonstrated – sprint after sprint – they can only keep 25 alive is going to lose a lot of fuzzy bunnies over the course of the project.

It is difficult for people new to scrum or Agile to grasp the purpose behind an abstract idea like story points. Consequently, they are unskilled in how to use them as a measure of performance and improvement. Developing this skill can take considerable time and effort. The care and feeding of fuzzy bunnies, however, they get. Particularly with teams that include non-technical domains of expertise, such as content development or learning strategy.

A note here for scrum masters. Unless you want to exchange your scrum master stripes for a saddle and spurs, be wary of your team turning story pointing into an animal farm. Sizing story cards to match the exact size and temperament from all manner of animals would be just as cumbersome as the sporting method of story points. So, watch where you throw your rope, Agile cowboys and cowgirls.

(This article cross-posted at LinkedIn)


Image credit: tsaiproject (Modified in accordance with Creative Commons Attribution 2.0 Generic license)

Autopilot Agile

There is a story about a bunch of corporate employees that have been working together for so long they’ve cataloged and numbered all the jokes they’ve told (and re-told) over the years. Eventually, no one need actually tell the joke. Someone simply yells out something like “Number Nine!” and everyone laughs in reply.

As Agile methodologies and practices become ubiquitous in the business world and jump more and more functional domain gaps, I’m seeing this type of cataloging and rote behavior emerge. Frameworks become reinforced structures. Practices become policies. “Stand-up” becomes code for “status meeting.” “Sprint Review” becomes code for “bigger status meeting.” Eventually, everyone is going through the motions and all that was Agile has drained from the project.

When you see this happening on any of your teams, start introducing small bits of randomness and pattern interruptions. In fact, do this anyway as a preventative measure.

  • One day a week, instead of the usual stand-up drill (Yesterday. Today. In the way.), have each team member answer the question “Why are you working on what today?” Or have each team member talk about what someone else on the team is working on.
  • Deliberately change the order in which team members “have the mic” during stand-ups.
  • Hold a sprint prospective. What are the specific things the team will be doing to further their success? What blockers or impediments can they foresee in the next sprint? Who will be dependent on what work to be completed by when?
  • Set aside story points or time estimates for several sprints. I guarantee the world won’t end. (And if it does, well, we’ve got bigger problems than my failed guarantee.) How did that impact performance? What was the impact on morale?
  • During a backlog refinement session, run the larger story cards through the 5 Whys. Begin with “Why are we doing this work?” This invariably ends up in smaller cards and additions to the backlog.

There’s no end to the small changes that can be introduced on the spur of the moment to shake things up just a bit without upsetting things a lot. The goal is to keep people in a mindset of fluidity, adaptability, and recalibration to the goal.

It’s more than a little ironic and somewhat funny to see autopilot-type behavior emerge in the name of Agile. But if you really want funny…Number Seven!