Creating Cycle Time Charts from TFS

Unfortunately, TFS doesn’t have ready-made or easily accessible metrics charts like I’m used to with Jira. There is a lot of clicking and waiting, in my experience. Queries have to be created then assigned to widgets then added to dashboards. Worse, sometimes things I need just aren’t available. For example, there is no cycle time chart built into TFS. There is a plug-in available for preview for VSTS, but as of this posting that plugin won’t be available for TFS. With Jira, cycle time and other charts are available and easily accessible out of the box.

Sigh. Nonetheless, we go to work with the tools we have. And if need be, we build the tools we need. In this post, I’ll give a high level description for how to create Cycle Time charts from TFS data using Python. I’ll leave it as an exercise for the reader to learn the value of cycle time charts and how to best use them.

To make life a little easier, I leveraged the open source (MIT License) TFS API Python client from the Open DevOps Community. To make it work for generating Cycle Time charts, I suggested a change to the code for accessing revisions. This was added in a subsequent version. For the remainder of this post I’m going to assume the reader has made themselves familiar with how this library works and resolved any dependencies.

OK, then. Time to roll up our sleeves and get to work.

First, let’s import a few things.

import csv
from requests_ntlm import HttpNtlmAuth
from tfs import TFSAPI
import numpy as np
import matplotlib.pyplot as plt

With that, we’re ready to make a call to TFS’s REST API. Again, to make things simple, I create a query in TFS to return the story cards I wish to look at. There are two slices I’m interested in, the past three and the past eight sprints worth of data. In the next block, I call the three sprint query.

# Anything in angle brackets "<>" is a placeholder for actual values.
# I keep the values in this block in a configuration file and load
# them when the script runs.

if __name__ == "__main__":

    project = TFSAPI("{}".format("<server_url>"), project="<Collection/Project>", user="<user>", password="<password>", auth_type=HttpNtlmAuth)

    query = project.run_query('My Queries/Last3Sprints')

With the result set in hand, the script then cycles through each of the returned work items (I’m interested in stories and bugs only) and analyzes the revision events. In my case, I’m interested when items first go into “In Progress,” when they move to “In Testing,” when they move to “Product Owner Approval,” and finally when they move to “Done.” Your status values may differ. The script is capable of displaying cycle time charts for each phase as well as the overall cycle time from when items first enter “In Progress” to when they reach “Done.” An example of overall cycle time is included at the end of this post.

When collected, I send that data to a function for creating the chart using matplotlib. I also save aside the chart data in a CSV file to help validate the chart’s accuracy based on the available data.

It’s a combination scatter plot (plt.scatter), rolling average (plt.plot), average (plt.plot), and standard deviation (plt.fill_between) of the data points. An example output (click for larger image):

The size variation in data points reflects occurrences of multiple items moving into a status of “Done” on the same date, the shaded area is the standard deviation (sample), the blue line is the rolling average, and the orange line is the overall average.

If you are interested in learning more about how to implement this script in your organization, please contact me directly.

OnAgile 2017

Attended the Agile Alliance‘s OnAgile 2017 Conference yesterday. This is always an excellent conference and you can’t beat the price!

I created a mind map throughout the course of the conference as a way to organize my thoughts and key points. This doesn’t capture all the points by any means. Just those that stood out as important or new for me. I missed most of the first session due to connectivity issues.

Clouds and Windmills

I recently resigned from the company I had been employed by for over 5 years. The reason? It was time.

During my tenure1 I had the opportunity to re-define my career several times within the organization in a way that added value and kept life productive, challenging, and rewarding. Each re-definition involved a rather extensive mind mapping exercises with hundreds of nodes to described what was working, what wasn’t working, what needed fixing, and where I believed I could add the highest value.

This past spring events prompted another iteration of this process. It began with the question “What wouldn’t happen if I didn’t go to work today?”2 This is the flip of asking “What do I do at work?” The latter is a little self-serving. We all want to believe we are adding value and are earning our pay. The answer is highly filtered through biases, justifications, excuses, and rationalizations. But if in the midsts of a meeting you ask yourself, “What would be different if I were not present or otherwise not participating?”, the answer can be a little unsettling.

This time around, in addition to mind mapping skills, I was equipped with the truly inspiring work of Tanmay Vora and his sketchnote project. Buy me a beer some day and I’ll let you in on a few of my discoveries. Suffice it to say, the overall picture wasn’t good. I was getting the feeling this re-definition cycle was going to include a new employer.

A cascade of follow-on questions flowed from this iteration’s initial question. At the top:

  1. Why am I staying?
  2. Is this work aligned with my purpose?
  3. Have my purpose and life goals changed?

The answers:

  1. The paycheck
  2. No.
  3. A little.

Of course, it wasn’t this simple. The organization changed, as did I, in a myriad of ways. While exploring these questions, I was reminded of a story my Aikido teacher, Gaku Homma, would tell when describing his school. He said it was like a rope. In the beginning, it had just a few threads that joined with him to form a simple string. Not very strong. Not very obvious. But very flexible. Over time, more and more students joined his school and wove their practice into Nippon Kan’s history. Each new thread subtlety changed the character of the emerging rope. More threads, more strength, and more visibility. Eventually, an equilibrium emerges. Some of those threads stop after a few short weeks of classes, other’s (like mine) are 25 years long before they stop, and for a few their thread ends in a much more significant way.

Homma Sensi has achieved something very difficult. The threads that form Nippon Kan’s history are very strong, very obvious, and yet remain very flexible. Even so, there came a time when the right decision for me was to leave, taking with me a powerful set of skills, many good memories, and friendships. The same was true for my previous employer. Their rope is bending in a way that is misaligned with my purpose and goals. Neither good nor bad. Just different. Better to leave with many friendships intact and a strong sense of having added value to the organization during my tenure.

The world is full of opportunities. And sometimes you have to deliberately and intentionally clear all the collected clutter from your mental workspace so those opportunities have a place to land. Be attentive to moments like this before your career is remembered only as someone who yells at clouds and tilts at windmills.


1 By the numbers…

1,788 Stand-ups
1,441 Wiki/Knowledgebase Contributions
311 Sprint/Release Planning Sessions
279 Reviews
189 Retrospectives
101 Projects
31 Internal Meet-ups
22 Agile Cafés
10 Newsletters
5.5 Years
3 Distinct Job Titles
1 Wild Ride

2 My thanks to colleague Lennie Noiles and his presentation on Powerful Questions. While Lennie didn’t ask me this particular question, it was inspired by his presentation.

The Passing of a Master

It has been several months since news of Emily Busch ‘s unexpected death and I still wrestle with the thought I shall never have the privilege of again practicing Aikido with her at Nippon Kan. The blow to Homa Sensei is undoubtedly far greater. I do not know his pain, but I am familiar with it.

Looking at the growing stack of draft posts, I see about a dozen on the subject of mastery. I feel I have a lot to contribute to this subject, particularly in regard to Agile practices. And yet, I hesitate due to a sense that I still have more to learn before I’m in a position to teach on the subject. In no small measure this hesitation is counseled by having met and studied with a great many truly masterful people across a wide variety of human experience.

Emily was one of those masters.

Not only was she a master of Aikido (6th degree black belt and Sensei at Nippon Kan), she was a master jeweler. She designed and made the wedding rings for both my first and second wife along with several beautiful pendants and a set of ear rings for one of my nieces. I never had a personal jeweler before Emily and shall not have another before my time is finished on this earth.

For all her skill and mastery, she very much understood the importance of service. There was no task that needed attention at the dojo or in preparation for a seminar that was beneath her rank. And I wonder how many patrons to Domo restaurant knew they had their order taken and served by a 6th degree black belt.

Emily had already achieved the rank of black belt by the time I began practicing at Nippon Kan in 1989. My very early memories from practicing with her are of her patience and ability to skillfully instruct a 6’5″ oaf like me in the ways of Aikido – both on and off the mat. I don’t know if Emily even weighed 120 pounds, but that never stopped her from putting my sorry ass on the mat or sending me over her shoulder. Even so, I never matched Emily’s skill, even on my good days.

Those mastery related posts will have to wait a while longer. And in the wider view, my respect for those whom make claim to be masters without having done the work and earned the title have lost a little more of my respect.

Emily Sensei will be missed.

Agile Interns

I had the privilege of presenting to a group of potential interns from the Colorado School of Mines interested in agile project management. Action shot…

The slide shows what we can offer them as interns: Failure, chaos, and confusion. I unpacked that as follows…

Failure

It’s important interns have the opportunity to learn how to fail in small, deliberate and safe increments along with the opportunity to learn how to extract every possible lesson from failures and how those failures lead to eventual success. Much of our business is driven by experimentation and hypothesis testing. Most of those experiments will fail, at least initially.

Chaos

We strive to be anti-fragile. One way to accomplish this is to be good at working under chaotic and ambiguous conditions. When involved with highly evolutionary design sessions, shifting sands can seem like the most solid ground around.

Confusion

One of the values for bringing interns into the organization is the fresh perspective they offer.  Why waste that on having them fetch coffee? However, interns can often be intimidated by working with people who have decades of experience under their belt. So it’s important they know they have the opportunity to work in an environment that expects questions and recognizes no one knows it all. They are in an environment that  seeks alternative points of view. In this organization, everyone gets their own coffee.