Show your work

A presentation I gave last week sparked the need to reach back into personal history and ask when I first programed a computer. That would be high school. On an HP 9320 using HP Educational Basic and an optical card reader. The cards looked like this:

(Click to enlarge)

What occurred to me was that in the early days – before persistent storage like cassette tapes, floppy disks, and hard drives – a software developer could actually hold their program in their hands. Much like a woodworker or a glass blower or a baker or a candlestick maker, we could actually show something to friends and family! Woe to the student who literally dropped their program in the hallway.

Then that went away. Keyboards soaked up our coding thoughts and stored them in places impossible to see. We could only tell people about what we had created, often using lots of hand waving and so much jargon that it undoubtedly must have seemed as if we were speaking a foreign language. In fact, the effort pretty much resembled the same fish-that-got-away story told by Uncle Bert every Thanksgiving. “I had to parse a data file THIIIIIIIIIS BIG using nothing but Python as an ETL tool!”

Yawn.

This is at the heart of why it is I burned out on writing code as a profession. There was no longer anything satisfying about it. At least, not in the way one gets satisfaction from working with wood or clay or fabric or cooking ingredients. The first time I created a predictive inventory control algorithm was a lot of fun and satisfying. But there were only 4-5 people on the planet who could appreciate what I’d done and since it was proprietary, I couldn’t share it. And just how many JavaScript-based menu systems can you write before the challenge becomes a task and eventually a tedious chore.

Way bigger yawn.

I’ve found my way back into coding. A little. Python, several JavaScript libraries, and SQL are where I spend most of my time. What I code is what serves me. Tools for my use only. Tools that free up my time or help me achieve greater things in other areas of my life.

I can compare this to woodworking. (Something I very much enjoy and from which I derive a great deal of satisfaction.) If I’m making something for someone else, I put in extra effort to make it beautiful and functional. To do that, I may need to make a number of tools to support the effort – saw fences, jigs, and clamps. These hand-made tools certainly don’t look very pretty. They may not even be distinguishable from scrap wood to anybody but myself. But they do a great job of helping me achieve greater things. Things I can actually show and handle. And if the power goes down in the neighborhood, they’ll still be there when the lights come back on.

Creating Cycle Time Charts from TFS

Unfortunately, TFS doesn’t have ready-made or easily accessible metrics charts like I’m used to with Jira. There is a lot of clicking and waiting, in my experience. Queries have to be created then assigned to widgets then added to dashboards. Worse, sometimes things I need just aren’t available. For example, there is no cycle time chart built into TFS. There is a plug-in available for preview for VSTS, but as of this posting that plugin won’t be available for TFS. With Jira, cycle time and other charts are available and easily accessible out of the box.

Sigh. Nonetheless, we go to work with the tools we have. And if need be, we build the tools we need. In this post, I’ll give a high level description for how to create Cycle Time charts from TFS data using Python. I’ll leave it as an exercise for the reader to learn the value of cycle time charts and how to best use them.

To make life a little easier, I leveraged the open source (MIT License) TFS API Python client from the Open DevOps Community. To make it work for generating Cycle Time charts, I suggested a change to the code for accessing revisions. This was added in a subsequent version. For the remainder of this post I’m going to assume the reader has made themselves familiar with how this library works and resolved any dependencies.

OK, then. Time to roll up our sleeves and get to work.

First, let’s import a few things.

With that, we’re ready to make a call to TFS’s REST API. Again, to make things simple, I create a query in TFS to return the story cards I wish to look at. There are two slices I’m interested in, the past three and the past eight sprints worth of data. In the next block, I call the three sprint query.

With the result set in hand, the script then cycles through each of the returned work items (I’m interested in stories and bugs only) and analyzes the revision events. In my case, I’m interested when items first go into “In Progress,” when they move to “In Testing,” when they move to “Product Owner Approval,” and finally when they move to “Done.” Your status values may differ. The script is capable of displaying cycle time charts for each phase as well as the overall cycle time from when items first enter “In Progress” to when they reach “Done.” An example of overall cycle time is included at the end of this post.

When collected, I send that data to a function for creating the chart using matplotlib. I also save aside the chart data in a CSV file to help validate the chart’s accuracy based on the available data.

It’s a combination scatter plot (plt.scatter), rolling average (plt.plot), average (plt.plot), and standard deviation (plt.fill_between) of the data points. An example output (click for larger image):

The size variation in data points reflects occurrences of multiple items moving into a status of “Done” on the same date, the shaded area is the standard deviation (sample), the blue line is the rolling average, and the orange line is the overall average.

If you are interested in learning more about how to impliment this script in your organization, please contact me directly.