Project Reporting Part 1 – Crucial Contribution Project Team Makes in Reporting

Recently I wrapped up a project where I spent a lot of time doing delivery reporting out of the old Team Foundation Service 2015 (TFS 2015). I became interested in how the reporting might have improved in the latest online version of Azure DevOps (formerly Visual Studio Team Services – VSTS). In this series of blog posts I shall dive into the reporting capabilities of Azure DevOps for project management and later the currently beta release of a Power BI connector to Azure DevOps.

Reporting is Only Ever as Good as the Data Your Team Input

No matter how much time and effort you put into project reporting, it will only as good as the data that your team input. If your project team members do not understand how the data about their work is used to track and improve the delivery, they will only see it as additional administration and therefore can be pushed aside and ignored. This can lead to a break down in group discipline and eventually the team ends up making poor decisions affected by time constraints and stress. The team might also find it hard to explain the reasons behind perceived project delays based on imperial evidence.

Classic Problems Arising from a Breakdown of Group Discipline can include:

• Inability to account for how time is spent on project tasks,
• Re-occurrence of bugs raised and inability to identify root cause of problems during testing,
• Inability to identify user stories or requirements suffering from scope creep due to loss of traceability between bugs raised and acceptance criteria in a user story/requirement.

Reporting is often viewed by project team as something that managers use but not useful to individual members. Often project team members do not understand how to use statistics to explain to managers or stakeholders how or where time was spent. What is the consequence of accepting a Change Request at the last minute of a sprint? People will always struggle without valid statistical based evidence when trying to have an informed discussion on the true cost of a change request or “treat change as bugs” mentality.

While some of the highlighted issues in a project team can be addressed by better tools supplied, most of the time things improve if and only if the project team carries out a retrospective and agree on a set of rules to follow regarding inputting data into the project tracking tool in use. In the following I list out several “ground rules” that might help if your team is struggling with project administration. Whilst I use Microsoft Azure DevOps (formerly Visual Studio Team Services) as the tool to run a fictitious project called “Smart Track” in this series of blog posts, the principle applies to any other tools available for project tracking.

Agree what each Team Member’s Personal Responsibilities are for updating Project Tasks

Agreeing upfront each member’s personal responsibility in updating tasks is vital to accurate reporting. For example, at the end of each working day all team members must update the work items assigned to time regarding time spent on the tasks or whether the tasks are completed or not. Without historic data on e.g. estimates, a team will never know whether they are over or under estimating tasks, and more importantly, whether they will reach sprint end successfully by completing all the tasks committed to the current sprint.

To support this change in behaviour regarding what many considers “boring admin tasks”, simply include these daily updates as part of the “definition of done” for each task. A team member has not finished a task until he or she has also updated e.g. time spent on the task and change the task status from e.g. Active to Completed. If this admin update is not completed, the team member has not finished the task assigned to him or her.

Share a Common Project Dashboard

Once everyone commit to daily task updates, the next goal is to ensure everyone have access to a shared Project Dashboard showing all the current ongoing work items and items coming “down the pipeline” for future sprints. It not only stops time wasted in verbally chasing members for updates, the team members themselves can see who is assigned to certain task and crucially, which member might be struggling or being blocked or delayed in progressing work items. This is where self-organising team can work together to support individual members to ensure all tasks will be completed by end of sprint.

For example, below is a screenshot of a Dashboard in Azure DevOps. Looking at the “Burndown” chart in the middle of the dashboard it is clear something has gone wrong in this sprint. Instead of burning “down”, the team is collectively adding more tasks to the sprint instead of completing them (chart gradient goes UP instead of DOWN). This signals to the team that something might have been missed in sprint planning session.

Agree on the Relationships between different Types of Work Items

Why is this important? Aside from ensuring you can report on project statistics accurately, it can help teams understand and control project scope and provide better estimation. Here’s how it can help.

Suppose your team decides to run in sprints and hope for a faster turn-around time in delivering functionalities per sprint. The natural pitfall many teams fall into is to begin development before the “acceptance criteria” of a requirement are agreed between the project team and product owner and stakeholders. The best analogy I use to describe this is how would you know where to “kick the ball” if you’re a football team when you have not even decided where the goal post (i.e. “definition of done” for a requirement) is?

Acceptance criteria for a requirement or a user story (depending on which methodology your team follows) is important as it helps clarify the scope of the requirement. Rushing to build a piece of functionality without understanding what “done” means to stakeholders is often a very costly way to build, as ambiguity and misunderstanding play a huge part in what stakeholders and testers later raise as functionalities not working as expected, and therefore, defects to be fixed by the team.

So what exactly is a defect? Basically a built functionality that is not working to the expectation of your stakeholders. Unfortunately depending on someone’s interpretation, their requirement can be “woolly” and very subjective. On many occasions I watched stakeholders and project team argue on whether a functionality is completed or not, given each person interpreted the original requirements slightly differently.

Here is where the relationship between a requirement and a bug comes into focus. If a bug is created without referencing original requirements or user stories, it can be difficult to identify whether a raised bug is a genuine defect, misinterpretation, or a desire for more functionalities than originally agreed which is really scope creep. If your team decides to treat bugs created as a child work item which must be linked to a parent Requirement or User Story, you can focus everyone’s attention on the Requirement’s acceptance criteria. That is (again with the football analogy) has the team “kicked the ball” towards the right “goal post”? If a bug raised is not covered by any of the requirement’s acceptance criteria, well, the team can argue that the bug is out of scope and should be treated as a new change request with the dedicated time needed for exploration and design prior to re-development.

In Azure DevOps, you can configure the project boards to treat bugs as children work items that are tied to parent User Stories (or Backlog item depending on which process template you select for your project). Setting go to the Kanban Board, click on Setting “wheel” on top right, then select “Bugs are managed with Tasks”. This ensures all bugs created and linked to a parent user story will show up as a child work item on the Kanban board.

Another handy work item that can help a project team break down, track and estimate the effort involved to complete a requirement or user story is children tasks. When you create a task in Azure DevOps, specify the parent User Story as well as the effort (Original Estimate) you think (in hours) might be required to complete this task. On the “Task Board” in Azure DevOps, if your team members know to update the tasks completion by putting down the number of hours worked on the task (Remaining Hours and Completed Work), by the end of the task completion, you should be able to compare the “Original Estimate” to the “Completed Work” in hours for each User Story.

Create Tasks in the Dashboard to Capture all Actions Team Members are Required to Complete

Why? Think of this is as a record of how you and your project team spend your time during sprint. If a definition of done that you and your team agreed to include e.g. update functional specifications after a user story is completed, you must have an additional “documentation” task and time allocated for the document updates as part of the children task breakdown for the user story.

Likewise, during a sprint you might suddenly discover you need more time to complete tasks that were not mapped at the beginning of the sprint planning session. Perhaps a deployment of this functionality might require you to additionally change a configuration file. Adding these new tasks to the parent User Story will help track the overall actual effort in completing the user story. Moreover, with these statistics, you can now account for delays or additional time spent on particular user stories when you run the delivery report.

Agree on how State of Work Items can Change

A work item such as a User Story can go through many different “States”. For example, when you first create the User Story, the state might be set to New. When it is ready to go into a sprint, the state might change from New to Active to denote that a team member is working on the user story. It’s important that all members of the project team understand under what conditions a work item changes state. For example, let’s say a User Story has completed development and is being tested by the QA team. The state of this User Story moves from Active to Test. If the user story fails testing and a bug is created, the bug work item might have the state “new” to denote it is raised recently but not yet been investigated. However the user story state should also move back from Test back to Active, to denote that this User Story needs more work. This ensures your reporting will always show exactly how far from completion the work items are in the current sprint.

Here is a screenshot in Azure DevOps of a Kanban Board showing all new User Stories on the left and all completed User Stories on the right.

For a drill down view in Azure DevOps you can go to the Task Board to see all the children Tasks and Bugs associated with a User Story

As I said at the beginning of this post, reporting will always only be as good as the data your team input. If your team does not understand the need to provide regular updates and record the different types of work done for a user story, all of the above will become “boring admin” and not worthy of time nor attention.

However if you are able to get your team to adopt at least some of the points above, it should provide you with the ability to do basic delivery reporting that are timely and accurate.

In the next blog post on delivery reporting, we shall discuss using Power BI to do delivery reporting.