Friday, 1 May 2015

Agile is a team game

Agile software development is a team sport. I see so many issues when working with software teams that I have also come across in a sports team context. Teams are collections of people and to get a collection of individuals motivated and organised to achieve a common goal is not easy. I am convinced there is much we can learn from  the sporting arena to help us perform better as software teams. 

This is the first in a series of posts exploring concepts like:

- Individual efficiency vs group efficiency
- Long lived teams v fluid team membership
- Learning/ knowing your system/ formation.
- Clarity of roles & responsibilities.
- Culture of continuous improvement
- Use of metrics to inform decisions
- Use of a coach
- Not using one size fits all management approaches

Post : Individual v team efficiency

A lot of what is important for the success of sports teams is also valuable for software teams. We create teams with a view to the Team achieving more together than the individuals would on their own or because the task/ deadline before us is unachievable without extra man power. 

Example: Cristiano Ronaldo is arguably the best footballer on the planet, however on his own he cannot achieve anything, he needs to operate within a team structure. 

Often organisations are trying to deliver projects to crazy deadlines which mean they need the extra manpower a team structure can provide. They cannot rely on the superstar, the Ronaldo, to do it on their own. For star players and gifted individuals, sometimes this means sacrifice. A sacrifice to how they would normally do things. A compromise of their individual efficiency for the sake of the team efficiency.

An example of this that we might be familiar with in software teams, might be where a team debates whether to break a large user story into multiple small stories. The star developer feels he can be individually more efficient by tackling the big story on his own in one go. However by breaking the work into smaller chunks the team gains a lot: 

  • ability to show incremental delivery
  • ability to spread the knowledge throughout the team
  • pride that the whole team can contribute rather than just one superstar
  • future reduction of key man dependency
  • shared knowledge of the system
  • early identification of risk/ complexity.

Often, using agile means sacrifice for the expert individual but it results in a stronger team, the herd is only as fast as its slowest member, not its fastest. This can cause conflict in some cases with the star player finding it difficult to sacrifice or to spend time coaching others to an enhanced level. The bottom line though, is that organisations need to have multiple, reliable delivery teams rather than a handful of exceptional individuals. This doesn't mean we don't want to have exceptional individuals in our companies or teams but ideally these should compliment outstanding delivery teams, not prop up poorly functioning teams. 

Thursday, 10 July 2014

Slice the peach

 When should you split a user story? Think "when would you slice a fruit?"

  • A banana is big but manageable you'll probably eat it whole. You might want to slice it in certain circumstances.
  • A peach could get messy so i'd rather slice it
  • A kiwi is small but its hairy, i'm definitely slicing that!
Deciding when to slice a user story is not always easy. Often its difficult to apply hard rules. Its usually a balance of size and complexity.

In the above examples, a banana represents a big story that isn't complex, should size alone determine whether to split it? The kiwi represents a story that is small but highly complex, we know its hairy, we know we shouldn't bite straight into it but how to slice it is not obvious either? peel first? slice first?


The peach represents the story I see most often. It represents uncertainty. Once you bite into a peach you discover if it is hard, soft, juicy, extremely juicy! If you slice it up first then you know exactly what you're getting and its much more manageable if things do get messy



Monday, 19 May 2014

Moving beyond tasks

One of the most common agile anti-patterns I witness is a compulsion towards breaking user stories into tasks.

Tasks take focus away from stories

The problem with tasks is that they take the focus away from user stories. Teams that focus on completing tasks risk losing sight of the real value which lies in the stories. They are often unable to see the wood for the trees.

How often have you seen sprints that are?
•             On day 9 of 10, but <30% of story points are complete
•             On day 5 of 10, but <10% of story points are complete
•             On day 9 of 10, with 90% of tasks complete, but still <30% of story points complete

I firmly believe this is a by-product of too much focus on tasks and not enough focus on stories.
Tasks themselves add little or no value on their own. I often see tasks such as “Manual QA”, “Code Review”, “Write test cases”, “Database work”, “Front end work”, “Back end work”. People often argue to me that the task breakdown ensures they don’t forget to do certain parts of a story. If we need a task to remember to do code reviews and testing then we are in big trouble J.  Also a simple check-list will suffice, that doesn't have any rally/ jira overhead.

Tasks are unlikely to help you to limit your work in progress

Let’s say a sprint starts with 10 user stories. Each story breaks down into 5 tasks giving us 50 tasks to get through.  Let’s suppose a breakdown of 20 front end tasks, 10 middle tier, 10 back end & 10 test and a team consisting of 1 FE, 1 Java, 1 DB and 1 QA.
Day 1 of sprint everyone is eager and wants to start a fresh, new piece of work. Everyone will naturally choose a task that fits their specialism. If we are lucky there will be some alignment so that we at least have multiple people working on the same user story. As people start finishing their tasks, they will, again, naturally choose the next task from their specialist area. They bring a new story into play, thus increasing the amount of work in progress. Nobody ever picks from outside their specialism which ultimately leads to a large portion of stories in play at any given time. This gives us the situation where lots of stories are starting but not many are finishing. Not to mention a massive tendency towards specialism where each person stays rigidly within their own skill set.

Why do we want to limit WIP?

CFD charts that look like this highlight a large amount of WIP. 















Halfway through the sprint there is very little work complete. This is a big risk that the sprint is going to fail. Or at the very least it doesn’t inspire confidence that the investment in the sprint is going to deliver.

CFD charts like this one provide more confidence that work is being delivered incrementally & early and the risk of failure is reduced. Its no coincidence that this team has limited work in progress throughout the sprint (the yellow section)















The more WIP, the more EXTRA UNNECESSARY work we generate for ourselves. There’s additional overhead of merging, managing the separate stories and tasks, discussions between developers and QAs about ALL the stories in play, discussions about integration of all the stories in play etc. It’s much more difficult to integrate with moving targets. Every new story you bring into play results in a delay in getting currently 'in progress' user stories into DONE!!

Finally, and quite simply... Managing tasks is waste! They introduce an unnecessary management overhead. Each task needs to be managed in rally/jira. Undoubtedly, precise estimates will be required (in hours) for how long each task will take. Time spent on a task versus the estimate will no doubt be tracked. All of this is time spent NOT adding value for the customer.

Limited WIP helps you delivery incrementally

Incremental and early delivery of your sprint user stories gives the following benefits:
  • Reduced cycle time (due to less unnecessary work mentioned above)
  • Reduced risk (earlier delivery)
  • Fast feedback (early integration/ early visibility of deliverables)
These are some of the main gains we should get with an agile development methodology. Otherwise we are doing big batches of work that inherently hold risk, and are delaying visibility of deliverables.

Tuesday, 21 May 2013

A short story about fish



About 6 months ago my team embarked on an initiative to upgrade our Continuous Integration (CI) & Automation pipeline. As an iOS team, the fact we already had a comprehensive suite of automated functional tests hooked up to Jenkins was impressive and gave us a solid foundation. However, our setup had started to creak at the seams. Inexplicable test failures, brittle tests, failing tests that miraculously worked if we gave them a swift kick in the nether regions and ran them again! We decided to evolve to a CI 2.0, which would be a lot more stable, massively reduce the amount of time we spent nursing our tests and ultimately give us better confidence in our system. We are not done yet but we have re-learned a number of basics along the way

Lesson 1: with CI & automation, fast feedback is king
Our monolithic test suites took hours to run. This discouraged developers from running them on every check-in, which is counter productive. Not running them means you increase the chances of breaking functionality or introducing defects with each commit.

To achieve faster feedback we have split our tests into smaller test suites each with its own Jenkins job which means we now start to get feedback in tens of minutes rather than hours.

Next steps :
We want to set up a grid system, to run the tests in parallel to further reduce the time to run

Lesson 2: Responsibility

When you don't have fast feedback You also get in a situation where breakages can't be easily traced back to a single code commit, as the 'window of culpability  is open for a few hours/ half a day. Anyone who commits in that period could be at fault. Who does the investigation to see what caused the breakage? Without trace-ability  it's often a case of thinking 'someone else will look into the failing tests' which means, probably, no one will look into it!

We introduced the concept of a 'Weatherman'. Someone who is responsible for investigating any breakages and getting them fixed as priority. This role is rotated on a daily basis so it doesn't become too much of a chore for any one individual.

This brought more visibility of breakages and helped change the culture of the team to have quality and visibility of quality as a priority on a daily basis.

Lesson 3: Granular feedback

I mentioned already that we broke our tests into smaller suites. Great for fast feedback, but also granular feedback which helps when trying to pinpoint the breakages. Also, if you have one failing suite you still have confidence in all the other passing suites. With the monolithic test suite we didn't have that confidence.

Lesson 4 : Keep your tools sharp & get new ones when you have to

We were using Apple's UI automation tool, but it started to show signs of flakiness and incompatibility with what we wanted to do. You would expect apple to produce a solid tool but further investigation showed that they don't really invest too much in UI automation which meant we had to start considering other tools.

We have now started to use FRANK for automated acceptance testing, which has led us down a behaviour driven development (bdd) path. (Bonus!)

Lesson 5: Appreciate legacy code

Appreciate when your legacy code is giving you value and not costing you much - you don't always have to migrate everything.

We continue to run our stable legacy tests in parallel with our shiny new frank automation tests. They are stable and don't cost us much time in maintenance and still give us good value. It would take months to fully migrate them so we won't bother unless we have a compelling case for making that investment.

Lesson 6: Adopt a culture of quality

As mentioned previously, the concept of a weatherman, who gives the team it's daily weather report, has massively helped change the culture of the team. Quality is everyone's responsibility but on a daily basis it's even more so the weatherman's responsibility!

Having experienced the pain of brittle tests packs, slow feedback, poor tools etc the team now knows the value of staying on top of our CI and automation packs and ensuring we have the speed and feedback that we need. .

Conclusion 

That's our story, we are more than halfway to having a slick CI pipeline. The improvement we have made is fantastic bearing in mind that all this was done on top of regular sprint work and deliveries. Great commitment from the team to improve the culture, environment and toolset.
Previously we were getting some of the value of CI & automation with a considerable maintenance cost. Now we are getting a lot more benefit with a lot less maintenance. Happy days.

Finally, To make sure nobody leaves disappointed ..... Here is a picture of some lovely fish!


Tuesday, 23 October 2012

What a makes a GREAT scrum master?




Part time scrum master?

Many software teams who claim to be "doing scrum" have people “stepping into” the role of scrum master.
I frequently see project managers, development managers, testers, developers taking the reins with varying degrees of success. Often, willing team members take on the role of scrum master in addition to their development or testing responsibilities! I also see a lot of teams going solo, i.e. doing scrum without a scrum master. This is a practice that I think ultimately prevents a team from improving and getting full benefit from the scrum process. Even if someone possesses all of the soft skills to the do the role, they still need to have the core skills and knowledge required to be an effective scrum master.

The scrum master responsibilities

  • Be the Guardian of the scrum process
  • Be a servant-leader to the team
  • Be an impediment remover
  • Be an interference shield for the team
  • Be a coach
  • Be an agent of change
I don't think that a part-time or temporary scrum master can fulfill all of these responsibilities and to do this role successfully you need to be all of the above things to the team. I think that someone who steps into the role can be a good scrum master, but will they be a great scrum master?

Two levels of scrum mastering?

I think there are 2 levels of scrum mastery.

Level 1: B.A.U.

This is the layer of scrum mastering that keeps a team ticking over. The part-time or temporary scrum master tends to operate at this level. The list of responsibilities include (but are not limited to):
  • Be the Guardian of the scrum process
    • Ensure PO has a backlog of stories for the team to work on
    • Ensure PO has prioritized a list of stories before planning
    • Ensure the team is clear about the requirements
    • Encourage the team to make a projection of the amount of work they can achieve in the sprint
    • Ensure team feels comfortable with projection versus current velocity
    • Protect the sprint backlog
    • Limit W.I.P.
    • Ensure the team focuses on quality
    • Monitor the stand up (make sure people don't waste time, keep updates short, stay on topic)
  • Be a servant-leader to the team
    • Set up all scrum ceremonies
    • Point of contact for Scrum updates.
    • Send out reports/ burn-up charts
  • Be an impediment remover
    • Remove barriers to productivity
    • Help to resolve issues/ arguments/ debate
  • Be an interference shield for the team
    • Maximize time available for developers to develop (protect dev time )
    • Reduce waste/ non sprint activity
    • Protect team members from being dragged away to work on non-sprint activities
    • Protect the team health and well being

Level 2: Strategic

This is where all the good stuff happens. A full time, "proper" scrum master will operate at this level.
  • Be a coach
    • Help team to self organize
    • Help the team to improve engineering practices so that each sprint/ story is potentially deploy-able
    • Help the team understand Scrum/ agile principles and techniques (rather than just mindlessly following a process)
    • Help the team understand the benefits of these techniques
    • Help the product owner manage the backlog
    • Remove the distance between the PO and the team
    • Teach the PO how to maximize R.O.I. and meet objectives through SCRUM
    • Ensure the team is learning in retrospectives
    • Promote continuous improvement
    • Coach a team to self sufficiency (e.g. how to remove own impediments)
  • Be an agent of change
    • Observe & identify areas for the team to improve
    • Create a culture of continuous improvement in the team
    • Facilitate creativity and empowerment
    • Determine where the Scrum process is compared to where it could be
    • Potentially look beyond scrum, for techniques that can help the team improve (e.g. Lean/ kanban)
    • Challenge the team to improve engineering practices (CI, TDD for example)
    • Look to improve things outside of the team that are slowing the team down (e.g. business stakeholder involvement, handover to dev ops, poor communication across the organisation)
    • Keep an eye on the future 

High performing teams through great scrum mastering

The scrum master who operates at the strategic level will give teams a massive boost over time. To have someone who is focused on improving a team and making a team awesome and improving a team's working environment means that inevitably, positive change will occur within a team. The alternative is stagnation. 

If your team has a level 1 scrum master or worse, no scrum master at all, who is going to constantly beat the drum for improvement? who will keep an eye on the future and the health of your scrum process? Who will promote constant improvement? Who will ensure the team is always learning?

In my experience, having a great scrum master with a strategic remit, facilitates the growth and nurturing of awesome, productive, high-performing teams. This stuff doesn't just happen!





Thursday, 4 October 2012

Expanded EPIC boards


What's the problem?

As scrum teams, we always look to the Product Owner as the fountain of all knowledge and requirements with respect to our supply of work. Yet, I've found that often the big ideas or concepts for projects are so light on detail that POs are desperately in need of a way to translate the high level idea into a backlog of stories to get the dev team up and running

We've all been part of projects where the BAs and POs lock themselves away for a few weeks with a high level understanding of a project and come back with a list of user stories for the dev team to work on. By the time the devs see the requirements, its hard to see the wood for the trees. We get bogged down in the
details, the stories are sliced poorly & are not delivering vertical slices of functionality. Often there's no wiggle room. Its all MMF. We cant cut anything out.

Additionally, defining an entire release worth of stories in advance which, ultimately, require change or may get dropped from scope, results in a waste of PO time and a delay in getting the dev team up and running. (Dan North touches on this release planning anti pattern in his blog post "The Perils of estimation" http://dannorth.net/2009/07/01/the-perils-of-estimation/)

So whats the solution?

Well, I've had great success recently using a variation of an "Epic board" as a tool for getting a project up and running. Developers, PO's and BA's collectively deconstruct EPICs into user stories that add value and are small enough to be used in a sprint. You don't need to write up the stories just yet, simply identifying valuable, vertical slices of functionality can be enough.

So Whats an EPIC board?

For those who are familiar with Epic boards as a concept, you may be used to the techniques outlined here: http://agile101.wordpress.com/category/programme-management/agile-epic-board/ , where EPIC boards are used as a programme management tool, tracking parallel deliveries across multiple teams.

To be honest, I think this technique is a bit too "project manager-ish", a throwback to the old "Gannt chart wallpaper" days.

So, why would I want to use an EPIC board?

Where Epic boards are most valuable, I find, is as a way of visualising the work in a project and also, as an acceptable way to discuss requirements at a very high level without the need for debate about whether or not the story is too big/ too small for a sprint. For kick off,  an EPIC should be a perfectly acceptable level of
granularity for a requirement

Benefits:

Representing your project as an EPIC board can provide a lot of immediate advantages by

1. helping you explore your project concept and discover what actually needs to happen in order to realise it
2. helping you visualize the work that needs to happen to achieve the project goals (piccie)
3. helping you get from "big concept" to "sprint level user stories" quickly. (This  helps you get your dev team up and running quickly)
4.  providing a framework to encourage scope management - facilitating meaningful prioritisation discussions
5. facilitating faster time to market by making "must have" functionality more visible & more obvious.
6. helping you de-risk project delivery by making "most important" features more visible & more obvious.

So, what's the process?

Identify project goal(s)
ask your PO "What is/are the goal(s) of this project" - write them on cards/ post them on your wall
(e.g. goal = sell flights online)

For each goal, ask the PO
What is the minimum we need to do to achieve this goal? & when do we know we have achieved this goal? - write them on cards/ post them on your wall

These will most probably be your 1st level of EPIC stories. Be ruthless, only identify the most important things you absolutely have to do to achieve the goal. (e.g. view available flights/ select flights/ enter payment details/ book flights/ view receipt)


Break it down - Expand your EPICS
Start breaking the Epics down into smaller chunks or "SUB-EPICS" - write them on cards/ post them on your wall
This step is the exploration step. You break down your high level requirements into something a bit more meaningful, you start to get at some of the detail that was previously uncovered.
e.g. for "view available flights" epic, you might identify the following sub-epics: filter available flights by airport, filter available flights by date, filter available flights by price, filter available flights by airline, filter available flights by special requirements (dietary, special needs etc)

We start to see there are a lot of different use cases or features hidden in the seemingly simple "View available flights" story



Depending on the size of your project and epics, you should find that at this stage you will already have identified some stories that are small enough to work on in a sprint. if not just keep on breaking a requirement down until you feel you have.

Prioritise
(I've used the MSCW technique, which has worked quite well in these exercises)

When you've broken the EPICS down, the expanded EPIC board serves as a great visualization tool. It becomes obvious when you see all requirements side by side, that there are some elements of the EPIC that we could "potentially" go live without (In Orange below). Finding flights by airport & date are probably critical, but in theory, filtering by airline, price or special requirements are all enhancements. Nice features that would give the user a better experience, but would not necessarily prevent them from finding a flight that they want.



Do the minimum
For the 1st iteration of your project, just do the minimum you have to do to book a flight. Take the risk out of your delivery.  (The purple and blues)

If you have time leftover you can add the extra features (the Oranges). Enhance the experience for the customer, provide bells & whistle but ensure you have done the highest priority work first!



There are lost of other things that could go into a project like this, e.g. show link to hotel booking, car booking, holiday insurance etc. You could capture these if you like, but make sure you aren't putting them on the critical path. If halfway through your project you haven't successfully booked a flight but you can cross sell holiday insurance its not going to give your PO any confidence that you are on track to achieve the goal of the project which is to enable flight booking online.

Focus on the main goal, then incrementally add other pieces of value later

TIPS:

- all groups should be represented in the EPIC boarding session (PO/BA, DEV, Test, Scrum master)
- Use a white board initially. This makes it easier to make changes when you are expanding your epics and identifying sub-epics or stories. Once you think you have your EPIC board in a fairly "expanded" state, put them on cards on the wall.
- Don't worry if some EPICS can't be broken down just yet. Early on in a project you will know more about some requirements than you do others. Accept this, but make sure you get the analysis done before the dev team is ready to work on the EPIC.
- treat the EPIC board as a living document. keep updating it when you get fresh information. just like your release plan.
- As soon as you have identified some sprint level user stories, take those away, write them up and get the team up and running. Once they are sprinting, you can focus on getting the other EPICS expanded! 10% sprint level user stories is fine if you have identified most of the EPICS.We don't need to know everything up front. We can drive out details along the way.

Wednesday, 5 September 2012

Mature agile estimation


Let's face it.... estimates are always going to be estimates! Not actuals.
No matter how much time we dedicate to getting the estimates "RIGHT", chances are they will still be wrong. They don't take into account the proverbial "unknown, unknowns" and in software development any item of work can contain unknowns or complexity that blow the estimates right out of the water!

In traditional project management the tendency is to chase reasons for "wrong" estimates and track actuals against estimates in some kind of attempt to get "better" at estimating. As a result, teams spend a lot of time putting hourly estimates on user stories, which still have a very high chance of being incorrect, so why bother? What a waste! Instead, spend a little bit of time estimating and be just as wrong/ right in the end!
I suggest a "mature" attitude to estimating. 
  • assume that the work will take as long as it takes
  • assume that everyone will do the best job they can on a user story
  • assume the team will not cut corners on quality
  • assume no unnecessary over-engineering
  • estimate "how big" user stories are using the fibonacci scale and comparatively estimate each story against other stories. (Is this bigger or smaller than the last one? How does it compare to our benchmark stories)
  • use the fibonacci scale because the gap between numbers increases as we go higher. This mimics software development in a way because the bigger the story, the higher the risk of unknowns or complexity creeping in which will increase the amount of time it takes to develop.
  • Based on the previous sprint's velocity, the current sprint's capacity and very importantly, gut feel. Project how much work the team can get done in the upcoming sprint
  • Use this projection to manage expectations of when you can deliver your features/ software/ product
  • If you "under-estimate", and have extra capacity left in a given sprint, then try to take extra stories into the sprint. (Pull system)
  • only take in additional stories if you can finish them by the end of the sprint otherwise you will affect your ability to be "potentially deployable" at the end of that sprint
Really, what management and product owners want to know is when we can deliver our products and they want predictability. The above method still gives us these things, but without the pain of hour estimation and actuals tracking.