Recently, a colleague sent me a link to a Agile Finland Seminar; Jim Coplien: Ten things that Scrum teams almost always do wrong . The talk is about 2 hours long, stuffed with excellent information, even if Coplien’s presentation style is a bit more laid back than I prefer. In it, he references that he does ‘Red Pill Scrum’ as opposed to ‘Blue Pill Scrum’.
Being an avid Matrix movie fan, I couldn’t help but think of Morpheus’s epic speech to Neo regarding if he wanted to know the ‘true’ meaning of the Matrix or not. If you have seen the movie, you know what I am talking about. Neo chose the Red pill… and the reality it lead to was certainly a game changer!
Coplien states Blue Pill Scrum is defined as simply passing the Nokia test (team focus). For many Scrum teams, ‘simply’ passing this test is difficult enough. (By the way, like the article states, many don’t even pass the Iterative Development part of the test!)
Red pill Scrum also integrates management and enterprise usage of Scrum from top to bottom.
Coplien states blue pill will give you 10% – 20% improvement. Red pill yields one to two orders of magnitude performance improvement.
I personally feel that organizations that are just starting to adopt Agile/Scrum need to experience team success to gain momentum and ‘social proof’ for other teams to see. Once that has been successfully done, moving to enterprise Scrum usage makes sense and the cross department synergy that will result will certainly yield crazy good organizational performance improvement.
I highly recommend giving the video seminar a view. Jim Coplien is one of the founders and proponents of Agile software development. He works closely with Ken Schwaber, Jeff Sutherland, David Starr and others to facilitate Scrum’s evolution as formalized in the Scrum Guide.
Once your team has decided to go forward with Group planning and estimating (and may have even decided to try Planning Poker as a estimation technique), often the next question is: “How much time should I expect a planning session to take?”
Best Practice: Time box 1-2 hours of planning per week in Sprint
A common rule of thumb quoted all over the internet is to time box sprint planning to 1-2 hours per week in the sprint. So expect 2-4 hours for a two week sprint, 4-8 hours for a four week sprint. In my experience, I have found that this amount of time is more than enough as long as your team has some experience with group planning. If your team is new to group planning (or a majority of your team is new to it), expect to add an additional hour or two just to allow for learning discussions to take their course. Expect 3-5 sprints to allow new team members to really get the hang of sprinting and the planning process that goes with it.
To some, 2-4 hours of planning time, an additional hour for a Retrospective, and also an hour for Review/Demo, seems like a lot to allocate for each team member every two weeks (assuming two weeks sprints). In essence, you have a full day of ‘Meetings’ for the team. My experience has been that the team can feel that this is a lot, but it really does pay off over the two week sprint, by ensuring all team members have a good understanding of all tasks and has participated in ensuring any unique knowledge a specific team member has about a task is shared and included in the estimation process. This also allows for the knowledge sharing necessary to allow team members to pick their tasks off the task board as the sprint progresses, allowing for a team ‘to do’ task board instead of a number of personal ‘to do’ queues.
Best Practice: Start your Sprints on Tuesday, Wednesday, or Thursday
One other important thing to consider: Attempt to start your sprint, and therefore your planning sessions, on a day other than the start or end of the week (Mon/Fri). There is a natural build up to the end of a sprint, and it’s just not fun to end a sprint on a Friday or Monday given the weekend in between. This also takes into account that often team members will extend their weekend by taking a vacation day on these edge days and might miss a planning meeting more often. Teams I participate on have started sprints on Tuesday, Wednesday, or Thursday and have found those days have worked well.
Next in this series: Bugs/Defects: How they fit into the process.
In my last post, I talked about the importance of tracking Remaining Work Hours instead of Completed Hours and some of the reasons why this can work for your team. Because of the relatively short nature of a Sprint (usually 2-4 weeks), one might think it’s not important to update the Remaining Work hours on their individual tasks, but instead just set the task to ‘in progress’ when work starts on them, and when completed, remove the Remaining Work hours and set to done. Truth be told, many times this would be just fine. It might make the Sprint burn down chart a bit more ‘up and down’, but the work would get done just fine.
I strongly encourage teams I work with to update the Remaining Work hours at least once a day on any active tasks. Some have accused me of attempting to ‘Micro-Manage’ the team by this request. I simply say to them now: “You can micro-manage yourself if you want, this request is for the benefit of the team, not me.” By updating the Remaining work hours daily, the whole team has a better picture of how all tasks are progressing, and it allows the team as a whole to quickly identify possible time line issues and adjust to the ebbs and flows of all development work.
The one big caveat that goes with asking the team to update their Remaining work hours daily, is the time entry mechanism has to be simple, fast, and readily available. If it takes more than 30 seconds to update this metric, it simply will not get done consistently.
Though I understand why Remaining work hour entry time (tool wise) can be a barrier to getting it entered, I believe the majority of the time Remaining Work hours don’t get updated daily has to do more with the thinking necessary to attempt to truthfully estimate what is left at the time of updating. Though that can be challenging, it’s worth the effort to checkpoint yourself each day given the new info (or lack there of) you are exposed to. Also, doing this checkpoint daily will get quicker and quicker as time goes on given the amount of practice you will get doing it.
Teams I Scrum Master do not track completed hours. Remaining work hours are the ticket. That is where the real information is held. This is not an easy transition for the Project Manager in all of us. Even for the team members, many of whom have been asked to track completed work hours for years and years, it can be difficult to give up the mentality of reducing a bank of hours predefined prior to the task starting.
So you ask: by tracking completed hours, don’t you simply subtract the completed hours from the original estimate and then you have Remaining work hours anyway? No. The only way that would be true is if all the original estimates were perfect from the start. That simply isn’t true, and contrary to Agile thinking where you expect change, embrace it actually.
Statements like “I am 80% done” go away. Any experienced Project Manager will cringe at the sound of that statement as we all know the PM truism that it’s that last 20% that seems to take longer than that first 80% reportedly done did. The remaining hours are all you hear about and doesn’t that really tell the story you want to know anyway, mainly: how long until the work is done?
A big question I have heard regarding this switch is: “If we don’t track actual hours, how do we ever know if our estimates are getting better?” Scrum’s answer: the Velocity metric will take care of that. It really isn’t about being ultra accurate estimation wise, but instead it’s more about sizing tasks well to allow for achievable Sprint goals when picking tasks to commit to for the upcoming sprint. Tracking actual completed work hours does not directly help with that goal what so ever.
When the team switches to reporting just Remaining work hours, they also need to switch their mentality to one that has them reevaluating how much work is left on the task at least once a day and updating the task to reflect this info. It’s important to stress that the tally can go up… it’s not a number to decrement only (an old habit of completed hours reporting that can be hard to break).
Next up in this series, the importance of making it E-A-S-Y for the team to update Remaining work hours at any time.
If you are using good old fashion index cards/post-its to record your User Stories and tasks, or using a electronic tool to keep track of them (maybe you have a distributed team), all team members need to be able to see the Sprint / Product backlog and should be able to look at each anytime. I would go one step further and say that you also need to be able to visually order items in addition to being able to see them.
Providing the ability to the Product Owner to visually prioritize User Stories is a must in my opinion.
To do this, all User Stories need to be easily viewed, side by side. Teams I have been part of have used software to track our User Stories and Tasks because we have some team members that are not at the same site as all other team members. We have tried to have the P.O. prioritize User Stories without a visual tool, and it simply requires too much memorization to do efficiently/effectively. Once we implemented a tool that allowed the P.O. to see all items on screen, next to each other, as well as drag and drop them to quickly prioritize, the process became much more streamlined and we were able to focus on the tradeoffs between User Stories instead of trying to remember what we set to what priority previously.
The Team needs to be able to visually see the backlog to be able to efficiently prepare for Sprint Planning.
This point is more a call to ensure that all team members have access to look at the backlog at their convenience. This empowers them to be able to prepare for upcoming planning meetings, as well as get a overall feel for what the P.O. currently feels are important User Stories to tackle in upcoming sprints. I add the ‘visual’ part to this statement for similar reasons as the P.O.’s need. Being able to see work items side by side is powerful from a relationship basis.
It’s imperative for the team to be able to see all User Stories while executing Sprint planning.
The team members need to see all User Stories side by side when committing to individual User Stories for the Sprint. If the team isn’t allowed a full view of choices, they will have difficulties choosing the best stories to commit to given the Sprint goals. Certainly the highest priority User Stories will be shown first, but those aren’t always the stories the team chooses to implement (for a number of reasons outside the scope of this post).
Next up in this series, the importance of tracking Remaining Work Hours as opposed to completed hours.
I have heard of many groups that will finish Sprint planning by taking all tasks committed to by the team for the sprint, and assigning them out to each team member. Teams I have been part of have tried this (especially the first few sprints when new to Agile planning) as it felt like the logic last step of planning. How else is a team member to know what they would be working on over the next two weeks (or whatever sprint timeframe was chosen)?
The opposite would be to not assign any tasks to individuals during the planning meeting, but instead have the team members select tasks as needed. Having tried it both ways, I would submit the following:
Best Practice: Don’t allocate all Sprint backlog tasks to individual team members at end of Sprint planning meeting, but rather have team members pick tasks from the Sprint backlog as they come to the point of needing more work during the Sprint.
This seems counterintuitive to most, myself included. Questions flood to mind like:
- How will we be sure that the ‘right’ people do the ‘right’ tasks?
- Aren’t there specific tasks that only the ‘experts’ on the team should do?
- What about situations where someone has specialized knowledge regarding a specific task or set of tasks?
Not only questions regarding task allocation, but also about individual commitment and getting the work done come out. Not having a single person responsible for each task brings questions like:
- Who do we look to for assurance each task will get completed?
- Doesn’t this open up opportunities for some team members to slack since they aren’t individually responsible for specific tasks?
- What about those team members that need to feel the time pressure of a number of tasks in their work queue to work at optimal speed? (Many of us work best ‘last minute’)
Based on my experience, and that of others I have spoken with, many of these questions are more pertaining to the cultural shift from traditional Project Manager led teams, to self organizing Agile teams. Though you may have the fear that some people might slack, or tasks might not get completed during the sprint because they isn’t a single person responsible for each task, the actual opposite happens. By leaving the tasks in the sprint backlog to be picked from as the need for more work arises, the team collectively takes on the responsibility to complete all tasks, instead of just ‘my’ tasks in the other scenario. This has proven to be a much more efficient way to enable the team to ensure they complete the sprint goals and do it in highest priority order. Also, the daily standup meetings help ensure all team members are ‘pulling their weight’, as there simply is no way to hide if you are updating your team members each day regarding task execution/completion.
Certainly there are tasks that naturally go to certain team members based on their past experience/etc. But more often than not, the team members know of these situations too, and therefore it works out just fine and in those rare instances that it doesn’t, the team works out the situation as it’s identified and grows from the experience.
To ensure we consciously search for situations that could get us in trouble task allocation wise, teams I have been on have gotten into the habit of asking at the end of Sprint planning: “Are there any tasks in this Sprint that someone feels they MUST specifically work on?” Very rarely do we identify such tasks. When we do, we discuss as a team the reasons and decide if the task should go to that person right there. Otherwise, once the planning meeting is over, each team member selects a task (from their desk) they can ‘Race to Done’ on, and enlist the help of others on the team if they need such help.
For those people that work best ‘last minute’ and feel the need to have a large task list assigned to them, they can simply shift their focus from an individual work queue to the team queue and race to done.
The team can use the Sprint Burn down chart to measure how well they are doing completing the Sprint tasks/goals and adjust accordingly if the slope of the Burn down is showing Sprint goals won’t be completed.
Use of the Burn down chart enables the team as a whole to measure progress and adjust as whole while the Sprint is progressing. This in conjunction with the Daily Standups keeping communication open and bringing issues out quickly, really does work well and eliminates the need to assign all Sprint tasks at the beginning of the Sprint.
Next up in this series, the importance of Visual tools for Sprint planning.
The best way teams I have been on have found to estimate via a group is Planning Poker.
There are many benefits related to planning as a group (see my previous post on this topic for more). One thing that was difficult to grasp for me at first, is the idea of group estimating. Have everyone estimate each task? Even tasks they know they aren’t going to be responsible for? Yes! The collaborative nature of group estimating helps further dig up hidden features/assumptions as well as provides other benefits.
Teams I have been part of have tried a number of different ways to group estimate like:
- Group determines (informally) who they think the task is going to be completed by, and defers to that person’s estimates.
- Group members all write down a hour estimate on a piece of paper, let everyone know what they thought, and then negotiate until some consensus is gained.
- Play planning poker for estimation.
- All were used to some success level, but the first two generally took much longer to complete the estimation process and also reduced group ownership of all tasks to more individual ownership of certain tasks. In one instance, the negotiation process for one estimate (not the details about the estimate) took over 30 minutes, and even after that time, we just took the highest estimate to be able to move on.
Looking for a better way, the team discovered Planning Poker. Here are the high level details:
- Each team member has a set of ‘cards’, each with a single number. We use a variant of the Fibonacci sequence for card values (1,2,3,5,8,13,20,40,100′).
- A item to be estimated is read to the group. Any team member who has questions about functionality/etc. are encouraged to ask their question(s). This continues until all questions are answered the best they can be.
- The group facilitator asks each team member to pick a single card. (Says: ‘Estimate!’) The card is not shown to other team members at this point and verbal estimates are avoided to reduce the chance of influencing other team members.
- All team members estimate (minus the Scrum Master and Product Owner if they are in the room)
- Once all team members have a estimate card, they are all flipped over so all can see.
- When there is a significant variance between estimates the person with the highest estimate and also the lowest estimate are asked to briefly explain why they picked the number they did. This usually exposes differing assumptions by each team member and allows for some quick discussion on which assumptions are valid.
- The team is then asked to re-estimate.
- This process if repeated until there is group consensus.
- Side note: This Planning Poker in detail page has a detailed outline of the process if it’s totally new to you.
A few tips/takeaways from my experience to help with playing Planning Poker:
- It rarely takes more than two or three rounds of estimating to have total group consensus. Yes… Really!
- Invariably when starting group estimating in general, team members will ask if they can ‘abstain’ from estimating some tasks. We have chosen as a team not to allow this. All team members need to participate and do the best that they can. This has helped all team members get better at estimating tasks they usually wouldn’t be asked to participate in as opposed to relying on specific experts and disengaging.
- If there is a small variance in estimates (3/4 of team picked 5 and 1/4 picked 2 for example), the team will discuss amongst themselves what they think is best and usually pick a number instead of doing another round of estimating.
- As a team, we have chosen to stick with estimates that correspond to numbers on cards. This eliminates the tendency to say: “you have 5, I have 2, lets just average them out to 3.5 and use that estimate”.
- We eliminated the 1 and 3 cards, as we found the difference between 1,2,3,5 to be too small to be concerned about as a team. This further encourages ‘bucket picking’ even at the low end of estimating to validate that we are really just sizing activities, not committing to specific time frames. (This decision has been debated a few times since in sprint retrospectives, asking to put the 1 and 3 back, but the team has decided to keep them out for now.)
- Have a way to limit question/discussion length prior to estimation. Some times team members can get caught in the details and ramble on for quite some time. I have seen some groups use a 2 minute timer that any team member can start, to limit the current discussion point of which when the timer is done, a round of estimation is required, keeping the process moving. Point here is: group is not trying to precisely estimate the tasks, but instead they are sizing them and that just needs to be ‘within the ballpark’ as the main value is gained in that short amount of time.
- Consider a ‘I need a break’ card. Some Planning Poker decks have a picture of a pie on one card, meaning ‘I need pie!’ When this card is shown, the group takes a mini-break. Group estimation can be quite taxing, so breaks are important to keep people fresh and avoid the pitfall of ‘lets just get this done’ estimation.
- Like most things, the first few times your group uses Planning Poker, it will take longer to get consensus, but over a number of sprints, estimation goes much faster as the group gets comfortable with the process in general.
Next up in this series, my take on how to best manage Sprint backlog task allocation to team members.
Decomposing a User Story involves taking the result your user is looking for (stated as a User Story) and breaking it down into a number of tasks that the team can work on individually. Here are five tips I have found to be very useful:
1) Decompose User Stories into tasks as a team
Group planning is a cornerstone of Agile development. Though it may feel inefficient at times, the benefits are well worth it. See my previous post: Agile Planning: Plan/Estimate As A Group, Really? for more information/details.
2) Attempt to size your tasks to take one team member between 1/2 day to 3-4 days to complete
Motto here equals: Allow a “Race to Done” situation
Creating tasks that are smaller than a handful of hours end up taking to much time administratively to create/track/update/etc. Tasks that are larger than 3/4 days (some would say that is too big, but teams I have been on have found it to be workable) really should be broken into a couple of tasks if possible. They just take too much time to be able to race to done effectively.
3) Create tasks that result in a deliverable unit of work when completed
When decomposing a user story, be sure to break the story down into tasks that can be completed in a small amount of time (Point 2) but don’t focus on the time so much as ensuring you are creating tasks that result in a deliverable unit of work.
Don’t break down a ‘Maintain User’ feature into (like you might have in the past):
- Build the UI
- Build the biz logic
- Build the data tier
Instead create vertical slices of functionality when possible:
- Implement Add User
- Implement Edit User
- Implement Delete User
- Implement Add/Edit/Delete User Automated UI tests
- When a team member takes on the ‘Implement Add User’ task, it’s a contained unit of work, straight forward to know when completed, and also not dependent on other tasks being completed to be able to be tested (like Build UI/Data tier/Biz logic all are dependent on each other to deliver functionality to the user).
4) Don’t get caught deep diving into the details of each task
This is more difficult in practice than in theory. Knowing that task estimation is just around the corner for the team, it’s natural for a experience developer to what to define all details possible, down to ‘how many stored procedures are we going to be creating’. This, in theory at least, helps ensure the estimation process will be more precise. I don’t buy into this theory, at least not from a time invested by the team to get this extra level of preciseness. Certainly attempt to ask the functional questions when decomposing a User Story, so hidden functional ‘gotcha’s are uncovered, but also realize the team is just defining/sizing effort at this point, not writing up a ‘development specification’.
5) Ensure testing/automation tasks are included
On the teams I have worked with over the past years, we have always had a group of professional Quality Assurance Analysts. Our Scrum teams are no different so these types of tasks don’t usually get forgotten, but for the many teams that don’t have QA pros integrated into their Agile teams, I can imagine this to be something missed. A motto of ‘get the functionality to the user as quick as possible’ would seem to lead to that. Just because the team is Agile, doesn’t mean there isn’t any testing that should go on! Automating tasks where appropriate is also very important, given the high amount of regression testing that is needed when sprinting via 2-4 week timeframes.
Next up in this series, group estimate via Planning Poker.
I don’t like Story Point estimating. There I said it. I know many have had success with Story Point estimating, and the Scrum guru Mike Cohn advocates it in his books/etc. I have just found it to be too abstract, and difficult for developers (and myself) to grasp when starting out using Agile techniques.
In my experience, when developers/engineers/etc. are asked to estimate in hours (which is very much the norm in software), they aren’t really thinking in hours. Truth be told, I don’t think many actually think in hour blocks when estimating, but instead think in terms of ‘days of work’ or partial days of work. Here’s an example of what a developer is thinking when giving an estimate: “Hmm.. I think this task should take me about a day, maybe day and half to complete, so lets make it 8*1.5 = 12 hours” Tell me you don’t do that? There wasn’t any self talk on thinking part one of the task would take 2 hours, part two, 6 hours, etc. but rather they would ‘chunk’ their time into days.
So in comes Story point estimating. We don’t want to be estimating User Stories from a calendar time perspective, but instead relatively against each other. This allows for quick estimates that give size, but not ‘commitment’ to time which is what most developers feel an estimate is. Hour estimates come during User Story decomposition, part of Sprint planning. Unfortunately, how do you define one Story Point? What is your logical point of reference?
This is where the ‘Ideal Day’ metric works better for me. This metric was shared with me by Pete Carroll and is really an abstraction of the number of hours you would normally expect a developer to be productive during a typical day, subtracting time for meetings, bathroom breaks, etc. This will vary from organization to organization, but has a large benefit over Story Point Estimation IMHO. Mainly it is the default metric the developers are already thinking in as I eluded to above. There isn’t any translation in their head, no trying to define an ambiguous metric. Instead it’s more gut feel that is natural to all developers with some experience while still allowing for the relative estimating of User Stories to take place. All Ideal Day estimates should be in round numbers, (ie 1,2,3. not 1.5, 2.34, etc).
Trick here is to realize the Ideal Day metric is still an abstraction of time estimates. We don’t plan off the Ideal Day on a timeline, but instead use team velocity matched with Ideal Days to equate to time-lining User Stories from a high level. The Velocity metric will help even out the group estimation variance just like with Story points, but it feels so much more natural to the team.
Next up in this series, decomposing user story tips.
So you decided to go ‘Agile’ with your team? Maybe you have read a book or two on Scrum, XP, or something similar. Many of the base ideals of Agile development make intuitive sense, but this idea of “Group Planning/Estimating”… Really? Surely that isn’t going to work with ‘my’ team.
I know this is where I was several months ago. The team I work with has been using a number of Agile concepts for years to get projects done, and doing it quite successfully I might add. I was reading up on Scrum, and intrigued by its simplicity yet didn’t really get the idea of how important group collaboration is to its success right away. I had originally balked at the idea of having Daily Standups (after having them for months now, I realize how wrong I was on this), so the idea of having the whole team get into a room for several hours to “plan” (which includes some estimating) every two weeks (our chosen sprint length) just sounded so inefficient. I mean… isn’t planning/estimating the PM’s job? Including all those people and taking them away from “development time” to plan?
Well, I am here to say that a transition to group planning can be difficult, but once you work through the growing pains, it’s well worth the effort. Don’t short-change yourself by going 1/2 way either. If you are planning for a 2 weeks sprint, and it’s only taking 1/2 hour to complete, you probably aren’t planning as a group, but more just planning individually and meeting for a short time to pick tasks individually for the sprint. Most people I have spoken with or read online say it should take around 2 hours per week in the sprint to complete the planning processes. (4 hours for a two week sprint) This is dependent on the size of the team, complexity of the project, etc… but it’s a good rule of thumb, and one that I have found to be about right given the number of sprints I have been involved with.
There certainly is work that needs to be done prior to the planning meeting, especially by the ScrumMaster and Product Owner, to groom the project backlog as well as ensure the User Stories are in a state ready to be handed to the team. But the process of taking a feature and decomposing it into development tasks (unless most of the features you are building are trivial in nature), should be done by the team as who knows better than the ones who will be doing the work? Picking the tasks as a team, golden. Estimating the work as a team, what better way to ensure all team members have at least a semi-good understanding of the work being committed to? Having a say in this process also breeds ownership by the team and it’s members. They have some ‘skin in the game’ from the planning stages, and that helps set them up for success to meet the goals of the sprint which is great for all involved.
My next post will cover the ‘Ideal Day’ metric, and how I have found it helps the team size User Stories (features).