I have finally blogged again Deadlines in Agile

I have not been able to produce new blog posts this fall. September 1 I started in my new position as PO in a completely different new domain for me – integrations in the health domain. It’s taken it’s toll on me, but now I am finally getting my head above water.

So now a new post on FrontRowAgile has been published. Here I discuss how to deal with deadlines. Hope you enjoy it! Eventually I hope to post some new stuff on this blog as well. However, sustainable pace is important, so I can’t make promises about when.

Yes, before I forget – Merry Christmas!

My very first guest post on FrontRowAgile "How to Create the Minimum Viable Agile" - an interesting topic

A couple of weeks ago my first guest post on another blog was published. I’d say I learned a lot from the experience with having someone actually reviewing my post before it actually got published. Typically I just read through my posts myself and fire away. My post got a whole lot better after having it reviewed and edited a couple of times. A great experience that I hope I get the chance to repeat!

The post can be read here. Please, leave a comment on the post if you have any thoughts on the concept of a “Minimum Viable Agile”.

5 reasons your Daily Standup isn’t working Most Agile teams have Daily Standups. How do you avoid the most common pitfalls?

The Daily Standup is a great opportunity for teams to inspect and adapt on a daily basis. The guidance from the Scrum Guide on the daily event is rather straight forward:

The Daily Scrum is a 15-minute time-boxed event for the Development Team to synchronize activities and create a plan for the next 24 hours. This is done by inspecting the work since the last Daily Scrum and forecasting the work that could be done before the next one.

The Scrum Guide also gives us three important clues for what you want the team to share in the standup:

  • What did I do yesterday that helped the Development Team meet the Sprint Goal?
  • What will I do today to help the Development Team meet the Sprint Goal?
  • Do I see any impediment that prevents me or the Development Team from meeting the Sprint Goal?

The terminology can be more generic by substituting the term Sprint with a more generic term like iteration, but other than that these three guiding questions works in settings outside of Scrum as well. Although the guidance seems quite simple there are a lot of common mistakes when doing the standup:

1. People show up late or don’t show up at all
Waiting for someone to show up is most likely to be wasted time and the opportunity to fully inspect and adapt won’t be realized if someone doesn’t even show up. It’s important, however, for the team to realize the importance of the event. The problem is a team problem and there are probably a lot of different solutions to it. One common way to try and solve this is by having some sort of punishment for being late. Having incentives for showing up on time is also a good idea. This could work, but then again it might not. A person consistently showing up late might have something outside work blocking them from making it on time; like f.ex having to drop off their kids to kindergarten or school. In that case it’s just going to hurt moral for this person since he/she has external obligations that can’t just be dropped. A better approach here might be to move the standup, so that everyone can certainly be there on time. Communication is important, so the team should figure out the best solution for their particular context.

2. The standup turns in to a status meeting
This must be one of the most common pitfalls. Only the two first questions of the three stated above are answered. In all earnest the first two questions might not even need to be addressed in a team that is working closely together. What was done yesterday and what you will do today may also be seen on the teams task board. Through pair or mob programming it follows that the team most likely know the answer to these questions already. This might in fact also be the case for the third question about potential blockers. However, both pair programming and mob programming is not something every team does – so the third question, the most important question for inspecting and adapting, really should be the focus of the standup. Only answering the two first questions turns the standup into a status meeting, which the standup is not supposed to be.

3. Problem solving and removing of blockers
Although the question about blockers is the most important one of the three in the guidance of the event – don’t try to solve these things during the standup! Instead a simple “I can help with the issue” from someone else is enough and the issue can be handled outside the standup. People are generally busy, so don’t involve everyone in everything. Of course, this doesn’t work so well if no one actually offers to help out – but in a self-managed team this shouldn’t be an issue.

4. Too much details
The three questions makes for excellent guidance. Once people goes outside those questions theres a good chance too much details will be communicated and people just ramble on. If you really have something more you would like to share – choose another opportunity to do so. If it’s really important information you could ask people to stay behind after the standup or just talk to the team when you get back to your team area. Too much details easily leads to too long standups – and engagement goes down. Also – people could stop really listening to what you say.

5. People are not really listening
If the standup is conducted such that the order of speakers is the same every day and the person leading/facilitating the standup is the same every day – people tend to stop listening, just awaiting for their turn to deliver their message. Also, if it’s the same facilitator every time people tend to start talking to the facilitator rather than the team, which only enforces the incentive to stop listening. As mentioned above – too much detail can also lead to less active listening. To effectively be able to utilize the standup for inspecting and adapting we need the whole team to actively participate – continuously looking for improvements. Some simple ways to mitigate this issue is to mix up the facilitator role – don’t let the Scrum Master be the one who always facilitates this event. To prevent the same pattern for who speaks a simple randomizing algorithm probably should be used. Passing a token (throwing a ball) or in the order of arrival could work. Probably some online tool for randomizing order exists. Experiment and don’t do the standup exactly the same way every time.

Conclusion
The daily standup shouldn’t be complicated and the guidance with the three questions that need to be answered is simple. If you get people to show up on time and mix up the order of speakers and facilitator the standups should be working. If they still don’t work for your team and the team questions why you are having them the team has to dig deeper. In teams with a high degree of pairing or using mob programming the standup might even not be needed. Through continually inspecting and adapting the team should discover and take actions to improve. Is your team doing that?

dilbert-standup

Trust is essential for teamwork Trust is the foundation of anyone working together - not only for agile teams

Often a key ingredience of making agile work for an organization is trust. For an organization used to more traditional ways of doing things it’s hard to let go of the command and control style and trusting the team to be self-organizing. Also, trusting that the team (who is actually doing the work) knows best what needs to be done and trusting their estimates of how much work needs to be done can also be a challenge. This also goes the other way – where the team trusts the organization as a whole to support it, remove impediments and to foster a healthy environment for trust to grow.

This being said, trust is important outside of agile as well. A brilliant take on this is from Patrick Lencionis book “The Five Dysfunctions of a Team”. The first dysfunction you need to handle to be able to get anywhere with teamwork is the abscence of trust.

Teams that lack trust waste inordinate amounts of time and energy managing their behaviors and interactions within the group.  They tend to dread team meetings, and are reluctant to take risks in asking for or offering assistance to others.  As a result, morale on distrusting teams is usually quite low, and unwanted turnover is high.

Whether or not you are in a self-organized agile team or you work in a more traditional way, micro management is a clear indicator of lack of trust. Not only is micro management something that affects motivation among people in the organization, but it also is wasteful. Letting the ones that actually have the technical skills make technical decisions is a proven good way to do things. This opposed to having the manager with little to no technical skills decide which database technology best fits your purpose. A traditional belief among managers is that whenever hands are not on keyboard – no work is being done. This of course is wrong since creating software is knowledge work. Knowledge work requires thinking, discussing and exploring options. Managers have to trust that the right things are being done to achieve the end goal.

Also, to really drive good communication and debate around decisions that needs to be made on architecture, design, requirements, team composition or whatever else that needs to be discussed to achieve results – trust is needed. A lack of debate around these topics or any other topic that is important for achieving results is a sign that trust is abscent. What really is needed is vulnerability-based trust. Trust of predicting behavior is of course important, that is trusting others to do what they are supposed to do, the best way they know how to. Vulnerability-based trust is even more important. A team that trusts each other to take risks and speaks their mind will drive healthy debate and fear of conflict will go away. To achieve this kind of trust it is a matter of parking your ego, and focusing on collaboration over competition with your peers.

When you achieve trust where people are willing to take risks and trusts each other to speaking their minds, people start to challenge each other and healthy conflict will occur. Going along with whatever the manager says or the CEO says in a meeting, just because you don’t want to stir things up is clearly not going to achieve good results in the end – particularly if his or her opinions are on something he or she really don’t know that much about.

All in all it is absolutely a shift in mindset that needs to be done. Trust of predicting behavior is step one and this should be fairly straight forward. The next step is vulnerability-based trust and this takes some work to achieve. Trust is essential to achieving results – no matter if you are agile or not.

Dilbert - trust

Characterisics of good metrics Only your imagination limits what you can measure. How does a good metric look like?

A lot of developers often are opposed to the idea of capturing metrics and being compared to others both on the team and to other teams. However, metrics are an important part of the whole process of continuous improvement. If you want to really know if you are improving, you got to have some sort of indicator telling you how you are doing. You also can’t escape the fact that management also want some metrics to make informed decisions.

A good metric first of all needs to be simple. Not just simple to understand, but also simple to collect. If a developer has to do a tedious manual process for collecting data for your metrics, it probably is going to do more harm than good. As far as it’s possible – find a way to automate data collecting. For most teams there’s a good chance your agile project management tool already is capable of capturing the data needed and maybe it even tracks the metric you are looking for already. If the metric isn’t tracked in the tool you could probably still query the data collected through the tools API. Having a decent tool could be important for making metrics simple.

Any metric you choose should also be comparable. This means the metric should be comparable with itself over time. Consistency is important. A metric that isn’t comparable with itself over time isn’t going to help you out much, so don’t bother collecting data for such metrics. An important aspect of metrics is that we aren’t necessarily focusing on a single data point, but rather the trend of that metric. The code coverage f.ex doesn’t provide that much value if you only measure it once. Yes, you get a percentage value, but you don’t know if that is going up or down – and that is what we want. We want the trends – and to be able to see trends we need metrics that are comparable to itself over time. Having said that it’s important to avoid common pitfalls such as comparing velocity across teams. Yes, velocity is comparable to itself over time – but that is within the team. Don’t compare velocity across teams, just don’t do it, please. Try to avoid comparing metrics across teams altogether. Look at trends for the individual teams as indicators for any potential problem areas instead.

A good metric must provide information that can be acted on – that is, it should be actionable. A clear plan of action and causality relation is a key element for successful metrics tracking. A metric that merely measures finite or completed actions, not ongoing activity, is not that interesting. Metrics should, as mentioned, be regarded as a trend and must trigger appropriate actions. The issue with many data points is that they are usually lagging indicators, in other words, they show what happened in the past. A metric that, when analyzed, can forecast the direction actions should take in the future is called a leading indicator.

Another key thing to take into consideration is that management will both track and use the metrics. Metrics in this respect should be honest, or said differently; you should be honest about how management uses metrics. The metrics need to be aligned with the business, in that when a metric trends upward (if thats the way you want it to go), the business value is also moving the right direction. Finding metrics that are truly honest can actually be really hard. F.ex, using lines of code to measure productivity. LOC is a terrible indicator of productivity because it is really easy to throw in extra lines of code if individual performance is measured on how many lines of code the developer is writing. Code coverage is also easily manipulated by not putting in the assert statement. Velocity as well – a team under pressure to “work faster” could easily inflate their story point estimations to please management. Finding truly honest metrics will prove to be hard if the metrics are used the wrong way by management. If metrics are used as probes and indicators of pain-points for teams as a starting point for team discussions, the need to manipulate metrics probably goes away. This means management has to realize the limitations of metrics, which might be easier said than done, right?

So which metrics should you use? I don’t really need to go to deep into that seeing as Ron Jeffries already has done an excellent job of describing his opinions on metrics. Just remember, you get what you measure.

measurement-cartoon

Work less, be more productive? Sustainable pace can sometimes be overlooked, but actually it's one of the twelve principles behind the Agile Manifesto. Why is sustainable pace so important?

Agile processes promote sustainable development.

The sponsors, developers, and users should be able

to maintain a constant pace indefinitely.

I feel there are several reasons for having a sustainable pace. When you love doing something working long hours won’t actually feel like a grind, but you’ll loose that important “down time” that spurs innovation and creativity. A sustainable pace isn’t necessarily a long marathon, but instead a series of sprints. In between these sprints you can get a chance to re-energize and get a couple of moments of quiet so your innovational juices gets flowing again. So the practice of having sprints/iterations/cycles isn’t just to shorten the feedback loop and releasing working software often, but also helps you clear your mind and re-energize.

The research also shows that every hour you work over 40 hours a week is making you less effective and productive – over both the short and long term perspective. A worker just don’t have more than a maximum of eight hours of good reliable work hours in them each day. For a very short sprint overtime can be effective, but it’s not like increasing the number of hours by 50 percent increases output by 50 percent. An increase of about 25-30 percent in output is more typical. If the period with overtime is carried on for several weeks it’s found that the team should just have stuck with the usual 40-hour week. Also, if the team works long hours for a long time getting burned out, it can take weeks to return to normal productivity after returning to the usual 40-hours week. As the linked article says, one hundred fifty years of research proves that shorter work hours raise productivity and profits, and overtime destroys them.

Rules of Productivity

For me personally I actually feel so strongly about this that if mandatory overtime is enforced I would really, really start to consider if I am really at the right place. Earlier in my career I was on projects where overtime was “optional”. They didn’t say it was mandatory, but it was strongly advised. Back then I was both younger and lived alone, so it only affected me personally when working those long hours. Fortunately this was only a short period of time, so I didn’t feel burned out. Also, I really loved what I was doing, so it didn’t really feel like work. If I had done some research back then I would definitively have had a talk with the managers about this. For one specific project it wasn’t just a short sprint with long hours, it was more like a constant state of overtime-mentality. I really think that project as a whole suffered because of that.

Now that I have both a wife and kids it’s more or less unthinkable to have a job where I must work long hours, even just once in a while. The research backs this up – you don’t really get that more produced by increasing the number of hours beyond those 8 hours a day, and both short term and long term consequences must be payed if you do. So sustainable pace to me is a given – no matter if your workplace is agile or not.

dilbert-google-20time

Success rate of agile projects Agile projects are far more successful than waterfall projects. How much more successful are agile projects - and is that enough?

The CHAOS Reports have been published every year since 1994 and are a snapshot of the state of the software development industry.  The 2015 edition studied 50,000 projects around the world FY2011-2015, ranging from tiny enhancements to massive systems re-engineering implementations.

For the 2015 report it has been redefined what “successful” means. Earlier a project was considered successful if it was on budget, on time and on target (e.g. scope). Standish Group admits now that “on target” was never a good measure because it lacked any measure of customer outcome. A lot of projects could meet all these constraints, but still leave the customer unsatisfied – and this is noted by Standish Group. “On target” is now replaced with a measure of customer perceived value, and other additions has been done, so that in all there are 6 factors to consider: on time, on budget, on target, on goal, value and satisfaction. This has resulted in a 7% decrease in the rate of successful projects from the year before.

If we look at the agile vs waterfall metrics this is what the report gives us:

Agile vs. Waterfall

What is very clear is that agile projects are a lot more successful than waterfall projects. Considering all project sizes agile is about 350% more likely to be successful. For large size projects agile projects are 600% more likely to be successful. Another key takeaway is that regardless of approach you really should be doing small projects. If you are trying to scale your projects, then waterfall seems to not scale very well. In any way – choose an agile approach. These numbers have been consistent over time now – abandon your waterfall approach!

Although this report tells us that agile clearly is the way to go, there is still lots of room for improvement. The average number of successful agile projects is at 39%. This leaves 61% of the projects challenged or failed. While this is clearly better than 89% (waterfall), it is still a high number and an indication that there is a lot of bad agile out there.

I have also taken a look at The State of Agile Report 2015 and The 2015 State of Scrum Report. There is a lot of things to take from these surveys, and maybe I’ll address more of my takings on these reports in later posts. For this post however, I am going to focus on the success and failure rates of agile projects.

In The State of Agile Report it is stated that only 1% of the respondents admitted that their agile implementation was unsuccessful. In the email I got from VersionOne (yes, the survey is sponsored by a vendor) the wording is a little different: “only 1% had experienced agile failure”. Up until the 2013 version of this report one of the options for leading reasons for failed agile projects was “None of our agile projects failed”, and it’s a little confusing that this has been removed as an option now, because we don’t really get an indication of the number of failed agile projects anymore (except that only 1% has experienced agile failure). For 2012 and 2013 the numbers where 18% and 15% respectively for this option – so quite low numbers. The State of Scrum Report says the overall success rate of projects delivered using Scrum is 62%. If the Scrum project was deployed and managed through a PMO 93% of them were considered successful.

The numbers from the surveys seems to be miles away from the Chaos Report numbers with regards to the success rate. One explanation is of course that the criteria for calling a project a success is not the same, and probably a lot more stringent in the Chaos Report. The State of Scrum Report even has 52% of their respondents stating that the most common challenge is identifying and measuring the success of Scrum projects. Still, I am having a bit trouble believing that only 1% actually has experienced agile failure and that 93% of Scrum projects deployed and managed through a PMO are considered a success. Having said that, these surveys are by no means scientific and the objectivity of the respondents with regards to the projects they have been on might not be the best.

So despite the numbers in The State of Agile Report 2015 and The 2015 State of Scrum Report being very optimistic for everyone that loves agile, it seems the numbers might be a bit over-optimistic. The Chaos Report probably has a bit more valid numbers – but the overall takeaway should be that agile rocks and that waterfall should already have been thrown away. Having said that, it seems to be a lot of work left before we can be happy about the overall situation. Overall 61% of agile projects are not considered a success – and that is a lot of waste in the long run. We can always do better, and that is exactly what we must do through continuously improving ourselves and our agile practices.

Continuous Integration, Continuous Delivery and Continuous Deployment Continuous Integration, Continuous Delivery and Continuous Deployment - what’s the differences and why should you have these approaches?

An engineering practice that seems to be more or less given if you are using any agile approach is continuous integration (CI), where a designated server or cloud based service polls your repository and kicks off a build whenever a change is detected. The main aim of CI is of course to prevent problems when integrating new code into the codebase, shortening the feedback loop for developers – so that the developers get almost instant feedback on whether or not they broke something with their last commit. One thing is feedback on whether or not the build of the code itself is doing ok after integration – the other is the suite of unit tests that also should run, getting feedback on whether the new code conflicts with existing functionality or have some unwanted side effects. Compare this approach to integration only at the time of release – a situation which begs for integration problems, or “integration hell” as it is called in the early descriptions of XP. When you also consider that a bug found 3 weeks from now takes up to 24 times longer to fix compared to a bug found today, you quickly realize that there is a lot to gain from using CI in your project.

If you take CI a step further and consider continuous delivery – that would be a series of practices designed to ensure that code can be rapidly and safely deployed to production by delivering every change to a production-like environment and ensuring business applications and services function as expected through rigorous automated testing. By deploying the newest changes to a staging environment using complete automation you will most probably get comfortable with the idea of doing the same to production, although this is not a necessary step in continuous delivery – instead a deployment to production is a manual decision, unless you have continuous deployment.

So once you are doing continuous delivery you can take the next step – continuous deployment. By using continuous deployment you will automatically deploy all new changes that passes all the automated tests (unit tests, acceptance tests ++) to production. Both continuous delivery and continuous deployment have similar benefits:

  • Reliable Releases: You learn by doing and get good at what you do over and over again. So if you have continuous delivery your deployment process and scripts are tested repeatedly before deployment to production, and also you get a lot of practice of deploying to production by having continuous deployment. Compare this to the very unreliable process of only deploying to production maybe 3-4 times a year – there is almost always a lot of complications by not deploying often. So many things could have changed since the last time you deployed: not only code or configuration changes, but changes in staffing as well. Half the developers could have moved on to other teams by the time the deployment to production happens. If anything goes wrong (as it usually does in this setting), you are in a world of hurt.
  • Shorter feedback loop: By releasing more frequent the development teams get user feedback more quickly. This enables the team to focus on features that are important to the users and helps building the right product. Doing things like A/B testing also becomes a lot easier, so that the team can really get to know what the user needs and wants.
  • Accelerated Time to Market: Continuous delivery/deployment enables the teams to  quickly deliver the business value inherent in new software releases to customers. This capability certainly could help the company stay a step ahead of the competition and could create a competitive advantage.
  • Improved Productivity and Efficiency: Automation saves time for developers, testers and DevOps
  • The quality of the product increases: Since new changes are continuously being delivered to staging by passing a rigorous set of automated tests the number of bugs that slips through the cracks will decrease. The mentality of a developer working in an environment where they are being held more accountable and where they see their changes being deployed to production continuously is different from the case where a change done now just sits in the codebase waiting to be released at a (often much) later time.

To sum up I would say CI really is more or less mandated if you have any agile approach. It does not take much effort to set it up, so not having CI is really not an option if you want to succeed with agile. Continuous Delivery on the other hand could be more tricky to set up, although a lot of out of the box cloud based services exists. You also really have to put an effort into automated tests, not only unit tests, but also feature level acceptance tests. There is really just not a possibility to have Continuous Delivery without a suite of automated tests. Once you manage the process of Continuous Delivery and pushing all new changes to a staging environment you are really mostly ready for Continuous Deployment. Continuous deployment should perhaps be the goal for most companies that are not constrained by any regulatory or other requirements. It should not be IT limitations that decides whether or not your company should do continuous deployment, but rather whether it is right for your company based on business needs.

cd

 

Definition of Done Driving your work to the done column is important for all Scrum teams - but what does “Done” mean?

A Product Backlog item described as done varies a lot per Scrum team, so it does not necessarily mean the same thing for your team as it does for another team, even if the other team is within your company. The most important thing about “Done” is that everyone must understand what “Done” means. All Scrum teams should have a clear Definition of Done (DoD) that they can use to decide whether work on a story really is complete or not. This way it’s transparent and clear what exactly is expected of everything the team delivers, and it also ensures quality fit for the purpose of the product and the organization.

A typical DoD in the software world could be something like:

  • the code is well written and follows our standards
  • the code has been reviewed or pair-programmed
  • it has been tested with 100% test automation at all appropriate test levels
  • it has been integrated and documented

Over time, as teams mature, it is expected that their DoD will improve and be more stringent so that the quality delivered will have even higher quality. Adding to the example DoD above, the team could maybe add “Deployed to production”. This extra criteria of being deployed to production is not an easy task for a lot of teams, but speed is important and Continuous Delivery is definitively a competitive edge for a lot of organizations nowadays.

According to Jeff Sutherland it’s proven through data analysis of a lot of Scrum teams through Openview Venture Partners, that teams that have a stringent definition of done that is also called “Done, done”, doubles their velocity. “Done, done” means the work has been code completed and is fully tested on feature level. It’s less stringent than the example given, but enough to drive velocity up significantly.

Code completed in Sutherlands case includes all bugfixes – which means that bugs found in a sprint should be fixed in that sprint. This is actually a question I have seen a lot lately, whether or not bugs found in a sprint should be fixed within that sprint or whether the bugs should be estimated and prioritized for later sprints. Of course bugs found in a sprint should be fixed within that sprint – given that you have a DoD which demands code being completed!

To sum up: A DoD is important for creating transparency, so that all team member know what is expected from their delivery. So having a DoD is a lot better than not having one at all. If you in addition have a DoD with the “Done, Done” criteria which means code is completed (with bugfixes) and the work has been fully tested on feature level – your velocity could double! And who doesn’t want that?

dilbert_almost_done

MoSCoW Difficulties prioritizing work is a recurring problem. An easy and simple tool to help out actually exists - the MoSCoW method.

I have talked about prioritization before on this blog, but it keeps coming back as a problem. The reason is, of course, that prioritization is hard. Everything feels important, you want to please everyone. The result is a long backlog of stuff to do – where everything in it is just as important as the next thing. This leads to all sorts of problems, which is discussed here.

A simple prioritization technique to help out is the MoSCoW method. The method originated from the DSDM methodology and, as mentioned, is a simple way to sort features/user stories into a prioritized order. This will enable the teams to quickly see and understand the customer’s view of what is important – and also what is not that important.

If the teams deliver features prioritized as “Must have”, “Should have” and “Could have”, the “Should have” and “Could have” are the first requirements to go if something needs to be dropped from a delivery. MoSCoW categories are outlined more broadly below:

Must have
Requirements labeled as MUST are critical to the current delivery timebox (sprint) in order for it to be a success. If even one MUST requirement is not included, the project delivery should be considered a failure.

Should have
Requirements labeled as SHOULD are important but not necessary for delivery in the current delivery timebox. SHOULD requirements can be as important as MUST, but they are often not as time sensitive or there may be another way to satisfy the requirement, so that it can be held back until a future delivery timebox.

Could have
Requirements labeled as COULD are desirable but not necessary, and could improve user experience or customer satisfaction for little development cost. These will typically be included if time and resources permit.

Won’t have
Requirements labeled as WON’T have been agreed by stakeholders as the least-critical, lowest-payback items, or not appropriate at that time. As a result, WON’T requirements are not planned into the schedule for the delivery timebox, and may be dropped from the backlog altogether to keep the backlog lean.

Using the terms in the MoSCoW method it should be fairly easy to categorize your company’s prioritizations. If everything falls into the “Must have” category even after applying this method you are back to square one, leaving the teams in the dark about what the priorities really are. MoSCow should be easy to start using and easy to use, although doing the prioritization itself might be hard. The categorization however should make the process a lot easier for POs and stakeholders not familiar with actually making these prioritizations.

caveman_rocks_sticks_prioritize

Engineering practices Scrum does not prescribe any engineering practices. This makes Scrum relatively adaptable - so which engineering practices should you choose?

If you compare agile methods, Kanban would be the least prescriptive of them. Kanban is just next to “do whatever” – with the only constraints being Visualize your workflow and Limit Your WIP. Scrum on the other hand has a few more rules, still leaving it relatively lightweight:

agilemethods_compared

XP and certainly RUP is more heavyweight when it comes to rules to follow. Regarding engineering practices, neither Scrum nor Kanban prescribe any that must be used. XP on the other hand prescribes practices such as test-driven development, pair programming, shared code and continuous integration. It’s quite common to adapt XP-practices (engineering practices prescribed in XP) in both Scrum and Kanban – but which ones should you choose?

As always it helps to see what others have done and are doing. In an award winning white paper from 2011 called “Scrum + Engineering Practices: Experiences of Three Microsoft Teams” we get to see which nine practices the Microsoft Engineering Excellence group suggests and the experiences of three separate Scrum-based teams at Microsoft. The nine practices used was:

  1. Planning poker
  2. Continuous integration
  3. Unit Test-Driven development
  4. Quality gates (aka “Done Criteria”)
  5. Source control
  6. Code coverage
  7. Static analysis tools
  8. Peer review
  9. XML documentation

Most of these are also practices in XP, but I feel one practice in particular is missing here: Pair Programming. This is a practice which often is used, however maybe not as a default. Maybe pair programming is used for certain stories where only one member of the team has the knowledge of how to solve the story. Using pair programming can be a good practice in such cases to make teams more robust.

Common for all of the practices is focus on the quality of the product. Of course there are downsides with some of the practices. TDD drives the development time up. However, the increase in quality with a lower defect rate certainly makes up for this. Continuous Integration could be problematic when it comes to build and test runs not always being successful due to bad merges, build system problems or source control integration problems. But also here the constant focus on quality makes up for the “downside”.

All the teams in the Microsoft study saw its productivity temporarily drop for three iterations when starting to use Scrum and these nine practices. From the fourth sprint on they experienced significant improvement in productivity without any increase in defects. Also the team that to a lesser degree followed the nine practices had by far the highest defect density.

Although all these practices were seen as beneficial for the teams in the study it is not necessarily so for your team. There isn’t a common recipe for success, which is exactly why the percentage of failed agile projects is as high as 49% according to Jim Johnson at the Standish Group. The number is 86% for waterfall projects, so agile is doing a lot better than waterfall. Still, 49% is a lot, so there is a lot of bad agile out there. And, as mentioned, some of the reason for this is that there isn’t a single recipe for success. Indeed there are a lot of things you really should do to be successful, but at some point you need to make some choices for your team with respect to what will work and what will not work – both when it comes to using Scrum, Kanban or other agile methods and when it comes to engineering practices.

dilbert_engineering

People don’t buy what you do; people buy why you do it Everyone can say what they do or what they make. Not many people or organizations can tell you why they do it.

This blog post isn’t specifically about Scrum. It is, however, at the core of running a business and making products. Your cause, belief or purpose for doing the things you do relates back to a previous post I had about motivation, where mastery, purpose and autonomy were highlighted as the drivers of Motivation 3.0.

dilbert_magicmotivation

To start with why in everything that we do we must ask ourselves what our cause is and what we believe in. If you don’t know these things chances are motivation won’t be all that great. Organizations that don’t really know why they exist will have a hard time both recruiting and motivating new employees and keeping the ones they already have motivated. Making money is just a result of what we do, not why we do it. To get high performing (scrum) teams it must have a purpose. Sure you can get a good implementation of Scrum – but it might very well just not “feel” right, and it will certainly not be a hyperproductive team. It’s not enough to recruit team members by saying what cool stuff your are building and how you are going to do it. It’s also necessary to state why the product you are going to build matters and its greater purpose. So initially the organization as a whole must have a reason for existing and then this must be a reason so clear and consistent in all that the organization do, that everyone in the company can share the same belief.

In his book Start with Why: How Great Great Leaders Inspire Everyone to Take Action, we get an explanation for why some people and organizations are more innovative, more influential and are able to inspire greater loyalty and engagement among their customers and emloyees than others. The reason is that they all think, act and communicate in the same way – and that is the opposite from almost everyone else. Every single organization on the planet works on three levels:

  • What we do (products or services offered)
  • how we do it (the organization’s or individual’s strength, values or guiding principles which they feel sets them apart from their competition)
  • why we do it (purpose, cause or belief)

When all these are in balance others will say with absolute clarity and certainty that they know who you are and what you stand for. Sinek calls his idea the Golden Circle.

goldencircle

 

The way people naturally communicate is from the outside-in, from what is easiest to understand to what is hardest to understand and explain. They tell what they do and how they do it different or better, and then expect people to buy their product or service, or get their vote or support. Leaders and organizations which inspire and creates loyalty and engagement does the exact opposite – they communicate from the inside-out.

The example which is perhaps easiest to understand is Apple. They start with why – they communicate in a way that drives decision-making and behavior, and literally taps the part of the brain that influences behavior. If Apple were to communicate like everyone else, this is could be how their sales pitch for their product ideas would look like:

  • We make great computers
  • They’re beautifully designed, simple to use, and user friendly.
  • Want to buy one?

This seems like a pretty standard pitch. You say what you make and how it’s better, then ask people to join/buy.

The way Apple actually pitches their products:

  • Everything we do, we believe in challenging the status quo thinking differently
  • The way we challenge the status quo, is we make our products beautifully designed, simple to use, and user friendly
  • We just happen to make computers – wanna buy one?

It’s of course not as easy as reversing the order in which you communicate your product ideas. It’s also of course about how you think and act. There must be clarity of why – you must know why you do what you do if anyone else is going to see this. How you do things must be aligned with these values, principles, strengths and beliefs. Lastly, everything you say and everything you do must be consistent with what you believe, because this is the only way people will know what you believe.

Sinek sums it up as this:

People don’t buy what you do, they buy why you do it.

So to get motivated and loyal employees and also loyal customers who “just feels” that your product or service is worth buying you should start with why. Any Scrum team that don’t have this clear sense of why will struggle in the end. Sure it’s the product owner’s responsibility for sharing a vision to the team and through that inspire and give a purpose for the team’s existence. This is not easy if the product owner is employed in an organization without a clear sense of why. So to every organization and every individual out there: Do you know your why?

Why you should estimate in points Estimating in points is more accurate, faster, important for release planning and will help your team improve performance

It’s easy to understand why Gantt charts initially are so appealing. They are easy to understand and you see the timespan for all tasks with a start and end date, leaving the project manager with the complete overview of what is being worked on right now and when it is going to be done. So using Gantt charts seems really easy to use when you do release planning. The problem is that the hours that these charts depend on assumes that you have a fixed amount of hours for some person and that none of the dependencies ever change. The hours to do a story of course rely heavily on the person doing it and is impacted by changes in dependencies. This means that to be valid a Gantt chart needs to be updated a lot. On big projects you might even need resources whose only task is to make sure the Gantt charts are up to date. Story points are more stable – and that makes planning with story points a lot easier. 5 points today will be 5 points a month from now or a year from now.

By doing estimation in story points a team can establish their velocity, a measure of the amount of work the team are able to handle during a single sprint. The nice thing about this is that the product owner can use this velocity to plan a release date. Knowing approximately how fast the team is moving s/he can calculate how many sprints it takes to implement the needed functionality for the product to be shipped. Using hours you don’t know the velocity of your team – and that also makes continuous improvement hard. How do you know if any of the teams process improvements work if you can’t measure how fast you are moving. Using points as a measure of output you can easily see if your improvements are working or not. When using hours, both input and measure of output will be hours. Since there are a fixed number of hours every day velocity based on hours can’t really be increased. There is a way it can be increased, but the only way to increase output when using hours is by increasing the number of hours put in. This means making your people “burn the midnight oil”. Studies shows that productivity dramatically decreases when working long hours, and of course – long hours is not the “sustainable pace” that we want.

So points are stable, making release planning possible. Being the measure of output it’s also possible to measure improvements done sprint by sprint. In addition story points are a lot more accurate. Humans are pretty terrible estimating stuff using hours. Studies done as early as the 1940s by the Rand Corporation showed this. A way to overcome this was by putting things in relative sized piles using the Fibonacci sequence when doing estimation, instead of using hours. This method is called the Wideband Delphi estimation method. The same technique is what Planning poker is based on – and most agile teams are familiar with this practice. A quite recent study at Microsoft shows that in the early stages of the project the estimation error using hours is as much as 400% vs 50% using story points.

cone_of_uncertainty

 

In addition to all this story point estimation takes a lot less time. According to data presented by Jeff Sutherland story point estimation would cut estimation time by 80% for a CMMI Level 5 company and a telecom company reported that story point estimation with planning poker was 48 times faster than waterfall estimation practices within that company. Since we are reasonably comfortable with relative sizing vs not too comfortable with estimating actual sizes we tend to spend a lot longer time to do those estimations, taking the level of discussions to a couple of levels deeper than you would with point estimations.

The takeaway here is that you should just stop estimating in hours. Numerous studies backs this up – use story points instead!

dilbert_estimation

 

Context switching causes brain damage It is well documented that multitasking kills productivity. Can Scrum help you reduce waste created by context switching?

That multitasking causes waste has been documented well and is to my opinion a well known fact. The most referenced book is probably “In Quality Software Management: Systems Thinking” by Gerald Weinberg. Whenever this is referenced the following chart is presented:

weinberg_contextswitch

 

Weinberg calculated that having just one project to focus on lets you spend 100% of your time on that. Adding just one project to two in total will cause you to lose 20% of your time to context switching. These numbers have proven to be true or even worse in real life scenarios. Another study done in 2007 with workers at Microsoft showed that even just responding to email or instant messaging lead to an average of 10 minutes or more resumption time. To truly get back to the task they were doing before the interruption it took an additional 10-15 minutes.

In addition to losing time to context switching it actually causes damage to your brain cells! Psychology Today reports the following:

A new body of research has found that multitasking makes people less efficient and reduces the level of brainpower used for each task. Also, people who overburden their minds with too many tasks at once can have problems with short-term memory.

Further it is reported that the mind slows down and adds a few seconds between each task. You get more stressed, causing a stress response. This stress response causes an adrenalin rush that can damage the cells that form new memories (short-term memory). In addition this stress response might weaken attentiveness and alertness.

It is pretty obvious what we should do – focus on on fewer tasks! In other words we should limit work in progress and avoid interrupts as much as possible. This is, as most of us know, easier said than done. For the most part it seems it is expected that you are available all day, on the phone, instant messaging, mail or in person. As an individual it isn’t easy to say no to management if they expect you to always be ready to shift your focus. Maybe Scrum can help us?

In Scrum the WIP limit is set per sprint, so that the team ideally only need to focus on the sprint backlog for any given sprint. Having a limited number of stories which are in play each sprint will help reduce context switching. However, it’s still possible that team members switch between backlog items. This could be because impediments limits team members from finishing the backlog item they are originally working on. If this is the case it is important to have a ScrumMaster that actively works to remove such impediments as soon as possible.

Pressure from external parties such as management or stakeholders could also cause team members to switch to and from backlog items. This interference with the team could also involve doing stuff that’s not even in the sprint backlog. In these cases it is important for the ScrumMaster to protect their team from such distractions. This protection could go as far as protecting team members for timeboxed bursts of work of 90-120 minutes followed by some recuperation time, so that during these bursts of work the team is not available for external interruptions.

The product owners also have an important job of demonstrating to management that focusing on fewer tasks and letting the team work uninterrupted is the best way to do things. Although the timeboxing of sprints in Scrum is one reason management might feel that Scrum is too rigid, there is a reason for this rigidness: Context switching is expensive. It is, as mentioned in an earlier blog post, possible to use an interrupt buffer each sprint. Here typical interrupts are calculated for. The thing about this buffer is that it should never overflow – if it does the sprint should be aborted. This is something management doesn’t want because aborts are visibly expensive – and thus interrupts will be kept at a minimum whenever they are kept visible through the use of an interrupt buffer.

All in all – Scrum can help reduce context switching. Sprint backlog reduces the number of items that can be worked on and ScrumMasters and Product Owners together protects the team from external interruptions. If you work on a team that is pretty much protected from external noise you also need to take action as an individual. Turn off your phone, exit Outlook (or whatever you are using) and exit whatever software for instant messaging you are using. Of course, I don’t think it’s realistic that you can always work like this, but having work bursts of 90-120 minutes where this strategy is used will without a doubt increase productivity.

Motivation 3.0 A framework for explaining why working in a Scrum team can feel so right

From the moment I started working on a Scrum team I knew this was a better way of working than I was used to. Scrum done right promises twice the work in half the time – that in itself does not explain why working on a Scrum team feels so right. Of course, being able to be that much more productive is appealing – but I think there is more to it than that.

One day after having read Drive: The Surprising Truth About What Motivates Us by Daniel Pink the pieces seemed to fall into place. In this book Pink talks a lot about what motivates us. He compares our motivation to that of an operating system which has evolved over time. Motivation 1.0 was survival, and worked well until society started to get more complex. Pink states an operating system based purely on the biological drive was inadequate and such Motivation 2.0 was introduced. This second drive stated that humans set out to seek reward and avoid punishments. This “carrots and sticks”-approach with “if-then” rewards to boost motivation is how things have been for a long time now. This is where Pink calls for an upgrade of the operating system – to Motivation 3.0.

With Motivation 3.0 humans still have a behavioral drive and also still seek rewards and avoids punishment. The third drive however is that humans have a drive to direct their own lives. The motivation formula (so to speak) is built on a trifecta: Mastery, autonomy and purpose.

motivation3pt0

These 3 things maps really well into Scrum and that’s a plausible explanation why working with a Scrum team feels really…motivating! My take on the mapping is done below – feel free to comment on this.

Continuous improvement through doing retrospectives and receiving feedback during review/demo is all about mastery. What impediments are in our way to achieve mastery of whatever we are creating in our Scrum team? Mastery can never be achieved and thus the improvement process can never stop.

Autonomy also maps very well to how Scrum works. Self-organizing teams is all about empowering the team members and letting them decide for themselves how to solve things, which tools to use and which team member to actually perform the task. Autonomy for the individual team member is very important in Scrum, and this is one of the top reasons why I like Scrum.

Part of the product owner responsibilities is to have a vision of what he or she wishes to build, and convey that vision to the scrum team. This is key to successfully starting and maintaining any agile software development project. By sharing this vision the purpose of the Scrum team will not be just about implementing that one specific feature or fixing that one specific bug. It will be communicated on a higher level. So it boils down to the stakeholders and ultimately the product owner to communicate the purpose of the Scrum team.

I could go into more details on the specific parts of Scrum that I like, but the point here is that having Motivation 3.0 as some kind of framework helps explaining that QWAN (quality without a name). Of course I like something that boosts my motivation! If you haven’t read Daniel Pinks book – I highly recommend it.

 

Prioritization and communication If everything is important then nothing is

The quote given is one I heard Patrick Lencioni use, although it probably can be traced to others as well. He describes well why a lot of companies fail to prioritize and instead have these really long lists of “top priorities”. They are afraid that if they focus on just a few important goals others will fall by the wayside. I believe this is absolutely true – and I even can see that in my personal life sometimes. However – once you realize this (probably some consultant or someone like that tells you this), you should do something about it. This is hard to do it seems – a lot of companies struggle with prioritization.

In Scrum or Kanban prioritization is often one of the most important parts of the implementation. There are a couple of ways prioritization might be screwed up. First of all – the product owner says that everything is important. This is classic and often gives the team trouble down the line because they aren’t working on the “right stuff”. The problem is of course – what is the right stuff? That is a business decision and shouldn’t be placed on the team. It is unfair to make the team make these business decisions for the organization.

Another way prioritization might be screwed up is by having multiple product owners – which actually translates to multiple stakeholders. If this is the case the team must once again make decisions about prioritization and which stories will result in most business value. This happens when the organization fails to decide or is unwilling to make one of the individuals the single product owner in charge of prioritization. This was nicely put by Mike Cohn is a recent newsletter. Here he also states that “When a decision cannot be made it should be pushed up the organization, not down”.

How can these situations be solved? Do not try to hide the organizational dysfunction – make it visible. Let the teams explain that they really have no idea how to decide how much business value the stories in the backlog represents – and that isn’t their job either! Communication is key – the word must be spread upwards in the organization. Lack of prioritization creates huge amounts of waste and can cause team moral and motivation to plummet.

If this sounds familiar – make changes – today!

dt120715

Important patterns What do you think the most important patterns are?

Yesterdays weather, interrupt pattern and scrumming the Scrum. Those are the 3 patterns Jeff Sutherland says he always implement in every team.

I don’t disagree. We have now implemented these patterns in my current team and we are seeing a 2-3x improvement after 3 sprints. If we had only heard of these patterns earlier! Yesterdays weather helps out in a lot of ways. First of all it’s really easy and saves time and effort in calculating how many story points the team can commit to for the next sprint. It also helps the team to not overcommit. Overcommitment is one of the worst things for morale and motivation. Not being able to finish a sprint with all stories moved to done time after time crushes team spirit. Together with the interrupt pattern which calculates for all kinds of typical interrupts during a sprint (high priority bugfix in production etc) a 4th pattern actually emerges – Teams that finish early, accelerate faster. By not overcommiting the team can start pulling stories and the morale and motivation gets a boost. Its not about velocity, its about acceleration!

The last pattern, scrumming the Scrum, is about continuous improvement. Take the most important improvement the team can do (the kaizen) and put it into the sprint backlog as a separate item. This causes the team to self-organize on removing that single most important impediment. This systematical approach to continuous improvement is very beneficial in the long run – helping the team to always find and remove impediments, which leads to the possibility of increasing the teams velocity.

I’m hoping our focus on these 3 patterns will help us achieve even higher productivity than we’re already seeing. We’re out to a good start at least!

 

Misunderstanding Scrum How common is it for senior management to misunderstand Scrum?

It’s probably common for everyone not familiar with Scrum to misunderstand what it really is. I have a feeling that these misunderstandings are most common among senior management. The reasons for this could be many, but of course limited time to learn what it’s all about is of the reasons (since they obviously have more important things to do!). I personally feel senior management should understand at least the basics of Scrum if any part of the organization has implemented or is going to implement Scrum.

Just recently a CEO of a company was trying to describe to a crowd of his customers how they work in relation with their outsourced projects. He started with saying that Scrum is pretty much all about estimating, and this is done by doing planning poker. In planning poker the product owner always want to have as low an estimate as possible, while the developers want as high an estimate as possible. In the end all the individual estimations are summed, and that sum is the estimation of that task. Yes, and they work in sprints too. He also consistently called the product owner a Scrum Master. It’s unclear whether he understands the difference of those two roles or not, but probably he doesn’t.

Lets not get into details about how wrong this CEO has gotten Scrum. Instead – lets just conclude that Scrum trainers and coaches aren’t going out of work any time soon.

Misunderstandings