Pipelining is not Lean

LeadingAnswers has a thought-provoking post today called "The Pipelining Anti-Pattern". Here's how Mike describes it:

If you have analysts working ahead of development, or have testers working significantly behind development, then you may have “Pipelining” problems. Pipelining is the term used to describe the situation when business analysts are working ahead on the requirements for a future iteration; the development team is working on the current iteration, and the test team is engaged on a previous iteration. In some circumstances analysis may be several iterations ahead and testing several iterations behind. To some people this may seem an efficient use of resources with each group running at their optimal speed, unfettered by the co-ordination constraints of different groups. However from an agile and lean perspective this is problem, a bad-smell that needs fixing.

Here are the problems with pipelining:

Three teams not one – in a project where pipelining is occurring we do not have one cohesive team we have three teams (or more). It is hard enough co-ordinating the members of one team towards a common goal aligned to business benefits. When there are three teams it is just too easy for people to claim that they did their bit and problems lie with other groups. Yet, the fact remains that if the software does not meet business satisfaction then it is everyone’s problem.

Increased Work In Progress (WIP) – Requirements whether they are in the form of user stories, use cases, or formal specifications all represent work invested that has not delivered value to the business. The same goes for code, until this functionality has been tested to the satisfaction of the business it is not valuable. As the time increases between capturing the requirement and finishing the last test two problems occur. The first is classic accounting, money have been invested for no return yet and there is a risk associated with future returns. The second is that requirements decay; the longer we keep requirements around for, the higher the likelihood that they will no longer be required or will have to change.

Increased time from defect injection to defect remediation – the cost of change increases the longer a defect goes undetected. In a pipelining project, defects introduced by faulty analysis could take months to be detected in testing or user review. Fixing the problem after this period of time will entail refactoring far more code (for the work happening in the interim) than if it was detected earlier, and will increase technical debt...

Time fragmentation costs when groups collaborate – on pipelining projects the test group could be working on iteration 2, the developers on iteration 4, and the analysts on iteration 5 or 6. If a developer has a question for an analyst or a tester for a developer then task switching costs will be incurred. Task switching is the time required to respond to an off-topic interruption. We must mentally “park” our current work, try to think back to whatever it is this person who has interrupted us is asking about, recall the details, answer their questions and then resume where we left off. The problem with this is that studies have shown people, while great at many things, are generally very poor at task switching. Our recall of past details is poor, defense mechanisms in our brain designed to dull unpleasant experiences erode memories of nasty problems, and we are terrible at remembering where we parked our current thoughts. The net result is that just a few requests to recall past events really mess up our performance and yield low quality answers to the enquirer.

Throughput optimization – three or more teams running through work at their own speeds with fewer dependencies on others may seem like it is working, but from a systems perspective it is a problem. Processes running at different speeds require work buffers to avoid running out of work and they build piles of unfinished work. This inventory is waste in the system and leads to scrap and more rework as processes change and partially completed work needs to be updated.

I really think Mike is on to something. I have seen this struggle in our own product teams. Read more of the post to understand this problem as well as see how Mike addresses it.

Commitment, Delivery and Customer Expectations

Since we have been on the road of adopting Agile and Lean in our organization, I have had increasing pressure from stakeholders to give them an idea of when we will deliver functionality beyond the current release. Our releases are currently around 4-6 months at this point, so they are asking for things that will be delivered towards the end of the year and into next year. One stakeholder wanted to go out even further than that (up to 24 months in advance)!

At first, I thought that they just wanted to get an idea internally by developing a high-level product road-map to help us with our priorities. However, my thoughts were that this road-map wasn't going to focus on dates. Why? Because that information would not have any sense of reality. This information is at best high-level and with that not enough knowledge and information to make a better prediction of what could be delivered. That better prediction won't happen until we do release planning for the release that we are considering the future functionality. Even with that said, we really won't be able to commit to dates until we get really into the details of the work during iteration planning.

So, there are really three levels of precision: "fine grain" in the current iteration (90% or greater chance of stories in the iteration getting done), "coarse grain" in the current release (60% or greater chance of stories in the release getting done - this number could go down if our releases were even shorter), then "swags" in future releases (40% or less of a chance of stories getting done in the expected time period - this number going down the further out you go). These aren't exact numbers but you get the idea.

What really worried me is that they wanted these long-term dates so they could make commitments to key customers on when future functionality would be delivered. When I discussed my concerns with the stakeholders and talked about the various levels of precision, they believed we could actually get better at our precision if we just planned a little more. However, that would become waste in Lean terms because we would have a lot of work (inventory) "in-progress", plus what we know now about something that will be started six or more months in the future WILL change once we get there. Even with all of that said, my greater concern was making commitments to customers before we can make commitments with the development teams that they can deliver. It should be the other way around in my opinion. Let's figure out what we can commit to and deliver, then start setting expectations to customers.

One of our vendors lives by this philosophy. If we ask for an enhancement, if it isn't something they are committing to for the next release, they will give the answer "we'll put this in our backlog of priorities and consider this for a future release". Then, they will focus my attention of things of value that are coming in the next release and get my feedback on them. Even if my particular request isn't there, I find great value in what is about to be delivered. Sure, there may be some features I don't plan on using or care about but there are usually features that I care very much about. As a customer, this vendor is setting my expectation based on what they know they can deliver, then based on what they hope they can deliver. Hopes don't cut it! If they had promised me a future date, and then miss that date I would have been much more frustrated as a customer. I would rather not know until they were really confident they could hit that date. I appreciate that this vendor has delivered what they promised and has built a reputation with me that they are making constant improvements to their product. Even if those improvements aren't everything that I have asked for!

This is just one of the bad habits we need to break by moving to Traditional development over to Agile. It wasn't that we were better at providing estimates and hitting dates using Traditional development, because we weren't (especially for features out several releases). Sure, we could try to hit those arbitrary dates by cutting scope, freezing scope changes, adding people, working overtime, cutting corners on quality, and other "tricks of the trade". However, would we really provide what the customer really wants? Isn't the goal of Agile to provide what the customer finds value, even if the cost of that value is sometimes difficult to determine? If the quality isn't there, they don't want it early. If the functionality doesn't really do what they expected, they don't want it early. Why this importance of hitting dates because of commitment to customers, when in the end the customers will be disappointed because what they got in return missed their expectations? Let's commit to dates first as a development team and "internal" customers, then set expectations to our "external" customers as we deliver what is important to them.

I believe we got into this dilemma because we developed a reputation of over-promising and under-delivering (because of the problems with Traditional development). Plus, what we would deliver (usually 8-12 months) didn't really cut it with the customer and left them disappointed and frustrated with how long things take. Therefore, their lack of trust in hitting dates causes them to even push harder for us to commit to completing functionality that is long overdue. We need to build trust again with our customers by delivering high-value and high-quality features more frequently in order to gain back a reputation of under-promising and over-delivering. This will take time, but in the interim we need to change the way we have set future expectations with our customers. This may cause some short-term frustration, but will quickly change as customers will change their focus as well on what is being delivered then what is promised to be delivered. After all, isn't that what really counts?

Iteration Planning for the Unplanned

On any project, you will have both planned and unplanned tasks. Planned are those tasks that you knew ahead of time and scheduled some of your work towards those efforts. Unplanned are everything else. In many software shops including mine, the people doing the development work also help internal operations with ongoing technical support for internal and external customers.

Though Agile helps adapt to change by accommodating unplanned tasks in future iterations, there is still a challenge of what happens in the current iteration. If you are doing Scrum, it is expected that at the start of each iteration that the team commits to the scope of that iteration and that any additional work requested by customers should be considered in future iterations. Sounds good, but most technical support is needed on a much frequent basis and turnaround must be at a minimum. Therefore, you can have support issues to resolve on an hourly basis at times and you must respond within the next 12 - 24 hours. So, how do teams address these issues? Mishkin Berteig from Agile Advice has identified a few in his latest post:

1) The part-time allocation to the iteration's work is most common. There are a couple things that need to be done to make this work well in the long term. The main thing is to make the allocation of time inflexible: if you allocate 50% to the iteration and 50% to support, then you should never be flexible about that allocation. This is necessary in order for the team to make a commitment at the start of each iteration.

2) Another common method is the "fluorescent note card" method which requires stakeholder negotiation around the impact of interruptions. With this method, any time a stakeholder comes to the team with a request, the Process Facilitator writes the request on a bright colored note card. The team then does a task breakdown on the card and using their normal process (whatever that is) estimates the work. The requesting stakeholder then has to negotiate with any other stakeholders about what work to remove from the iteration in order to make room for the new work. The trick here is that the team has to be involved because they have already started on some of the work and it might be difficult to dis-entangle things enough. This process works well primarily because it makes the tradeoffs visible. It does not work so well with letting the team make their commitments.

3) A third common method is to form two separate teams: one doing new work, one doing support work. This is simple, effective, and annoying for the people on the team doing the support work! Please don't consider a rotation system since this destroys the process of team development and makes it nearly impossible for the team doing new work to learn their capacity for the purposes of commitment.

4) A less common, but fourth method is to have extremely short iterations. In this method, choose your iteration length to be so short that you can always start work on urgent interruptions before anyone gets impatient! This can be exhausting, but it is one of the best ways to get the team and the organization to understand the large toll that these interruptions take.

We have tried both #1 and #3. The problem with #3 is that developers that are put on the support work would love to do some development work. Plus, it is often difficult to have the knowledge (at least in our organization) to have a select few people who can address all of the support issues. However, if you try and combine support issues with ongoing development, you will find yourself trying #1 next. Unfortunately, it is very difficult to estimate what percentage of time should be put towards Support vs. Development as Support issues will come and go at an expected rate.

I don't think we can manage one week iterations (idea #4) effectively at this point at least until we become more mature at Agile and have the right infrastructure in place to turnaround iterations this quickly.

This leaves us with #2, which is what we are trying to do at this point. On our current Iteration Schedule board, we will have a Story called Issues that will be at the top of the list. Then, every issue will be identified as a Task and will have a color designation to help it stand out against "Planned" tasks and stories. I like this idea because it helps the team address what is reality but also balance the priorities of the Issues against Development stories. It also makes since that the Development team OWN the support issues since they have the most knowledge and skills to address issues in their own product and technical domain.

Can you really plan for the long-term future?

I attended a "planning conference" last night as part of my involvement in Cub Scouts with my son. The purpose of this meeting according to the leader was to "knock out the schedule" through summer of 2008 and we had the next 4 hours to do so. Initially, as we focused on the next few months things went pretty well. However, things begin to break down quickly the further out we went. One of the primary reasons is that the next scouting year would begin in the Fall of 2007 and that will cause some changes with kids, parents and leaders in that process. Therefore, it was difficult to firm up dates because the people involved by that time will be different. Yet, we were told by the leader to just give it our best shot and go on. The other reason that became frustrating is that the leader wanted particular dates and times that we would have different events. It was easy for the group to determine particular months, but much harder to nail down days and almost impossible to determine times. Why? Because who knows their own schedules a year from now (much less the next month). In the end, we created our "project plan" of activities but it left everybody a little frustrated, very tired, and most everybody skeptical that the plan would reflect reality a year from now.

I don't place blame on the leader of this because she was doing what had been done before. It was my first time, and knowing the problems I have had in the past with long term software project planning I was perhaps a little more sensitive. But, I had to ask myself, was it really worth spending hours on something that was probably only good for a few months of accuracy? Sure, there are probably some events we need to reserve well in advance because others are asking for it. However, we really didn't know if we would do these events until we get closer. There were just too many unknowns in the equation and things that could happen between now and summer of 2008. Despite our best planning, we can't really predict what will happen to the day and time. Plus, we already know that the people will change and they may want to do some things differently later on.

That's the beauty with Agile. It puts a shorter "time box" on everything. Let's focus on the next few months with a greater precision in the short term, and intentionally leave the future as fuzzy as possible. When we absolutely have to make decisions, let's make them at the last responsible moment. All the time changing the plan to better reflect the current state of things. Then, when people change (or change their minds), you can change the plan with it. I don't know what I will be doing at 10 am on June 1st of 2008 but I can give you a better idea of what I will be doing in the next six weeks.

Games that you can plan with

Mike Griffiths from Leading Answers has a great post called "Release and Iteration Planning with Innovation Games". Here is his introduction:

In this post I outline some really useful techniques for planning releases and iterations. They are adapted from a great book called “Innovation Games: Creating Breakthrough Products through Collaborative Play" by Luke Hohmann

I have never been a fan of suggesting the use of “games” in the enterprise workplace, as in XP’s “Planning Game”. The term does not sit well with some traditional-type folks; to them it sounds unprofessional and not serious enough for important work. Yet the Innovation Games described by Hohmann are high performance facilitated workshop exercises that produce great results. If the project is serious enough to engage busy stakeholders then I think we owe it to the business to use the most effective tools at our disposal. If calling them “facilitated workshop exercises” eases their acceptance then I’m all for it, because it is the results I’m really interested in, not so much what we call them.

In Luke’s book, he outlines a number of games (exercises) that can be used in a variety of settings. Some like “Design the Product Box” and “Buy a Feature” are probably familiar to many people working on agile projects, others will likely be new. To keep this post a reasonable length I will focus on adapting three exercises for use in release and iteration planning.

The three we will look at are “Remember the Future”, “Prune the Product Tree”, and “Speedboat”. I use slight variations on the last two, I call them “Shape the Product Tree” and “Sailboat” and I will explain the differences when we get to them...

Read more in his post on these three tools. Personally, I like tools like these especially for people who think more visually. Plus, by using stickers you have plenty of flexibility as you brainstorm with others and determine what the release will look like. I definitely plan on buying the book.