Changing the way you do long term planning

We are in the process of implementing a pull scheduling system for each of our product development teams. No longer do we think in terms of projects, but in terms of releases and iterations. No longer do we figure out the entire scope in detail of what will be contained in the product, but we determine the details when we are ready in a particular iteration in a release. No longer do we have rigid control of scope for the entire project, but we try to contain what we do in a 2-week iteration once it has been planned and committed by the team. No longer do we try to figure out exactly what we will do in the next 1-2 years, but we "pull" enough work to determine the next release when we are ready to think about it. No longer do we wait for customer feedback only at the end of the project through alpha and beta cycles, but we get internal and external feedback through demos at the end of each iteration. No longer do we conduct vague and long weekly project status meetings, but we conduct quick and concise daily standup meetings. Our world has definitely changed, but in the minds of our executive stakeholders they have struggled a little with it. In particular, if you are only planning within short iterations and the current release cycle, how do you do long-term product planning for customers that want to have some idea of a delivery date for something "down the road"?

I have no problem in planning long-term if the planning is focused more on priorities than dates. Though the team is pulling just enough work for the next release, the more work that is prioritized and thought out (at least to a high level) will make the release and iteration planning go much smoother. And, actually I don't even have a problem committing to a particular date for the next release or any iteration in the release because those are fixed. However, committing to particular scope outside of the current iteration is something that I find challenging. Why?

Because we know only look at the details of the release within the current iteration, we haven't given enough thought about the remaining stories in the release. Sure, we have done some high-level estimates but we have learned that what you think you are going to do after thinking about it for a few minutes is much different when you think about it for a few hours in iteration planning. Until you really start looking at particular tasks and tests that the team will do in fulfilling each story, you don't really have an accurate estimate. And even with that said, you may find surprises within the iteration after planning.

The other challenge is that you are dealing with change in a whole different way. When you find a defect, you don't "log" it in a repository automatically because you don't have the time to fix it during the project. Instead, you deal with defects as they happen. When you show functionality in a demo, you don't log future enhancements automatically to be considered in future projects but you make appropriate changes if they are valuable and while they are fresh in the team's minds. When new issues come up, you create new stories to deal with them if they are deemed the right time. All of these things change scope, which changes what is done in a particular release. This changes the expected delivery dates many times. Which makes it very difficult to predict the scope of a release, and makes it almost impossible to predict when new functionality will be available in 12-18 months.

Now, this may sound bad initially to your stakeholders, because they want to make promises to customers as soon as possible. So, this may sound like Agile and Lean is the wrong way to go right? While I can't predict what will be done in 18 months, I can show you progress every two weeks. This progress will have higher quality because it has been tested and could be released to customers if desired. What we work on has been determined the most important to our customers, so this is a good thing. The best thing of all? The fact that we make sure that every deliverable, minimal useful functionality, really is what the customer wants because they have been more involved more frequently. So does it really matter if we try to fix scope, resources and schedule but having such as strict change control process if in the end we don't deliver what is really wanted?

So, what I have agreed on with the stakeholders is that we will go out beyond a release but strictly from a planning and goals perspective. At the point where it is determined that the goals are not obtainable in the process, I will make sure that the stakeholders are notified and can adjust the scope or define new goals based on this information. What's interesting is that I will never have to have that kind of conversation because the stakeholders will be involved in every part of the process. They will see progress being made, they will set priorities of work, they will attend demos and make changes, and they will see the impact of those changes. In the end, they will learn that they can draw the "line in the sand" when they feel most appropriate. Based on the confidence that they are gaining with the teams, they will change the long-term discussion with the customer to what is being developed than what is being planned. When that isn't enough, they can give the customer an idea of the priority of work being considered. If the customer feels that the priority isn't meeting their needs, they can voice their opinion. Then the stakeholders can determine what to do with that feedback and make priority changes as necessary. As long as customers see steady progress with frequent releases that are meeting needs, they should be satisfied with this approach.