Breaking Scrum: Scope changes within the sprint...

A discussion I started on Linked:

Breaking Scrum: Scope changes within the sprint...
Hi,
Looking a "list of reasons" on why product-owner-pushed scope change during the sprint breaks scrum...

Few I can think of (please add):
- Takes away the focus from the current functionality
- Reduces team ownership
- Impacts Quality
- Impacts Velocity
- .......

Thanks!
Rahul

3 months ago

Prasanna Raghavendra • The basic essense of agile to me is getting joy in the journey (see here: http://2srp.blogspot.com/2010/06/joy-of-journey.html). One should be successful in the small steps to make sure there is continuous passion and teaming, If this fails, everything breaks!

Mike Caddell • A scope change during a sprint - which is a contradiction in terms - invalidates the fundamental agreement and commitment that bounds a Scrum iteration. The team commits to complete a set of functionality as prioritized by the Product Owner and agreed to by the team. The Product Owner in turn commits to helping define and elaborate the agreed on scope in a timely manner to enable the team to do it's work; and only the agreed on scope, nothing else unless the team finishes early and then picks the next highest priority Release Backlog item.

If the team is asked/required to accomplish a different scope of work during the sprint, the mutual commitment to the previously agreed to scope is thus invalidated. At this point, the sprint should be terminated and a new sprint with the new scope begun - starting with Backlog Grooming and Sprint Planning.

Julie Hendry • I agree Mike, although it often depends on what is meant by 'scope change' - is is the overall deliverable or is it simply that we discover more detail?

I've seen successful teams accept some changes within scrum as long as the changes are agree and communicated, and do not change the underlying goal of that sprint.

Rahul Sawhney • Thanks all for comments.
@Julie, To clarify I mean "goal changing" and "deliverable changing" sort of changes. I do not mean discovery of some more details.

Mike Caddell • @Julie - a great clarification!

I meant as Rahul indicates, a change in the objective or the actual content.

As this discussion clearly illustrates conversation almost always illuminates more detail - which is why we value Individuals and Interactions more than Process and Tools!

Derek Neighbors • Is the reason why they do? or the reason why they shouldn't be allowed to?

The reason they do... the team allows them. The don't enforce some penalty in breaking the contract.

The reason they shouldn't be allowed to. A key premise of agile resolves around trust. Changing course mid sprint breaks trust and deteriorates the relationship.

Prasanna Raghavendra • I am seeing this discussion going similar to building a stringent chinese wall between the product owners and the team and closing doors on genuine yet possible changes within iterations. Like I mentioned earlier (brewity killed clarity in my earlier note, I guess!), the focus should be on small but quick win at the end every iteration for the entire team (including product owners). That would mean there should be constant communication and acceptability of what can be done for the changes coming-in while appreciating concerns from both end. A "genuine" change coming in within an iteration should be discussed and accepted when possible while showcasing what it means for the dev teams to pick it up. This is when teaming and binding becomes stronger. Product owners should also have strong reasons why the change envisaged cannot wait until the next iteration.

I would discourage a generic and rigid process coming in the way of teaming. Agile is not a process but it is a set of natural tendancies teams follow to bond, and that should take precendence. it is interesting to note that we are losing our this natural behaviour and find it hard to let those take precendence.

Rahul Sawhney • @Prassana - I agree with you completely.

Just to bring the discussion back towards my question, refining it a bit....
What are the after-effects (short term and long term) of having the "sprint-goal changing" scope changes, which would take the focus away from the team to deliver what they committed?

So far, we have seen some points indicated above, would like to learn more perspectives, experiences when this happened etc..

Jeff Smathers, CSM • Rahul, If I understand your question, I think that Derek and Mike have great answers. I will try to summarize for my own benefit. If the scope change is so large it changes the sprint goal then the team cannot meet its commitments and the sprint is no longer valid and should be ended. A new sprint needs to be planned and started to address the changes. Depending on the maturity of the team, the trust between members and the quality of their relationships could suffer. Keeping to the commitments defined by the sprint goal is very important.

Jason Little • I wouldn't worry so much about the after-effects as I would in trying to figure out why the scope keeps changing every sprint.

Some context would be helpful. Is this team supporting an existing product? Are the interruptions due to customer complaints or support issues? If so, Scrum doesn't work well in that situation.

Is the PO having problems grooming the backlog to get stuff ready for the sprint? Is there a lack of overall product direction? Are different stakeholders pushing conflicting priorities?

A sprint being constantly interrupted is often the symptom of a deeper problem. Focusing on Scrum as dogma isn't going to solve the problem and you'll end up creating more dysfunction and trust issues between the team and the PO.

John Clifford • In Scrum, the team's commitment to accomplish the items on the sprint backlog that starts the sprint is matched by the Product Owner's commitment to observe and respect that commitment by not trying to change the agreed-upon scope for the duration of the sprint. One can't work without the other. I would argue that a Product Owner who continually wants to change scope during the sprint isn't following the Scrum process and doesn't respect the team or their commitment. In my experience, this happens because the Product Owner fails to take ownership of the product backlog; either he procrastinates on prioritization until the last moment and then overlooks stakeholder commitments that drive prioritization needs, and/or fails to take responsibility for ensuring backlog items are groomed and prepared prior to sprint planning.

Not having objective, quantitative, and binary acceptance criteria for sprint backlog items is one of the problems leading to scope change; if the team has no defined criteria for the complete story then what is to stop the Product Owner from whipsawing them around? Therefore, the Product Owner should ensure that sufficiently-defined acceptance criteria exists for all stories that she will bring to the sprint planning meeting... and the team should refuse to consider any item lacking this. This isn't to say that the team and the Product Owner should be at each others' throats, but instead to say the team may need to spend some time during the sprint planning meeting working with the Product Owner to create sufficiently-defined acceptance criteria.

(Product backlog items with valid acceptance criteria and that have been estimated are often referred to as 'Ready-Ready', in the way that sprint backlog items that are completed according to the Definition of Done and all acceptance criteria are referred to as 'Done-Done-Done-Done.' Effective Product Owners always have at least one sprint's worth of backlog items in a 'Ready-Ready' state before the start of a sprint planning meeting.)

I'm a huge believer in not hiding dysfunction, because you can't fix it if you refuse to see it. In the above situation, the sprint planning meeting timebox will most likely end with only one or two items that have valid acceptance criteria; so be it. The Product Owner will need to accept that bandwidth may need to be reserved during the sprint to work on additional items' acceptance criteria in order to be ready for the next sprint; if the team finishes with the committed items early, they can either drag additional items into the sprint or end the sprint early and re-plan/re-launch. At any rate, this should be discussed at the sprint retrospective and a solution devised that will prevent this from happening again should be implemented.

As others have mentioned, if the team believes that an item's scope has changed to the extent that the original commitment is invalidated, then they have the authority to reject the changes and not complete the item... and this should be discussed at the retrospective. Or, if the Product Owner really wants to change scope, e.g., substitute a new sprint backlog item for an existing one, even if the item estimates are identical, then the team has the authority to accept or reject the change, to the extent of cancelling the sprint and re-planning. Again, to expose dysfunction, I counsel against accepting ANY sprint backlog changes, not to be a jerk or uncooperative, but in order to force the issue to the surface... Product Owners who continually fail to prioritize and want another bite at the apple, for whatever reason, are a common issue.

Andy Dent • A few thoughts on this, against a background of multiple scrum teams collaborating internationally, spanning 24 hour time differences. We have architectural separation of much of the work but that translates into one team having the other two as customers.

We could take the bloody-minded approach of just saying "no" but that risks some bugs being blockers to another team for an entire sprint.

An alternative is for the team to identify one or two people as responders to handle urgent cross-team requests. They can have a task allocated in the sprint which is basically a budget of their time, to be whittled down over the course of the sprint and leaving them free to do other tasks of moderate size.

When it comes to tasks being inserted into the sprint, another pet idea of mine is to mandate that tasks with twice the number of story points have to be removed.

This reflects the loss of velocity from something unplanned being included and emphasises the seriousness of the interruption to the product owner. I'd love to know if anyone applies this, how they get on!

Raul Xavier Neto • I´ve got a little intrigued with that many arguments being so dogmatic about scrum. I think that one of the best points of agile (in this case, scrum) is to help teams cope with change. Change that is a big reality on every software project.

I´ll go with Jason and Prasanna telling you that the best thing for your team is to find out why the owner is changing that much and try to bring him to the team, where those changes should be agreed by the team with good discussion.

Speaking of dogmas, maybe the initial phase is too dark in the owner´s head, maybe the team is not sure about the concept or something else. To ease that kind of thing, that usually comes in the beginning of the project, you can try to reduce the sprint duration, bring the owner closer to the team, try different approaches to put the team as one peace. Scrum is not a Religion, it only guides teams to achieve good results, but the principles are the important thing, not the "rules". If you have to, adjust the "rules" to your needs.

Prasanna Raghavendra • @ Raul, @Jason: Bingo!

@Rahul: It is less about worrying about after effects than accepting it as another pragmatic anamoly and see why it occurs and how it can be fixed.

Rahul Sawhney • Thanks John - I enjoyed reading what you have written here.
Thanks to everyone else too - great insight!

Cuan Mulligan • @Rahul, the impact for me is one of cost.

for example, lets say the PO has aprpoval to spend $x on the delivery of a project. It is the role of the PO to ensure the best spend of the companies money.

If the PO keeps changing their mind, this will consume time (cost) and reduce the teams ability to deliver the project benifit for the agreed amount of money.

If the changes are small , ie refeinments, then this should not be an issue. However I have worked with PO, who make quite dramatic changes during a sprint, and then wonder why the project did not deliver. The upside was that they learnt a valuable lesson, and stopped this behaviour.

Gail E. Harris • The fundamental question in this discussion thread is about product-owner driven scope changes during a sprint breaking Scrum. Well, as someone else wrote, Agile approaches like Scrum are all about being agile, and, as it turns out, changing the scope during the sprint doesn't break Scrum at all.

In Ken Schwaber's book "Agile Project Management with Scrum" he describes exactly how to handle this. He says that "management could abnormally terminate the Sprint" at which point there would be a new Sprint planning meeting. Ken further notes that this new planning meeting would make the action highly visible and thus hopefully product owners would not make a habit of it.

It's also worthwhile to note that a new opportunity to change the scope is never more than four weeks away. Once they realize this many product owner's find that four weeks from now is more than soon enough.

Griffin Jones • We have a very small team, working in a start-up.
Priorities change.

When targets-of-opportunity appear, ownership will pause the sprint as we deal with the new higher priority. The owner makes the informed choice.

Long pauses force a clean re-plan of the sprint. Too much switching reduces velocity. But the owner values responsiveness from development, and is willing to accept reduced velocity in exchange.

Rahul Sawhney • @Cuan,Gail and Griffin, Thanks I was looking for exactly these inputs!

I do believe that this does not break Scrum as a framework. When Scrum is done incorrectly, (i.e. when PO makes significant changes without agreeing with the team and does not allow terminating or pausing the sprint, and does so regularly), the team gets demotivated and starts falling apart. And as Cuan mentioned, not learning from experience escalates costs and reduces team's ability to deliver and impacts collaboration.

Have an Initial Conversation to Start an Agile Project

This article was published on Scrum Alliance in Oct 2010

In this article, Prashant Patel and I present:
• The different aspects of this conversation - the parameters that we consider.
• Why these parameters are relevant in the context of Baker Hughes Inc.
• The impact of these parameters on solution delivery

Have an Initial Conversation to Start an Agile Project

The Baker Hughes Product Development Lifecycle starts with discovery and inception phases. The delivery teams start engaging with the customers during the inception stage. By the end of inception, business requirements are understood at a high level and stakeholders from different teams who will be involved in the project are identified. This is when discussions are held for selecting the appropriate solution delivery methodology.

The choice of methodology impacts how the different stakeholders engage with the project. We have seen that in order to improve the chances of success for any project, the solution delivery methodology and the level of commitment needed should be understood upfront by all stakeholders. To that end, it is important that there is representation from multiple stakeholders, including but not limited to business, development, testing, agile coach, project management and resource management.

The stakeholders should be aware of the agile methodology and scrum framework for having this conversation. It is beneficial if they understand the rigor agile brings with respect to solution delivery and how it is different from waterfall.

Parameters

As part of our methodology selection criteria, the team examines the following basic parameters. During this conversation, the parameters and their implications are discussed in detail and a rating is given to each sub-parameter. All parameters need to be rated to consolidate the results, but it is not necessary that all parameters need to be rated favorably as a criteria for going agile; however, parameters with lower scores are considered risks associated with project execution. It is more important for stakeholders to have the conversation, understand their constraints and make a conscious selection of the solution delivery methodology, than the actual number results from this assessment. At the end of inception, the team needs to agree on the solution methodology based on its own judgment.

These parameters are customized to work in our environment and can be discussed in any order of preference. Organizations are encouraged to devise their own criteria based on the guiding principles of agile and assign their own weights to individual parameters.

Collaboration with Business

Collaboration with customers is critical to successfully execute an agile project. That is why it has been explicitly called out in the Agile Manifesto (www.agile manifesto.org) – “Customer collaboration over contract negotiation”. As part of collaboration with business we examine the following:

• Availability of Business/Customer for clarifications
  Some customers prefer to go away after the initial discussions on project scope and requirements and be available only when the complete scope has been developed to check and provide feedback. This approach does not work very well with agile and Scrum teams.  Short feedback cycles of two to four weeks require that customers are engaged throughout the project lifecycle and are available to provide feedback at the end of each sprint. During the sprint planning, teams may identify certain open points, questions and reminders for discussion during the sprint execution. These may also need clarifications from the customer even while the sprint is in progress. In order to meet the sprint commitment, the stakeholders need to ensure that passing on these clarifications to the team does not require too much time.

• Flexibility towards scope reprioritization 
  Agile teams thrive when they can deliver what the customer needs. Their focus is to continue to deliver maximum business value in shortest possible time. This is only possible if the scope is revisited frequently and not cast in stone at the beginning of the project. As part of revisiting the scope, it is essential that the priorities for the remaining work are revisited as well.

• Identification of acceptance criteria
  The stakeholders discuss and agree that the team needs to be given fair visibility and understanding of the scope items at the beginning of every sprint so that they can make reasonable commitments on what can be delivered. For this reason, we encourage the stakeholders to identify the acceptance criteria for each scope item at the time of sprint planning.

Project Constraints

Projects may have multiple constraints that may need to be satisfied. Each organization, group or team may have their own set of constraints that may be evaluated during the conversation – These should be heavily customized depending on the project situation. Below are the criteria that we use at Baker Hughes:

• Outgoing and incoming dependencies
  Dependencies with external teams add to system complexity. Ensuring that the project progresses as required, requires coordinating multiple dependencies. At times, dependencies may impose constraints such as: force special schedule requirements for certain scope items, make a part of scope definition more rigid, and make certain resources available to the team only in a particular time window. A large number of dependencies require allocating a significant amount of effort towards synchronizing the work of different teams. In that case, stakeholders need to discuss how these dependencies would be managed in the context of agile.

• Documentation needsThe agile manifesto calls for “Working software over comprehensive documentation”. Stakeholders need to discuss and agree about the level of documentation that will be created. For agile projects, stakeholders should agree to create only that documentation which will have value and will be easily maintained. Regulatory aspects, which may be relevant to some projects, should be considered too.

• Resource allocation and availabilityFor agile projects, we encourage the team members to be allocated 100% to the project instead of allocating percentages. We also encourage them to retain resources for multiple versions of the product and we discourage teams from changing resources while a sprint is in progress. This helps sustain delivering to sprint level commitments and makes resources own the sprint results and a predictable team velocity.

• CollocationCollocation of team members improves intra-team communication and improves velocity. Multi-located teams can do agile with the aid of communication and collaboration tools.

• Build automationTeams are encouraged to put build automation in place as early as possible during development sprints. If teams do not automate their builds, a lot of time is spent in manual builds and continuous integration cannot be accomplished.

• Unit and System Test automationDeveloping in short cycles and sustaining good quality and high velocity requires that team should be able to test what it develops quickly and effectively. This becomes difficult with manual tests as system becomes more and more complex with each passing sprint and regular addition of new functionality. Creating a good set of automated tests helps in ensuring that system health is maintained even after multiple sprints without sacrificing speed of delivery.

• EmpowermentAn empowered team performs faster and better, and is more accountable for its results as compared to teams that have not been empowered to make certain decisions regarding their work.

• Product Owner empowermentDuring the conversation, a product owner is clearly identified and expectations from the role are discussed. For agile projects, it is important that the stakeholders empower the product owner to make quick decisions on the project scope and provide direction to the team about what needs to be done in the project. The product owner is also empowered to decide on the priorities of the items in the product backlog.

• Scrum Master empowermentA Scrum Master is also identified clearly to help the team in removing impediments. The stakeholders and the product owner should empower the Scrum Master to act in the interest of the team, represent the team and say “No” to disturbances.

• Team empowermentThe stakeholders empower the team to go into agile and Scrum with an understanding that they are empowered to provide fresh commitment at the beginning of each sprint. Since the details for many items in the scope may be discovered while the project is in project, initial estimates of the team are only based on the visibility of the team into the requirements at the beginning of the project. Stakeholders discuss and agree that these estimates may be revised later on in the project as the team gains more visibility on what needs to be done.

More of the same

To be able to deliver new functionality in short cycles, it helps if the team members are well aware of the business domain and technology.

• Team’s acquaintance with business domainThe stakeholders discuss about the team’s awareness of the business domain and if it has executed similar projects in the past. If business domain is not well understood by the team, the gaps are identified and team is coached during the project.

• Team acquaintance with TechnologyThe stakeholders also discuss the need for the team members’ acquaintance with the technology. At times the technology itself may be new or the team members may not be acquainted with advanced concepts that might be used in the project. In such cases, or where there are gaps, the team should be coached. This might involve planning for additional technical trainings for the team and/or on the job coaching by experienced technical staff.

Team characteristics

Fast- paced agile development requires giving importance to “Individual and Interactions over processes and tools“. It becomes crucial to assess if the team is well suited to perform in an agile setting.

• Team sizeThe stakeholders discuss about number of developers and testers that will be on the team. If there are too many people in the team, it impacts the team’s focus and activities (such as daily scrum meeting, making a sprint goal commitment) start taking longer time. This is attributed to the number of possible communication channels within the team, which increases substantially with team size. Based on our projects at Baker Hughes Inc., we have laid down the appropriate range from five to ten, with five to seven being the ideal range.

• Agile coaching and awarenessSince the organization is new to agile, it is recommended to have an agile coach associated with the team to guide the team members with agile and Scrum. In the context of Baker Hughes Inc., the delivery organization’s Program Management Office helps the teams with agile coaching.

• Number of sub-teamsThe stakeholders discuss the internal structure of the team. If the project is too big, sub-teams may be created to keep the size of each team below ten. As of now, none of the agile projects have had to have multiple sub-teams.

Project type and execution

These parameters, in addition to the ones specified earlier, help apply agile to situations where it will yield advantageous results for the organization compared to other methodologies. During the conversation stakeholders assess the project type and try to identify the benefits expected by using Scrum and agile in the project.

• Type of project and Need for changeIn our organization, we encourage use of agile for projects for significant enhancements, entirely new development and initial prototyping of a new concept. These projects are considered more suitable as they typically have an initial scope to start with, which can be refined over a period of time. Projects that are of production support (continuously changing scope – that cannot be controlled) and minor enhancements (having very low effort estimate) are not encouraged.
  
  The stakeholders also discuss the need for scope change in the projects and how agile will benefit the customer in such a situation. In cases where the scope is more or less understood and not much change is expected, the project could be executed in iterations. And in other situations, the scope can evolve as the project progresses making agile more suitable for the project.

• Project durationContrary to the perception of many people, agile works best when the duration of the project spans multiple months. Such duration provides the team an opportunity to inspect and adapt their ways of working. It also provides the customer an opportunity to get the most value through regular reviews of the functionality delivered.

• Mission criticalityThe stakeholders deliberate whether or not the requirements of the project have any tolerance for defects. In the ideal world, the team should deliver defect-free product at the end of each sprint. However, in the practical world of agile, the product evolves over time and there is always scope for perfecting the product over time. There are some exceptions though where there is no room for errors, in the delivered product – examples include life-critical healthcare applications, really expensive applications (like ones used in space programs) and so on. Having executed agile projects in such situations ourselves, we do not say that agile cannot or should not be applied in such situations. We say that in such situations, it is important to realize that quality needs to be given a prime importance and extra measures need to be put in place to validate the product before its Deployment.

• Resource availability for Sprint zeroIn our organization, we execute a sprint zero (we call it the formation sprint) for each agile project to build an initial product backlog and do the groundwork for executing the project such as plan number of iterations, set up the initial infrastructure and prepare a statement of work. To execute the sprint zero properly, it is necessary that the resources needed are made available. This means allocating key team members to the project and making the necessary hardware, software and tools available to them.

Conclusion

Agile and Scrum can be applied to vast variety of projects and situations. To execute projects well using agile, the need for collaboration and understanding between the customer/business and the delivery organization cannot be underscored. Basic preparations are needed in this regard and this conversation goes a long way in ensuring that happens.