Over-commitment

When I wrote the “Long time, no see” post in December I was planning to resume writing regularly (at least bi-weekly) but since then I’ve only managed one. I thought I would have some spare time that could be put towards it but other things got in the way.

Does that sound familiar? How many teams start a sprint having planned to complete a number of things, only for that plan to go awry? If your team hasn’t experienced it yet, then it’s just a matter of time before you do… or maybe you’re playing it safe and you have a second planning session mid-sprint.

I’m not saying this is a bad thing – stuff happens and not adapting would be worse. Sticking blindly to a plan and ignoring reality sounds ridiculous and yet many organisations still do precisely this.

So how can we reduce the chances of our plans being impacted? Plan smaller steps; discuss risks and dependencies as part of planning; understand your capacity; and say “no”, or at least “not right now”.

Plan smaller steps

If you plan once a year (e.g. an annual budget cycle) then try moving to quarterly planning. If you create quarterly roadmaps that describe what needs to be delivered, then break it down into smaller feedback loops. If you’re a scrum team that plans four-week sprints but finds yourself replanning during sprints, then try two-week sprints. Whatever your cadence, if your plans often need updating then try a shorter timeline. It’s also possible that your plans are too detailed or rigid; there needs to be room to adapt based on learning, discovery and feedback. This is also why there are multiple “planning layers” – the team meet (at least) once a day to coordinate their efforts, share new insights, identify obstacles and make adjustments as needed. The shorter these feedback loops are, the sooner the team can respond.

Discuss risks and dependencies as part of planning

Sprint planning isn’t simply for the team to decide what “fits” – there should be (or have been in preparation for planning) discussions about the risks, assumptions, issues and dependencies. The somewhat-controversial Definition Of Ready is a good place to keep a list of reminders of the types of things to consider, e.g. if it takes a while to get access to a particular system, then make sure the team knows to start that process ASAP (in extreme cases, it may mean postponing a story until the access is granted). What else might throw the plan off course? It’s not possible to predict and anticipate everything but the team should learn from past “gotchas” and try to avoid repetition.

Understand your capacity

Even if everything goes smoothly, there’s no point in planning for more than could be realistically achieved. If a team uses story points consistently, then they probably have a good guide as to their capacity. Even without putting a number on it, stable teams should have a good feel for what is possible. The trouble with both those approaches is that teams can believe (or be cajoled into believing) that they can do more “just this once”. Forecasting (e.g. using cycle time) is based on the reality of historic data and is less open to “interpretation” or wishful thinking.

Say “no”, or at least “not right now”

The previous steps can help improve the likelihood of success, but one other defining factor is whether the team feels they can say “no” to taking “just one more thing”. If they feel they have to say yes to everything then there is no true commitment. (I know commitment is a controversial term; I’m using it as a shorthand for what the team tell those outside the team what they believe is achievable.)

In short: don’t make promises, because things change. Small steps with frequent feedback let your customer/sponsor see progress (which should lead to increased confidence) and collectively you can make adjustments based on new information.

I’m not committing (out loud or even just in my head) to updating this blog every two weeks, but I will try to write when I have time and have something to say. 🙂

Describe Your Ideal Development Process

All too often discussions about how to improve a team’s/company’s software development process are met with “It won’t work here”, so I find it often helps to discuss the ideal process and then look at the next step(s) that could be taken to move closer to it. Yes, you might not get there in its entirety, but that’s why it’s an ideal process and we’re just trying to move closer to it.

This video by Dave Farley is a presentation of what an ideal process might be, so I’m going to step through it and expand on some of his points. Feel free to disagree with what that ideal looks like, but for now, suspend any thoughts of “it won’t work here” and just focus on what you would like it to be.

The Goal of Software Development

Solve Problems for Others, with Software
It’s hard to disagree with this premise:
- if you’re not solving a problem, then why are you building something? Now it could be that you’re building something truly revolutionary and your target audience doesn’t know they have a problem yet, but someone has identified an opportunity in the market. Maybe we need a broader understanding of what “a problem” is but if we’re not building a solution then we should probably stop.
- Dave mentioned that you’re being paid to do it, so it must be “for others” unless you have some convoluted cross-billing system for your personal finances! (Even if I’m building something for myself I tend to use a simplified version of this process but let’s focus on professional software development organisations for this post.)
- I think “with software” is a reasonable assumption given we’re talking about a software development process but I will just add that one of my favourite solutions to a customer problem was when the team laminated some instructions rather than build a new help tool.

Understand the Problem
Dave mentions requirements. I find teams often misunderstand what this word means. A “requirement” is just a description of the problem… and the description could be in the form of diagrams and conversations. It does not have to be specified in great detail, written down and signed off – in fact, that’s exactly what we don’t want!
Solve the Problem
Write some code. I don’t think this needs an explanation.
Confirm the Problem is Solved
I’ve encountered disagreement when discussing the need for writing tests but (remember this is an ideal process we’re discussing) everyone wants tests to exist. Rapid, reliable, repeatable tests. There was unanimous agreement that flaky tests are worse than not having them.

I agree with Dave’s summary that the Ideal Dev Process results in Great Quality with Minimal Work, but this is where I’ve seen people get confused by “minimal work”. Often they take it to mean that they need to know everything before they start so that they only do exactly what’s needed, but (the video will get to this in a minute) we cannot know everything upfront – sometimes the customer doesn’t know exactly what they want (or need), but even when they do it’s hard to convey that precisely. But let’s assume the dev team perfectly understand the goal; there’s more than one way to reach that goal so we may need to try some alternatives before we pick one. Oh, and of course, everyone writes perfect bug-free code the first time. So let’s agree that “minimal work” means “based on what we know at the time we’re doing it” which leads nicely into Dave’s discussion about continuous learning during the process.

For those reasons (and more) we should be discovering new information throughout the development process. We could be learning more about what the problem is, how to solve it, or how we work together to solve it. Any new information should result in the team deciding how to incorporate it; that may result in changing work which was previously considered finished, but what’s the alternative? If your customer initially said they wanted the screen to be blue but now they want it to be green, would ignoring that result in delivering Great Quality? I won’t get into the practicalities here but on my backlog of blog topics, I have the sunk cost fallacy and what to do if your customer keeps changing their mind.

The ability to incorporate new information requires the willingness and ability to change our minds and change our software.

How do we make it “Easy to Spot the Need for Change”? Frequent feedback from the right people. We should be asking our customers (and end-users) for feedback on whether we’re solving the problem; sometimes that leads them to realise the problem they thought they had isn’t the one we need to address first. We should also be asking the people involved in creating the solution (the dev team but also the people we are working with who are outside the team) for feedback on how we’re collaborating. In both cases that means taking small steps.

We also need to make it “Easy & Safe to Make Changes” – we need safety nets to make it easier to spot when we’ve made a mistake and to fix it. Those safety nets can consist of tools, tests, processes and procedures; together they help developers move with more confidence.

Dave goes on to explain why setting up a deployment pipeline is essential, and I completely agree with him – if you build a system by hand then you run the risk of making mistakes, leading to inconsistencies from build to build. This is as bad as having flaky tests – you cannot rely on the results, so no matter how good your code is you cannot be confident in what you deliver to your customer. Automating the integration process also reduces the effort it takes, which means you can build more often, which in turn means you can give your users a new release more often and thus get feedback more frequently.

Part of the safety net needed to bolster developers’ confidence when making changes is another feedback loop: does your new code work with the rest of the system? There needs to be a way to let the team know if a change has broken the system, and then the team need to make it their priority to fix it. If the system is broken, why would you continue making changes before you fix it? You won’t know if your new changes work until you test them and you can’t test without a working system, so all you’re doing is postponing that knowledge, which in turn means you may need to backtrack further to fix your code – that doesn’t meet our aim of Minimal Work.

So the principles of our ideal development process are:

minimal work (minimize unnecessary work)
great quality
ability to change our minds
make changes in small steps
regularly verify progress (frequently seek feedback)
work incrementally (incorporate feedback)
creates working, usable system
keeps the system working

The video goes into more detail about Continuous Integration and Continuous Deployment, but I think that if we can agree on these principles of an ideal development process then CI/CD is a given.

I read some of the comments under the video on YouTube and a major thread was around estimation. Given that the focus of Dave’s presentation was on the development process, I think estimation is out of scope (you can develop great quality software without estimation) but it probably deserves similar treatment, i.e. thinking through what an ideal product/project management process might look like. Estimation is a possible solution depending on the problem, but often it’s assumed that it’s needed instead of asking “Why, what will the estimates be used for? What problem are you facing for which you think estimation is the solution?” Often it’s considered necessary because someone wants to know “when will we be done?” but that should be a question to the customer as part of that frequent feedback loop – “are we getting closer to what you want?”. If we don’t know where the finish line is, how can we hope to say when we’ll reach it? Instead, focus on making small incremental steps and also asking whether taking another step down that path is the most valuable thing to be doing – sometimes we’ve gone far enough that we should park this work and move on to something else.

—
I linked to it a couple of times in my post, but in case it’s not clear the video I’m referencing is “The Most Powerful Software Development Process Is The Easiest” by Dave Farley of Continuous Delivery.

Start With The End In Mind

We have completed another lap around the sun, and that’s often a time to think about our plans for the new year. In my previous post, I talked about New Year’s Resolutions and thinking in smaller steps rather than a goal that might take 12 months to complete. But if you only think in small steps and focus on where you want to be in a few days or weeks, then it’s often hard to know if you’re going in the right direction – there can be such a big gap between where you are and where you want to be.

If you were planning on travelling from Toronto to Montreal then you need to plan further ahead than just “which street do I take now?” You might think that was a bad example as the 401 covers the vast majority of the distance, but that assumes you’re optimising for time and that your method of travel is by road, i.e. you have already ruled out many alternatives. Take a step back and consider your goals: if minimising travel time is the primary aim, then flying is significantly faster. If you want the freedom to move around and have a meal without it impacting your travel time, then the train could be your best option.

But let’s go back to driving between the two cities, and assume we have picked the route we would like to take. We set off but before long we discover there’s a problem ahead: an accident is causing a long delay. If we had only been inspecting our progress over small steps, then we might not see the impact of the accident until we’re sitting in the traffic jam and it’s too late to do anything else. We would see a similar outcome if we set the route and followed it regardless of what we discovered along the way. One of the reasons we chose to drive is to have flexibility – we can’t easily change our route if we chose to fly or take the train.

Choosing flexibility (aka agility) may not be the cheapest, fastest or most comfortable path, but it gives us options when new information (whether that’s delays or shortcuts) is available … so we should ensure that we take advantage of that discovery, otherwise we probably shouldn’t have picked this approach!

The plans we create should provide clarity on where we want to be in the future (and why!) but we also should revise them when we see a better alternative. Sometimes it’s a detour that still gets us to our destination but in a better (faster / cheaper / safer) way, but sometimes the discovery is that our goal needs to change – maybe the reason we were heading to Montreal can also be achieved in Ottawa, saving about an hour of driving. If we don’t respond to that new information, then why didn’t we just get on the train, put our feet up, and just enjoy being passengers?

Incidentally, this is why we call some plans “roadmaps” – their purpose is to show potential alternative routes and to allow for changing course while still heading towards the destination. If our plan has only one path from A to B, then it’s not a roadmap. If the plan says “build X” rather than “solve problem Y”, then it’s not a roadmap.

I’m not saying you always have to drive (i.e. use an agile approach) – there are times when the train makes sense, for example when I travel to Montreal for a concert I know the destination isn’t going to change and I would rather get there with minimal stress. In the same way, if we know exactly what we need to build and don’t expect any changes along the way, then we could create a plan, write detailed specifications, and do all the design upfront … but, in our industry, how often can we go from A to B with no learning, no discoveries, no need to adapt?

So: start with the end in mind (understand where you want to go and why); consider what your constraints are (e.g. time) and how you expect to react to new information (do you want to accommodate discoveries or follow the plan); then choose your vehicle and route accordingly.

*Yes, I know the second habit that Stephen Covey describes in “The 7 Habits of Highly Effective People” is “Begin…” but he has a registered trademark for that phrase so I’m avoiding it, and anyway I’m not using the phrase in the way he does.

New Year’s Resolutions

I’m sure most people will be happy to see the end of 2021 but also trying not to build up too much expectation for 2022. There won’t be a sudden end to COVID – it’s going to take a continued effort from everyone. It won’t go away just because we’re all tired of dealing with it. Stay strong; hang in there; support each other; and keep safe, please.

The New Year is a time when many people set Resolutions – things they want to improve, e.g. eating better, getting more exercise. It’s a handy reminder to review how happy we are with various things, but is doing it once a year really enough? It often leads to huge goals because it’s going to be so long until the new review… and, of course, huge goals aren’t usually achievable. That’s why most resolutions are abandoned within a few weeks. It probably helps to have a vision of how you would like things to be in a few months (even a year), but set smaller goals – what are the steps that you think will take you towards that vision?

Hopefully that sounds familiar: regular retrospection, identifying where you want to improve, deciding on small steps to go in that direction; take a step, then go back to retrospection, rinse and repeat.

Engineers and product owners should be well practised in this – retrospectives to look at how we work, and experiments to probe what to build. So how do we convince HR that annual appraisals should be replaced with frequent feedback, and Finance that regular (e.g. monthly) planning trumps attempting to predict six or even twelve months ahead? That’s a challenge I’d like to add to my backlog for next year. Suggestions are welcome!

p.s.
2014: Didn’t jog
2015: Didn’t jog
2016: Didn’t jog
2017: Didn’t jog
2018: Didn’t jog
2019: Didn’t jog
2020: Didn’t jog
2021: Still haven’t jogged
This is a running joke! 🙂