All too often discussions about how to improve a team’s/company’s software development process are met with “It won’t work here”, so I find it often helps to discuss the ideal process and then look at the next step(s) that could be taken to move closer to it. Yes, you might not get there in its entirety, but that’s why it’s an ideal process and we’re just trying to move closer to it.

This video by Dave Farley is a presentation of what an ideal process might be, so I’m going to step through it and expand on some of his points. Feel free to disagree with what that ideal looks like, but for now, suspend any thoughts of “it won’t work here” and just focus on what you would like it to be.

The Goal of Software Development

• Solve Problems for Others, with Software
  It’s hard to disagree with this premise:
  
  • if you’re not solving a problem, then why are you building something? Now it could be that you’re building something truly revolutionary and your target audience doesn’t know they have a problem yet, but someone has identified an opportunity in the market. Maybe we need a broader understanding of what “a problem” is but if we’re not building a solution then we should probably stop.
  • Dave mentioned that you’re being paid to do it, so it must be “for others” unless you have some convoluted cross-billing system for your personal finances! (Even if I’m building something for myself I tend to use a simplified version of this process but let’s focus on professional software development organisations for this post.)
  • I think “with software” is a reasonable assumption given we’re talking about a software development process but I will just add that one of my favourite solutions to a customer problem was when the team laminated some instructions rather than build a new help tool.

1. Understand the Problem
   Dave mentions requirements. I find teams often misunderstand what this word means. A “requirement” is just a description of the problem… and the description could be in the form of diagrams and conversations. It does not have to be specified in great detail, written down and signed off – in fact, that’s exactly what we don’t want!
2. Solve the Problem
   Write some code. I don’t think this needs an explanation.
3. Confirm the Problem is Solved
   I’ve encountered disagreement when discussing the need for writing tests but (remember this is an ideal process we’re discussing) everyone wants tests to exist. Rapid, reliable, repeatable tests. There was unanimous agreement that flaky tests are worse than not having them.

I agree with Dave’s summary that the Ideal Dev Process results in Great Quality with Minimal Work, but this is where I’ve seen people get confused by “minimal work”. Often they take it to mean that they need to know everything before they start so that they only do exactly what’s needed, but (the video will get to this in a minute) we cannot know everything upfront – sometimes the customer doesn’t know exactly what they want (or need), but even when they do it’s hard to convey that precisely. But let’s assume the dev team perfectly understand the goal; there’s more than one way to reach that goal so we may need to try some alternatives before we pick one. Oh, and of course, everyone writes perfect bug-free code the first time. So let’s agree that “minimal work” means “based on what we know at the time we’re doing it” which leads nicely into Dave’s discussion about continuous learning during the process.

For those reasons (and more) we should be discovering new information throughout the development process. We could be learning more about what the problem is, how to solve it, or how we work together to solve it. Any new information should result in the team deciding how to incorporate it; that may result in changing work which was previously considered finished, but what’s the alternative? If your customer initially said they wanted the screen to be blue but now they want it to be green, would ignoring that result in delivering Great Quality? I won’t get into the practicalities here but on my backlog of blog topics, I have the sunk cost fallacy and what to do if your customer keeps changing their mind.

The ability to incorporate new information requires the willingness and ability to change our minds and change our software.

How do we make it “Easy to Spot the Need for Change”? Frequent feedback from the right people. We should be asking our customers (and end-users) for feedback on whether we’re solving the problem; sometimes that leads them to realise the problem they thought they had isn’t the one we need to address first. We should also be asking the people involved in creating the solution (the dev team but also the people we are working with who are outside the team) for feedback on how we’re collaborating. In both cases that means taking small steps.

We also need to make it “Easy & Safe to Make Changes” – we need safety nets to make it easier to spot when we’ve made a mistake and to fix it. Those safety nets can consist of tools, tests, processes and procedures; together they help developers move with more confidence.

Dave goes on to explain why setting up a deployment pipeline is essential, and I completely agree with him – if you build a system by hand then you run the risk of making mistakes, leading to inconsistencies from build to build. This is as bad as having flaky tests – you cannot rely on the results, so no matter how good your code is you cannot be confident in what you deliver to your customer. Automating the integration process also reduces the effort it takes, which means you can build more often, which in turn means you can give your users a new release more often and thus get feedback more frequently.

Part of the safety net needed to bolster developers’ confidence when making changes is another feedback loop: does your new code work with the rest of the system? There needs to be a way to let the team know if a change has broken the system, and then the team need to make it their priority to fix it. If the system is broken, why would you continue making changes before you fix it? You won’t know if your new changes work until you test them and you can’t test without a working system, so all you’re doing is postponing that knowledge, which in turn means you may need to backtrack further to fix your code – that doesn’t meet our aim of Minimal Work.

So the principles of our ideal development process are:

• minimal work (minimize unnecessary work)
• great quality
• ability to change our minds
• make changes in small steps
• regularly verify progress (frequently seek feedback)
• work incrementally (incorporate feedback)
• creates working, usable system
• keeps the system working

The video goes into more detail about Continuous Integration and Continuous Deployment, but I think that if we can agree on these principles of an ideal development process then CI/CD is a given.

I read some of the comments under the video on YouTube and a major thread was around estimation. Given that the focus of Dave’s presentation was on the development process, I think estimation is out of scope (you can develop great quality software without estimation) but it probably deserves similar treatment, i.e. thinking through what an ideal product/project management process might look like. Estimation is a possible solution depending on the problem, but often it’s assumed that it’s needed instead of asking “Why, what will the estimates be used for? What problem are you facing for which you think estimation is the solution?” Often it’s considered necessary because someone wants to know “when will we be done?” but that should be a question to the customer as part of that frequent feedback loop – “are we getting closer to what you want?”. If we don’t know where the finish line is, how can we hope to say when we’ll reach it? Instead, focus on making small incremental steps and also asking whether taking another step down that path is the most valuable thing to be doing – sometimes we’ve gone far enough that we should park this work and move on to something else.

—
I linked to it a couple of times in my post, but in case it’s not clear the video I’m referencing is “The Most Powerful Software Development Process Is The Easiest” by Dave Farley of Continuous Delivery.