How to Cultivate a Great Engineering Process

Because process is so central to everything we do, having a good process is one of the highest-leverage ways to help a team perform at their best. Having a bad process, on the other hand, is one of the easiest ways to grind a team to a halt, destroying morale in the process.

As an engineering leader, learning to cultivate a good engineering process is one of the most important ways you can invest in your team’s skills.

Cultivating vs. installing a process

Even if you’re in your very first leadership role, you’ve likely seen a few different ways to organize a team’s work over the course of your career. You may have read books, attended courses, even gotten certifications in different software engineering methodologies. You might be tempted to take one of the methodologies you’ve seen work well in the past and install it wholesale into your current team. But this is almost certainly a bad idea.

Every team is unique along multiple axes: the individuals that comprise the team, the product the team is building, the technology stack they’re using, even the company they’re part of. This uniqueness means that no two teams have identical work patterns or process needs. What’s more, process needs are constantly evolving as individuals come and go and projects begin and end.

You might be able to install an off-the-shelf methodology and get decent results, but the best engineering leaders think of themselves as process cultivators. They help the teams they support to build a process that fits them, drawing from the methodologies they’ve used or learned about in the past like a toolbox.

Getting started

The good news is that you can start cultivating a better process for your team from wherever you are in the development cycle. There are a few discrete steps you should take:

1. Observe the existing process: Whether you’re brand new to the team or have been leading them for years, the first step is to observe how work gets done. Even if the team follows an off-the-shelf methodology like Scrum or SAFe, there’s a pretty good chance that they’ve put their own spin on it in subtle ways, and these variances are important to understand. Avoid the urge to introduce changes until you really understand what’s going on./>

2. Write it down: Next, write down the critical steps in the current process. You’re looking for where work enters the process, how it gets broken down, how it’s assigned to individual engineers, how it gets validated, and how it gets released. You also need to understand the differences between how new feature work, bug remediation, and engineering-driven work (like observability or tech debt reduction) are handled within the process. Once you’ve got a rough version of this documented, share it with the team to see what you might’ve missed.

3. Find the friction: I’ve listed this as a discrete step, but by the time you’ve observed and documented the current process, you’ve likely got a pretty good idea of where it obviously isn’t supporting the team well. It’s worth running these areas past an engineer or two on the team to see if they agree or have other areas to point out.

4. Experiment: This is where you get to dig into the toolbox of process components you’ve accumulated over your career and find a change that you think will help address one of the points of friction. Instead of telling the team you’re making a change, propose it as an experiment and get their buy-in. You want them to start taking ownership over the process, and framing changes as experiments makes proposing their own changes much lower risk. It’s also important to only change one piece at a time so you can clearly see the result.

5. Repeat: If the experiment is a success, celebrate and keep it! If not, celebrate what you’ve learned and roll it back. Either way, figure out your next experiment and implement it.

A retrospective culture supports continuous improvement

In order for the team to really own the process and feel empowered to improve it on an ongoing basis, you need to give them a mechanism to raise problems and propose changes. The most natural way to do this is by fostering a highly functional retrospective culture. Specifically, your team needs retros that are:

1. On a regular cadence (ideally at the end of each engineering cycle/iteration/sprint)

2. Full of honest and candid sharing (both positive and constructive)

3. Complete with feedback (actually feeding back into the engineering process with action items that are consistently acted on) Honest and candid sharing is by far the most challenging part of this. Having retros on a regular cadence and making sure the feedback is consistently acted on will encourage the team to do the hard work of sharing because they’ll be confident it will result in change. You can also try a retro format with an anonymous component to give your team an even safer way to share their frustrations.

Many engineers have lived their careers having bad engineering processes imposed upon them with very little room to change anything, so it may take some time to help your team feel safe and confident enough to really own their process. You may have to seed changes at first, but when they really take ownership, you’ll be amazed at the great solutions your team will come up with that you might never have considered.

Responding to common objections As you propose and begin rolling out this thinking with your team, you may get pushback from your leadership or the engineers on your team. Here are the three most common objections I’ve come across, and my suggestions for how to respond:

1. ‘But we have a company-wide process we have to follow.’

First, a side note: senior leaders, please stop doing this! Imposing a mandatory company-wide process is operating at the wrong level of abstraction. You should be defining the interfaces you need for input to and feedback from your teams and then encouraging your teams to come up with the processes that fit their needs as well as the company’s. If you don’t trust your leaders to do this, you should be investing more in training them.

But, if you’re a manager in this situation, there are always parts of the process where you still have some flexibility. Figure out where those parts are, and see how much improvement you can find in them. If your team is feeling friction from a mandatory part of the process, work with your management chain to understand why it’s important. If you think the team can fulfill that need while also reducing friction, propose it to your manager as a time-bound experiment (reducing their risk in letting you try it). If not, helping your team to understand why it’s important will usually mitigate the pain.

1. ‘But [Methodology X] is a best practice validated across hundreds of teams and we should just follow it.’

While there’s lots of value in formal Agile methodologies and the research that went into them, they’re also something of a lowest common denominator. There will be practices in them that are helpful, but also practices that at best you don’t need and at worst are actively harmful to your team. They’re designed to work in the broadest range of possible scenarios, when actually what you need is a process that works well for your specific, unique scenario. Cultivation is how you get there.

1. ‘But I’m starting a new team and don’t have anywhere to iterate from.’

While your new team may not be following a process yet, chances are everyone on the team has opinions and preferences about the process. Take a little time to understand those preferences and then help the team put together a barebones process that will give you a base to start getting work done and to iterate from. As long as you’ve got a regular planning meeting, periodic coordination on work (like a standup), and a way to make changes to the process (like a retro), you’ve given yourself a good start.

Reflections

The goal with any process is for the team to feel supported by it in doing their work rather than doing work just to support the process. While this path is less straightforward than just grabbing a methodology off the shelf and running with it, over time you’ll give yourself and your team the precious gift of a process that fits them perfectly. They’ll also have the tools and confidence to change the process when they do find points of friction.

Giving your team this level of agency and control over their process is one of the biggest things you can do to improve both overall efficiency and happiness, so it’s absolutely worth whatever work it takes to get there.