Software Development Processes: Less is More

Following a software development process is good for your team. You might have heard the famous saying: "Any process is better than no process". On the other hand, have you thought about what's disproportionately more important than any process? A good team. Give me a team of capable individuals and I can't care less about the process.

This is precisely why I love the very first line of the Agile Manifesto and I'm sure it's not there at the top by coincidence.

Individuals and interactions over processes and tools

I believe a software development process should be as simple as possible. It should for example, fit on a single A4 page, or a team member should be able to explain their entire development process in an elevator pitch. When it comes to processes, I believe in "less is more" – because a process is just a tool to empower individuals and their interactions, nothing more.

While simple & lean processes can be empowering for individuals and interactions, excessive processes do the opposite. Having too much process very often creates more problems than it claims to solve.

The Problem with Grandiosity

More often in the enterprise, process oriented mindset is attractive to the management. That's because managers like to have a sense of control and order. Frankly, you can't blame them for it, can you? After all, in traditional organizations, they're the ones who are held accountable when everything is said and done. Well, that's another problem on its own but it sets the stage for the next scene.

For every problem that has been (or potentially can be) encountered; some process, policy or a rule is made. Defense mechanisms are created. Decisions are centralized. In his talk, Architecture Without an End State, Michael Nygard calls this phenomenon "grandiosity". Grandiose, big scale policies and processes (and architectures) have the problem of quickly getting out of context and out of touch with reality. This creates bottlenecks and confusion within the teams where the work is actually happening.

Turpentine

While I was gazing into the Twitterverse, I saw an interesting tweet. When Brianna Wolfson was leaving Stripe, she tweeted about her reflections on what made working at Stripe feel different than working at other places. She named the first item as "Turpentine". The quote from Picasso is hilarious.

The thread goes on with an example of a gemba walk. Let me give you another example that I think fits nicely to this "turpentine" analogy.

If you read the Agile Manifesto and its 12 principles (go read it, they're all very short), you'll quickly realise that it's a result of a grassroots movement. It's created by developers who are actually carrying out the work - by building stuff, solving problems and shipping software.

One of the main themes of Agile is allowing people who are actually carrying out the work to make the decisions about their work. This happens in self organizing teams. Not by managers in high level management meetings, not by fixed attendees of weekly guild meetings. Not by representatives of any sort. No high level discussions or assumptions. Decisions should be made by the people who are on the ground, doing the actual work at one-inch altitude. Why? Because those people know the most about the topic at hand and the challenges they face. Period.

Building a culture where people who know their craft are encouraged to get together on an ad-hoc basis and are facilitated to make informed decisions is key to a company's long term success. No gates or formalities. No "first you should talk to my manager or scrum master" nonsense.

Illusion of Control & Misunderstanding of Software Development

Another problem is related to the nature of how software is built. Software development is not merely about moving work from "To Do" to "Done", but it is about overcoming complexity. Rather than man-days, you need passion, talent and creativity.

Process oriented mindset fails to grasp this true nature and tries to construct a set of predefined steps or strict workflows to create repeatable success while maintaining control along the way - an illusion of control that is.

This shows a clear misunderstanding of how quality software is built. It shows a misunderstanding of the job of the programmer and denies the existence of an engineering culture. Maybe we can take lessons from Boeing's recent disaster and Hertz' problematic relationship with Accenture.

Top Down vs Bottom Up

The essence of all the points discussed above comes down to creating a software development culture that's "bottom up" rather than "top down". This is true for the ability to be "Agile", to be able to create good "Software Architectures" as well as practicing "DevOps" in an effective way. Excellence in these areas can only be achieved by the side effect of this freedom (and desire) of doing things bottom up. Obviously, there has to be capable people on board who can drive these grassroots movements, in fact that's the whole point.

In his remarkable lecture, How Does Tech Excellence Look Like?, Martin Fowler talks about an analogy to describe the software development process in a business environment: "A vehicle that needs to make a long journey". What's important in such a journey is that you need to keep your vehicle healthy and you need to know where you're going. In the software development context, the team and the codebase represents the vehicle. And what business provides is the direction of travel. So the way I see it is that for the direction and vision it probably makes sense to be top-down from the business. But everything related to the team and the codebase should be bottom up. So harmonizing these two forces using short cycles is the key.

Scaling Agile? Beware Grandiosity

Scaling Agile? Sounds like an oxymoron. I've always watched this space with a little bit of pessimism. Here's what Mary Poppendieck has to say about it in "An Interview about Lean and Agile":

There's lots of talk now around scaled Agile frameworks such as SAFe, Nexus, LESS, etc. with mixed results. How do you approach the challenge of scaling this way of working?

Every large agile framework that I know of is an excuse to avoid the difficult and challenging work of sorting out the organization’s system architecture so that small agile teams can work independently. You do not create smart, innovative teams by adding more process, you create them by breaking dependencies.

What we have learned from the Internet and from the Cloud is very simple – really serious scale can only happen when small teams independently leverage local intelligence and creativity. Companies that think scaled agile processes will help them scale will discover that these processes are not the right path to truly serious scale.

The End

I sat down to write about my observations & experiences in Scrum but went completely off track. I guess I had a few things to say about software development processes and autonomy in general. So now that this one is out of the way, I'll get on with the next post about Scrum.

Thanks for reading.

~Hakan