Facilitating Software Architecture with Andrew Harmel-Law

Transcript

Thomas Betts: Hello and welcome to another episode of The InfoQ Podcast. Today I’m joined by Andrew Harmel-Law. He’s a tech principal at ThoughtWorks, he’s a trainer with O’Reilly, and most recently he’s the author of Facilitating Software Architecture. Andrew was on the podcast a couple years ago talking about the architecture advice process, which is a core component of his book, so we’ll get to that a little bit later. But Andrew, welcome back to the podcast.

Andrew Harmel-Law: Yes. Thanks for having me, Thomas. It’s great to be back on.

What qualifies as an architecturally significant decision? [01:12]

Thomas Betts: So the book starts with the concept of making decisions. And you go into a little bit of analysis, and I think this is useful to talk about, what are important decisions and what are specifically architecturally significant decisions?

Andrew Harmel-Law: It came about because I was spending a lot of time… Because I’m a consultant, I get to go in and help people do software architecture. And I was realizing more and more as I was doing software architecture that the technology would change, the frameworks would change, the patterns would change or to a certain extent change. Lots of stuff would change, but there was a kind of fundamental thing that would underpin things and that was decisions, the investigation of a bunch of possibilities, and then the selecting of one of those things and moving forward.

And then I did a bit of research. Because it’s the classic what is software architecture question, which I actually dodge in the book. But some of the classic quotes around about this and you get… There’s a quote from Grady Booch, there’s a great quote from Martin Fowler. And they would both talk about the fact that decisions are the thing that underpins software architecture.

I can’t remember which one, but one of them says that your actual architecture is the sum of all of your decisions that have been taken either consciously or unconsciously through the life of a software system. And that’s what gets you to where you are now, but it’s also the environment in which you live. You live in the context of previous decisions.

So then I was looking at that and then that got me interested in the actual practice of making decisions and then I realized, which is what turned into the article, which then turned into the book, we talk a lot about the architectures, but we don’t talk so much about the practice of getting to the decisions which pick microservices over a modular monolith or which pick Kafka and Kubernetes over. And so I realized that was probably something that worth thinking about and worth talking about.

Thomas Betts: Yes, I think architects love to say it depends, and that’s where the role of being an architect doing the architecture comes in is what does it depend on and how do you do that analysis and what are your options?

Andrew Harmel-Law: And that’s exactly it. And so then I was realizing that we have a bunch of things that we do where we realize… Because like you said, the answer is it depends, but the question is what does it depend on? Lots of the work of architecture is figuring out what does it depend on, and that’s where you bring all your expertise and all of these kind of things. And then I spent a little bit of time thinking that… I would talk about this out loud to colleagues, and they would say, “Is this an architectural decision? Is this an architectural decision?”

And I realized that while again there are some things that are written down about this, there’s definitely not a even vague consensus on lots of things, what makes something architectural or an architectural decision and what makes it a significant architectural decision, i.e., something that if you’re an architect, you might want to focus on and spend time with as opposed to something which might be architectural but maybe isn’t significant. So it’s a way of, to begin with at least for myself, just to make sure that I was spending the right effort and attention in the right place at the right time.

Thomas Betts: So what are the attributes of those architecturally significant? How can we tell like, “Oh, I’m spending a lot of time thinking about this, but is that the best use of my time?” Where is the architect going to gain the most bang for the buck on the decision making?

Andrew Harmel-Law: There’s the classic architectural decision records post. It’s a long time ago anyway from Michael Nygard, I think it’s called Documenting Architectural Decisions, where I believe they were introducing ADRs. Then Ruth Malan messaged me behind the scenes and pointed out that someone in the 1980s had talked about them, but Michael had written a blog which everyone read, including me. And in that he highlights a bunch of things which I thought were quite interesting and pulled out.

So there’s decisions which affect the structure, and in that I would include placement of function. Where does this thing belong? Does it belong in that component over there? Does it belong in that component over there?

Things which define interfaces and dependencies, which are two sides to the same coin, but it’s like things which I use and things with use me.

Things which affect cross-functional requirements, which is a ThoughtWorks term, which although everyone else calls non-functional requirements or systemic qualities or something. It’s like the qualities we expect the system to have.

And the last one, which I like, which you don’t see as much but I think is super important. So Michael Nygard’s got construction technologies.

And again, I expanded that based on having spent too long working with certain ThoughtWorkers where if you’re doing continuous deployment, I think if the only route to production is down some continuous deployment pathway and originating from source control, I think the choice of tooling on your pipeline, to quote a ThoughtWorker, I think it’s Kief Morris, is if you’re doing continuous deployment, your pipelines need to have the same availability as production because otherwise you can’t fix stuff. So choosing technologies for that, that all feels like architecture.

And then for significance, I tried to identify and I go into great lengths of this in the book, I tried to identify what would make something a significant change to an interface. For example, on that one I realized that for me if it was going to make a minor version or a major version change using semantic versioning, that would be significant. If it’s going to do a bug fixture, like a level three kind of change, I don’t care. Someone else can worry about that kind of stuff. And so, for dependencies. If I’m going to select a new major minor version of something or if I’m going to choose something completely new.

Placement of function, what I did was I said, typically if it’s going to involve one of the classic patterns which we know about, and I list off a bunch of all of the classic sources where you think about things, if it’s going to evolve an adoption of one of those, then that’s probably relevant.

And then construction technologies again and CFRs, it was if it was going to have a material impact on those kind of things, then it was worth paying attention. But if someone’s upgrading something to a minor version of something or whatever, then it’s probably less relevant.

I did feel like I was stepping into the unknown a little bit, so I made sure I got lots of feedback about it, so I didn’t want to arbitrarily make statements which people would then blindly follow, but it seems to resonated. It’s written to be cut and pastable, I’m not sure O’Reilly like you cutting and pasting. But it’s meant to be… You could cut and paste a bit of it and put it in your wiki and say, “This is what we’re defining as significant”, it’s a pretty good starter for ten. And I used it on a few engagements with a few clients and I got some good feedback there around about clarity and things, so.

The goal is to help people focus their attention because the point of the book is there’s decisions happening everywhere. It’s a bit busy, so we need to make sure we’re in the right place at the right time if we’re doing architecture.

Building on five revolutions in software delivery [07:46]

Thomas Betts: Right, right. Time is usually the limiting factor of any architect being able to do everything. I want to go to that idea. You talked about having the CI/CD is the way we deploy software. We’ve talked about having platform systems and Team Topologies talks about having a platform team and those decisions. This makes a decision that makes it easy for teams to follow the golden road and get stuff done.

So yes, I can see how you make one of those choices. It’s like, we’re going to use this database or this storage technology or these servers and if you do that it’s easy. But if you do something off that path, it gets more challenging. I think that having those pipelines gets to another idea you talked about in the book at the beginning that this is built on top of layers and layers of revolutions we’ve gone through with… I can’t remember what they all were, but Agile and DevOps and maybe you have the list in front of you.

Andrew Harmel-Law: So yes, so the list I’ve got just a case. I called them revolutions and there’s a footnote which points this out because I didn’t want to pick a fight. The revolution term is like the things came along which reset us our practice in software delivery, which includes software architecture. It reset the standard way of doing things. So the Agile Manifesto, the point was we’d done a bunch of stuff, people had built software, but then the Agile Manifesto came and reset how we thought about focusing on running and testing code.

Then the cloud computing comes along and it’s how you think about where your runtimes are and whether you own them. DevOps comes along, which is how you think about the running of the systems as opposed to just the building. Product thinking comes along and helps you think about when you factor in feedback. And then stream-aligned teams helps you focus on how you align the human beings to the work that needs to be done.

In every single step along the way, we’re getting out of the way of the software. Someone has an idea and then that idea turns into software and is running somewhere in production. The key thing is to get as much as possible out of the way of that so you can then do the work, get the code into production and learn from the feedback. That was a big thing I realized. It’s very easy to just put stuff in prod, but if you’re not learning from all of these things.

Traditional software architecture decision making [09:52]

Thomas Betts: So if we go back to maybe an earlier point before those revolutions or somewhere along the line, how did you see software architecture decisions being made traditionally versus after all these revolutions have occurred?

Andrew Harmel-Law: So it was really interesting. So traditionally I saw them… And this was for valid reasons. Because software was hard and because multiple people were building it, you would see certain nominated people being responsible for deciding and therefore accountable for deciding, which is a good thing. So you want the big decisions, like I said, the significant decisions, you want those to be made explicitly, ideally, and by people who are taking them seriously and putting the right level of effort into them. And you also want those people to be accountable for things.

Traditionally the way that we structured things was that it made sense because we weren’t moving fast because maybe less of the revolutions had happened. So we would want the right people, typically middle-aged men of my description, and in some central place where they would do work. And it would appear… For so long, you’d get these memes coming up. So the ivory tower would be the first meme and there are problems with the ivory tower. So then rather than fundamentally changing how it’s done, the people came from… And this is what I did a lot. You’d come down from the ivory tower and you’d work with teams.

The key thing which both of those shared, which I was experiencing problems with more and more, was the responsibility and accountability was still in a named, few people for valid reasons because you want people to be accountable and you want accountability for decisions, and therefore as the revolutions come along and everything else is speeding up. And working for ThoughtWorks, we would for many clients come in and make their software delivery go faster.

By doing that, as you know from systems thinking, if you speed up a bunch of stuff in the system, it just exposes another bottleneck. And I rapidly realized that I was… Me doing architectural deciding was a proper bottleneck. And the more ThoughtWorkers and clients moved faster, the more I was like, “Oh my god, there’s more decisions coming my way because everything’s faster and then I’m a bottleneck because there’s more teams making decisions”. It was untenable.

And I wondered if it was just me and me being incompetent, but the more people I spoke to I’m like, “Okay, this is a thing. Lots of people are trying to do the right thing, trying to do the right job, trying to make good decisions, but in the context of just more and more coming at them all the time”. And that felt like that’s the force, which is pushing this into an alternative-

Option making and decision taking [12:16]

Thomas Betts: Right. So you called out two different variations, the ivory tower, just go over there and get your decision and it comes back, or this is an embedded architect, but it’s still the person responsible for doing all of the work. I think you can talk about language a little bit. I liked how in the book you talked about option making and decision taking, decide what we can decide upon, what are the paths we can go and then make it. You talked about the accountability as the person accountable for the decision. That’s the decision taking aspect. But I think you need to talk a little bit more about the option making, is a key aspect to that as well.

Andrew Harmel-Law: Yes, exactly. So I didn’t realize this because I again, through privilege, I had access to these things because I’ve been part of them and I’ve been on projects and seen the making happen, the hard work of architecture. So the weighing up of the context and of the various options and the uncovering of the constraints, and then like you said, figuring out what it depends on and then balancing those things and then using my expertise and knowledge of patterns and all this kind of stuff to figure out the right options.

But I realized that that’s… Especially when it happened in the ivory tower. But to a certain extent when architects are down with teams, that would be something that was a bit of a magical trick people didn’t realize because nobody was saying out loud what they were doing.

So they would do all of this hard work and draw diagrams on whiteboards and all of this kind of stuff, but that’s the bit people would see. What they wouldn’t see or wouldn’t see explicitly, trade-off analysis, the pros and cons, the fact that there is no perfect option, the fact that there’s always some downsides to the option that you select, there’s always some upsides to the ones you ignore. And colleagues came to me over and over again and that’s why.

The book is aimed at two audiences: stressed architects, who I’ve just described, and also lead developers who are like, “It feels like I could make some of these things. I could become an architect, move into architecture, but I don’t know how”.

And so narrating all of that kind of stuff around about like, “This is what we’re doing, this is how I’m doing trade-off analysis. This is where my gut feel comes in, this is where it smells bad, I feel a bit uneasy about it”. All of that kind of stuff, the softer stuff which just comes with experience. Putting all of that into things and acknowledging the fact that it’s a creative act, it’s not just like I go and do a bunch of research and then I do whatever’s obvious. That felt important to say out loud. And hopefully have captured the intellectual aspects of it, but also the creative and the gut feel.

Architects need to communicate the “why” behind decisions [14:41]

Thomas Betts: And I think it’s important to talk about that whole process and for people to see that process. The ivory tower meme, you don’t see it. They’re off from the tower, you don’t see what’s happening. All you see is that result. Here’s the diagram, here’s the architecture as if it’s written in stone. And the interesting bits, at least to me is why? Why did you choose this and why did you not choose something else? We had these trade-offs, there were other options that had some very valid good points, but in the scheme of things, we had to trade them off.

When you have the people in the office and there’s a whiteboard and everyone can get around, that’s when it’s easier for that lead developer to start doing the architecture. Maybe they’re not working on the biggest architecturally significant decisions, they’re not designing the platform, but maybe within their team they have decisions to be made and say, “Hey, I’m just trying to figure out how do I write this code? How do I structure these smaller interfaces, not the interface of the API?”

Andrew Harmel-Law: So there was three things that motivated the book. The third one is the famous Alberto Brandolini quote, which is like, “It’s the developer’s assumptions what gets into prod”, and this is what pushed me over the edge. I was already stressed and then I realized that I might do an awesome job.

I might still be designing great architecture and making great architectural decisions, but if I fumbled the last bit, which is explaining it to the developers, or it just doesn’t get into the heads of the people who are going to write the code, because it’s not me writing the code because I’m too busy, then who knows what gets into prod. And it’s the running architecture that’s the real one. I can wave at Miro boards and murals and whiteboard pictures as much as I want, but if that’s not what’s gone into prod.

Thomas Betts: There’s another quote and I can’t trace it back, but the, “You always have an architecture, it’s whether you explicitly decided what it was and the sum of all those decisions”. Alberto Brandolini has a great quote. That’s the insight. If you just give them one diagram, it’s like, “Here’s what you’re supposed to do”. What about the thing that’s one layer down on the diagram that’s not drawn?

Maybe you didn’t do the full C4 analysis all the way down to the bottom and say, “Here’s the code you’re supposed to write” because you assumed anyone looking at this diagram is going to be able to interpret and make the decisions underneath that reflect that. But if they don’t understand the why, then they might make something that looks the same as what that diagram does, but doesn’t actually do the core thing you were trying to accomplish.

Andrew Harmel-Law: There’s a great quote which didn’t make it into the book, but it’s from Domain-Driven Design, the Blue Book by Eric Evans. There is something around about taking whatever the design is down to the level of code and then trying to put it into code. We know that it’s not just typing.

That wrestling with that level of detail will frequently fundamentally reframe your understanding of the problem, which might then completely change all of these things. And the realization that there’s a feedback loop to the design, not just to the implementation all the way down from the code, again, terrified me. Because I was trying to be a hands-on architect, but the more of things were moving fast, the less I was going to be there.

Balancing the speed of responding with decentralization of decision rights [17:31]

Thomas Betts: That bottleneck you described, the, oh, once we made the system fast, we realized that I am the bottleneck. There’s a great chart that you referred to a few times that has two axes and I think it’s how fast you’re able to respond and make a decision, and then also who has the ability to make that decision, who’s in charge? And the ivory tower architect was one way on that. The hands-on architect was somewhere else. And you had different variations of who interacts with them. Can you describe what you were trying to accomplish with showing those… I know audio podcast, hard to see the axes.

Andrew Harmel-Law: It seems to be working quite well, although whenever I show it in… I’ve got a presentation where I talk about this bit explicitly, and so I always get feedback as to where I put the dots. But fundamentally, like you said, it takes a bunch of ways to decide, standard ways to decide in groups. So autocratic where you just one person says, like the ivory tower, “Do this”. And the other end of the spectrum is consensus where everybody has to agree.

And so what I wanted to point out was there are pros and cons to both in the dynamic system of how we currently like to deliver software. So there are other pros and cons, but like you said, I’ve mentioned in a previous opening chapter, the forces I felt I was being subjected to, which were the five revolutions. So the goal is to get good decisions into production as quickly as possible into code.

What I did was I looked at the various ways of making decisions or making options and taking decisions and then evaluated them on these two criteria. Because you definitely want things to be fast because you want… If a team needs to push a feature and there needs to be some architecture done behind it, you want that decision to not block them. But also the fastest way to do it is for one person to have a think in the shower and just say, “Do it”. But that’s incredibly risky and nobody in the right minds, I don’t think, would really do that very often.

So what you want to do is get the right people in the right minds on that thing with the right expertise, the people who know about the code, people who know about the infrastructure, the people who remember when we tried this 10 years ago and it blew up in our face. So therefore the other hypothesis is the more people you get involved, the better the decision.

And then I pitched those two things together and realize that depending on if you have… The more people you have, the slower it goes because especially if it’s consensus, there’s always one person who doesn’t agree or half agrees or whatever. And if it’s autocratic, then one human being thinking really hard definitely isn’t a good way to do software these days. So I pitched those. And then, this was the hook, which again, I got feedback from Rita, my editor, that I shouldn’t bury.

The fact that it was one or the other was predicated on the classic you can have one thing, but you can’t have both. But as they pointed out in Accelerate, the old story used to be you can’t have high quality and frequent deployment. You can either have one or the other. And then Accelerate by Dr. Nicole Forsgren and co comes along, goes, “No, you can have both and it’s probably good”.

So my thing, and this is, like you mentioned, the fundamental part of the book, which is a very simple core, which is the architecture advice process, is you can have almost as fast as possible and almost as distributed decision-making as possible if you want to sacrifice some things, which I would argue in the book are less important if you’re shipping code and iterating and getting feedback on it.

Facilitating software architecture with the architecture advice process [20:39]

Thomas Betts: This is where I wanted to get to the architecture advice process and I will make sure we have a link to the previous episode you were on and the article you wrote on Martin Fowler.

Andrew Harmel-Law: Yes, the article is the 8-page version of the 460-page book.

Thomas Betts: There’s a lot of detail on that. So that’s available if you don’t have the book. But the architecture advice process goes to those premises that you’re going to have the right people involved in the process, but they’re not, as you said, the consensus. They’re not all making the one agreed-upon decision because if you have 10 people in the room, you’re going to have one person who holds up the entire thing. You can’t go fast at that. So how do you avoid that situation? What does the advice process do differently than saying, “Oh, we’re going to get everybody together to make a decision”?

Andrew Harmel-Law: This is the super smart part, which to be clear, I didn’t come up with the architecture advice process, I just put the word architecture in front of it. There’s a thing called the advice process, which comes from trust organizations and it is simple as anyone can make any decision, so therefore it’s not anyone in the ivory tower, a team lead or whoever, could make a decision.

Typically, you’ll end up with architects who look across teams deciding cross-team things, and teams or team leads deciding within-team things. So anyone can make a decision or make the options and take a decision provided, and there’s two rules, they seek advice from those who’ll be affected and those with expertise.

So those who’ll be affected are… The way I take into explaining it is if your decision will make them put a story on their backlog. It means it’s caused them work to do. And people with expertise are just classically the people who’ve got the war stories, are the people who you’d want to include, but they might not be the people who are affected. And it’s incredibly simple, but it’s incredibly powerful.

And that’s what changes the practice of software architecture from the doing of it to the facilitating of it, which is where the title of the book comes from, because architecture is still important and the major fear is suddenly people who aren’t well-practiced in architecture are going to be doing it, and the answer is yes. So therefore if you are an architect who is practiced to architecture, you can magnify your impact by making sure that the right advice is offered to the right people and making the right decisions in the right way.

The key point about advice is, like you mentioned, it’s not permission. No one’s giving permission or nobody’s agreeing, they’re just offering advice. And so the one thing I added to the standard advice process is if you were going to do only two things. You do the advice process and architecture decision records because you would write it down. Because if someone’s writing it down, then it holds them accountable. And so that’s like I said.

Previously, there was responsibility and accountability for decisions was in one place. You still need responsibility and accountability to be there. And the open group in their Open Agile approach to architecture, O-AA framework, they maintain and that’s still important, but the key point they realize is that as long as accountability and responsibility move together, then that’s fine.

So if I can decide and I’m accountable for the decision, that’s good. So therefore it’s a check to the reckless. I can’t just randomly decide to do stuff like suddenly adopt Kotlin or Go even though the rest of the team are on C# or something because I’d have to write an ADR and I’d have all of the people in the comments on the ADR saying, “This is a terrible decision. It’s completely against the strategic direction of everything we do. We have no developers with any of these skills, et cetera”.

So whoever made the decision in the face of that advice would be a pretty brave person. Although it’s interestingly, sometimes it lets people do the decisions which make more sense in reality because they’ve got information which everybody isn’t party to.

Thomas Betts: And if you’ve been able to solicit that feedback, solicit those opinions, then you understand we made this decision in this case because of this reason. It wasn’t just I chose out of the blue. Which goes to some of the… You mentioned Accelerate, but also those revolutions you talked about, Agile and cloud. Some of the arguments against them, the people who have done traditional waterfall things like, “Oh, it’s going to be chaos. Everyone’s going to be just deploying services left and right.

They’re going to write in whatever language they want. Kotlin just shows up as a thing we have to support. And teams are going to do who knows what and we’re going to lose control”. As Accelerate has shown, you trade off, you make more specific control, but you put those guardrails in place. Let’s make these significant decisions to help those people then be successful, not just let anarchy and chaos ensue.

Andrew Harmel-Law: Yes, Exactly. And this is what’s interesting. So there’s a bunch of stuff… Because just to your point, if anyone can do anything, how do we stay aligned? Because there’s very few companies where one team is building a product, so you’ll have more than one team and you all need to head in the same direction. Although as Diana Montalion points out, direction is a very linear way of thinking, and we’re in a socio-technical systems world. But how do you know you’re all headed for the same goal or the same outcome or whatever?

And that’s interesting because then it places more emphasis on what we should be doing as architects on the tech strategy, which should be the guiding light. It puts more emphasis on platform teams. So we’re not just building a platform because it’s fun and we like building platforms. The platform, to take the classic Netflix approach, and Netflix, you could still use any language you wanted to write stuff, but Netflix are investing time and human resources in making certain languages and certain frameworks and certain ways of working.

So you can by all means go off and do your thing in your circumstances, but you’ll have no support. You’ll have to build your own pipelines, you’ll have to build your own monitoring, you’ll have to all of this stuff. But if you stay on the paved road, then you’re good. But that means sometimes you want to go off the paved road, and teams do at Netflix and they get the benefits if they’ve done the cost-benefit analysis.

So that’s why I like it, but like I said, there is a benefit to having more than one or two smart people thinking about things. Because it turns out most people are smart if you give them the opportunity to bring their expertise and things. And there’ll be people who want to learn and to get smarter. So by doing this, deciding out in the open, like you said, the problem in the past was it was all hidden behind closed doors, there’ll be this kind of people who aren’t maybe necessarily confident enough to start deciding, but you will see people who are like, “I’ve got a strong opinion on that”.

Or, as I learned from Rita, my editor again, will ask amazing questions. I spend lots of my time coaching people these days. They’ll be like, “Oh, I don’t know enough to say”. I’m like, “But you could ask very good questions because your questions when you ask me behind the scenes are really good”.

Because again, that goes back to this thing needs to be understood so it can be coded up and deployed and run. So making sure that things make sense to people and making sure that things are coherent and all of that kind of stuff, it helps. Things that are always been important, like conceptual integrity and things, if you involve people and lift the lid on the option making, then suddenly things like conceptual integrity and all of those important aspects become, in my experience, a lot more attainable.

How to get started [27:25]

Thomas Betts: So how do we get started with that? You mentioned people see these benefits, and it’s one thing to say, “Hey, here’s the advice process. Here’s how we get all the people in the room”. But I’ve done this with Agile and DevOps, getting companies to actually shift the mindset as opposed to, “I’m just going to spray on the we’re doing Agile now because we have scrum masters, so we must be Agile”. How do you actually make the transition to doing this effectively?

Andrew Harmel-Law: There’s a bunch of suggestions in the book or advice in the book. Again, this was a thing where I learned a lot from Rita, my editor. I was very aware that if this book did become successful, which hopefully it does, it’s telling people things that they need to hear or want to hear, I didn’t want it to be a recipe book because otherwise you’d fall into the Agile trap, which is like, we’re doing all the things, so we must be.

So what we ended up was there’s the very core things, like I said, which is basically do the advice process and write down ADRs, but then there’s a lot of advice. So there’s lots of advice, that’s 400 of the 462 pages which are like, “Here are the things I have learned. Here are the things that are important”. A large chunk of that is adoption techniques and things that will probably trip you up.

The key thing I’ve learned is if you’re someone who has the power to distribute your authority and accountability, then you can do that, but it makes sense to do it with one or two or three, a small amount of teams working on a similar piece of work just so the cognitive load doesn’t blow up. But you can also do it, and I’ve done this with teams of clients before. It’s a hard sell to say to an architect, especially a client architect. All these ThoughtWorkers are going to come along and maybe some of your people as well are going to come along and suddenly you are going to stop doing deciding, and they’re going to do it? They’ll be terrified.

But what we do instead is we model the process. So a team will do everything up to the point of deciding and even say, “We would make this decision if we were going to do it, but we acknowledge that that’s not your process, so therefore we would like to propose to this, but we’ll do whatever you tell us to do”. And so that works really well because architects then realized that there’s less and less that they have to do. I’ve had some really good feedback from an architect, the current client where I’m working, where they said, “These decisions are getting better and better. I don’t even have to read them, I just trust the team now”. And you’re like, “Okay, this is good”.

So they’re moving towards rubber-stamping so they know that things are happening and then they’re aware, and then… So they can slowly build the trust in the team and the team or teams can build their capabilities,. And that’s quite nice. So you can either, like I say, you can give your power away if you have the power to decide, if you’re an architect probably, and then mentor people, or teams can do it and then ask for feedback. And both of those work quite well. Things that definitely don’t work well are when you go big bang because this is, like you said… You’re pulling away a lot of… Like Agile if it’s done properly or DevOps or cloud or whatever, you’re removing a lot.

And what you want to do is not suddenly fill that space with more things, what you want to do is fill that space with the practice of architecture, which is now going to be happening in more places. So holding that space and especially if you’re an architect, not rushing in to tell people what to do, or strongly suggest. It’s very easy to think you’ve given everybody autonomy and accountability, and then realize that they just do everything that you tell them to do. Then you’re not really scaling yourself, they’re waiting for you to tell them what to do.

Thomas Betts: Right. You need to give them the tool so they’re set up for success. You alluded to the idea of we’re going to go with microservices because everybody’s doing microservices, but we don’t actually re-architect our system appropriately and we have a distributed monolith, and then we have networking problems that we didn’t have before and we haven’t actually fixed any of our core things and we’re still tightly coupled and the teams can’t move along. We got slower doing what was supposed to speed us up. I can see the same thing of, “Oh, we’ve our architecture decisions, we should be faster”, or, “Maybe we’re moving faster, but in the wrong way because now it’s just..”. Again, the chaos-ensuing scenario.

Andrew Harmel-Law: So people use the word anarchy sometimes. Technically it is anarchistic. If you read about political anarchy, it’s a small scale implementation of anarcho-syndicalism. Or there’s a bunch of work that was done in Norway and I think in English coalmines or something where they realized that the more self-management and self-sustainability you gave to teams, the more teams would do the right thing. So there are precedents for this, but chaos is definitely not what you want. While there are emergent properties from our distributed systems, I don’t think you want to entice chaos into the mix. Just a little bit of decentralization.

Thomas Betts: It sounds like the advice for implementing this is similar to how I might roll out a new platform, a new CI/CD pipeline. You don’t say everybody switches over and does this today, you experiment. Let’s try with a few services that we think were a good example, and we find, “Oh, this does and doesn’t work”. And by the time we can ramp it up to more and more teams, we’ve figured out where the guardrails are, people are going to be successful. Similar for rolling out the advice process like, “Here’s what we want to get to. That’s our end goal. How do we try this out?”

Andrew Harmel-Law: Yes, exactly. I mentioned a space a minute ago. You clear out the space and then you’re like, “Right, so that’s the space where..”. And I got this from Matthias Verraes who runs DDD Europe. He’s like, “There’s a space where architecture practice is happening. So you’ve cleared out that space”. You mentioned guardrails. What you want to do is put in place as few as possible because you want as much of that space to be open as possible for creativity and people to design. So you clear out that space and then you’re basically trying to protect the space and do it.

But what’s interesting is culture is the thing that fills the space, and culture is good and bad. So what you want is a good form of architectural culture to fill that space, and you want it to be informed by the people who are doing the deciding, the systems that you’ve got in place at the moment, the technologies that everything is based on, the dynamics of the market, your users and all these kind of things. Because that’s what’s different. That’s why I was trying not to be prescriptive in the book. Because those are so many things which mean you can’t just cookie cutter and do exactly like someone else because it’ll be different.

And I’ve done this with numerous clients. What is most exciting is how different this is because they always take things and do it differently. If you read the article, it sounds very chatty and very synchronous, and then we rolled it out at one client, and they’re actually written up a case study on Martin Fowler’s website as well, which we can put a link to.

So they were a blockchain banks, so obviously they were like, “We’re very decentralized”. They loved all the theory of it, but also they didn’t want meetings. And I was like, “That’s going to be problematic because I’ve got this heavy layer of human beings talking to each other”. But they with their async culture figured out how to do it incredibly well using just written communication, which is now my default actually because it works so well.

Thomas Betts: The async or writing the ADR for making the decision process?

Andrew Harmel-Law: Yes, the whole thing. So now, which I don’t think is even in the book, but now I’m more defaulting to… There’s a thing called the Architecture Advice Forum, which is where you get together and people share things. It’s less advice and more like notification because the advice is happening before, afterwards, in the gaps, asynchronously, in comments on the bottom of wiki pages or in pull requests or whatever. The richness of the communications that you see, and the fact…

There’s always one or two people, like the experts who know about the old systems. They can scale better because they can be in the right place at the right time. They can offer the right advice in the right context. Super powerful. As opposed to those people having to be in meetings for the entirety of their lives, it feels like a very useful way of using people’s expertise and things, and they can refer to, “Please refer to my comment on this previous ADR and all this kind of stuff”. As I said three years ago over here, it’s good. Works really well.

Conclusion [34:55]

Thomas Betts: Fantastic stuff. Like I said, I highly recommend the book. I enjoyed talking to you a few years ago. I was looking forward to the book coming out. I learned a lot more. Things we talked about today, made me think about the practice of architecture in ways I really hadn’t dived into, so I recommend people pick it up for that. If people need to reach out to you, Andrew, for any questions, how should they do that?

Andrew Harmel-Law: Questions are great and advice and feedback, which is good based on that’s how the book’s supposed to work, but they can find me on LinkedIn because I’m the only Andrew Harmel-Law on LinkedIn, or I’m on Bluesky, the only Andrew Harmel-Law on Bluesky. On Mastodon as well, I think as well. And again, Mastodon’s a bit harder to find people.

But all of those things, plus a bunch of supporting stuff is on the website for the book, which is facilitatingsoftwarearchitecture.com. If folks go over there, there’s a free chapter they can download, which is Chapter 11, on how to make a tech radar. There’s a load of other excerpts, there’s some templates. There’s a worked example of an ADR being written using the advice process. Loads of different stuff.

If people want to go there and ask questions. And I think that actually, I’ve don’t know if anyone’s used it yet, but there’s a GitHub community site, so I’d love people to ask stuff there if they want to ask a question in public. Because this is the thing, I’ve collated a bunch of stuff from my experience and the experience of other people who’ve tried this. There are far more ways of doing this than just me.

One of the best things is somebody came to me and said, “I think I’m doing this, but I just had a different name”. They arrived at the same approach from first principles, which means to me that it’s probably more likely to be something. Because if other people can get to the same place without any knowledge of it, that means I haven’t invented it, I’ve just glued a bunch of thoughts. So I’d love to hear people’s thoughts.

Thomas Betts: Once again, Andrew Harmel-Law, thank you for being on The InfoQ Podcast.

Andrew Harmel-Law: Thank you.

Thomas Betts: Listeners, we hope you’ll join us again soon for another episode.

Mentioned: