Cultivating Healthy Engineering Cultures: Insights from Sophie Weston

Transcript

Shane Hastie: Good day, folks. This is Shane Hastie for the InfoQ Engineering Culture Podcast. Today I have the privilege of sitting down with Sophie Weston. Sophie, thanks very much for taking the time to talk to us.

Sophie Weston: You’re welcome. Nice to be there.

Shane Hastie: My normal starting point with these conversations is who’s Sophie?

Introductions [00:49]

Sophie Weston: Sophie is someone who’s been working in tech for about 30 years. It’s terrifying to say out loud these days. I spent many years working as a software developer, mostly in Java, mostly on Unix, but doing all sorts of other things as well, like Python and TCL and lots of scripting stuff. But I had the privilege of working in a DevOps way even before the word had been invented. So that was my natural kind of shift then and more and more sideways into DevOps and how we actually do work and how teams are working and how we make it better. And here I am today still doing it, working as a principal engineer. But very focused on the people side of that rather than actual, not hands-on with tech anymore.

Shane Hastie: So at QCon London last year, you hosted a track, How Do Teams Really Work? So can I ask you to answer that question? How do teams really work in software engineering today?

Teams and teamwork in software engineering [01:49]

Sophie Weston: I don’t think there’s any one way, but I think there’s some ideas have taken hold around the idea that teams need to be autonomous and make their own decisions, or at least some degree. And we talk about having long-lived product teams these days. But I think some teams are doing it more and it’s more about the words than actually doing it. I think sadly, too many people are still kind of just doing projects and it’s still feature factory and there’s still too much of that going on. So it varies, I guess. There’s no one answer. But I think we all think we’re doing agile. Again, I’m not sure how true that is. So there’s some ideas are prevalent, but I think it varies from team to team.

I worry that teams aren’t getting enough time to do the things that make a good software team. I think there’s too much focus on getting stuff out the door because obviously that’s what we need. We need to deliver value to our customers, it’s why we’re there. But I think some of the ideas around how we do that well are still counterintuitive to people who aren’t involved in software and other parts of the business. And I think we struggle to kind of make a good case sometimes for how we do software well. I think it’s more of an art than a science, and it’s not a feature factory. You can’t just churn it out, because it’s knowledge work.

Shane Hastie: A lot of points I’d like to dig into there and pick some of these apart, so agile versus tragile.

Sophie Weston: I haven’t heard that. I like that. I’m going to steal that.

Shane Hastie: I have to give credit Phil Abernathy for that one. And tragile was one that I picked up from him. And certainly in my experience looking back over 20 plus years of the Agile Manifesto, I have seen a small group of organizations who really got it, and lots and lots of the agile veneer and no real culture shift.

Agile or tragile [03:40]

Sophie Weston: Yes, what I see is we talk about agile only in development teams. So I wish I could show you, there’s a cartoon that came from Jon Smart’s book about Sooner Safer Happier, where you see a dev team in the middle of the whole process and they’re going, “Yay, we’re so agile”. But there’s all this stuff either side of them that isn’t, taking months and months and months to get this idea to the point where it’s in a dev team. And then there’s all this stuff afterwards that we don’t think about, about getting into production, but then operating it and supporting it and then decommissioning it. And then there’s this tiny little slice in the middle and we’re going, “Yay, we’re so agile. Woo. Two weeks sprint”. And when you take that step back and look at it in a more systemic way, you just go, “Well, yes, okay, great. Woo, we’re agile”.

And I think, like I said, because I’d been working in that DevOps way, even before Patrick Debois called it DevOps, just because I was working in a team where we were dev and ops together working for a single customer, so it just seemed natural that you would talk to the ops engineers about how you were going to develop your software so they could support it well. We weren’t even doing you build it, you run it, but we were kind of integrated. We just don’t think about that. We just think about getting stuff out the door as fast as we can. And it’s not just about dev. We need to look at the wider picture.

And even if it is just about getting stuff out the door, even when you kind of do want to focus in on the software development part, like I said, it’s knowledge work. So it’s not just churn it out, churn it out. Because the code is not the hard bit in some ways. And we know that if you can solve a problem with less code, that’s good because the more lines of code you have, the more complicated it is to support, the more hard it is. So we want to be writing less code and solving problems. This is why I think AI is going to be interesting because the way we’re looking at AI and software development, it all seems to be about copilot and more code and, “Yes, great, just bung it in and get it out the door”. And I don’t think that’s where the problem is or where the benefits are.

Shane Hastie: So what are some of those good team practices that we are neglecting in our focus on get the code out and more code and more code?

Good team practices often neglected in the rush to deliver features [05:57]

Sophie Weston: I don’t think we’re making time or space for the other things that we need to do. Things like documentation. I know it’s dull and everybody goes, “Ooh”, it’s almost like a joke that engineers don’t like to write it. But we need it. It’s important. Especially when we are working remotely or working across distributed teams and across time zones. You need that because otherwise you’re just creating bottlenecks in the system.

You need to be able to self-service that information. And particular things like onboarding, when you get a new engineer into your team, you want them to be able to be productive as quickly as possible and not have them A, wasting time and B, being miserable. Because it’s horrible when you’re sat there and you think you don’t want to go and have to ask everybody. So it’s both. It’s about making it a good experience for them and also about being good for the business because you want them to be useful. So that’s one particular example.

Runbooks is another one. Again, it’s like that classic thing of incidents at 3:00 AM, because they always happen at 3:00 AM. That’s always the time we have to quote. You want people to have access to the information they need and it’s no good at being in people’s heads. So you have to write it down and it’s documentation, and it’s very boring, but we need it and we don’t make time or space for it.

We always kind of say, “Oh yes, we’ll come back to that”. But we don’t because then we have the next feature to get out the door. Just things like retros, team retros, time together to reflect, to learn, to take that step back and say, “How can we do this better?” And also personal time to just not be flat out, just to be curious and to learn and just to take a look at what you’ve got, what your system is and just go, “Huh, why does that look like that? I wonder if I could..”. You don’t give people that space, they don’t have that continuous improvement time.

And then also more things like more organizational things like tech talks, weekly tech talks or guilds or things. Again, it’s time away from writing code, but you give people space to learn. They meet people outside of their team, so you create new connections, you create new ideas and people next time they have a problem, they might go, “Oh yes, I was talking to somebody at that guild or that, they might know about that”. And you create different bonds. But this is all stuff that doesn’t have immediate return on investment. It’s slower and it’s less visible, but it’s massively important that you make space and time for these things to happen.

Shane Hastie: Meaning slow down to speed up.

Sophie Weston: Yes, exactly. And it is less visible, especially things like tech talks. You kind of go, “Oh, give engineers 45 minutes or an hour every week to go and just sit around”. But they’re not, they’re learning things and they’re creating connections. And some of the people who are doing the talking, that’s another skill, that’s another good skill that people get. And a chance to practice it and learn it in a safe environment, public speaking, putting a talk together, it’s all good stuff.

Shane Hastie: So how did we get into this picture factory rut?

Getting out of the feature factory rut [09:01]

Sophie Weston: I don’t think we’re very good at making a case for ourselves somehow. No, I mean that in the broadest sense, not just dev. It’s not the same as, I’m not going to try and belittle the construction industry, but it’s not the same as building a bridge. It’s not the same as doing that. Because it is different every time. It is knowledge work. And I don’t think we make a good case to other parts of the business that we need to work in this particular way and that it’s not just a case of everything’s the same, just churn it out. Every problem is different every time you come to write it. I mean obviously we have patterns, we have architectural patterns and we have software patterns. But you’re applying them in a different way each time.

And for the stuff that you’re not applying in the same way each time, you should just be making that as painless as possible. And this is the kind of things like platform engineering, which is where DevOps is evolving into platform engineering, and the whole idea of team topologies where you have layers of platforms so that the people that are higher in the stack are solving business problems, they don’t have to think about how to provision a database or they can just self-serve that stuff and solve the business problem. But I don’t know, it just feels like we’re always trying to have to argue that we need to work in this particular way and that space and time for these things are as important.

Shane Hastie: How do we advocate?

Measurement is often misused [10:27]

Sophie Weston: I don’t know. I don’t know how we get people to understand the kind of counter intuitive nature of some of this, the slowdown to speed up. Because we seem to have an obsession as well now with quantitative data. And it’s like unless you can show hard data and hard numbers for it, people kind of aren’t interested. And it’s like it’s somehow qualitative data and telling stories and having people’s experience is seen as less worthy or having less weight to it than hard numbers. So it’s like the DORA metrics. When they first came out of State of DevOps report and Accelerate, it was amazing, frequency of deployment, change, failure rate, time to recover and I can’t remember what the last one is. But they were kind of revolution. They were amazing. And now another thing that’s kind of a bit like agile is like they’ve been taken and not abused, but a little bit.

It’s like now they’re seen as the only things you have to care about. But there’s an awful lot more to creating software and writing software, which is why I like where things like the SPACE framework and the DevX framework are advocating for using the qualitative as well as those hard quantitative metrics. Because you could have amazing DORA metrics, you could be doing 1,000 releases a day, but what value are you actually giving to your customers? And what experience your engineers having while producing those amazing DORA metrics? They could be burnt out, they could be totally miserable, they could be looking for new jobs every night on LinkedIn. That quality of experience is important as well as the hard quantitative data that you get from things like DORA metrics.

Shane Hastie: One of the most misquoted adages of management today is, “That which gets measured gets managed”. Because if you actually look at the whole quote, the author, it wasn’t Demming but it was one of those experts, is actually saying, “That which gets measured gets managed even if it shouldn’t be”.

Sophie Weston: Yes.

Shane Hastie: We also all too often miss the rest of it.

Making things better for people in tech [12:36]

Sophie Weston: Yes. All of this is about making it better for the people in tech. That’s kind of where when DevOps started to become more of a movement and people were talking about and the idea of continuous integration and continuous deployment began to become a thing, it made a lot of sense to me because not only is it good for the business, obviously you want to be able to release and get more value out more quickly, and you don’t have to do releases out of hours and the weekends and that kind of thing.

But that is also really good for the people doing it because no one wants to be doing releases at the weekend and in the evenings. I’ve been there and done it and it’s horrible. And big scary releases where you’re releasing six months worth of stuff and you’ve got this massive plan written down on paper that you do this, do this step, do this step, do this step. But you’ve got no rollback plan and you’ve got nowhere to test it and you’ve just got to kind keep going. That’s no fun, that’s no good for anybody.

So the ideas of DevOps where we can automate and do tiny little releases as frequently as we like and get faster feedback, that’s better for everybody. So it’s kind of win-win. And that’s why I like it. And I see the same with the platform engineering stuff coming through where… Because the whole idea of shift left was a good one, shift left along, get testing left, but earlier in the development process. But as things like Docker containerization kind of started to come along and that started to shift that left, into dev world, it felt like more and more stuff was becoming the responsibility of devs.

It’s like when I was developing and I worked with ops engineers, I wrote applications and then yes, I kind of handed it over. I didn’t have to worry about provisioning the infrastructure for it to live on. I didn’t have to understand about networking beyond the simplicity of, “Please can you open this firewall port so the database can speak to my app”, kind of level of understanding it. But as more and more stuff started to shift left, it felt like more and more stuff was becoming responsibility of devs. So cognitive load and it’s just doing everything.

So platform engineering, I can’t remember who said it, it was something about don’t shift left, it’s abstract down. But it’s that idea of push it down rather than left so that it becomes a platform so that the software engineers, the devs solving the business problems can just self-serve. Again, that to me is about making it better for people in that kind of slightly abstract way. It’s miserable when you’ve got to try and think about everything and do everything and be responsible for everything and it’s overwhelming. So that to me is also about making it better.

Shane Hastie: So what makes a good platform engineering team?

What makes a good platform engineering team [15:20]

Sophie Weston: So I think it’s all just software, but it’s kind of what interests you. If you think vertical stack, some people like doing web UIs, they like that whole UX and making things look pretty. I hated it. I was the world’s worst. Never get me to do web UX, awful. I just don’t care enough. I appreciate it when someone else has done a good job. Likewise, I didn’t want to be low level. My first kind of job when I graduated was doing very low level stuff, like working with assembler creating software for mobile radios. Likewise, I hated that. It was too low level and too kind of niggity.

My perfect mid-spot for me was working with databases. It was backend stuff, talk to databases, doing the kind of middleware. And I think people just have a natural affinity to where they want to live in that stack of software. So I think a good platform engineer is just a software engineer who somehow is drawn to that level of abstraction or involvement in the tech stack. But I think the principles are the same. They’re still just software teams. They still need to think about their customers and work in a good way and try and make things so that they’re delivering value in that self-service idea of delivering what people need. I don’t think it’s a lot different other than that.

Shane Hastie: One of the challenges we often see is the great individual contributor or great engineer who’s then made a manager and not given the support or training they need and suddenly you’ve got a really frustrated poor manager.

The transition from individual contributor to manager [17:03]

Sophie Weston: It’s a tricky problem because they’re different skill sets, aren’t they? They really are different skill sets. But I think to be a good engineering manager you have to have a techie mind. I don’t know whether I’d need to say you absolutely have to have been the hands-on software engineer or an engineer of some kind because I do think they’re very, very different. But you do need someone who’s in the tech space enough that they understand the problems and they understand… It’s like they understand the people. Because at the risk of offending my fellow software engineers, we are a quirky bunch quite often. We are renowned for not having very good social skills and that kind of… I’m making stereotypes here. But I think as a good engineering manager you need to understand and appreciate techies for who we are.

But I don’t know if you do actually need to have been a software engineer, but for those people who do start as engineers and want to make that transition to managers, yes, they need more support. They absolutely need more training. Again, it’s about having space and time, isn’t it, to learn the skills that you need to learn to be a good manager. I think it’s better now than it was earlier in my career. I think it’s easier to stay as an individual contributor if you want to. There is less pressure I think to make that jump into management. I think that was stronger when I was younger. So I think that there’s more of a dual track now. But I do think we need to support people who want to aim at that jump and kind of…

Also Charity Majors talked about the engineering manager pendulum. She wrote some really interesting stuff saying that for some people that’s almost how their career is going to be. They start as an engineer and they go and be a manager for a bit and then they want to go back and be an individual contributor because they want to get immersed in the tech again and then they want to go and work with people. So we should also be supporting that because I don’t think it should be a once and only choice for the people who do want to do that. They’ve got things to offer and contribute on both sides. It’s why not let them do that?

Shane Hastie: And your role currently is principal engineer, what was the journey to get there?

The journey to becoming a principal engineer [19:16]

Sophie Weston: A squiggly one. So I did spend 20 odd years just doing straight software development. But as I started to get more interested in the systems of work themselves, I got very into Kanban and DevOps and actually how we do the work. I started to move more into… I had a job title for a while as DevOps Advocate. So I was trying to work with teams to bring some of those ideas. And specifically at that point I was working with ops teams where I was working. They kind of had two streams to their business. One was bespoke software development, the other’s managed service. So they had teams of just ops engineers supporting other people’s platforms. And it was interesting working with them because they seemed very far behind software development in terms of ideas like agile and doing retros and thinking about how much work and progress and all that kind of stuff.

So I was working with teams of ops engineers, kind of trying to get them thinking about that kind of stuff. Yes, so kind of just kept going sideways a bit from actual hands-on tech, getting more into the people side and the culture side, which being a principal engineer is more about… I guess in some ways it’s a bit like an engineering manager role in that you need to have had that experience in some ways more so than a manager. But you’re not actually writing tech anymore, you’re working with people who are writing code.

Shane Hastie: What is a good engineering culture?

Good engineering culture [20:47]

Sophie Weston: I think it goes back to what I was saying at the start. It’s about understanding that it’s not just about churning out code, it’s about making it a good place to do that. But understanding that part of what makes good code isn’t actually about writing code. Again, it’s counterintuitive. It’s somewhere that has space for learning and space for doing dull stuff like writing documentation. It’s little things. Kent Beck gave an amazing talk, he was talking about fix the little things, but he was talking about code, fix that broken test and do that bit of refactoring and don’t promise to come back and do it and then never do it. And I kind of think engineering culture also is all little things.

And DevOps was a little things, they used to look at the unicorns who were doing amazing things, it’s because they’d automated this little problem, fixed that little problem and automated this, and improved that test there. It’s always little things that make up the big things and you have to kind of pay attention to them. So making space to your documentation, making it matter. Having things like definition of ready and definition of done. Paying attention to how much work and progress you’ve got. Giving people time to have retros, giving people space to go to conferences. Like I said, it feels counterintuitive but then you’ll get better code and you’ll get better value and you’ll get people solving problems in better ways.

Shane Hastie: You touched on burnout. It’s endemic. How do we, I don’t know if we can prevent, but how do we mitigate, how do we reduce the level of burnout in our industry?

Sophie Weston: Doing some of the stuff I’ve just talked about, making it less about just getting stuff out the door. There’s always so much pressure. Making more time for people, I guess. Those engineering managers that we talked about, they need to have those skills as well about being able to recognize it, about having time to do proper one-to-ones to get to know the people they’re managing so that they can recognize the signs when it does start to happen and do something about it. Having it so that engineering managers aren’t themselves burnt out because they’re trying to manage too many people and they have their own, not quite feature factory, but whatever factory that you want to call that.

Shane Hastie: Sophie, a lot of great ideas, lots of food for thought here. If people want to continue the conversation, where do they find you?

Sophie Weston: They can find me on LinkedIn. And I recently made the jump to Bluesky, so you can find me there as well.

Shane Hastie: We’ll make sure we include those links. Thanks so much for taking the time to talk with us today.

Sophie Weston: You welcome. It’s been really enjoyable. Thank you for inviting me.

Mentioned: