Building Engineering Culture Through Autonomy and Ownership

Transcript

Shane Hastie: Good day, folks. This is Shane Hastie for the InfoQ Engineering Culture podcast. Today, I’m sitting down with Marcos Arribas. Marcos, welcome. Thank you very much for taking the time to talk to us.

Marcos Arribas: Thank you so much. I’m excited to be here.

Shane Hastie: My normal starting point with these conversations is, who’s Marcos?

Introductions [00:41]

Marcos Arribas: I lead engineering here at Statsig. We’re a currently series C company. I’ve been here since we started about four and a half years ago. We started with eight people, all from Facebook, and so I’ve seen the whole journey from the beginning. I’ve been the lead for engineering since the start. Not many people to lead at the beginning, but engineering has grown to around 40 people now. So, definitely learned a lot of things along the way.

A lot of my growth has been in leading the team but also establishing a lot of technical direction since the beginning, no customers, no infra, no nothing at all. And so, needed to pretty quickly figure out how to build all these things, scale pretty quickly with all the customers we were getting. Not much time to learn these things, but we did pretty well.

Shane Hastie: What’s it like to grow and maintain the culture when you’re growing from starting from eight to what you’re now 40, and I’m assuming planning to grow further?

Maintaining Culture Through Hypergrowth [01:58]

Marcos Arribas: I think we got pretty lucky, I guess. So we started with eight people, all eight of us worked together at Facebook and so we kind of carried over that culture of Facebook that we all appreciated. Made sure to take that culture but then continue building it in the way that we thought was right.

A lot of the initial people we’ve hired was also from Facebook, and so it was relatively easy at the beginning to maintain the culture, we were all speaking the exact same language, and so pretty easy to maintain that. But you can’t just keep hiring the people that you know, you got to start hiring other people, and I think that’s a pretty easy way for companies, startups to fail or fail to really scale.

It’s just like once you start bringing in people that aren’t speaking your language, people that are from different company, a lot of times it’s easy to just steamroll them and say, “Your culture is wrong, our culture is right.” I think the really important thing is, how do we adapt as we grow? And so, just because coming from a different background, doesn’t mean your opinions are wrong. So figuring out, what are the right ways to maintain that has been pretty important.

Shane Hastie: So, what is the core of a good culture?

Autonomy at the Core of Good Engineering Culture [03:00]

Marcos Arribas: I think an important part about it is the autonomy that people get. The CEO, the leads for a company, for teams, will set the top-level direction of what we should go and do. But at the end of the day, it’s the engineers, the individual contributors that are actually going and doing these things. A lot of times the top-level direction you get is good, but you’re going to have to change direction quickly.

What you thought was the right thing to do might not end up being the right thing to do. And so, the individual contributor is actually building the product, working with customers day to day, are going to know in a much deeper way how the product actually works, what is working, what is not working. And so, they need the autonomy to make decisions without always needing to go back to leadership and go like, “Hey, this isn’t working. I need to change it.”

People should be able to just go and make the changes they think are right. For me personally, I don’t know if this is important for engineering, but it’s important for me, is enjoying the people I work with. I’ve worked with a lot of people in the past that are extremely talented but not enjoyable to work with. I do not want to work in a company where my engineers are brilliant but suck to work with, and so it’s important for everyone that we hire is next to each other, they’re fun to collaborate with, respectful.

I think that’s extremely important to grow. As you grow and you keep hiring more people, you definitely have to adapt to that kind of thing of, what jokes are appropriate to make? How do you actually interact with people? You need to learn how to grow that as well. I think those are the main things I can think of.

Shane Hastie: So in that autonomous space, how do you avoid anarchy?

Balancing Autonomy with Structure [04:36]

Marcos Arribas: I think part of it is, you do need very frequent check-ins. Depending on the level of what an engineer working on a project is, you need more or less check-ins. So if an engineer that you’re giving a project to is a new grad engineer, you probably should be checking in on them every single day. Check to make sure they’re not getting stuck technically.

Make sure that they’re not just doing the completely wrong thing. If the engineer you’re working with is a senior engineer, like a staff engineer, sometimes I think some anarchy is good. I think a very important part about it is, especially for more senior engineers, the only way they can grow is if they can make mistakes, if they have the ability to choose the wrong thing to do.

And so, maybe you let them have a little bit of anarchy for a few weeks and then you check in and go like, “Hey, this is the wrong thing to do. We should probably correct this a bit.” But I think a little bit of anarchy is not too bad, as long as overall you’re driving in the right direction, the turns don’t have to be perfect.

Shane Hastie: This brings us close to the, move fast and break things. How do we move fast and not break things?

Moving Fast Without Breaking Things [05:44]

Marcos Arribas: One thing that Statsig provides is feature gating. A lot of companies use feature flagging for product development and honestly, that is a pretty important thing for moving fast without breaking things. I think part of it is maybe moving fast and breaking things a little bit instead of breaking things a lot. If you’re building a new product, you’re building a new feature, it’s pretty unlikely you’re going to build it perfectly from the beginning.

You’re going to have a lot of assumptions you made that will be wrong once you actually launch it. There’s just so many things that you can’t possibly account for. So, really important thing is you build a new feature, you put it behind the flag, you start testing it first locally. All of your team is dog-fooding it, understanding, okay, for the common scenarios, for the uncommon scenarios, test it out, make sure things are working fine.

Then you start testing it for a group of users, maybe 1% of your users start getting this experience. You see, hey, are there any new regressions from this feature? Are there any issues? You definitely don’t want to go build this completely new feature, release it to everyone, and then hopefully it works, because it probably won’t work from the beginning. There’s things that you can do to mitigate that risk.

Of course, you can create tests, you can do all these kinds of things, but I think all these kinds of things really do is just mitigate the risk. Nothing will ever guarantee you don’t break things. And so, the really important thing to focus on is when things go wrong, how do you quickly address them?

Shane Hastie: That brings in, I’m going to say quality standards, things like observability testing and so forth. How do you define that? How do you set these guidelines in a way that your autonomous and hopefully aligned teams apply them?

Quality Standards That Match Product Maturity [07:30]

Marcos Arribas: It’s dependent on what the area the team owns, what the feature is, what the maturity level of the feature is. If you’re only building a product from zero one, there’s no users. Honestly, I think quality is not that important at the beginning. You should be prototyping things quickly. I think you should be okay writing code that you are okay to throw away in a few months.

There’s no way if you’re building a new infrastructure, the infrastructure that you have in mind from the beginning will work once you have 10X the users or 100X the users. So, I think a lot of times there’s no point in trying to make the perfect implementation, you want to just get something that works out there.

Once you actually start getting metrics, seeing how it works, you iterate on it and figure out, okay, these are the bottlenecks, these are the things I should be improving over time. Of course, you still want to have good design practices, good quality standards, but not focus so much on the perfection piece. But if you have a product that’s used by thousands of users, hundreds of thousands of users, you need to start becoming much more careful there, of depending on what their expectation is of the product, you can’t just launch new features and then they break constantly.

Also depending on, is this a consumer application, is this a business application? Consumers might be a little bit more okay with some bugs, your server crashing a few times. If businesses are relying on it, can’t really be as okay with that. Pretty quickly, I think teams will understand, hey, there are these people that are dependent on this feature on this product. I need to still move fast, I need to write new features, and what are the right ways to release that safely?

The right way to release safely isn’t, not release it because I’m scared to do it and not make sure I test every single possible edge case, because you’re not going to be able to enumerate every single edge case. If I’m releasing the feature, do I have the right metrics that I’m logging to understand is this working in production? Do I have the right way to roll it back in the case that the feature doesn’t work? Engrain this in the engineers and the teams the right way to build these things.

Shane Hastie: How do we train new people in these ways of thinking and working?

Training Engineers Through Leading by Example [09:41]

Marcos Arribas: The best way there is, is just leading by example. And so one benefit that we had was, a lot of people that joined, like Statsig for example, came from Facebook and there it’s exactly this kind of mentality.

At Facebook, every single feature that you build, you always have a flag behind it. I would even do, if I have a bug fix that I’m going out, I’m super confident in this bug fix, I would still put it behind the flag or a gate. There’s a little bit of overhead to out of a gate. Worst case, I release this bug fix and actually my bug fix introduces a new issue that’s even worse. And so, leading by example of, now we came over to Statsig, everyone that has joined from Facebook knew this kind of mentality, they understood the value of it.

As new engineers joined, they see, hey, my manager when they’re building new features, is building new things in these ways. Everyone is building things in these ways, and so it’s pretty clear that I should be building in these ways as well. When you join, you see everyone is doing it this way, I see the value of it, and so I’ll do it as well.

Shane Hastie: The lead by example.

Marcos Arribas: Yes.

Shane Hastie: We can’t talk today about engineering teams without talking about AI. How are you at Statsig incorporating the generative AI tools into your work, or are you?

Incorporate AI Tools Strategically [11:05]

Marcos Arribas: We definitely encourage people to use AI tools, like ChatGPT, Cursor, all these things that go and improve your productivity. I think a major thing that AI provides teams is, at least right now, AI can’t really build perfect products, perfect infrastructure, there’ll be a lot of issues if you just let it go rampant. But I think right now one thing that it does really well is help out with POCs, of I have an idea for a new feature, but I don’t really know how this new API works, how library works.

You can usually ask ChatGPT of, can you give me an example of using this library or this feature? And then much more quickly go and test it out versus you go to documentation, read it, take a few days to understand what’s going on. You can just apply it immediately, and so it’s much easier, much quicker for people to pick up new things, just try it out, see if things work, without huge amount of investment into super understanding things before they can actually start testing it out.

I think there’s definitely incremental benefits that people get with AI right now, on just increased productivity from better auto complete, maybe just automated test writing for unit tests, things like that. But I feel like speeding up things like POCs is one of the better outcomes I see right now.

Shane Hastie: One of the things that we’ve seen a fair amount of evidence of is that with the adoption of the generative AI tools and the coding level, pull requests have got bigger.

Keeping Pull Requests Small and Focused [12:42]

Marcos Arribas: One thing that we do there is also coming from Facebook, is we are big proponents of splitting diffs, of stacking diffs. Some people hate this because they feel it’s just a way to pad how many pull requests you’re putting out there and improve your performance review.

I think splitting diffs is actually pretty important in order to help that diff review process or that PR review process. If you release this huge PR with thousands of lines of code touching a bunch of different things, there’s no way anyone can really understand what’s going on in this pull request, and so any review you get on it will be pretty low quality.

One practice that we do encourage people here is you should be splitting up your pull requests into modules, into you are making one change, you make that change. Probably something like 50, 100 lines. If you need to iterate on that change, you stack it on top of that change, and that is a separate PR. You never just do a bunch of different things in one PR.

This kind of process definitely increases the quality of the review, because now when the reviewer is looking at this PR, they can see, hey, this diff is doing this one change. I can understand, what are the implications of this change, and appropriately review it, rather than needing to, in my mind, maintain, what are these 20 different things doing all at once?

Shane Hastie: So, encouraging that good practice of smaller pull requests.

Marcos Arribas: Yes.

Shane Hastie: Something we’re hearing, and I’m not sure how much hype it is, but the junior developers, people coming into the industry are struggling to find roles because we’ve got AI for that now. How are you tackling that one?

Why Junior Engineers Still Matter [14:32]

Marcos Arribas: We still hire junior engineers. I feel it’s still pretty important to hire junior engineers. I don’t think AI can really completely replace that. There’s also the whole forward-looking, if you replace junior engineers right now, you’re not going to have senior engineers in the future.

You need to have people that will continue to grow, continue to be able to build a product in the future. And so, it’s really important to invest in people right now so that you have those people in the future. I think if you just think about it like, okay, I can have AI do some things that juniors do right now, and then that’s good enough, that’s very short-term thinking.

So, I think it’s critical to remember a year from now, two years from now, I need these people to still be here and actually be growing. There’s the perspective for senior engineers. A lot of things that seniors want to do is also do mentoring, have people that they can go and lead. You can’t do that if everyone you have is a senior engineer.

Senior engineers can’t learn how to lead if there’s no one to lead. Think from the whole perspective of continuing to grow the quality of engineers, it’s extremely important to have a pipeline of junior engineers keep joining, are able to learn. Senior engineers are able to mentor them so they can learn and you can make that more sustainable.

Shane Hastie: If you had a crystal ball, where are we headed as an industry in software engineering?

The Future of Software Engineering [15:54]

Marcos Arribas: Oh, that’s a good question. I mean, I feel like with AI and the way that is developing, it’s just continued to abstract away how development works of, at the beginning, you start having punch cards for programming. Then you have assembly.

Then you have higher level programming languages. I feel like AI is another higher level programming language of, instead of worrying about the exact implementation details of how you are implementing things, you start worrying about different things of, what are the right ways to structure the product? What are the right ways to make sure that AI is building things the right way?

And so I feel like a lot of the same principles of software engineering will still remain, but the exact implementation details will likely change. Just similar way of how engineering has evolved in the past.

Shane Hastie: And what about the merger or crossover in roles in engineering?

Marcos Arribas: I think we’ve already started seeing that. We have PMs here that are doing some coding, using AI, especially for things like POCs or things that don’t always require super in-depth knowledge, making it easier for people to prove out their concepts, prove out, this thing does work. A lot of times when you have a POC, then it’s easier to get investment into, yes, this feature does make sense.

So before, maybe if you’re a designer or PM you are saying, “These are the ideas I think we should be working on,” and maybe if you’re a designer, you have a Figma mock of what it looks like and people can understand a little bit. But if you’re able to go and actually produce a workable example of it, it’s much easier to start expressing that idea as well.

Shane Hastie: What’s the important question that I haven’t asked you today?

The Ownership Mentality That Drives Growth [17:41]

One thing that I’ve seen for engineers that grow fast, engineers that perform really well, is the ownership mentality of the product. Of a lot of engineers or people that are working on a product on a team are given a task and told, “Hey, can you go implement this?” And they just go and implement the one thing that they’re told to do.

I think it’s pretty important of a concept for people to understand, if you’re working on a product, if you’re working at a company, on a team, you should feel like you are responsible for that team, for that product, for that company. If you have that feeling of responsibility, the actions that you take will probably change because of it. Rather than going, someone told me to do this, implement this feature, maybe implement it in this way.

You start thinking, is this the right thing to implement? Is this the right way to implement? Should I be doing other things instead? If people just think about, I am only supposed to do this one task, all those questions go away, all the optimizations based on that kind of thinking just go away.

For a lot of engineers I’ve seen that have grown really quickly, they take this mentality to heart, and so if they see a bug in the code as they’re writing it for a feature, they’ll go and fix that bug or at least post like, “Hey, I noticed this bug. Can someone go and take a look?” They’ll say like, “Hey, for this feature that someone else built, I feel like if we did X, Y, or Z, it would be better,” and then they just go and implement X, Y, or Z.

A lot of times this gives them opportunity to understand how things in the code base or in different products in the company work, rather than limited understanding of what they have been working on.

If you’re able to understand how things work across the company, rather than just a siloed scope, as you start growing and working on more products, it’s usually easier for you to continue to keep making changes.

If you only work on one siloed piece of the code base or product, and then you’re asked to go and work on other things in the future, you’re always going to have a ramp-up time.

If you’re constantly growing and expanding the things that you are familiar with, it’s much easier for you to go and implement those changes.

Shane Hastie: Marcos, a lot of really interesting and insightful ideas there. If people want to continue the conversation, where do I find you?

Marcos Arribas: Yes, the best way, probably to reach out on LinkedIn.

Shane Hastie: Well, thank you so much for taking the time to talk to us today.

Marcos Arribas: Thank you so much.

Mentioned: