Transcript
Fabien Deshayes: What are we going to talk about? My name is Fabien Deshayes. I’m an engineering manager at Monzo. If you’re from the UK, you have probably heard of Monzo. Basically, we’re a digital bank that operates here. We have more than 11 million customers. We’ve achieved that with a fairly small engineering organization. We’re roughly around 400, I think 412 is the last time I checked, is the number of engineers at Monzo. You’d think it’s fairly small for a bank, especially with 11 million customers. I’d like to think of us as not a bank, actually, but a tech company that happens to run a bank. It’s really about how I think and reason about Monzo.
Monzo’s Dev Velocity Team
I want to talk to you about a DevEx team that we’ve built at Monzo, and what that team has achieved. In December 2023, we formed the Developer Velocity squad, these guys on that screen. It’s roughly a team of three engineers, plus an engineering manager. This team has worked on something called the Experimentation Platform. What’s the Experimentation Platform? I’m going to give you a simple example. This is a screen of a feature we called roundups. Every time you make a payment, it basically rounds up your payment to the highest pound, and puts that difference into an investment pot. It’s very simple, it’s pretty cool. Makes all of your numbers round, and makes it very easy to reason around. What we did for that experiment was have two versions.
On the left-hand side, our control version, which is basically explain what that feature does. On the right-hand side, it adds a little graph that says, based on the payments you’ve made over the past year, you would have put that much amount of money on an investment pot, and after 10 years, this would have grown to that much money. It’s telling a story about what will happen with your money. What we did was present one of these two variations to our customers, and the experiment is basically trying to figure out which one will get the most conversion. How many people will click on that create investment pot button at the bottom? That’s basically the experimentation platform.
What has the team achieved? Initially, we had an old in-house experimentation platform, which was ok, but not great, especially it was really hard to do data analysis on top of it. We’ve replaced that in seven months, and as a result, we doubled the number of experiments that we run in the company, from 70 to 134, looking in 2024 to 2025 for the first quarter. We’ve also halved the time to decision. Before, it would take 43 days to run an experiment and get to a decision and trying to know which option is the best. Now, it’s only 21 days, and it cannot be really much less than that, because you still need a little bit of time to exposition and to gather enough data to make your good decision.
Assemble Your DevEx Team
How did we pull that off? That’s what I’m going to talk to you about today. I think there’s three elements that are part of how we made this a success in such a little time. First is being able to assemble a great dev team, so I’m going to talk about that. We built very impactful products, and here I’m going to insist on the product side, and communicate about the impact of your DevEx products. Let’s dig into the first of them, assemble your DevEx team. When I did put together my slideshow, I thought, my team is a team of superheroes. I’m just going to call them the DEVEX-MEN. I thought that was a very nice thing.
Then I realized that it’s not very inclusive and it’s not very true. It needs to be diverse, and also the engineers are not superheroes. They’re just like engineers you find in any company. We just manage to get the right skills in the right team, working the right way. I want to demystify the fact that you’ve got some ninja engineer or top star or rockstar. That’s not the case. You don’t need these people in order to build a great developer experience. You just need the right people. First, I think you need to find engineers who have some product acumen. That’s basically the ability to make good judgments and make some quick decisions.
Basically, you have to think of other engineers in the company as your customers. They’re going to be the ones that are going to use your product. Really think about them as customers. You also need to be able to make quick tradeoffs. You need to move fast. It’s not because you’re building an internal tool and you don’t have a hard deadline because some marketing team said by that date we’re going to release this new feature. It’s not because of that, that you shouldn’t move quickly and iterate fast. You still have people that are going to look at what you’re doing, and say, what have you done last half, last quarter? You need to be able to think about how you move fast and make tradeoff decisions.
Finally, you need to be able to make this decision and think about long-term. It might not be intuitive when you’re working on internal developer tools because you just want to test something or hack on the side. If it’s successful, it’s going to be here for like 10 years, 15 years. You really need to think about these decisions that are going to be potentially reversible or potentially irreversible. If they are irreversible, you better think really hard about them and you better have a good argument about why you took this decision instead of another. Really think of your platform as a product and all the consequence that comes with it.
The second thing that I think is super important in a DevEx team is to identify engineers who have tenure in the company. Why is that? Usually, if you’ve been in the company for some years, you probably have a little bit of a network with other engineers that you’ve worked in the past. You’ve joined the team, people have moved on to another team. You’ve done the same, maybe two, three teams over a span of a few years. That can give you free user research for your DevEx team. Again, thinking about you’re building a product and you need to learn about your customers.
The best way to learn about that is just to speak to them. If you know someone that knows an engineer that is facing a problem with their developer experience, this is fantastic, you just have to talk to them. Also, people with tenure are more likely to be a little bit more senior and they’ll be able to have a breadth of ambitions and ideas of what can be achieved in the company. Think about all of that and make sure that they bring all of these ideas with them to your team. I’ll add a caveat to that. You could have engineers which have been here for a very long time and maybe they were stuck at some point in one team that had this very specific problem. Then they come to your DevEx team, very opinionated, thinking, “We should solve that problem. I’ve seen it in action. It was really painful”. Maybe that problem was only happening in one team and not all the other teams. Really also think about the entirety of your engineering organization and not just the teams that your engineers have worked in previously.
Finally, soft skills. You’re building a product. You’re going to have to sell your product. I don’t think you’re going to get marketing teams working with you to sell your product. I don’t think you’ll even get one person to help with that. You’re going to have to do the selling yourself, and selling is hard. You just need to be able to talk about what you’re doing and convince people. If you manage to find people with great soft skills, bring them on. They’re going to be so useful.
One thing I wanted to add on that is it might be counterintuitive because you think, my engineers who have the best soft skills, the one that speaks really well and you can put them in front of an audience, they might be the one that are closer to your end customers. They might sit down with some user research sessions. They might be just talking to users. I think there’s also an argument for having them sitting in developer experience team, like platform, looking inward towards the company because without adoption and without people talking about your product, they’re just not going to succeed as a developer experience team.
What does this give us? Basically, three aspects of what would be an ideal person in a developer experience team, and ideally, you would have someone who’s bang on the middle. Don’t worry, you don’t have to have three, four engineers just in the middle. It’s going to be very rare if you find that magic combination.
For example, at Monzo, what we’ve done is we had someone I consider was bang in the middle. We had someone with great soft skills and someone with tenure. That mix worked really well. We managed to build a well-rounded platform and product, and see a lot of great adoption within the company. I also think it’s absolutely fine to have some people who are not within these circles. Some people may be less experienced or coming from a different background, as long as they’re surrounded with people with these skills because then they can grow, they can absorb, and they can move within these circles.
Build Impactful Products
You’ve got your perfect developer experience team, now it’s time to build, and build impactful products. Again, I’m going to insist and repeat, we’re talking here about products, not projects, not solutions. The first thing I think is important is to align with your company values. Why? This will make it resonate with your user. One of our company values at Monzo, I’m not sure if it’s a value in itself, but we keep talking about Monzo magic, how we make things magically simple. For example, you’re at the restaurant with some friends, you’re paying the bill, and then you want to split the bill. It’s as simple as a touch in a mobile app. We really want to make these small things very easy. We try to replicate that in our experimentation platform.
An example of what we’ve done is just try to make cleaning up experiments as magical and as simple as possible. If you worked in experimentation platform, or even feature toggles, it’s very easy to create new stuff, but it’s always a lot harder to think about cleaning up, removing the code, understanding if it’s safe to remove or not. Are we finished with our experiments or not? What we’ve done in our developer velocity team is built a little app, that little bot on Slack that just tells teams, you’ve got an experiment here that is ready to be cleaned up. How does it know it can be cleaned up? It’s looking at all the evaluation. It’s looking, is your experiment still evaluated, yes or no? If it’s not evaluated, probably can be cleaned up. If it’s always evaluated, but always returns the same value, it’s probably that we’ve made a decision to ship an experiment, as we say. Because of all these heuristics, we know that your experiment can be cleaned up. We’re sending a little nudge, great. That’s not super useful either. We still need to know like, what do I need to do to clean up? We’ve built this cleanup guide. There’s quite a lot of details here.
Some of it explain the reasoning around why we think you can clean up an experiment. The really important part is these things at the bottom, which is basically telling you, click on that link here, and it’s going to show you the code where your experiments is referenced. The only thing you need to do is click on that, delete the line of code, commit that. You don’t have to worry about anything else. That’s a great thing that we’ve done, is that as we are monitoring these experiments, we know that if it stops being evaluated, it means you’ve removed all of the code. After 30 days, we’re going to quarantine your experiment. It’s still going to be there, but just not going to be visible. Then we’re going to self-delete it after 90 days without activity.
At that point, we’re fairly safe. Nobody is going to use it. Nobody is going to call it. We’re going to archive it because we don’t want to delete it. We want to keep trace of everything. It’s super simple for you as an engineer. If you’re doing experimentation with our platform, you just have to read the guide, click on the link, delete that line of code, and the rest, not up to you. It’s going to be automated.
The next thing is about solving real problems. You need to make sure that the solution you’re going to create is adding a lot more value. You just don’t want to create yet another tool that engineers will have to adopt and spend time learning about, and they will have to switch from that to all the other tools. You don’t want to create yet another standard, for example. When we built the experimentation platform, we knew that it was a real pain for data scientists to analyze the result of experiments. We knew there was a big use case here to make things much simpler, and it was a big problem that we wanted to solve. What we’ve done is just replacing our old system, which was forcing you to jump between BigQuery and our custom UI and some debugging tools, to just one platform where you can have everything in one place. Find your champions. I think another very important one. You can build the best solution or the best product in the world, if you can’t talk about it and have people relaying the fact that your product is great in the company, you’re not going to get the impact that you should. Really think about leveraging your leaders. That’s the first thing.
By leaders, I mean it could be managers, head of technologies, VPs of engineering. Could be people in product. Could be people working in data. You just need to identify these people and get them on board so that they can echo your purpose. To me, that’s one of the best ways to get engagement is by having someone who’s going to be listened to just saying, “Have you seen this new experimentation platform? We should definitely adopt it”. How do you convince them to say that in front of an audience? I think the first thing is you need to talk to them. Understand what is their problem. What are the problems they are facing or they know that their organizations are facing? Solve it for them. I think a very good example would be, a leader at Monzo once asked me, I don’t know exactly why our time to pull request is so slow.
The only thing I did was pull a little dashboard that showed him that depending on where the engineers, which team they’re working on, and if they have colleagues in the same discipline in their same team or not, that would make quite a difference. I gave the dashboard. That way, you build some trust with that person. They know that you can act on the problems that you’re being given, and they’ll be more likely to echo your sentiments. Really try to build trust by delivering on projects that they care about. I think the second type of champion that you need to find and convince are your early adopters. Here, that might be where you’re going to leverage your tenured engineers that we talked about before. Get them to reach out to their network and identify who is feeling the most pain with the problem you’re trying to solve.
For example, in experimentation platform, there was a team who didn’t have the time to learn the platform, the old platform, set up an experiment, and then there was quite a lot of work to actually get to analyze and make decisions. They didn’t have the time to do that. What we did was just tell them, we’re building something new. It’s going to be great, it’s going to solve your problems, but are you happy to spend a little bit of time with us when the edges are a little bit rough? Then you’re going to build it with us. They were really happy and engaged with us and they gave us tons of feedback. We call them our early adopters or alpha adopters. These are the two types of champions that you should really leverage when building products.
Have and use your tech strategy. At Monzo, we’ve got a tech strategy for our platform organization, and that defines key elements about how we operate. I’m going to mention two of these elements. The first one is raising the bar on abstraction. I mentioned we’ve built a new platform, but that platform is built on a third party. We didn’t want just to give the third party to all of our engineers and say, yes, the tool is here, just feel free to use it. No, we built an abstraction on top of it so that we can add all of the logic that we don’t want these engineers to think about every time they create an experiment.
For example, things like, you should not be able to delete an experiment because we need to keep a trace of everything we’ve built and we need to be auditable. Also, we don’t want to allow engineers to programmatically create thousands of new experiments because then the cost will go really high and then we’re going to have some problems. All of these things we’re building directly within our API and within our interface. We’re not allowing our engineers to call directly the third party.
Then, the second part of our tech strategy that we’re following, and that’s probably going to sound familiar by now, is platform as a product. It’s really something that we care about, and we basically treat our engineers in the company as our users. No big surprise here. Meet your users where they are, even if it means going to the beach with a hat and a false beard. Do you know what tools your users are using day-to-day? Where are they spending their time? Is it in Slack? Is it in emails? Is it in Notion? Is it face-to-face in the office? Where are they spending time? Make sure that you make the extra effort to meet them where they are, because you want to reduce the cost or the effort for them to go to your product.
An example at Monzo, we’re a fairly Slack-heavy company. We do a lot of our day-to-day collaboration on Slack. That’s why we’ve built so many of these Slack bots that allows you to just send Slack nudges to teams or to individuals. You’ve seen that previously in one of our examples, but all of our other teams as well. Here in platform, we built a lot of tools that allows you to create new bots as well. It’s like a multiplying effect that allows us to leverage all of that many times. Super important to go where your users are. If you create one more channel, you’re just going to be lost.
This one is pretty straightforward, but gather early feedback, super important. It’s not because we’re building an internal tool that we shouldn’t iterate and gather early feedback. Again, what I said before is still true, that you need to find your early adopters and work with them. Just make sure that you build on top of that. You’ve got your nice feedback loop. One thing I will add is, think beyond engineering. You’re a developer experience team. Yes, that’s fine, you’re going to impact developers. Developers collaborate with product managers. They collaborate with designers. They collaborate with data scientists. If you talk to these data scientists, if you talk to designers, and you understand that the designer, developer workflow is broken, it takes a lot of time to synchronize or for them to be able to double check that the design matches the final result. Then there’s something broken, and there’s something that you, in a developer experience team, can do. The earlier you know about that, the better. Make sure you keep talking way beyond just engineering.
In our case, in the experimentation platform, we worked very closely with data scientists, just because there was a lot of data analysis to do. To the point that we had the data scientists working within the team for a few months, just to help us set up and make sure we were on track. That’s absolutely fine for a developer experience team. If anything, that’s better. Because too often, you would see a developer experience team that just cater for the most common use case, like the backend engineers in your company, and then the frontend engineers are on the side because there’s only 30 of them, or the web engineers. I think you really need to think across discipline and just go beyond the discipline and also go to the other discipline, like data and design. Iterate aggressively within reason. Make sure you align with your risk appetite. How much risk can you accept when you iterate on your product? You might think that because it’s an internal product, you can just put something in production, and even if it’s broken, it doesn’t matter. It’s just internal. Nobody is going to care about. You’re not going to have to talk to your customers about it anyway. I don’t think that’s true.
At Monzo, we are working in a regulated industry, so we need to be very careful about what we’re putting in production. If I put an experiment platform in production, people will start using it, we present some things to some users, and it results in payments being made that’s not the intention of the customer, we’re going to have big troubles. I need to be able to understand why this happened, and I need to be able to evidence why it happened. All of this to say, we understood our risk, and we understood what we could build. What we’ve done for our platform was look at the vertical slice of what can we do to make experimentation easier, and how can we iterate on top of that. What we did was just have a backend API. We had only one way to segment our customers. It was by user ID, so looking at the whole population.
If you want to think about segmentation, it’s about saying, I want an experiment that only targets people on the Android app, or I only want to target customers that are using one of our paid features. This we couldn’t do at the beginning of the experimentation platform. What we could do, though, was observe all of the logs, audit everything, and make sure we had redundancy. We could debug everything as well, there was logs for everything. All of this is the minimum things that we wanted to put together before we got our early adopters to use the platform in production. Again, it’s all about tradeoff, and all about the risks that you’re taking, how much are you willing to accept before you put things live.
Consider adoption. There’s the carrot, and there’s the stick. The carrot here is just about building a great product that everybody will want to adopt, because it’s super simple to use. The stick here is about your head of technology saying to everyone, you have to use this new tool by the end of April 2025, because we’re going to turn off everything, so you don’t have a choice. You might think that the carrot is the best, and you’d never want to have the stick. I would say that a mix of both can be useful sometimes, and sometimes you just don’t have a choice. You’ll have to make sure that you think about the adoption in both ways.
A little nudge from your head of technology saying this is the new platform, we would like to get people to adopt it, that’s nice. If they say something like, we want everyone to adopt it by that date, you’re probably going to have more effect. Really, the incentive to use the new platform is so that we can shut down that old platform which could be a cause of incidents, and which is costing us more money. There are incentives to use the stick and say, let’s move on. You also need to make sure that your carrot is very appetizing, and you really want to make sure that you build a great product. All of these are things you need to consider altogether. Finally, measure continuously. This is no surprise.
If you’re doing product development, you’re probably going to have some key metrics that you want to measure. I think it’s the same for developer experience team, and it’s probably even more important. We’ll talk a little bit about communicating around the impact of developer experience after. I think that links up really well. When we built our experimentation platform, in our initial discussions, we were wondering, what are the key metrics that we want to move? What are the things where we want to see the needle change? These were the two things I presented at the top. Number of experiments that we run every month, and the time it takes to get to decision. This came from discussions we had with our leaders back in January, December 2023, 2024. We started building a dashboard straight away. We had a very nice local dashboard, and we could just look at it from months to months, and we see the things moving. We had adoption of the new platform, and usage of the old platform going down as well. There was a nice mix, and that allowed us to tell a story, which is the topic of my third section.
Communicate About Impact
You’ve built a great product. Fantastic. It’s ready to be rolled out. It’s in production. Then, no one understands what it does. Nobody talks about it. It doesn’t get adopted. Why? Maybe you didn’t communicate enough about it. I’m talking about communicating about your DevEx product, but it’s mainly talking about the impact it can have. Also think about how you’re going to talk and convince people that you should be working on this. I just want to make a quick call-out to a company called DX. DX is a developer insights platform. It allows you to run a survey and gets you a ton of data. They’ve done a lot of thinking around measuring productivity and measuring the impact of developer experience. I would definitely recommend you to look at their website. They’ve also got a podcast. Their CTO, Laura Tacho, is doing a lot of conferences, and their CEO, Abi Noda, has this great podcast, as I mentioned. Just a quick call-out to say that a lot of what I’m going to talk about is influenced by some of the great work that they’ve done.
Why would you need to talk about developer experience? I think there’s three reasons why you need to talk about developer experience and the impact of developer experience. First, get buy-in from the company. If you start working on something, and then nobody understands what you’re doing, I do hope that you’re going to be asked to work on something else, because you really need to optimize your time. That’s one thing. I think the second part is, it will allow you to prioritize more efficiently what you’re doing. If you’re thinking about two potential projects that you’re going to work on, which one should you tackle first? If you don’t have a good idea of what the impact of these potential products is going to be, it’s going to be hard to make that decision. Early on, think about the potential impact of your products, and think about quantifying that, and how can you compare these two products that you might build, and then just go with the one that has the highest potential impact.
Finally, it’s just going to improve communication with leadership. Whenever you’re going to need to ask for a little bit of extra help, or if you need to talk about moving engineers from one team to another, if you need to put together a promotion package for one of your engineers, or if you need to ask for an extension on a certain deadline, you need to be able to communicate and talk about the impact of your product. All of this is very important to have a healthy organization. A healthy organization for your engineers as well. It’s not just for the company as a whole.
It’s all about impact. If you’ve been working in platform and developer experience, you’ve probably heard that word a lot. Let’s dig into the details. Let’s try to find, what are we talking about when we talk about impact? In my brain, I think about impact in two ways. It’s like the very straightforward, easy to understand, and easy to measure impact. Then there’s the other things, the messy part. Let’s unwrap all of that. A very straightforward one is time. Time is basically, you are working on a selective test. Maybe you’re using dev velocity. That will reduce the time it takes to run your test from four minutes to one minute every time you run your test. That’s three minutes saved every time you have to do that. Maybe every engineer is doing that 10 times a day, that’s 30 minutes per engineer. Maybe you have 100 engineers, so you’re saving 3,000 minutes every day. Roughly 3,000 minutes is two days. That’s a lot of time that you’re saving just by doing this small optimization from four minutes to one minute. The great thing with time is it’s very easy to understand. Just leverage that. Talk about time, if possible.
The second aspect, another very straightforward one, is money. If you’re optimizing a flow, if you have less data to store or if any operation that you’re making needs less computing, you’re going to save some money. Maybe you’re going to migrate to another provider for your experimentation platform, and you’re going to save £5,000 per month. This is very easy to quantify. I’m sure that’s a great way to convince people that what you’re working on is actually impacting the revenue of your company. Time and money are the two very easy base units you can think of when you talk about the impact of developer experience.
I think now there’s another category of that, which is a little bit more complicated. Let’s look at it. First one is probably the hardest one to unwrap, it’s productivity. By productivity, what do we mean? It’s like making things faster, but it’s not just fast, it’s also about quality. There are tons of things. I’m not here to talk about how to measure productivity. There are great frameworks that exist, like Core 4 or the SPACE framework, which are doing this really well. I would definitely encourage everyone to look at what’s existing to measure productivity. I think when you talk about productivity and you’re saying things like, my product is going to make other engineers more productive, make sure to anchor this into the life of your engineers, the day-to-day life. Maybe you’re optimizing part of their day-to-day workflow or the way they work within their IDE.
That’s something that people are going to reason around easily, and they’re going to be able to relate to as well. Like, my IDE, yes, an engineer probably has their IDE open all day, and they spend maybe 30%, 40% of their time actually playing with their IDE. It’s super important to talk about productivity in the context of the engineer experience. A good way to talk about productivity as well is by combining sentiment data, so survey data, with some hard data that you might have from your sources, like GitHub. Number of pull requests that you make per month. We don’t really want to talk about lines of code, but that could be something you might want to measure. You don’t.
In our experimentation platform, we talked about productivity by really talking about the number of experiments we were doing and the time it takes to get to a decision. We’re saying that because you’re spending half the time to get to a decision, you’re going to get to better product outcome faster, and that’s going to give us an edge in being able to iterate on our overall product faster. How do we go from experimentation platform and just being able to be a little bit faster to how it leads to better company outcomes?
The next thing you want to talk about is adoption. Adoption means people are using your solution. It’s important to talk about it, because I think intrinsically it means that your product is good. Unless you’ve used a stick and everybody is adopting it, but then you probably don’t want to talk about adoption in that case. If you don’t enforce people to use it and people organically start using your product, that’s probably a good thing, and that reflects really well. It also gives an idea of the scale of the impact that you’re going to have. Because you could have built something that takes down the time to deploy from one hour to two minutes, but if it’s only for a certain type of deployment that you’re only doing twice a month, that’s not very impactful.
If you manage to say every team in the company is actually using that and they’re using it two times a month, that’s much better. Future enablement. I think this is a very tricky one, because not every product needs to have a direct impact on time or money or any of the things we’ve already talked about. Sometimes it’s just something that enables future things to be built on top of it. A quick example. Maybe you want to start building metadata for your repository and all the projects and components that you have. Does it bring a lot of value in itself? Not really. It doesn’t make your teams faster. They have one more file to update whenever they’re making changes or moving something to another team. It’s just a little bit extra boilerplate. Not great.
If you think about the value it can bring in the future, this system can help you build some automated alerting. When a component is failing, you know which team is owning that component and you can alert them directly. Similarly, if you need to talk to an engineer about an API or an endpoint or a behavior in a certain part of your system, you just use that file to talk to them. Think about the future enablement and what you can do based on the product that you’re building as well.
Reducing risk. Reducing risk is tricky, because it’s always difficult to talk in negative terms. It’s not going to be as impactful to say I’m reducing risk, or there’s this not very nice thing that we have that we just want to hide or we just want to remove. That’s not great. You really want to talk about the positive aspect, like how you make things faster, how you make things safer. Try to think about how can you phrase this in a positive manner. I think an example would be, we used to run experiments with our old platform and we had three incidents that came because of that. It was difficult to debug. It took us a few hours to get to a resolution. We lost a lot of money because of that. Basically, being able to say we’re ensuring that we are not going to have these incidents anymore by promoting a new way of doing things. Always try to phrase this a little bit better or a little bit more on the positive side. A personal favorite of mine, reducing cognitive load.
On this one, I’m going to dig a little bit into more details. This is a very simplified representation of the day-to-day life of an engineer. Every circle is a tool that they’re going to use. Every arrow is a hop that they will need to do, which will add a little bit of cognitive load and just they have to switch context. Start from an issue tracker to an IDE, back and forth with your terminal, you’re ready, you shift to a PR. How can you impact that? The first is you can make this hop a little bit smaller, a little bit easier. Very straightforward. In your IDE, you’re raising a pull request. Maybe you just want to see that pull request. You want to just click on something and that opens that pull request directly.
Instead of having to open a new tab in your browser, go to github.com, click on your repository and then click on the open PRs, and then click on the PR. Having these little shortcuts is going to be a little bit impactful, but all of these will add up. Another way you can improve that is about removing the need to jump from one tool to another. That’s why the arrow is a little bit smaller and thinner. Basically, again, if you manage to do a lot of the work that you could get in your pull request, like linting, formatting errors, some warnings, if you manage to move that to your IDE and have this feedback loop within the IDE in real time, you’re going to save a lot of time and that’s going to be really impactful.
The final example, probably the more impactful, but probably the hardest one to get to, is about merging these tools together and reducing the number of context switching that an engineer has to do. An example here would just be maybe in your PR review, you also have the results of your test and the results of your build all within one tool and one page. Really what you’re trying to achieve here is to make sure that you don’t end up with a browser with 53 tabs open by the end of the day. Because the amount of context switching and the amount of effort it’s going to need is way too much. You really want to reduce that and remove all the noise so your engineers can focus on delivering better value.
Developer happiness. I’m really happy about that because it relates to what we discussed around joy and having happy engineers. It’s super important to understand the impact that your tool is going to have on your engineers. Happiness is one of the ways of doing that. I think you can definitely measure that using surveys and being able to look at the satisfaction certain users have doing experimentation, for example. This is something we measured as well. Not a metric that we wanted to move directly, but we measured that and we wanted to make sure that people were happy or were satisfied with experimenting at Monzo as well. Have some sentiment data is also a great complement to what you’re doing.
Finally, retaining talent. That’s a little bit hard to reason around. It’s very hard to directly correlate some of the developer experience product that you’re going to build with the ability to retain talent. In the long run, that’s going to be very impactful. If you have people exiting the company saying that they complain about a bad experience and that’s the main reason why they’re leaving, this is a great call to action to do something about it. Replacing engineers in a company is really expensive. You need to hire people. You need to train them. You need to do knowledge sharing. It’s a lot of time wasted. It’s a lot of budget. Actually, being able to build a product that just helps retaining talent within your engineering organization is a fantastic way to talk about the impact of your developer experience team.
Conclusion
We’re just going to recap what I’ve mentioned so far. First of all, you need to assemble your DevEx team. Look for engineers who have a good product acumen, who have some tenure in the company, and who have great soft skills to talk about your products. Build impactful products. Align with your company values, solve real problems, rely on your tech strategy, meet your users where they are, gather early feedback, iterate aggressively within reason, consider the adoption of your product, and measure continuously. Finally, know how to communicate about the impact of your products, whether it’s about time, whether it’s about money, productivity, adoption, future enablement, reducing risk, reducing cognitive load, happiness of developers, or retaining talents. I think if you have to remember one thing about that presentation, I would just say, engage with your company’s engineering culture. This will put you in the best position to greatly improve developer experience.
Questions and Answers
Participant 1: Developer experience is really a big and essential topic within any organization. You gave an example of the experimentation platform, which looks like an A/B testing tool in the end. Did you also get to build other products? I also see in your diagram like a developer portal. I think that also might fall on the developer experience team. Did you get to build other products in your suite of products that would actually improve on the developer experience? Also, maybe measuring tools to measure the impact in the end.
Fabien Deshayes: Basically, developer experience is a big topic in companies. Are we just building things like experimentation platform or other products? How does that fit? Also, do we build tools to measure the impact of our developer experience products? In the case of Monzo, yes, obviously we’re doing a lot of developer experience tools. For example, we’ve got a lot of CLI tools. It’s interesting because not every company is built the same. When I joined, I realized that we are a very CLI-heavy company. That fits really well with the day-to-day approach of how engineers are working.
One thing I loved about what you said is about suite of products. I think it’s super important to have consistency within all the DevEx products that you’re building. Whether it’s having a CLI for everything or having Slack notification that are similar for every product, for example. When it comes to measuring the impact of that, we have an insights platform in the company that helps us measure things like productivity of engineers around certain metrics, like the Core 4 metrics. Looking at satisfaction, looking at productivity, number of PRs that are being raised. We’re also looking at other potential metrics. I think we can do a lot better on that. We’re getting there.
Participant 2: Can you talk a bit about how you got buy-in from Monzo to build the developer velocity team in the first place, without tools already to talk about the impact that they gave?
Fabien Deshayes: Basically, we had a very senior engineer in the company that really understood that there was a need for developer velocity. What they did was just talking a lot to our VP of engineering and try to understand and build a case as to why there needs to be a team. We had initially one problem, which was that experimentation platform. You don’t build a team just to build one platform. You build a team to solve problems in the longer term.
I think what that engineer managed to do and did to convince people and actually to hire me, was really to just portray the need for the engineering organization getting to a scale where it was necessary to have a sustained investment into developer experience. I think the scale here is important. Because I think when you’re a small team, you can just do a little bit of developer experience, doing a hackathon. You can do some improvements here and there. We were really reaching a scale where we were in the hundreds of engineers, where we needed some sustained investment. I think that’s part of the talk that convinced people to say, yes, let’s have three engineers and an EM working on that.
Participant 3: What you showed us looks a lot like product management and customer satisfaction. Did you get any insight or is there any correlation between your work at Monzo and the work other people do for the customer management?
Fabien Deshayes: Basically, your question is about we’re doing some work which is very much product management and are we talking with the product side of Monzo, and is there any sharing happening? One thing actually at Monzo which is very interesting in that in our platform organization, we don’t have a product manager. In a way that forces us to have this product mindset that we probably wouldn’t have if we were working directly with a product manager. I think that’s one thing.
The other thing I would say, it relates back to what I said about aligning with your company values. We’re really keen on things like early feedback and iterating and offering things to our users as early as possible. We do a lot of A/B testing for that as well. I think that’s something we try to replicate in our platform organization as well. I actually don’t see a lot of difference in the way we operate, we’re just really mimicking what they’re doing but we’re adapting it to the context of what we’re doing. Obviously, different users, different customers, and no product managers in our case.
Participant 4: I’m assuming you are using the same language and tech stack across the company, and how would it work if you have the teams on different stacks, different generations, talking in the technological sense?
Fabien Deshayes: The question is about there’s an assumption that at Monzo we have a unified tech stack, let’s say, which is true. We do a lot of Go, for example. How would that evolve if we start adopting more tools and different things? I think it’s super important to be able to find the rationale to keep using what you would call golden path, for example, or golden standard, golden technologies, which is basically saying that there’s one set of technologies that we encourage everyone to use.
Then build your tools for that use case. It’s not always going to be possible. I think there might be time when you need to switch your approach and say, we’re going to support more use cases. I think there’s a lot of value in being able to say 90% of the time you should use the golden standard or the golden technology and just rely on that. At a very large scale and in companies that have been here for 100 years I know it’s not always possible. Then what I would say here is try to identify where you can maximize your impact. Maybe you need to think about the layer where you need to work on, so whether it’s closer to the data or maybe closer to the frontend. Also, maybe just try to encourage people to converge toward a certain technology if possible. Otherwise, it’s going to be messy.
See more presentations with transcripts
