Las Vegas 2023

DevEx Essentials: Igniting Change, Delivering Results

Ever wondered about Developer Experience (DevEx) and how it can truly impact your work? Join us for a chat where we break down what DevEx is and why it's relevant for everyone, not just devs. DevsOps and productivity expert Dr. Nicole Forsgren will reveal how DevEx can ignite cultural change and deliver real results in today's rapid software development landscape, backed by the latest research findings. She will share how Microsoft is leveraging a DevEx perspective to drive cultural shifts and enable AI-powered innovations that make expertise available to all teams. It's not just about code; it's about fostering better vibes and achieving outstanding outcomes for all. Don't miss out on this journey into the magic of DevEx.

NF

Nicole Forsgren

Partner, Microsoft Research

Transcript

00:00:00

<silence>

00:00:12

Uh, this morning I described the, uh, o Brian Eno's concept of cs, uh, and how one of the characteristics of a productive and great CS is a rapid dissemination of tools and techniques. And so without a doubt, I think one of the tools, uh, and techniques that have been spread most rapidly throughout this community are the DORA metrics. So the, uh, deployment frequency, deployment, lead time change, success rate, and meantime prepare. So these metrics, of course, come from the state of DevOps research that I had the privilege collaborating, uh, with, uh, Dr. Nicole Forsgren, who was the lead researcher for five years. Uh, we worked with Judge Humble, and again, without a doubt, uh, this is the professional achievement that I'm most proud of. So she's the lead, uh, lead author of the Accelerate Book, uh, which I routinely cite. She's a co-author of on the, uh, second edition of the DevOps Handbook.

00:00:54

So, uh, Dr. Forsgren was the CEO of Dora for six years, which was sold to, uh, Google Cloud in 2018. She was, uh, later the VP of research for GitHub, um, and where she contin and she's now a partner at Microsoft Research where she continues her fantastic work, which includes the space Deliverer productivity framework and so much more. And so here's a fun fact that Space Paper is now the, one of the most cited articles in ACMs history. So I'm so delighted that Nicole is back here at DevOps Enterprise sharing what she's been working on lately. Here's Nicole.

00:01:39

Hi everyone. Thank you so much for being here today. Um, today I'll be talking a little bit about DevX and developer experience, and I'm not sure if anyone has heard, but you know, apparently DevX is a little bit of hotness. So for those of us who are kind of in the DevX space, so DevX is short for developer experience and it's kind of had this resurgence. So it's kind of some hotness. Is anyone in Dev X doing some Dev X, like three of us? Cool. Um, I don't know if anyone, for those of us not in Dev x, I don't know if anyone's heard of AI or LLMs. I don't know if you've heard. That's kind of the hotness. So here's, here's, here's the dirt. It can be the hotness, but you can't actually ship your generative AI enabled programs unless you can actually ship software.

00:02:34

So we come back to where DevX is the hotness. Welcome to my talk. So today we'll be chatting about, uh, DevX, what it is and why we should care. Um, I've got a little bit of research to drop some brand new stuff, and then I'll be talking about another reason we should care about DevX because you can actually use this as a lens to shift culture because, um, I don't know if anyone else has ever hit this, but sometimes it can be a little tough to shift culture to do this whole like transformation or like get everyone aligned with change. And if not, then um, please come see me and probably a bunch of us, let's create a new birds of a feather session. Um, and then I'll wrap up chatting about DevX and AI and how we can really enable, um, a bunch of teams and some of the promises that are, uh, promising new tech that's, um, existing in this space.

00:03:32

So starting off with a little bit of 1 0 1 really, um, you know, what is DevX, right? So developer experience at its core really is kind of the satisfaction with tools and processes and what it takes to deliver code and what it takes to really even do our work as developers and technologists. What it is not is tooling. This is not just about tooling. Sounds a lot like DevOps, right? But what's a little bit different here is that we're really centering our users, right? Why should we even care? It's taking this really people and customer centered approach. And when we talk about dev DevX, it's that developers are our customers. So we can get some really valuable insights to the work that we're doing here and why it allows us to improve our systems in a way that can highlight the bottlenecks, it can highlight the blockers, it can really surface the things that allow us to improve and accelerate and amplify the work that needs to be done.

00:04:34

And many times this work is speed, sure, but it can also be reliability, it can be security, it can be compliance. And the thing that I love about this is that it's security and compliance in ways that are user friendly. Because then when we mean user, user-friendly, it's baked into the systems. So it's fast, so it's easy maybe. So it's invisible so that everyone really wants their systems and their software to be safe and secure and compliant instead of, oh, well, we'll just make them do it. I don't know if anyone's ever heard that before and how successful that is. It's not right. And so when we do this, what I like to call the right way, we find ways to make it delightful to do the right thing. Now, there are lots of ways to think about developer experience, but here is one way.

00:05:23

And so when I talk about this, here are a few dimensions to think about contributing to and having a good developer experience. One is flow state. We've all probably hit this in our work. So this isn't unique to developer experience, but when it's, it's when we have this work that we do where we can hit deep work, right? Where we can have this incredible, you know, ability to be fully immersed in our work, right? It's joyful even when it's incredibly challenging. Another one is having feedback loops that are both fast or the right fast, right? And they're high quality, right? I don't, I don't want fast feedback that's wrong. Um, and then the last piece here is cognitive load. Now this is how much mental processing is required for the task. We generally want this to be low, but I will mention that occasionally we want to indu introduce just a little bit of friction to help people think through things.

00:06:27

But in again, in general, we want to have low cognitive load. Now, we used these dimensions to frame this latest research that I'll be talking about today. Now, I will mention this data is hot off the presses, so I just have a teaser. Uh, the fall paper will be dropping end of the year, but I did wanna share some of what we were doing. Now, first question is why more research? Um, if people ask me this question, I will usually just stare at you blankly and say, because I love research, but it's worth asking. So I think there are a few things to think about here, because I will say it's a fair question, right? First of all, I sometimes joke here that it's nice, right? Good vines only. I love bringing this up in terms of DevOps DevX because there is this misunderstanding that developer experience is just about making people happy.

00:07:23

That's not true. Now, is it worthwhile? Absolutely. I love that Chuck just talked about engagement and how absolutely worthwhile engagement is, but there's also more to it than that. What else is there? There are impacts. And what do those, what do we mean when we talk about impacts and how do we get these impacts and for whom do these impacts fall? And so these were some of the grounding questions, motivations to doing some of these, this research, right? And by the way, the impacts are for a variety of stakeholders. We're talking about developers, we're talking about teams, and yes, we're talking about the organization. And I wanted to do some of this work because so often people are like, oh, well, and I by the way, preface, I hate this answer, but when they're like, oh, well it just makes my devs happy, first of all, you should care deeply about making your devs happy and making their work easier to do.

00:08:19

And there's so much more here, right? So when the question is, but why should I invest? Now we have some really good and really compelling answers, and these act, these answers fall into actionable insights that anyone can do. Um, and then so that anyone can kind of deploy some of this, uh, research or, you know, uh, quick surveys where you are, um, at the end of the year, we will be sharing all of the survey questions as well. So, uh, here's what we found really quickly. What is it that you actually get? What are these impacts for developers? We find we find that it boosts creativity, productivity, and the opportunity to learn at work. I don't know about you, but that sounds pretty dope. Um, for teams, we find better code quality and less technical debt. Now, for an organization, we see that it drives a culture for innovation, it improves retention and it improves their goals and their profit. I don't know about you, but I'm in Now how do we get there? Again, we followed this three dimension framework. So for flow state, now, how did we define flow state? By the way, this is a quick infographic. I will, I have all of it at the end. You can take a quick picture for Flow State. This was defined as fewer interruptions, deeper work and engaging tasks. Now

00:09:44

For deep work folks, were 50% more productive versus those without dedicated time. This is huge. This echoes some of our previous work that we found in Good Day project. For those that had more engaging work versus those that had kind of boring work, we see a 30% boost to productivity when we look at cognitive load. Now, how did we define cognitive load, intuitive processes, understandable code and easier deployment. Here we see a 40% increase in productivity for those with great code understandability. When we look at, uh, intuitive process, we see increased innovation to the tune of 50%. Now finally, when we come to feedback loops, fast feedback loops are fast responses for developer questions and code reviews. Here we see fast code review and turn turnaround times, see a 20% increase in innovation and fast responses to developer questions. See a 50% decrease in tech debt.

00:10:46

I think this is really interesting and really, really promising. Quick take a picture. Here's your infographic. I also have a Bitly leak at the end of the, uh, talk. Now we will have a full paper coming out at the end of the year, but I at least wanted to share some of our early findings because I think it's really exciting, not just for what it means for our developers and our teams and our organizations, but what it means for the software and for the experiences that we wanna put out. So listen, if anyone's only on this generative AI hype train, here's how to get that hype out the door. Okay? Now, when we want to change the culture, the thing I also love about developer experience is that we can use a DevX lens to help align people to this as well. And I'm really excited, incredibly fortunate that I've been able to work on some of this on a cross Microsoft effort over the past year. And this quote from Brian Chesky really sums it up well for one definition of culture is that it's simply a shared way of doing something with passion.

00:11:59

Now, folks who know me will not be surprised that I am now going to start talking about metrics.

00:12:07

Metrics can be incredibly powerful because they can be a communication tool. Data gives us wonderful opportunities to clarify and define what it is that we're talking about, regardless of the organization that we're in, regardless of. And by org I kind of mean like different business units, right? Regardless of the technology that we work on, it helps align and clarify. It also creates shared language among teams. Sometimes we find that we can be talking about something totally different and use the same word or talking about something that's exactly the same and use different words. This gives us an opportunity to align. The other thing I love about this is it gives us an opportunity to move from intuition and just shooting from the hip to use data informed insights. I also love data informed here, not data driven, because we probably don't have something that just like auto executes when a certain number pops up, I wanna surface insights and then let that guide decisions because we still have experts where we work. But I want this to be informed. So across Microsoft, we created something called Engineering Thrive. Uh, this is an actual example, uh, visual that we started sharing across the whole company. This is a handful of data points

00:13:31

That fall into broad categories of speed, ease, quality, and then Microsoft, uh, culture metrics that we use. This has presented a wonderful opportunity to surface conversations, invite engineering leaders to get curious, think about what's happening across their organization. Think about the trade-offs that are happening. Think about the strategies they will take to improve. Now also look at what's happening here.

00:14:11

It invites you to think about what's happening there. Now, yes, I have, I have redacted a few specific labels, but speedies and quality are durable. Something I want you to notice as well. There is no one metric. There will never be just one metric. There is not just one metric. It is a suite of metrics across durable categories. It really helps you think about what trade-offs happen. And by when I say trade-offs, it helps engineering leaders put themselves in the seat of the developers that are there, right? Again, this is a very strong developer experience framework. What are the trade-offs that the developers in their organization are making between speedies and quality and culture? What does that look like? What do these data points represent? By the way, some of these are objective metrics. They come from telemetry and instrumentation. Some of them are subjective metrics, they come from surveys and people, it's an explicit representation of this developer experience throughout the org.

00:15:11

Also, remember the visual that was there? They show these tradeoffs and constraints. We know that we work in a resource constrained world. And by constrained, I mean the answer isn't Give me more headcount, gimme more. I don't know about you. Does anyone else work in this environment? Can we, do we just get to ask for like the unlimited budget? I would love this world, right? But when we think about making strategic investments, what will these investments be? Where is the most strategic place to invest? First, what does this look like? Because the leaders know that this is the world in which we live. Also, look and notice that there were categories, speed, ease, quality, and culture. These will remain durable. This is also an explicit, and we've made this very explicit acknowledgement that the specific data points within will likely iterate as we continue to improve in our data access data fidelity, data quality data instrumentation. But the categories of speed, ease, and quality will continue. These exist in many companies. And then again, this creates a shared language and understanding for improvement for change. There are no comparisons. The only comparison is against yourself, your own basement, your own baseline, sorry, <laugh>.

00:16:36

Listen, if if it's not great, embrace your own baseline. And then what matters is improvement over time and what that graph looks like over time. So this uh, spider chart, let's call it a spider chart, it's not quite, that is paired with raw values. And then one of those columns includes comparison to last period and last periods. So like, you know, spark lines and stuff. And so that's what really matters and I love that that is the culture and the conversations that it brings. So this is leveraging kind of this dev X lens to embrace change, to motivate this understanding of how we improve. Now the other thing I love about this is it creates new opportunities to think about improving. Not just what you know, kind of surfacing these strategic pieces of what, but then thinking about how and thinking about how we can improve that, how faster and better and allowing everyone to do that. And so my team at MSR has been looking at how we could maybe use AI and large language models to allow everyone to do this. So, um, is any of this familiar? So as we sort of talked about, efficient infrastructure is really important for any of us who want to be able to ship software. Um, does anyone hit complexity that slow us down?

00:18:20

No. Maybe. Okay, perfect. Um, does anyone have situations where problem diagnosis is just like really hard or really complex and we need like 27 people in a room and then we can't schedule it on the calendar, right? Um, do we ever find that we have repeated solutions because two or three different teams didn't realize what we were all working on? The same problem? I'm seeing hands. Um, does it ever, does the complexity ever make it difficult to understand code and context? Oh, we all work together. We are all on the same team. Okay, what if you had a dream team? What if you could all call on any one of us in this room, any one of us at this conference just to come like sit next to you for a minute or an hour to help figure out and solve this problem? Oh, and by, so me, Courtney, Jim, Jason, Chuck, would that be great? Oh, and by the way, we happen to have already been working with you for a year. So we knew your code. I mean, I sign me up. Are we on for that? Okay. What if every single engineer and team in your company also had that dream team?

00:19:34

That was the

00:19:37

Question that my team had because once we know what we need to solve, we still actually have to solve it or like figure it out what our next steps are. So we wanted to come up with our personal l and m powered experts, consultant expert, guidance analysis, system analysis, tech lead. What if you could have a code-based tour stack trace insights? What if you need to figure out all of the things that are in your code that's not limited to a file, but it's the whole repo or a data scientist expert software engineering expertise? I want Steve McGill to come help me all the time. These are the types of questions you could ask

00:20:17

For consulting. How can I improve onboarding? Who else has improved build times in similar contexts? What's happening in my systems that I should know about? What patterns are there for a tech lead? How does authorization work in this code base? <laugh>, walk me through the build process in this repo. I got a lot of build questions. Listen, um, help me understand this staff trace for data scientists. Does distributed development affect code quality in my organization? How does build time affect developer satisfaction? Does branching strategy affect my PR time? Do any of these questions come up for any of you? Do these sound familiar at all? I'm seeing some hands. Super quick race. We built some prototypes that actually help with this. So, and if you want to come find me later, I can walk you through more detailed prototypes of this. But we have a consulting expert that will actually create an expert panel and come up with a ranked list of matches for you in a number of minutes versus weeks or months.

00:21:18

For the tech lead expert, you can ask it a question and it will select and summarize a mountain of context and it will give you a summary as well as a diagram and the right kind of diagram that links between the code. And then finally, for the analysis too, you can pick any paper that you want. It will surface the assumptions for you. It will create an analysis plan and it will generate the code for you. So what's next? Let us know what problems do you have that LLM experts could maybe help, um, co-innovate with us. We would love to create solutions with you. Um, watch for the latest research around end of the year and let's co-create the future of developer experience. Um, shout out to everyone who was involved in all of this work because it took a whole bunch of people to pull all of this off. So many, many thanks to all of them. And, um, thanks to everyone, please feel free to reach out and access the research. Thanks everyone.