00:00:02

Hasn't this been an amazing conference? I, right. This is really one of my favorite places to be. And as I was pulling together this talk, I was thinking back about the journey and where I've been. And the talk title I submitted is the key to high performance and what the data says, but I was going through all of it. I realized it's really how to become a high performer. And I was thinking back over the last few years, as Jean said, it's kind of been this four year journey. And I thought back to the first DevOps enterprise summit, and I stood on main stage and I had this quick 15 minute almost Ted talk about what we found in that first state of dev ops report. And we talked about what it meant to be a high performer and how we found that dev ops actually works and how the ability to develop and deliver software with both speed and stability could really drive organizational outcomes.

00:01:17

And by the way, this was kind of a big deal. I'll get to that in just a minute. Okay. So then 2015 rolled around and we really kind of expanded our study. So we talked about how lean management practices and principles drove excellence as well. In 2016, we branched out again, we really kind of shifted left again, talking about shifting left on security test data management, 20 17, 20 17. We did a lot, but in 2017, we expanded again into what organizational performance really meant because for some reason, profitability, productivity and market share just wasn't enough. We also wanted to know about effectiveness and efficiency and customer satisfaction. And as Jane mentioned, this research has been done with the core team at puppet, me and Jean and Jess humble.

00:02:16

Also the group of puppet, as you mentioned. So this journey has also taken us other places, right? So we have some white papers talking about forecasting, the value of DevOps transformations, which, you know, some of us call ROI who here cares about ROI or as an executive that cares about ROI that's for free, right? So you can go out on our website and it talks about how to access this. I've got a page at the end that lists a whole bunch of websites and stuff. We worked with teams like capital one. We have this book, which if you voted, like really dig into it, we had a signing last night, but this really digs into the detail of the four years. But for people who don't want to dig into for years and want watch a video, instead I was thinking, this might be a great opportunity to meet a race through some of the highlights, but I'll warn you. This can't cover everything. So these are some of my favorites over the last four years. Here's a quick outline. So technology matters, maturity models don't work. It's fine. Trust me. I'll get to that one. Transformations, need technology and process and culture, also architecture and technology. You can accelerate your journey, leadership matters and then how you can help. So let's get started.

00:03:43

Okay. First of all, technology matters, which I'm sure everyone here is like, yeah, okay, Nicole, seriously. We know, but guess what? It didn't always matter. Or at least not in the way we thought it did or we want it to matter. So in 2003, Nicholas Carr wrote this paper called it doesn't matter. And the challenge is that he wrote it in Harvard business review. So all of our executives read this paper and then believed it and then decided it was a cost center who here's her this oh, so that sucks, right? Like this sucks really bad, but it was sort of true. I'll get back to that in just a second. So it was actually based on the last 10 or 20, the previous 10 or 20 or 30 or 40 years of research.

00:04:36

And so suddenly we had done this research in 2014, that was different. It was finding different things. And here's why we were finding that dev ops is good for technology. So we had to start with the definition. Okay. So dev ops is good for technology and software delivery performance. What do I mean by that? I mean, the ability to develop and deliver software with both speed and stability, we're in dev ops, right? So develop and deliver software with both speed and stability. This is how we measure it. Speed is deployment frequency or when the business demands that's important, right? Because now it's a business decision and not a technical constraint. Also lead time for changes, code commit to code deploy. This is the most, most predictable part of your pipeline. Now, stability, MTTR, and change fail rate. Now here's the really, really cool part. Definitely pay attention to this.

00:05:39

We find that software delivery performance is through putting stability and both are possible without trade-offs four years of data, four years of analysis, over 23,000 data points. They all moved together. High-performers I do this analysis. It's called cluster analysis. They all cluster, they all cluster together. And then I see a gap and then the medium performers all clustered together. And then I see a gap. And then there's a group that all kind of sucks. That's fine. And I'm so there's three groups and I am super creative. So I call them high, medium, and low performers, but they all grouped together there aren't trade-offs I don't see it. So anyone who's been like telling you that in order to go fast, you have to like sacrifice stability or, or in order to be stable, you have to sacrifice speed. It's not true. I don't see it anywhere in the data.

00:06:43

Also DevOps is good for organizations. So here's what we find high performers. Year over year are twice as likely to achieve or exceed their commercial goals. Again, profitability, productivity, and market share. I like money. Any organizations, anyone here works for an organization that likes money, profit driven, right? So this speaks to us, but it's interesting. After a few years, some people came back to us and they said, I work for a not-for-profit. I work for government. I work for some educational institutions, or I worked for a for-profit institution that has broader goals. We also care about other things. Other things matter to us. What about non-commercial goals? Even if I'm in the for-profit sector. So in the 2017 study, we added additional measures like non-commercial goals. And the crazy thing is we find this same two X multiplier high-performing organizations are twice as likely to achieve or exceed non-commercial goals, effectiveness, efficiency, customer satisfaction, by the way, the measures for these are drawn from the academic literature, highly, highly rigorous measures. Okay. For bonus points in the 2014 study, we had a large enough data set that I could actually pull from stock market data. And we find that high performers had a 50% higher market cap growth over the previous three years. So this is great. Okay. The ability to develop and deliver software technology is driving organizational performance.

00:08:26

Okay. The next one for me, maturity models don't work. They're not a great fit for us who here has played world of Warcraft. Okay. Who has a friend that's played world of Warcraft. That's fine. When world of Warcraft first came out early two thousands. This was the world. This was the land. It went to level 60, right? This was about good as you could get. So let's create a maturity model for world of Warcraft, right? This is sweet. So let's say like level 40 is good. Low 50 is rad level sixties. Amazing. I'm making this up. Let's roll with it. That's great. Fast forward a few years. And the landscape has changed the world. We have extra worlds. We have extra lands. You can now also get to level 110. There are extra tooling and technologies available. Does this sound like technology?

00:09:40

If we create maturity models, they go out of date too fast. This is a challenge. The technology landscape has changed in the last 10 years. Is this sounding familiar? There's new tooling, there's new technology. The competitive landscape has changed. If we rely on a maturity model that was created five years ago, 10 years ago, it's out of date. Also, if we reach that magical level four or magical level five, what happens when resources disappear? Those resources can be money. They can be time. What if their attention? What happens when our executives or leadership lose attention to our transformation? The same is true in the data that I've collected over. Only the last four years. Let's not even go back a decade. Let's go back four years. The year over year, data trends are drastically different. What is good enough in one year is completely out of date.

00:10:43

The next, if I had created a maturity model of software development delivery in 2014, it would have been wrong. And out of date in 2015, it would have been old in 2016 and it would have been old in 2017. I'm not doing it in 2018 either. So let's look at 2017 data. Okay? High-performing DevOps teams are more agile. We see 46 times more frequent code deployments. This is the difference between deploying code multiple times a day and once a week or less. And by the way, this is just typical performance. We still have people that are doing this much less. We also see 440 times faster lead time from code commit to code deploy. It's a difference between less than an hour and more than a week. How fast can you get those changes through your pipeline? This is how we can delight our customers, beat our competitors to market, or on the off chance you have no competitors. How you can keep with compliance and security changes. This is the power that this gives us. So let's meet, move over to stability. Our high-performing DevOps teams are also more reliable. We see 96 times faster, meantime to recover from downtime. So high performers are recovering and less than an hour instead of several days, y'all days don't raise your hand, who is down for days, don't raise your hand.

00:12:27

Slow is the new down right? Being down is tough. We also see that high-performing teams are one fifth is likely that their changes will fail. And when they do fail, they have a much slower downtime. They have a much smaller blast radius, and it's much, much easier to identify what those changes are because the changes that went through that pipeline are so much smaller. It's very, very easy to identify what went wrong. This is my favorite. I chart. Luckily we have very big screens here. We can see what the high performers are doing, what the medium performers are doing and what the low performers are doing in 2017. The thing to take away here is to notice that the high performers are really maximizing across all dimensions. Remember how I said there's? No, trade-offs I don't see trade-offs

00:13:34

So take a look, see where your teams are. And by the way, I do categorize this at the team level, because particularly in large organizations, different teams will perform at different levels. And we expect that, right? And we've, we've seen that. And we've heard that here across so many of our talks also, I'm going to mention this data that we've collected over all of these years covers organizations across all industries, all sizes around the world, subtext there's no excuse. I see all industries in high-performers. I also see all industries and low performers. You can do this. And dev ops is there to help. Okay? Next up transformations need technology and process and culture.

00:14:39

So how we've categorized. It is technical processes seen in continuous delivery management practices and capabilities seen in lean and agile, right? We've drawn from these movements and organizational culture and leadership. And again, we've seen that these drive organizational performance and technology performance, remember Nicholas car guy that said it doesn't matter. He sort of had a point, but here's why back then, back then eighties and nineties, his data really drew from the eighties and nineties. We used to buy technology, throw it in a closet, check a box and walk away. Does anyone remember that? None of us here are old enough for that, but maybe I'm remember that. Right? Totally. I'm fine. Fine. I remember that. The challenge is that when we do technology alone for technology sake, it's not really impactful. It's not super strategic, right? If we only do the technology and then walk away, it doesn't have these big impacts.

00:15:45

We only do technology. We need technology and process and culture to really get this strong synergy, because then we only have a point of parody we can keep up. Right? Of course we should be using technology because we shouldn't be doing things by hand. Don't do accounting by hand, don't be doing math by hand. Don't be doing HR systems by hand. It gives us a point of parody, but everyone else is buying computer systems. It doesn't get us a point of distinction. It doesn't get us ahead. Who here has maybe heard of buying DevOps in a box or the mistaken belief that dev ops is buying a new technology or tooling system. Right? That's getting us back to the eighties and nineties again. That's why it's not going to work. Dev ops is a challenge. Dev ops is hard, but that's also why suddenly we're seeing it deliver real value to organizations.

00:16:36

So always remember it's tech plus. Okay. So has anyone here heard cams, this original definition really kind of coined by Damon Edwards, John Willis at DevOps days, mountain view 2010. I'm also a fan of calms culture, automation, lean measurement, and sharing. I like that. Ellen there, the 2014 to 2017 state of DevOps reports has found this as well. That tech pluses, what drives performance gains technology plus lean management plus culture. Now, if you don't want to trust me, look at a very recent article by Besson. He comes from Boston university. I love this, a summary, a really accessible summary. It was just published in HBR Harvard business review. The title is the real reason superstar firms are pulling ahead and it's because of it combined with other things like management brand or IP. And it's that the strategic use of technology combined explains revenue and productivity gains more than MNA and entrepreneurship.

00:17:40

Again, it's tech plus. Okay. Next up architecture matters technology doesn't we find that low performers are more likely to be working on software developed by an outsourcing partner. They're more likely to be working on mainframe systems. We found this in the 2015 study, but guess what? It wasn't statistically significant. There was no significant correlation with performance from working on a mainframe system. There was also no statistically significant correlation for working on Greenfield and brownfield systems. Either Joel was correlated having a good architecture. If you have well-architected systems, it doesn't matter what type of platform you're working on. So in 2017, we dug into this. Here's what we find. If you can answer these questions positively, this is what counts can your team make? Large scale changes to the design of your systems without the permission of someone outside of your team or without, depending on other teams, can your team complete its work without fine grained communication and without fine-grained coordination, can your team deploy and release your product or service on demand independently of other services? Can your team do most of your testing on demand without requiring an integrated test environment and can your team perform deployments during normal business hours and with negligible downtime?

00:19:27

So the thing that is most important is having a loosely coupled architecture, both at the team level and at the technology level. It's interesting sounds a lot like Conway's law organizations, which design systems are constrained to produce designs, which are copies of the communication structures of those organizations.

00:19:57

Having smartly designed API APIs, having good service oriented architectures are good investments in organizations. And then it doesn't matter what type of architecture you have. We've seen. Great, great organizations have mainframes that are doing fantastic, fantastic work. Okay, next up, you can accelerate your journey. You can accelerate your transformation. And these are the key capabilities that really drive software delivery performance. They fall into four categories, tech and automation, process measurement and monitoring and culture. Quick, take a picture. These are the ones that we have seen are predictive of your ability to develop and deliver software with both speed and stability. This is really kind of the core of our research version control for all production artifacts, deployment automation, continuous integration, trunk based development, test automation, test data management, shifting left on security, continuous delivery, having a loosely coupled architecture that I just talked about and then architecting for empowered teams, process gathering and implementing customer feedback, working in small batches, having a lightweight change approval process and allowing teams to experiment.

00:21:15

So one example here, we did some work with capital one and they found that by focusing on two core things for them, it was trunk based development and streamlining that change approval process. They saw stunning improvements, right? In just two months, they saw a 20 X increase in the number of releases with zero increase in incident, moving on to measurement and monitoring visual management monitoring for business decisions, checking system health, proactively use of WIP limits and visualizations. These are all predictive again of that ability to develop and deliver software with both speed and stability. Now back to capital one, they have Hygieia, which is an open source solution to help communicate visualizations throughout teams across the organization.

00:22:10

Okay. Culture. I'm sure we've all heard that tech is easy and people are hard and culture super important. Culture is important. So Western organizational culture, which I'll get back to having a climate for learning. So learning is seen as an investment, not a cost, have a good collaboration among teams making work that's meaningful and transformational leadership. So Western organizational culture in particular has been a really strong indication and prediction for performance. And the reason that we chose this was because it had such strong, predictive ability among highly, highly complex work environments. And if anyone has ever seen me talk, you've probably heard me talk about this. Ron Westrum is actually a sociologist. He's a PhD and he studies high-risk complex work environments like healthcare aviation. And we see that this really kind of maps to what we mean when we talk about a dev ops culture, right?

00:23:14

We tend to talk about doing blameless postmortems. So failure leads to inquiry. We talk about breaking down silos and talking to people in other groups. So bridging is encouraged. We talk about tackling problems and working out novel solutions and innovating. So novelty implemented. Um, so when we implement this and when we tested this, we found that this was highly correlated with each of our performance profiles. So the pathological group ends up being the most highly correlated with low performance. Bureaucratic is most highly correlated with medium performance and generative. The generative group is most highly correlated with high performance. We also see that this is highly predictive of both software delivery performance and organizational performance. So investing in creating a good high trust culture is a great way to impact your organization.

00:24:13

Okay. The question I often get here is where should I start and the right answer. I'm sure we all know is it depends. Everyone's different, but based on the data that I see, here's a couple like hints or cliff notes based on the 2017 data architecture was the highest contributor can to continuous delivery. So I actually kind of dedicated that whole section to having a loosely coupled architecture based on the Dora assessments that we do. We also see that loosely coupled architecture and a trunk based development ends up being a constraint for so many teams that we work with. So that's probably going to be a great place to start looking probably number one or number two biggest constraint for most teams is a lightweight change approval process. So always look there, right? Do you have a lot of handoffs? Do you end up losing a lot of time there, check it out. The last one is probably continuous integration, right? And not full compliment. Also pro tip go dig into continuous integration and what it actually means, not what you have maybe defined continuous integration. To me, turns out almost everyone has their own definition for CEI.

00:25:31

So what do you do here to improve, identify your constraints, figure out what's slowing you down. What's your bottleneck. Pick a few, be aware of death by initiative. There are a handful of companies I talked to that have 50 initiatives, whoa, pick a few work to eliminate those constraints. And then reevaluate. Once you've eliminated four or five, number six is probably no longer. Number one. We work in complex systems. Once we've eliminated a few, everything will reshuffle then rinse and repeat. Okay, next up leadership matters. We dug into transformational leadership in 2017 and we selected this model because this is the one that is the strongest, most strongly predictive. This is born out in a lot of the research. It's a five dimensional model it's based on intellectual stimulation, inspirational communication, supportive leadership, personal recognition, and vision.

00:26:34

And what we find is that transformational leaders are really able to drive support and amplify the work that happens in organizations. We also see that leadership is necessary, but not sufficient teams with the least transformational leaders that bottom third, we're only one half as likely to be a high performer, but leaders can't do it alone teams at the very, very, very top performed no better than the median. So what does it actually look like? This was the research model for 2017. I won't walk you through the whole thing, but anytime you see an arrow, you can read that as a prediction relationship. We go well beyond correlation into prediction, but look at where transformational leadership lies. That doesn't mean it's the most important thing, but it does mean that it supports and it underlies and it sort of amplifies everything else that happens. Look at how important we all are in here. We have the ability to drive and support all of that technical work in test and deployment automation, continuous integration, trunk based development, shifting left on security, that whole technical stack. We also have the ability to drive and support lean product management. And then look at all of the arrows that come after that work, after that comes it performance, a decrease in deployment plane, and then organizational performance. We support and amplify all of that work.

00:28:15

We're at the core. So good practices and smart investment make the work better too. Okay. The work, what do I mean? The work less deployment pain, less burnout, higher employee net promoter score who hears hiring, whose whose organization is hiring. Okay. Every hand, right? We're all hiring employees and high performing organizations are 2.2 times more likely to recommend their organization as a great place to work. And other research coming out of HBR shows that high E NPS scores that employee net promoter score drives revenue as well. The investments are worth it. Okay. You can help be the transformational owners or sorry, transformational leaders own this. We can really make a big difference. Start your own measurement journey, which now someone might say weight measurement. She hasn't been talking about measurement. We don't know where we are unless we measure a few things so we can track where we are on our journey.

00:29:28

Start with five things, make sure you focus a little bit on outcomes, maybe software delivery, performance, that ability to develop and deliver software with speed and stability. Those were outlined earlier, the slides will be up. And then also thinking about driving performance improvements tech, for sure, but also think about things that are not tech, something that's process something that's culture, and then share your stories, share with each other. For sure. This community is super important and then reach out, share it with me. Let me know what's working. Let me know what's not working. Let me know what your headaches are. It's time to design the 2018 state of DevOps report and state of dev ops survey. I'm dying to know what's happening. I always love to know what's happening for more information, you can always reach out to our website, dev stash, research.com. We have white papers. We have case studies. We have links to all of the research. I just went through some of my favorites because Gina only gave me 30 minutes. There's so much there. Thank you so much for all of your time.