Host Intro (Gene Kim)

[0:02] This morning, I had described Brian Eno's concept of genius and the characteristics that mark great geniuses, one of which is a rapid dissemination of tools and techniques.

[0:13] I also mentioned how the goal of science is to explain the most amount of observable phenomena with the fewest number of principles, confirm deeply held intuitions, and reveal surprising insights.

[0:22] Incidentally, I was just reminded backstage 20 minutes ago that this is what PhD candidates and philosophy majors refer to as the principle of parsimoniousness. So the next talk is a great embodiment of both genius and science.

[0:36] I've had the pleasure of working with Dr. Nicole Forsgren and Jez Humble on the State of DevOps Report for five years.

[0:41] It is without doubt one of the things I'm most professionally proud of, and it's difficult to overstate how much I've learned from that effort.

[0:47] I'm also delighted that both Nicole and Jez were able to commercialize this research into a company called the DevOps Research and Assessment Company, which we affectionately refer to as DORA, and that all the research and the underlying theory and methodology were put into the book "Accelerate: The Science of Lean Software and DevOps: Building and Scaling High-Performing Technology Organizations." It's a book that I routinely cite from.

[1:05] My favorite page is page 138, where it describes under what conditions you can assert prediction, not just correlation. Without further ado, Dr.

[1:12] Nicole Forsgren and Jez Humble. Thanks, everyone, for joining us.

Dr. Nicole Forsgren and Jez Humble

[1:19] We pulled a quick switch and decided to actually talk about if you don't know where you're going, it doesn't matter how fast you get there.

[1:34] So let's start by talking about getting better and measuring performance. So we'll start by outlining some common mistakes that we see.

[1:47] First, talking about outputs and outcomes, and what's really important here is focusing on outcomes, not just outputs. Also, individual and local measures versus team and global measures.

[2:05] And then we'll walk through some common examples really quickly. So the most common mistake we see here is lines of code.

[2:13] I'm sure we all know that this is a bad idea, right? I also get asked about this every single month.

[2:21] I can't blame anyone for this because it's really easy to measure, but it's really tough, right? More is better.

[2:29] Not necessarily. It gives us bloat software, gives us higher maintenance costs, higher cost of change.

[2:36] Okay. Well, on the inverse, less isn't necessarily better either, because all that can possibly lead to is one line of magical code that can do a million things, and then no one can actually read it, so no one can ever maintain it.

[2:56] The best thing that we can possibly do sometimes is to solve a business problem with the most efficient code possible.

[3:03] And actually, the best thing you can do is delete code and solve a problem. That's my favorite.

[3:07] That really is the very best. Delete code. But how are we going to measure that and reward someone for it?

[3:18] There is actually a story in the story of how the Mac was made about someone who deleted several tens of thousands of lines of code and put that in their weekly report about how much code they'd written. They were asked not to enter a number in that box again.

[3:32] I like it. Okay. Here's another example. Velocity. So who here is doing Agile?

[3:42] Okay. Who here has their velocity measured and compared to other teams?

[3:49] Okay. Sorry- I'm not saying you should feel bad.

[3:52] But your manager should feel bad. Yes.

[3:55] So this is a terrible idea. Velocity is a capacity planning tool. It's not a measure of productivity.

[4:01] And the idea of velocity is we'll see how much we can get done this week so we can not commit to too much work next week or however long your sprint is so we can make sure we can predict roughly when we're going to be done.

[4:15] But when you use that to compare teams, that's hugely problematic for a couple of reasons. Firstly, velocity is extremely context dependent, so you can't actually take a velocity in one team and compare it to another team.

[4:26] Secondly, if you're measured by your velocity and some other team asks you to help them with their task, but your manager is like, "Get your velocity story points this week," how likely are you to help them actually achieve their goals?

[4:39] Not very, I'd suggest. So measuring team velocity tends to cause people to not work very effectively together, and in an enterprise where we have dependencies between teams, that's a terrible idea because I might get my points, but the organization as a whole probably won't get what it wants. So for this reason, bad idea.

[5:00] Is this me too? Yeah.

[5:01] Utilization. So people love to measure utilization, especially CFOs, because they're paying a lot of money for their expensive resources and they want to know those people are busy.

[5:13] Hence my friend had a T-shirt that said, "Jesus is coming, look busy." That's your CFO.

[5:20] However, you don't actually want to have utilization at 100%. Results from queue theory and math show us that if a system is 100% utilized, you will experience the maximum possible lead times.

[5:34] What we actually want to optimize for is throughput, not utilization, and in order to achieve the highest throughput, utilization must be well below 100%.

[5:43] So to sum up, remember outcomes, not just output.

[5:48] Not lines of code. And global measures, not just local measures. So no matter how quickly we're improving performance, and I promise I'll give you an example quickly, where are we going?

[6:00] So here's one good example for performance metric that we have used. We've been using software development and delivery performance. It's an outcome measure, not an output measure. It measures how well we're developing and delivering code, and it also captures measures intention, right? This is also really nice.

[6:21] To quote my good friend Sam Guckenheimer, he says, "If you ever capture only one metric, you know exactly which metric will be gamed." So it's nice having measures intention.

[6:33] We have speed and stability here, both throughput and stability. Our speed measures are deployment frequency or when the business demands.

[6:41] Lead time for changes, code commit to code deploy. Notice, again, these are global, right?

[6:47] You have to collaborate with people all the way through that value stream. We also have time to recover and then change fail rate.

[6:55] These are our stability metrics. So I know everyone here is thinking this is nice, but what do we know when we compare our high and our low performers?

[7:04] So here's what we see in our 2018 study. Our highest performers see 46 times more frequent code deployments and over 2,500 times faster. When we talk about stability, we see 2,600 times faster time to recover from incidents and seven times lower change fail rate.

[7:26] Why do we care? It lets us beat our competitors to market. It helps us pivot when we need to pivot.

[7:32] It helps us get our features to our customers, whether our customers are end users, internal, external, if we're a commercial enterprise, if we're government or not-for-profit.

[7:41] This is all very important. Again, speed and stability. By the way, these go together.

[7:49] We don't see trade-offs, and this is true for five years of analysis and data.

[7:55] They go together. And if you take one thing away from this talk, please let it be that.

[8:03] Now, this is another thing that I really, really love.

[8:08] Okay, I'm going to change my mind. If you only take one thing away, let it be this. JK.

[8:14] Take a look at this high-performing group. In this year's data set, well, in all the data sets, we take a very data-driven approach. When I run the analysis to identify who is in the data performance profile categories, I take those four metrics, right? The two throughput and the two stability metrics, and I kind of throw them in and we see where the data cut points are.

[8:35] Take a look at that high-performance data set. It's really large. This should be incredibly encouraging.

[8:43] High performance is available to everyone. That elite performance group kind of up at the top, it's a subset.

[8:48] The highest performers are continuing to optimize on all measures. So yeah, the best are still killing it. But you know what?

[8:56] The industry is moving. The industry is changing. Excellence is available to everyone. You just have to execute.

[9:03] Yeah, it's kind of hard and yeah, it's kind of tough, but you know what? This is not reserved for just a handful of tiny, annoying unicorn companies hanging out in the Bay. Everyone can do this.

[9:16] Okay, here's an eye chart. Sorry, it's hard. I promise we'll send out all the slides.

[9:21] If you want to know what this looks like, again, the elite performers are optimizing on speed and stability, deploying daily.

[9:28] Lead time is less than an hour. Time to restore is less than an hour. Change fail rate is 0 to 15%.

[9:35] Low performers are struggling. Deploy frequency is between once a week and once a month, and so is lead time for changes.

[9:42] Time to restore has slipped. Time to restore is between one week and one month.

[9:48] The change fail is between 46 and 60%. Now, don't forget availability.

[9:58] This year we extended our outcome measurements to also include availability in production.

[10:07] So we looked at measures of software development like lead time.

[10:11] Deploy frequency kind of is partly to do with development, partly to do with release or deployment. Change fail rate is a measure of the quality of your release process, if you like. Time to restore is about how quickly, obviously, we can respond and remediate incidents into production.

[10:27] But we also wanted to look at availability as well. So availability is obviously the extent to which users can access your service, normally measured in terms of service level objectives such as uptime or downtime per unit time, such as downtime in a month.

[10:43] It's also about your ability to make promises about your uptime to your customers and to be able to keep those promises as well.

[10:51] So we looked at both of those things together. What we find, again, is that people are not making trade-offs.

[10:58] As you might expect, but also very cool to see, people in that elite performing group are 3.55 times more likely to have good outcomes in terms of availability.

[11:10] So we can say that throughput and stability and availability move together, and together they predict better outcomes in terms of organizational performance, both commercial and non-commercial measures. So we know what good looks like. The next question is: how do we get there?

[11:31] So how do we get there? Is anyone here familiar with my maturity model rant?

[11:37] Yeah. So if you're not, maturity models are dumb.

[11:42] If anyone tries to sell you a maturity model that ends in, oh, let's probably say level five, they're just selling you something.

[11:49] So I don't have time for this. Gene's in a hurry and he only gave me 30 minutes. So find me, I'll tell you why they're dumb.

[11:56] There's lots of reasons they're dumb. So how do we get better? We need to identify our constraints in terms of processes, capabilities, things like technology, process, culture, continuously improve. Okay? Now, what does this look like?

[12:13] So- What do I mean when I say capabilities?

[12:16] Well, Nicole, it just so happens we recently released a book where we talk about our research program.

[12:22] And you can get a free one tonight. 7:15 to 8:00, we are giving away copies of our book, signed, sponsored by...

[12:31] XebiaLabs? I feel like- You'll be here all night.

[12:34] Yeah. Tip your servers.

[12:38] Thank you. Sorry.

[12:41] So there's a- Because no one can read this, sorry.

[12:43] Yeah, sorry, you can't read this at the back. Doesn't matter. It's a big diagram. It's on our website. You can find it.

[12:49] We lovingly call this the BFD. The BFD.

[12:50] That's what that means. The big and fulsomely researched diagram.

[12:58] So the key things that I want to point out on this diagram, obviously you've got software delivery performance, speed, stability, availability.

[13:04] Predicts organizational performance, commercial and non-commercial. What predicts software delivery performance?

[13:10] Well, there's four key things we're going to mention. Number one is culture. We measure culture. We'll talk about that later.

[13:16] Culture predicts both software delivery performance and organizational performance. And then we've got three groups of capabilities that predict both software delivery performance and culture. So how do we change culture?

[13:29] By implementing practices, by changing behavior. Three key groups of practices. Number one, technical practices. Things like the practices that make up continuous delivery.

[13:40] Lean management practices like managing work in process, having visual displays that allow us to see quality in our system and respond to that quality information, being able to get feedback from production and have a lightweight change approval process.

[13:56] And then some lean product development capabilities, such as working in small batches, making the flow of work visible by using, for example, Kanban boards or burnup diagrams, cumulative flow diagrams.

[14:09] Gathering and then implementing customer feedback, and then enabling teams to experiment with ideas for new features and for process changes. If you're working in an agile organization and you get requirements and you must implement those requirements and you're not allowed to change them or add your own, it's not agile.

[14:27] Same for process improvement work as well. So all those things together impact both culture and software delivery performance. So this year, we found some new technical practices.

[14:36] We extended our model. Really quickly, anytime you see an arrow, that's a predictive relationship.

[14:41] This goes beyond correlation. So as Jez mentioned, but wait, there's more. Yeah, predicts or impacts are the words you can use when you see arrows like that.

[14:49] So this year we looked at a whole bunch of new stuff and there's way too much to go through in the time available. So we're just going to give you some key highlights.

[14:55] But we looked at continuous testing, which is shifting left on testing, not just automating it. And we've got into trouble for this, for talking about test automation from the testing community, and we want to make really clear we're not talking about getting rid of testers. Absolutely not.

[15:13] Testers are critical to effective software delivery. Manual testing, like exploratory testing, usability testing, those are critical activities. The crucial thing is they should not be a phase after dev complete. They should be built into the development process and performed continuously throughout the delivery life cycle.

[15:30] And then some other things that we're going to talk about in a little more depth right now.

[15:34] Really quickly, everything in bold is things that we've added this year. Anything not in bold, we have reconfirmed from prior research.

[15:41] So a couple things we want to touch on because we're running out of time. Database. So integrating database work, again, positively contributes to the software development delivery and availability, which this year we called SDO performance. So this is super important, right?

[15:55] The database change process is important. Databases are hard. So what does this look like? Interestingly or not, it looks a lot like shifting left, right? What does DevOps do?

[16:05] It integrates operations into the chain. Communications, config management, including teams, visibility, so we all know what's happening when we include the database, when we have these schema changes.

[16:18] Another thing we looked at this year... Monitoring and observability.

[16:23] So we actually took some care to define both monitoring and observability because experts in the field will tell you that they are two different things. So these are our kind of short definitions that we used.

[16:34] We find that if you are doing these things, you are 1.3 times more likely to be an elite performer. So having monitoring and observability solutions in place predicts software delivery and operational performance, speed, stability, availability.

[16:49] Interestingly, what we found is that people perceive monitoring and observability as being the same thing. They don't perceive them as being different things.

[16:59] If you want to fight, come find me. We can stats fight about this. It's fine. It's really interesting.

[17:05] We talk about the implications of this from a research standpoint in this year's 2018 Accelerate State of DevOps Report.

[17:11] And it doesn't mean they aren't different things. It just means that people don't perceive them as different things.

[17:15] Among the study, stuff, stuff, stuff. We can research.

[17:19] Words, words. Words, words. Okay.

[17:22] Also, cloud infrastructure helps. Of course.

[17:27] Yay, the cloud. But only if you do it right.

[17:31] Who here is in the cloud? Working on stuff in the cloud? We have some cloud people here.

[17:36] That's a lot of people. Okay. Everybody, we're going to play a game.

[17:42] Play along. This is your exercise for the afternoon.

[17:44] This is your exercise. Everyone put your hands up if you're in the cloud. Keep your hands up.

[17:47] Keep your hands up. So, this is a bit like PE class where you have to hold your hand up for a really long time, so get ready.

[17:54] Hopefully- Have a support hand maybe ... you're going to hold your hand up for a really long time.

[17:57] Yeah. If you're doing well, you will suffer more. Such is life. I love this game.

[18:04] The National Institute of Standards and Technology-- sorry. The National Institute of Standards and Technology defines five essential characteristics of cloud in what is their smallest and shortest paper that I have seen of theirs. So characteristic number one, if you do not meet this characteristic, please put your hand down.

[18:21] So characteristic one is that anyone in the organization must be able to self-service the cloud resources they want on demand. So we love ServiceNow. ServiceNow is great, but if you have to create a ticket in ServiceNow in order to get your server in the cloud, it's not a cloud.

[18:39] Put your hands down. It's just a very expensive data center.

[18:42] So cloud, hands down if developers can't self-service the resources they need on demand.

[18:47] Okay, good work y'all. All right.

[18:49] That rules out most people. Okay, next up. Number two, if you cannot access your cloud stuff on all the devices you would like to be able to access it on, put your hands down. So the whole point of cloud- All devices. Yeah, so many devices, workstations, tablets, phones.

[19:04] You should be able to get it on all those devices. Yes.

[19:06] Broad network access. Okay. Number three, resource pooling. So a single physical piece of hardware should be able to support many different virtual instances. If that's not true, if you're not seeing resource pooling, most people kind of have that, so I wouldn't expect to see many hands go down.

[19:23] Okay. Next up. Rapidly- Rapid elasticity. So you should have the illusion of infinite resources, or as Nicole said in this slide- Bursting like magic.

[19:34] Yes. Okay, last one. Okay. Measured service. So what this means is the service is metered.

[19:38] You only get charged for what you use. You're not paying the capital costs. Yes. Okay.

[19:43] Okay. We're about ...

[19:45] Ooh, that's not good. What?

[19:48] Let's give all these people a round of applause. By the way, y'all match our sample because of the people that said they were in the cloud, only 22% of them are actually in the cloud.

[20:02] It's fine. So- But do you ever talk to people? Yes.

[20:05] Yeah, this happens all the time. This is why we did this research, because it winds us both up. The people are like, "We're in the cloud." And it's like, really?

[20:11] But I don't see the benefits because- Because you're not doing it right ... I love you, but you're not in the cloud.

[20:16] It's just a very expensive data center. As Corey Quinn said, "You can't slap a sticker on your data center and call it the cloud."

[20:24] However, if you are actually doing all these things, what we find is that you are 23 times more likely to be an elite performer.

[20:31] So cloud can really make a huge difference if you meet those essential characteristics that NIST defines.

[20:38] Again, remember earlier when I said you should be super encouraged? High performance and elite performance is absolutely possible, you just have to execute. We know how, just do it. It's hard, right? I know it's difficult, but you can do it.

[20:51] We know how. NIST tells us how. Short white paper. Okay, next up. Open source is good. Who here is using open source?

[21:00] Woo. Okay. Highly correlated with SDO performance.

[21:04] Elite performers are 1.7 times more likely to be using extensive use of open source components, libraries, platforms, and they're more likely to plan on expanding this. So you're in good company.

[21:20] Also, outsourcing is bad. Now we're talking about functional outsourcing here.

[21:25] If you take all of dev, all of test and QA, all of ops, and you just throw it out somewhere else, it's not going to work.

[21:32] Low performers are almost four times as likely to be using functional outsourcing. And if you think it's going to save you money, not you, if someone in your organization thinks it's going to save you money, the costs often far exceed this, and we outline this in the paper, in the report this year.

[21:50] You can do the math basically using cost of delay. When you do functional outsourcing, you tend to have to batch up work into big batches. Your high and low value features get mixed together into one enormous batch that takes months to deploy.

[22:02] You can calculate the cost of delay, of delaying those high value features, and that will typically outweigh the money you'll save from outsourcing.

[22:10] And you've probably seen this, right? You're probably waiting for some dope feature because they're not done with the project. That sucks.

[22:17] Or the release. Okay.

[22:18] Slow and overly cautious is also bad. This is where Professor Nicole gets all scoldy with people. I'm sure you've heard of someone who thinks that deploying twice a year is still a good idea because we're going to be real careful. We're going to do lots of testing, we're going to do tons of quality control. Everything's going to be amazing.

[22:38] Who here knows of someone who's done this? Maybe you've got a friend.

[22:42] We have friends in other organizations. Super not us, it's fine. Not us. It's not me.

[22:49] I heard of a person. I know a guy.

[22:57] We did additional analysis and I found this group that's, they're adorable. They mean well.

[23:04] It's fine, except it's not fine. Here's the thing. They deploy between once and six months.

[23:10] Lead time is between once and six months. Here's the thing. Their change fail rate is actually better than low performers.

[23:17] They do see evidence of this. This is great. I'm real happy for them. Also, when they go down, they go down hard.

[23:23] They go down between once and six months, right? Because their blast radius is massive.

[23:29] So it takes them months to restore service. Takes them months to come back. Blast radius is huge.

[23:34] They can't debug it. They don't know what to do. Their customers might be able to access something, but how long does it take to restore service in the back?

[23:43] Data is down, systems are down, infrastructure is down. Everyone's putting out fires forever. It's a bad idea.

[23:50] And a great example of this is when you have security issues where someone's broken into your system. It could take months for you to do all the analysis and triage, and forensics you need to actually work out what's going on before you can fully restore service. For me, security is one of the biggest use cases for what we're talking about, the ability to deploy with speed and stability. When there's a CVE announced, how long would it take

[24:15] you to identify all the systems impacted by that CVE, apply the patches, get those patches deployed into production?

[24:21] That's a key goal for, and a key use case for short lead times and reliable releases. If you cannot repeatably and reliably release changes, you can't respond quickly to security issues, and that's a huge risk.

[24:37] Also, who here's heard of healthcare.gov, right? Or Sony? This is a thing. Okay, and 5% of teams hit this.

[24:45] By the way, this group has the highest use of outsourcing of all groups.

[24:52] Most highly correlated. When we look at outsourcing in the diagram, it's negatively predictive of software delivery performance, okay?

[24:59] It has a negative impact. Quickly, we can influence culture. So often people ask us what this looks like.

[25:08] I want to quickly win DevOps Bingo. We know that bad practices hurt, right?

[25:16] Now, what do I mean by culture? If anyone's been to one of our talks, this should be familiar.

[25:20] Yeah, we've been talking about this for five years. But if you haven't seen it, this is a model created by Dr.

[25:26] Ron Westrum, who's a sociologist studying safety outcomes in healthcare and aviation. He has a typology which you can use to look at organizational culture. How do we deal with cooperation between different teams and within teams?

[25:40] How do we deal with people who bring us bad news? Do we shoot people who bring us bad news? Do we ignore them?

[25:45] Or do we actually train people to bring us bad news as soon as possible so that we can react to it before we have catastrophic or cascading outages or failures? Are responsibilities ignored because we'll get into trouble if something goes wrong, we don't want responsibilities? Are they defined narrowly so we know who to blame when things go wrong? Or do we all share risks because we know that we succeed or fail as a team?

[26:06] Do we encourage or discourage bridging between departments, between different parts of the organization? And then the two critical things, how do we deal with failure and how do we deal with novelty? And these two things are connected.

[26:17] In organizations where failure leads to punishment, no one will ever innovate, because by definition, innovation is doing things that have not been done before. If you know that if something goes wrong, you're going to be punished, you will not innovate, which is why psychological safety is so important if you want a team which will innovate. So if you're on the left, and you probably know where you are, and you want to move to the right, how do we do that?

[26:39] Well, that's what the practices we talked about do. We've shown that those technical lean management and product management practices all positively impact culture.

[26:47] So that's how you change culture. Although that's really interesting because we've been telling people that for years. They're like, "No, what else can I do?

[26:54] I don't just want to make smart investments in tech and process. I know. What else can I do?" Okay, so we investigated that this year.

[27:01] The first thing we dug into is what can leaders do, right? What can we do?

[27:08] A couple things we can do is give our teams autonomy. We can set targets and outcomes, and then we can give our teams autonomy to decide how to achieve those outcomes.

[27:20] What we found is that similar to other environments, similar to other contexts, this works in technology as well.

[27:27] By giving our teams autonomy, it then contributes to both voice and trust. What does that mean? It contributes to voice.

[27:34] Our teams feel safer to speak out about what's working, what's not working, both in the tech and in the teams, and it makes them feel more trust in their leaders.

[27:45] Those two things then in turn contribute to that culture that Jez just described and then in turn it contributes to software delivery performance and organizational performance.

[27:54] And just one little note on that. Autonomy also means that teams are involved in setting those goals and targets.

[27:59] Goals and targets that are imposed top-down can be problematic if the teams are like, "Well, that's completely unachievable." So autonomy includes teams being involved in setting the goals and targets that they're measured by.

[28:12] The other thing we looked at is retrospectives. So these are called sometimes learning reviews in the Agile movement, we talk about retrospectives.

[28:21] We also talk about blameless postmortems. And the idea is reflecting on what we've done so we can learn and get better in the future in a way that's focused on system improvement, not blaming people, basically.

[28:34] So we find that instituting retrospectives and making sure we're learning from them positively predicts culture. It also impacts a climate for learning, which is essentially creating an organization where learning is treated as an important investment that the organization will invest resources into, not as something which is, well, you should be doing that in the weekends on

[28:55] your own time. Make sure you're working really hard to get your story points this week.

[29:01] So retrospectives predicts climate for learning.

[29:04] Both retrospectives and climate for learning predicts organizational culture. By the way, climate for learning is a nice revalidation from the 2014 study, and we have found in our work at DORA, our assessment work, that this is particularly useful and impactful because we work in technology.

[29:17] Things are moving and changing and racing ahead so, so quickly that when organizations can leverage their climate for learning, their transformations are much more successful.

[29:28] Now, quickly, in the minute and a half we have left, some of our other favorite data findings.

[29:34] Change advisory boards are useless. Asterisk, they're good for notification systems.

[29:46] They're not good for approvals. It's actually negatively correlated with stability. So if you think you're helping, you're not, sorry.

[29:53] Things like lightweight approval processes and peer review are super helpful. So like peer code reviews, for example, or pair programming.

[30:03] And the other thing I would add to change advisory boards, we think they should move to a governance function. Thinking about how can we create more effective change management processes should be about governance not inspecting individual changes, which is a terrible idea.

[30:16] Absolutely. Industry doesn't matter. I took a much more rigorous approach to this analysis this year, and I know a few people out there are thinking, "Oh, but that won't work for me.

[30:24] I'm highly regulated." And then somebody else is thinking, "But I'm even more highly regulated." I love you. Also, no excuses because there is no statistical significance for any industry, and I tested over a dozen industries. So there's no statistical significance for software development or delivery across any industry.

[30:43] And then finally, integration times and branch lifetimes.

[30:47] So the most controversial thing I have talked about for 10 years is saying that it's better to develop off trunk than on long-lived feature branches.

[30:56] So the data has borne this out in many different ways, but basically you can work on feature branches, you can use GitHub branches or pull requests. The crucial thing is that you're merging into a shared trunk or master at least every day, and ideally within hours rather than days at a time.

[31:16] Yes. Okay. TLDR, for anyone who was busy on Facebook or Twitter instead, technology matters. Cloud works if you're actually doing cloud, the good and the bad, and we can influence culture. Okay, if you want the latest goodness because we did not even cover it, you can get that in this year's Accelerate State of DevOps Report.

[31:34] Go to DevOps-research.com, go to research, go here, or go to bit.ly/2018-devops-report.

[31:41] Or if that's way too hard, quick, take a picture of this. Get a bunch of free stuff.

[31:47] And my magic will send you all sorts of good stuff. nicolefv@sendyourslides.com with the subject DevOps.

[31:55] Thank you so much, and we will see you all tonight.