00:00:04

Good afternoon. Just afternoon. And I'm the only thing in between you and lunch <laugh>. So, uh, so I'm not sure what that does, but, uh, for, for this. But anyway, we'll get through this and, uh, and hope you enjoy, hope you're enjoying the DevOps Summit so far. Uh, I've been, uh, involved with this conference since the beginning. I was on the inaugural program committee and, uh, have really, and been in DevOps space for, uh, many years. And, uh, this has just been transformative, uh, not only for the companies I've been involved with, but in my own personal life. And so I really can, uh, attest to the fact that this can change, uh, lives and make, uh, make, uh, the workers and the employees lives better. So, with that, uh, I wanted to talk today about, this is an interesting topic. Uh, I called it avoiding the infamous DevOps team.

00:01:04

And, uh, uh, first of all, I'm, uh, most recently, uh, was with a company called Advanced md. We did medical software, uh, practice management and electronic health records. And in that space, uh, and most recently I shifted, uh, we had an exit, uh, acquisition and an exit event. And so I decided to shift into consulting. And really, um, what happened was, over the years, I would do kind of small engagements here and there with, uh, my background and expertise. And, uh, just really wanted to be able to, uh, spread this DevOps love, if you will, around a little bit more and, uh, be able to help more companies. And so, uh, I shifted into the consulting, and I'm with a company called Veracity Solutions, and we look at, uh, the whole software delivery spectrum. Um, it was DevOps days, salt Lake City, 2017. So last year.

00:02:01

And, uh, I was invited to be on a panel. And, uh, so there were four of us, uh, on this panel. Uh, and the general, uh, audience, uh, was, uh, um, you know, listening and participating. Anyway, the first question that came out was, uh, let's see here, move forward there. What was your biggest regret or mistake in your DevOps transformation? And, uh, I think I was third in the list. And so I heard some of the other, uh, comments, and it got to me. And really as I thought through that, the answer that I gave was I created a DevOps team.

00:02:49

And at that moment, I thought, I don't, you know, I mean, that, that was just the honest answer. And, uh, this is what happened. <laugh>, I was shocked. I, uh, I mean, basically the entire, uh, crowd started clapping. Several people, not everyone stood up, but several people did stand up subsequent to that, once kind of the panel was done, you know, kind of answering these, uh, pre, uh, conceived questions. They opened it for q and a, and pretty much the rest of the q and a session was dominated by this things like, we have a DevOps team, how do we get rid of it? <laugh>, uh, you know, why did you create one in the first play? I mean, just all of these kinds of things. So I just wanna, if you feel anywhere similar to this about a DevOps team, would you mind applauding <laugh>, just by a raise of hand?

00:03:59

How many have seen this anti-pattern, this dysfunction occur in an organization? Yeah. Um, lots of hands. And anyway, it was this, it was the reaction that I'm like, whoa, I just tapped into something that I really wasn't aware of. And, uh, but I had, uh, just recently joined another company. And, uh, this was accentuated for me because there was the infamous DevOps team that was in that company, and I was, uh, dealing with that issue as well. So this was even more heightened, you know, uh, I don't wanna belabor it. Uh, I think we all understand, but, you know, just, there's a couple of slides in here. I just, you know, would want you to, as a math teacher, I had once said, just burn this into your eyelids. Um, DevOps is a set of principles and practices and culture, right? And I mean, they're, they're awesome.

00:05:02

But at the end of the day, you can't take that and embed that into a single team or group or even a person, right? I mean, even the, the, the DevOps engineer, and I understand, you know, there's a power behind, uh, that, you know, as we're looking for talent, but even that, uh, is kinda, you know, pushing the edge. And so, uh, we just, we really need to be careful about this. So what I wanna do in this talk is kind of look at one, why do they get created in the first place? What are the forces that are happening? And this took me some time to kind of think about this and understand what is going on that these keep popping up. And as I've been also consulting and working with other companies, I mean, I'm seeing this again and again, this isn't right, as you all raised your hands, this is a very common thing that's occurring.

00:06:00

Signs of a DevOps team. Obviously, there's a team over there called DevOps. They're right, some centralized group, some whatever they're called. DevOps. I mean, that's obviously the most, this was an interesting, uh, uh, thing that I'd never encountered before. Uh, I came in as the head of operations, so that, that last mile that Damon talked about today. And, uh, as I sat there and talked about, Hey, we're, you know, we, we wanna embrace DevOps. We want to embrace those practices and principles. There were people in that group and on that team on my team that said, John, we can't do DevOps. And I'm like, why? What? Like, what's the Well, we don't do that. They do. And I was just really shocked about that, that mentality. It had been so ingrained into the culture that that group over there did it. And we can't participate.

00:07:02

We don't even know how to participate. Um, obviously the, you see silos of responsibility. Um, I mean, even to some extent, kingdom building, right? The well that's under the DevOps purview, or that's our, uh, responsibility. That's what we do, et cetera. Um, I think Damon today really hit on the head when you talk about self-service platforms, okay? What happens is that, uh, I think, uh, I don't know if they intend to start a self-service platforms, but they certainly don't become self-service platforms. They become teams that actually embed themselves into the value stream itself. They become a part of that flow. In other words, they perhaps maybe are doing release engineering. They're releasing code, they're, uh, participating some way in what's being actually created. And so they become another part of that, uh, value stream, and they end up really, most of the time that you see them, they end up becoming exactly what you wanted to eliminate in the first place, which was a bottleneck.

00:08:11

And they themselves become a bottleneck. Um, a case in point, this particular team that I came in, and eventually I had to actually dismantle this, I mean, completely dismantle it. I mean, just like, okay, this is going away. It was a pretty big pull the bandaid event off, but it had gotten so dysfunctional that this and the DevOps team was involved in the release of code, principally that they were even merging developers' code into branches. Developers were not even merging their own code into branches. That was something the DevOps team did, because it was part of the release process. Now, that is really, really messed up. Okay? I hope you can see that, uh, this is right, the two organizations, dev and ops, and, uh, what happens, right? Is that somehow through the formation of a group or a team, we get this, uh, that is put between the two groups, okay?

00:09:27

And things that are flowing across dev and operations, right? Have to somehow end up going through this DevOps entity, right? That actual flow of value. Um, you know, dev and I, I, I wanna just say this, dev, the DevOps movement, there's two components. There's no question. There's dev and ops, and the whole movement is around how do I bring those two together? How do I streamline the flow of value across that whole, uh, that, that, that whole continuum. And, uh, um, but for whatever reason, right? We end up creating yet another entity, you know, that somehow gets involved in this. And, uh, so why does that happen? Okay? Um, so DevOps, if it describes a cultural context with practices and tools, et cetera, right? Why does it happen? Well, one, the easiest thing is, well, we're just going to rename a team. This team maybe was doing release engineering or some other function.

00:10:39

Maybe there's some kind of a tools, well, let's rebrand it into DevOps, but we really haven't changed anything, okay? That's, that's, that's probably fairly common or reasonably common. I think one of the most common things though, is that this, as a leader, as a transformational leader and not understanding what's really going on here, you have, you really have this new set of work that you've got to make happen in the organization, right? You've got, uh, right, new toolings, new capabilities, you've got change to drive through the organization. Well, the existing organization is already doing a whole bunch of stuff. In fact, everybody's all very busy just doing what they do day to day. I mean, how many of us really, are we really, uh, able to just take on new stuff with, uh, with no thought, with no direction, with no priority? No, we're all, by definition, we're all busy.

00:11:39

We all got something to do. So at the end of the day, really to be able to drive transformation, we've either gotta create some kind of capacity in the system. We've got to free up resources to create change. We either have to free up resources, free up capacity, or we need to get new capacity. We need to augment capacity. Okay? So either we need new people. Oftentimes this requires different skills, requires different skill sets, okay? And so at the end of the day, when I look at this, I look at really DevOps in a lot of ways as what I would call innovation work. It may not be new product innovation, but it's new technical innovation within an organization. There. We're trying to do something different today than we did yesterday. And typically to do innovation work requires that we have new additional resources, or that we make, uh, capacity available, okay? Different from the ongoing day-to-Day work of the business.

00:12:44

So, um, if you've, uh, maybe you've heard of, uh, this book or these couple of books, the other side of innovation, um, this was kind of the first, uh, seminal work of VJ Govin, um, a professor I think out of Dartmouth. Um, but he talked about, uh, really, you know, I mean, it's the idea that yeah, there's, uh, ideas are plentiful, but once you have an idea, how do you actually make it happen? It's, that's what he called the other side of innovation. And there was another, uh, kind of a leadership parable called How Stella Saved the Farm, that kind of illustrated this in a, in, in a fable setting. But for me, um, this, the follow on work called Beyond the Ideas, really where this became, uh, more practical and more implementable for me, because he really described a framework of how you actually execute innovation within an organization.

00:13:43

And this is where the light bulb went on for me, about, uh, what is going on here, okay? And, uh, what he describes is that there are, there are, um, what he called the performance engine or the core business. There's the ongoing operations. It's the business that we're in today, what brings in the money, what brings in the cash, and what actually is what funds the innovations that we're going to, uh, do. In other words, without the current, uh, system, without the current business, we wouldn't even have the ability to fund new innovations. So it's very critical that we do maintain that ongoing operations, and that we don't disrupt that. But then you have this, right, this non-routine, uncertain new thing that you're trying to do. And in this case, DevOps related activities that are uncertain. And so you want, you need to be able to, uh, do that and integrate that, uh, into your, uh, overall, um, production and over your overall system.

00:14:55

So innovation leaders need to think differently about how to do innovation. And this is, uh, the model that he describes. Uh, he describes, uh, three different styles of innovation. Um, I'm not gonna go completely in depth, but for our purposes, the two that I think are the most applicable are what is called Model S type innovation, which is, uh, simple innovation, okay? And fundamentally simple innovation is that I can actually do the innovation within the ongoing operation. I've got enough slack, or I've got enough ability within the existing operations to drive change, uh, into that operations, okay? Or into the existing operational, uh, context. The other one, however, is a much more disruptive, he calls it a custom innovation. It's basically a high risk. It's basically a non-linear type disruption or event. And if you think about it, many of the DevOps practices that we attempt to incorporate are just that for many organizations, they are, they're non-linear, they are disruptive, they're completely, you know, outside of what we do day to day.

00:16:18

And so, really many of them fall into this ca, this model C category. Um, and so, uh, it takes a different model about how to execute a model C. It's not, you don't execute it the same way as you do A model S, but what happens is this, and this is what I tried to do, this is what, uh, uh, many, you know, try to do, is that normally model S or simple innovations, right? These things that can be absorbed normally by the business can be absorbed, right? In just kind of normal working, uh, leaders, making some space for those things to happen. But, uh, they're just, uh, done within the normal sequence, uh, of production. But we try to take the model C and we try to shove it into the performance engine, and of course, it doesn't fit. So that doesn't work. So what do we do?

00:17:18

We create another team, which by the way, is actually a viable alternative. We need to somehow do something different. And so a viable way is to create another team. But what happens then, I, I'll say this, when I did create the DevOps team initially, I mean, it was to solve this problem, I didn't understand this, but it was like, you know, I need some additional ability. I need to get some stuff done. So let me get another team, let me start building the tools, et cetera. So for a while it worked, but as it longer it went, the worse it became. So the secret is not necessarily that you can't create a team, it's that how long do you keep it around? Why do you keep it around? What's the future of that? Okay? So the secret here is that when you're doing custom innovations and you end up creating a, a, a team to be able to drive that innovation, right?

00:18:23

So you've got that capacity, um, you actually do need a team of some sort to drive that innovation. What he would say, or what, uh, Govinder Raj recommends, and what I've learned subsequent is that the link between the core engine, the performance engine, and the innovation team have to be very tight. In fact, you can see here, there's a shared aspect, there's a shared staff. If you wanna drive innovation, you don't wanna completely copy, uh, the personnel or the mindset of the core system because you'll get just another copy of what you're doing. So you need to seed the new team with new ideas and new, uh, uh, uh, capabilities and skills. But you also need to embed in that team a very strong link back into the performance engine, because eventually that innovation team or that dedicated team must go away. It, you must, when you start and create the innovation team, you must start it with the express understanding that it will go away within a, a, a certain amount of time.

00:19:43

Okay? So, and all, what do I just generally would recommend? First, just don't name any team a DevOps team, okay? Whether they're an innovation team or whatever, just don't name them DevOps team, period. Now, I have a little less heartburn about hiring people called DevOps engineers. I understand kinda how that works. But at the end of the day, um, just don't call it DevOps, because, uh, it's going to create overall this dysfunction of, oh, they do it, we don't, et cetera. I think there's a couple of exceptions for that. If one of, if the team you're gonna create to help drive the innovation is like, uh, a center of excellence or like a Dojo type organization where they're helping and enabling, then that may be okay. But again, at some point, uh, you're gonna have to figure out, okay, how, what is the ultimate, uh, end state of that group?

00:20:43

Um, and how does that translate into the overall organization? Okay? Um, if you do need to create teams around self-service, create it, name the team, or give the team the purpose around what the service is they're creating. So if you need a continuous delivery platform, created the contin, you know, call it the continuous delivery tools or something, or, you know, tools team, a service team, right? Very similar to a feature team, but just name it what it does, okay? Not the ideal of DevOps. Um, think about selecting a long-term topology strategy. And I have a couple of examples of that, of that here in a minute. Um, a really good website to go to that talks about different, uh, team, uh, patterns and structures for DevOps, uh, for implementing DevOps is found at, uh, web dot DevOps topologies.com. Really, really good information there. Describes the pros and cons of each, uh, a model.

00:21:44

Um, and as Damon, uh, I hope, I mean, that, that talk of Damon Evers today was just fabulous, but create these innovation teams, and if they are intended to be long lived for services or platforms, right? Those self-service platforms, um, you know, then, then go ahead and do that. Um, if you need to drive innovation, DevOps innovation forward in another way other than in with, uh, service platform teams, then again, plan on how are they going to get reabsorbed back into the core, uh, engine. And lastly, um, I think this is probably the most key, is that cultural and behavioral changes, right? Those things about how people are gonna change their work, change their mindset, okay? Um, you can enable that with an external team, like a Dojo concept, but at the end of the day, figure out how to drive those directly into the performance engine through a simple type s execution, okay?

00:22:57

Do not leave the cultural transformation to outside of the core engine. This might take more time, it's gonna take more planning, right? But as you identify those different practices and cultural changes that need to be made, drive those directly into using small, uh, changes and small increments, driving them directly into the performance engine, using an external team perhaps to help aid that process. Okay? Okay. So, um, one model, right, is, yeah, I created DevOps, quote a DevOps style team to maybe enable this, but one pattern is that you actually set an expiration date. In other words, this thing is going to collapse and go away. However, there is danger, danger will Robertson, right? This, the problem is, is you, it may not go away. So you have to be very, this is, this is what I call the ig, the, the ignorant, right? This is what gets created because of the forces, but people don't understand how to, that they need to expire it. Okay?

00:24:06

There's the, uh, this is what was talked about today. Again, self-service, uh, right ops or DevOps as a platform or as a service. Uh, and actually this chart shows it in dev, but it can live on either side, it can live in dev, it can be a part of ops, depending on what the service is. So a very, a very effective model. Okay? Uh, getting a lot SRE getting a lot of, uh, attention. Uh, at this conference last year, there was some, this year much more. Uh, talk about SRE. The fundamental idea here is that, I mean, SRE really embodies a lot of the DevOps concepts. Um, the key part of the SRE, I mean, is an, it's more of an operational, uh, discipline. Um, and as, uh, you, you've probably heard the SRE model is of course around reducing toil, but the one key aspect of the SRE model is that it promotes, uh, shared responsibility and ownership with the development team.

00:25:12

So what you will find in a lot of organizations is that developers, especially your more traditional, uh, organization, is that developers are, they really don't want to take on operational responsibility. They really have an aversion to that. And now over time, you might be able to change that, but in the short term, what do you do? You want them to take more ownership. In other words, you just don't want it to be thrown over the wall. But how do you make that happen, right? As you evolve, and I think the SRE model's an effective model, it takes some time and work to put in place, but the SRE model pushes that responsibility back on development saying, Hey, we will be happy to operate this and free you up to do the things that you need to do. However, we expect that it's going to come into operations with a certain minimal standard of quality.

00:26:10

And so then we gain this shared, uh, responsibility and collaboration. So I think a very, very powerful model, especially in the transition from traditional to, um, what I'd call modern operations, okay? And then we, I've already mentioned this, the evangelist or the Dojo model, right? This group that really spans both groups that enables and helps, um, in many cases provides an immersive experience. So now I'm running the actual change through the core engine, but I'm actually supporting that, uh, through, uh, an enablement or a right, an immersive, uh, experience, right? For those teams. And then ultimately at the end of the day, we want that full, uh, collaboration between dev and operations to handle, to, to, to happen. Those are two fundamentally different, um, entities. They have fundamentally different purposes. And that's why, again, why DevOps is created to bring those two worlds together and, uh, to make them more effective with that. Thank you.