Join Mark Schwartz, author of: "The (Delicate) Art of Bureaucracy" "War and Peace and IT" "A Seat at the Table" "The Art of Business Value"
Author, The (Delicate) Art of Bureaucracy
Have any of you ever been frustrated by bureaucracy? That's it? Go ahead and raise your hands. Nobody can see you. All right. Looking around. I see pretty much. Everybody's got their hands up, though. For those of you who don't, um, you might want to broaden your definition of bureaucracy a little bit. So for example, when you were a child growing up, did your parents perhaps have some kinds of rules about how much TV you could watch each day or what time you have to go to bed? And despite the fact that you had really good reasons why you needed an acception, why you should be staying up late or your parents just wouldn't listen for a reason, right? The rules were the rules. Okay. Well, that's pretty much what bureaucracy looks like to most of us in most cases. My name is Mark Schwartz and I'm an enterprise strategist at AWS.
And I want to talk to you today a little bit about bureaucracy and what it is and how we might think about overcoming it when it gets in our way. All right. So my next question for you, I think some of you probably are involved in some kind of organizational transformation. That's a transformation to dev ops or digital ways of working in general. And, uh, you're probably running into some bureaucratic impediments. I think I know I was, I was a CIO of us citizenship and immigration services in the department of Homeland security. As you can imagine, we had some bureaucracy to contend with in the federal government, but from what I've heard, it's, it's really not just the government that is filled with bureaucracy. Uh, I talked to executives from large corporations from around the world in different industries. Uh, w from what I can say, it's, uh, large enterprises in general are very good at bureaucracy, small companies sometimes also, but as a company becomes bigger, it tends to wrap itself in your aquatic process.
Also, there are some industries that are especially good at it. I would say financial services is a pretty good example. So at Homeland security, we have this wonderful piece of bureaucracy that was called MD 1 0 2 management directive, 1 0 2, I've talked about it before. And some of my other speeches, I love to talk about MD 1 0 2. I find it fascinating. And, uh, it kind of imposed, uh, uh, SDLC, a software delivery life cycle on all it projects that involves something like writing something like 87 documents going through 11 stage gate reviews, it defined 21 oversight roles in the oversight process. It said that all requirements have to be finalized before any other work started. And in fact, it said that all the requirements have to be written as shell statements. It said that we couldn't start testing any code until all the code was finished. And somebody had issued a letter saying they've reviewed it and determined that all the code was finished and it was ready for testing.
So, you know, it's a, it's not so surprising that government, it projects sometimes take years to complete sometimes costs billions of dollars. We actually had one that had spent developed five years, uh, at, at the point that I'm going to tell you about. And, uh, it had spent about a billion dollars and it didn't actually have any results to show for it. It had some results, but they turned out to be unusable, uh, with a process like MD 1 0 2. You can, you can kind of understand how things expand into monstrosity is like that. We found that our average release time, our cycled cycle time for finishing a release on most of our systems was about 18 months, 18 months between releases. And we were in immigration during the Obama administration. So it was kind of a fast changing area and that kind of speed. Maybe it wasn't a very good match for what we needed to do.
Nevertheless, we moved to the cloud and microservices, we rolled out dev ops across the agency. We went from that 18 month release cycle to deploying new features about three times a day on our major systems on each of our major systems. And we did it still complying with MD 1 0 2 sort of. So I I'll tell you a little bit about, uh, what I learned along the way. I actually just wrote a book on the subject. My book is called the delicate art of bureaucracy, digital transformation with the monkey, the razor, I'm gonna assume a wrestler.
You might wonder a little bit at the title I realized as we were doing our transformation, or as I, as I look back on it, I realized that the techniques that we had used in order to overcome our bureaucratic impediments, there were a number of them that I could identify as things that seem to have worked for us. They, they fell into three categories pretty easily. Uh, the category of the kind of things a monkey would do the kind of things a razor would do and the kind of things a simple wrestler would do. So in my book, I present a number of plays or, uh, actions you can take to try to overcome bureaucratic impediments. And I classify them into these three categories. You can sort of think of them as personas that we adopted in order to change bureaucracy. Uh, so that's the monkey, the razor and the Sumo wrestler.
So to illustrate for you what the monkey does, what the razor does and what the Sumo wrestler does. I'm going to tell you the story of MD 1 0 2 and our struggle against M 1 0 2 and, uh, break it down for you a little bit. So to begin with, we realized that MD 1 0 2 really was not written to support DevOps. So we knew we had to do something about that because we had to comply with MD 1 0 2. Fortunately, first thing we noticed is that like a lot of other pieces of bureaucracy and D 1 0 2 had an exception process, they knew that it wasn't going to be able to cover every possible contingency. So they put an exception process into it. In this case, they said that it would be okay to Taylor MD 1 0 2, whenever it was justified by the specific needs of a project. So we figured our project required some tailoring of because MD 1 0 2 didn't make the rules didn't make very much sense, really.
Uh, that was, that was kind of our argument, but, um, I think we worded it a bit differently, but we started by writing a tailoring plan. And, uh, we found when we looked at our tailoring plan that we had tailored MD one to, to say exactly the opposite of, and do you want go to now, uh, the, the, uh, the gotcha of course, is that, uh, you can't just write a tailoring plan and you have to get somebody to sign off on it. So that was what we had to do next, and that took a little bit of work, but we have big advantage. Our big advantage was that our project had already wasted a billion dollars in five years. So people were desperate for us to figure out a way to turn it around. And, uh, I don't think anybody had any new ideas on how to do so except what we were proposing.
So we had a discussion, a little argument around it. In the end, we agreed that our team would pilot a new approach and, uh, that we would be allowed to use this tailoring plan. Right. So that was a start. So we went ahead at the time we were using scrum, we didn't know anything about dev ops at the time. So we were, we were using more or less a standard scrum kind of a practice. And, uh, so we did, you know, we had the stand-ups and we had a backlog and it got roomed by a product owner. And we had retrospectives in some of those retrospectives. We realized that, uh, we had a little bit of a challenge with product ownership. Um, it's a, it's a long story, but, uh, we had a huge project ahead of us, and it was hard to have one product owner who really could, could own the entire thing cause it cross everything the agency did.
But at the same time, when we had multiple product owners for different pieces of it, their pieces tended to expand and we weren't able to move on to others so easily. So we figured in the spirit of continuous improvement, we would try a few experiments to see what we could do about product ownership. And, uh, we came up with something that seemed to be working for us at the moment. Uh, so we were pretty proud of ourselves. I would say now, honestly, we, we didn't really know how, how this whole thing was going to work out. It seemed a little bit too easy to tailor MD 1 0 2 into saying the opposite of what it said. Um, so we, we figured at some point we're going to hit bureaucratic impediments, but we didn't know what they were going to be. And we figured the best way to find out is just essentially provoke and see what happens.
So that was our strategy and that is the classic strategy of the monkey. The monkey, as we all know, is mischievous and playful and monkeys provoke. So by tailoring MD 1 0 2 in this way, we were issuing sort of a, a provocation to the bureaucracy. And we figured that when we saw its reaction, then we would learn something and we would learn about what we had to do in order to consistently be able to work around the bureaucracy. So without thinking about it at the time we had, we had essentially, um, begun down the road of the, of the monkey, the monkey persona. Well, so we waited to see what the bureaucracy would do and how it would react and, uh, it reacted. So the first thing that happened was, um, you have to understand that this was a big failing government initiative. Uh, and when you have a big failing government initiative, auditor's get very interested in it.
So, uh, they decided to audit us. This one is the GAO, the government accountability office. And they audited us. They then called us in to explain their findings. And we were, we were kind of surprised by the results because they, they sat us down and strangely, they said that the problem with our, our project was we weren't being agile enough. This was puzzling for a couple of reasons. The first was that as far as we knew, GAO didn't really know a lot about it development. And they certainly didn't know a lot about agile. This was, this was a new concept that was just being introduced to the government at that point. So, uh, we thought about it and we realized that, um, it was unlikely that they understood agile better than we did after all. We had some agile thought leaders on the team. Some of us had been doing projects using agile approaches for awhile outside of this project.
So, uh, we were interested in what the GA would say and the way they explained it was sorta like this. They said they read our documentation and they saw that we were using scrum. And so they looked it up and they read some things about scrum and they found that our practices didn't exactly correspond to scrums practices in particular, that continuous improvement we'd done on the product owner role didn't didn't match what the scrum books said. So they listed the points where we weren't doing what scrum said. And, uh, it turned out that, uh, in, in some of their books and papers, Ken Schwaber and Jeff Sutherland, the creators of scrum said that you can't change anything in scrum or else it's no longer scrum. They did. They said that, and GAO pointed that out to us. And they said, uh, you know, uh, you guys, you said you were doing scrum.
You're not doing what you said you were going to do. Therefore, you are not agile. And we argued, we said, continuous improvement is part of being agile. And we had continuously improved. We fought. Um, but they said, no, no good. We didn't do what we said we were going to do. So, uh, the bureaucracy kind of one round one, you might say, and part of why it was so difficult for us at that point is that we were not only fighting the government bureaucracy. We were fighting scrum bureaucracy as well. I mean, scrum is a bunch of, uh, formalized artifacts and practices and has a number of formalized roles like product owner. And there are rules more or less about how they operate and Schreiber in Sutherland had said you can't change anything. So, uh, it was, you know, two, two enemies to fight against.
It was a little bit too much. We backed up a little bit and we thought about it. And we thought about how we could get out of this little bind. Well, it was kind of obvious when you think about it, we realized that what GAO really cared about is that we did whatever we said we were going to do. They read our plans and they compared what we did against the plans. So it's obvious, I think how you deal with it. Um, let me introduce the Sumo wrestler. So Sumo, if you're not familiar with it is a kind of a, a wrestling match where you have two very large men typically, uh, I mean, typically large, I think they're always men and they're in a small ring and they, uh, uh, crash into each other and the object is to push your opponent out of the ring, or actually you can win by pushing your opponent out of the, out of the ring or by making them touch the ground with anything besides their feet.
So the two of them, the two Sumo wrestler sorta crashed into each other, and there's a bunch of pushing and, and, uh, you know, often it seems like nothing's happening for a couple of minutes and this sounds, it doesn't really sound like a very subtle art form here, but it actually is. If you think about it, if you push, if your opponent pushes really hard against you, let's say, and you don't push as hard, they might push you out of the ring. Then again, if they push really hard and you just completely yield, they might go flying out of the ring. Right? So in a way, the game here is to carefully sense what's going on with your opponent and then try to use their own strength against them. So that's what we did with the bureaucracy. I mean, the bureaucracy is also a big object and, uh, we figured if we can be even bigger and if we can use the bureaucracy is strength against itself, then maybe we can win.
So what we tried to do is to create our own bureaucracy. That was, uh, let's say, even more powerful than the government bureaucracy. We figured we'll write a policy that says we're going to be agile. And, uh, and then we'll tell everybody in our organization that they have to follow this policy. Okay, good idea. It turned out we couldn't do that because policy is very formally defined in the government. There are certain ways to create policy and number one, we didn't have the authority and number two, it's a lengthy involved process and we didn't really want to go through all that. So we couldn't write a policy. Uh, we also couldn't write a management directive, which is what MD 1 0 2 was management directive 1 0 2. We couldn't create a rival a management directive because we didn't have the authority for that either. But I did have some really good bureaucracy experts on the team.
And they thought about it and came back to me and said, you know what? We can actually write a management instruction, not a management directive, but a management instruction saying that everybody had to be agile and everybody in the organization would have to follow it. So that's what we decided to do. We created M I C I S O I T O O one, our first policy, not a policy, our first management instruction. Sometimes people ask me how long our transformation at USC has took. And I love to tell them it took an entire morning, which was pretty much how long it took for me to read the last draft of the policy and sign it. After that we were provably agile. We released our later on that, uh, essentially said everybody should use dev ops practices. It explained in more detail what we meant by those practices.
And it had an appendix that we said was going to change often as good practices changed, but we mandated being agile and later being what we're using. So we could prove, you know, we're, we're agile now. And that seemed really good. The Sumo wrestler had come through for us. Um, but we kind of suspected that this, you know, this was not gonna solve all our problems either. So, uh, we, we went on with this for a little while and then GAO decided to audit us again. And, uh, when they did, they came up with another interesting finding. They said, the problem with what we're doing is that we don't have a good way to make sure that everybody's following our policy. So great. You've got those management and instruction and great people are, are doing what it says, but you don't have a control mechanism to make sure they do.
Now MD 1 0 2 did actually have such a control mechanism. It was something called V and V or independent verification and validation, and the way they had set it up, it, uh, essentially involved a third party, uh, impartial, you know, objective party coming in and checking to make sure that the project team had actually met all of the requirements of the original requirements documents. So they went requirement by requirement and made sure each of them had been implemented, whether, you know, it made sense or, or anything. Um, and so we didn't have such a thing. And we had to admit that GAO was right about that. We didn't have a good way to force people to follow the policy. So we wrote another management instruction D uh, defining, describing what I V and V should look like in our environment. And, uh, IBMB said, uh, well, the, and the instructions said that IBM, they should check to make sure that all of the teams were using good dev ops practices.
And, uh, it also had a couple of other interesting things to it. Um, if you think about MD 1 0 2, essentially it's Ivy and V checked to make sure that the maximum amount of work had been done right. That every requirement had been implemented. So we decided that our instruction would take a different approach. Our IVF IVM, the instruction essentially said the outside observers should make sure that the maximum amount of work had not been done. In other words, the minimum amount had be done, been done. They were to enforce the principle of maximizing the amount of work not done. And, uh, they also were to enforce the idea of doing as little documentation as necessary. So part of their job was to review any documents that had been written, make sure that they were as short as possible and make sure that they actually served a purpose.
So our IBM V process was a little different from the MD one or two, one, what it made a lot of sense given the direction that we were trying to go. So this is the technique of the razor. That's the third persona that I wanted to introduce. The razor is named after the principle of Ockham's razor, which, uh, essentially there's some debate in philosophy about what exactly it says, but let's say it says that if you have a complicated explanation and a simple explanation of something, you should generally prefer the simple explanation. So we, we created essentially a bureaucratic version of Ockham's razor, and it said, given the control that you want to have, if there's an easy way to do it and a harder way to do it prefer the easier way or more exactly prefer the process that is shorter and faster. So IBM V in our definition was there to apply the razor to everything we did to make sure that anything we did was the shortest path, the fastest path, and the simplest way to get things done.
Given the level of control that we had specified in our other instruction. We also, uh, applied the razor in the way that we tailored M 1 0 2 or the way that our process set up, uh, the way that our instructions set up the processes where MD 1 0 2 had 87 documents, our management instructions specified 15 or so. And we were working to get the number down from there where MD 1 0 2 had 11 stage gate reviews. Ours had two. And, uh, again, we figured maybe we can even on that, but these are examples of how the razor can take bureaucracy and without eliminating the bureaucracy to make it leaner. So putting these techniques together, you've got the monkey who provokes and observes and try things and learns. You've got the razor that makes things lean even within the bureaucratic context. And you've got the Sumo wrestler who tries to use the, um, the weight of the bureaucracy against it. Uh, and, and to pull off a reversal, you might say, so by combining these three approaches, these three personas, we had managed to create this, uh, process within the bureaucracy and fully acceptable to the bureaucracy. That's still supported our approach, which we knew to be the best approach of doing frequent releases with cross-functional teams that let us do three deployments a day of new capabilities. And let us do all of that within the context of the bureaucratic rules within even, you might say within the context of .
So when I thought about it and thought about why all of this worked, I realized that there are three characteristics that bureaucracies tend to take on, but that are not really essential to bureaucracy. And those three things are, they become coercive. They become bloated and they become petrified. They don't change. Now, bureaucracies don't have to be those three things. What you're really trying to do in a bureaucratic transformation, which is going to accompany your digital transformation. You're trying to turn those three characteristics of bureaucracy into their opposites yet maintain the, uh, let's say, the, the controls that the needed controls that might be embedded in the bureaucracy already. Why do you need controls what you need controls in some cases, because you have compliance requirements, for example, because you have security to think about, and you need to put some limitations on what people do.
There are a number of reasons why you might need the structure of a bureaucracy, but what you don't want is for it to be coercive, petrified, and bloated. So the idea of a bureaucratic transformation is to turn what is a coercive yearocracy into an enabling bureaucracy. You want to turn what is a petrified bureaucracy into a learning bureaucracy that's constantly able to improve itself, and you want to turn a bloated bureaucracy into a lean bureaucracy. Now that might sound impossible, but as we, uh, as we showed with the technique of the raiser, we could still work within that framework and just look at the value stream, cut out the pieces that were adding waste and make the processes that were important to the overseers, much leaner and still have them fulfill the same needs. So that's what we try to do when we're transformer transforming you're on Chrissy with the help of the monkey, the razor and the Sumo wrestler. And when, when you do it successfully, then the bureaucracy is actually on your side and helping you to enforce the good practices that you are trying to spread across the organization. So bureaucracy is quite wonderful in that way.
So what I, what I really want to, um, let me put it this way. Our digital transformations are really important and it's crazy to let them get stopped by ineffective. Yearocracy, it's important to learn these techniques of, instead of panicking in the face of bureaucracy or, or getting upset by it instead to see a way to working with that bureaucracy and succeeding in the transformation, even while the bureaucracy is there and watching in a way, um, unless it's so important because the potential of our digital world, the potential of dev ops and the cloud and user centric design is enormous. And there's no reason why it has to be held back by bureaucracy. So I'd ask you to take a look at the plays that I suggest in the book. And, uh, I think there are about 30 of them altogether and see if maybe some of them will actually help you dealing with the things that are frustrating you in an organization. I hope you will. Thank you for listening.
Unlimited users from organization
Gene Kim’s SRE Playlist