Industrial DevOps – "What are the barriers?"

In 2009 Patrick Debois coined the term DevOps at a Velocity event in Belgium. 12 years later there have been countless books to describe this cooperation between development and operations to deliver capability rapidly to the user. We further have extended that term into Industrial DevOps to account for complex system of systems which include hardware, firmware, and software. Many of the practices such as small batch sizes, limit work in progress, and organizing around value are not new. The benefits of these practices in quality, schedule, cost, transparency, value are undisputed facts that have been shown repeatedly in periodicals such as the DORA report. The question is if the ideas are not new and the benefits are proven why is it so difficult for organizations to move to DevOps and almost impossible to move to Industrial DevOps.


Robin Yeman, Chief Technical Officer at Catalyst Campus and Suzette Johnson Senior Northrop Grumman Fellow will walk through the common barriers to adoption they see in these two large scale companies as well as invite the audience to participate in potential root causes of these barriers. The barriers we will discuss are Organizational Structure, Psychological Safety, Access to Common Language, Understanding the Value Stream, Lack of Trust, Access to patterns to break down systems, and exclusive over inclusive behaviors. In this session you will not only hear stories of programs who are experiencing these barriers you will have the opportunity to collaborate on solutions.

RY

Robin Yeman

Chief Technical Officer, Catalyst Campus for Technology and Innovation

SJ

Suzette Johnson

Northrop Grumman Fellow, Northrop Grumman

Transcript

00:00:13

Welcome, thank you for the opportunity to be here and joining us for this session, industrial DevOps, what are the barriers and this presentation, Robin, and I will continue the industrial DevOps journey. We will briefly revisit the principles of industrial DevOps and then discuss the barriers to an industrial DevOps transformation along with providing some recommendations based on research and some of our own experiences that we want to continue to validate and believe that you likely have some of these experiences and would welcome that conversation following this presentation. So with that, I am Suzette Johnson. I'm a fellow currently leading our enterprise lean agile strategy. And with me today, we have Robin Robin, would you like to do a quick intro? Yeah.

00:01:12

Yes. Uh, so nice to be here as well. My name is Robin human and I have, uh, like I said, worked at Lockheed for 26 years and recently I've had the chance to transition to a not-for-profit. So I am now the chief technical officer for catalyst campus Here. You can see Suzanne and Reed. This is, uh, I want to say three or four years ago, but Suzanne and I have been, uh, working and presenting together for over eight, nine years now. Um, we actually had the opportunity to meet at a conference and was listening to each other's presentation and it sounded like we were both talking on stereo. Like we were saying the same thing. We had the same stories. Um, so after that we decided to team up and actually tell some of those stories together.

00:02:13

So with that, we want to get into our conversation today about industrial DevOps. Um, first we'd like to start by giving a little bit of context of what is industrial DevOps in case this is a new concept for you. Um, industrial DevOps is designed for large cyber-physical solutions. We have identified eight principles of industrial DevOps, and as we talk today about the barriers to, uh, industrial DevOps transformation, we were going to stay centered on principle number one, which is to ensure that our organizations stay centered on the delivery of value. And while there's many practices and tactics and behaviors that we'll discuss, we want to be sure that we stay focused, the outcomes that we're after, which is delivery of value in the shortest sustainable lead time.

00:03:07

So we have a scenario, um, that we were going to kind of put things in context of this ad, um, all set transport company. They're a fit fictitious company who produces vehicles to support assisted and autonomous vehicles, both commercial and consumer markets. Um, in this context, they have been implementing industrial DevOps principles for nearly a year, and they're experiencing some challenges, including multiple competitors, rapidly changing technologies, security and compliance issues. And they're also struggling with their adoption of the principles in terms of getting it to stick within their culture and really getting it to apply and be part of how they do business. So with that, um, Robin, do you want to tell us where all set is right now and their adoption?

00:04:01

So, absolutely. Um, so basically what we're looking at here is all set has been awarded a federal dev ops program, um, autonomous transformation communication system. And as you can see, this is very much a place or a situation that Suzanne and I would find ourselves in, um, both at Lockheed and Northrop Grumman in these, these large cyber-physical systems that require multiple different suppliers, right? So, um, in the context of all set, what I'm going to tell you is they really, really still have a functional organizational structure. Not sure if you guys have ever experienced that, but we're trying to make the transformation, but they still have one and they haven't had the chance to change because they've got this great big contract and they actually need to get started quickly. So they'll, they'll get back to it later. And actually what they're thinking about is, Hey, maybe let's just have some IPTs or integrated product teams until they can reorganize the engineers are still allocating all of the requirements.

00:05:12

So when we look at that, um, we have not had the chance to create these cross-functional teams, these teams that actually can deliver value all the way through the capabilities. Um, they're, they're still staffed organizationally, uh, by systems engineers, software engineers, test engineers. Maybe you've had this problem before the team thinks that in order to get started quickly, maybe they'll just use, uh, the sprints and the timeboxing to build their architectural artifacts. If we build those first, then maybe we can get started and actually really move into a dev ops program. But we know that this could possibly be a trap. Um, they have developed a detailed master schedule and they need that scheduled, um, so that the customer can see what their, uh, their milestones are and, and what their timelines are. But we know that they really need to use a product backlog and move from predictive planning to empirical planning.

00:06:19

They have to start immediately. So instead of incrementally staffing up teams, they're going to hold a hiring event, bring everybody on all at one time. And we know that's also a problem. Um, they have detailed statement of work for all of their suppliers. So they've already taken out all of the creativity, even though they really want to enable, um, they don't know how to make the change. And because they have promised a really tight schedule, they believe that starting all of the features at one time is the way to go. Now, the reason I bring this up is because just like all set, I have had this experience over and over and over again. Um, the most well-intentioned programs want to make these changes to move to a full dev sec ops life cycle. They want to deliver at this be irrelevant, but they're really struggling because this underlying organizational structure or architecture that they have is creating these barriers.

00:07:32

Okay. So now we will take you on this journey with all set and we'll talk about specifically, what are some of the barriers that they're experiencing as part of their transformation and what are some of the recommendations that they are going to try and experiment with to help overcome these barriers and to form this new way of working? Right? So we have six barriers that we are working with with all set. Um, the first one was kind of focused on their organizational structure. Um, then we'll talk about the challenges around the lack of a common language, not staying focused on that value stream. We're really understanding what their value stream is, um, patterns to breaking down the system and what challenges they're struggling with there to really understand the system and how it decomposes also Val valuing exclusivity over inclusivity. And what is the impact of that? And how do we make some changes so that we can have a more inclusive environment. And lastly, we'll talk on the lack of psychological safety. So Robin was going to say, do you want to talk to us about organization?

00:08:44

Yes, this is near and dear to my heart, the biggest challenge. And it almost looks like everything comes back to, this is the existing organizational, right? So here, um, the executive is saying, Hey, we really need specialization because of efficiency. And we need clear roles and responsibilities. This is what our executives have been taught. And because of that, they're really struggling with the recommendations that the coaches are making, which is, Hey, Conway's law for one, um, you are doomed to build a system that matches your organizational architecture, right? We're going to create those hierarchies. And then with all of these functional organizations, we have an incentives mismatch, right? It's highly unlikely that somebody that's spent their entire career becoming the VP of software engineering or the VP of systems engineering wants to give that up and actually look at a product delivery organization, right? Their incentives are completely opposite.

00:09:51

Um, the handoffs are continuously causing delays, but they can't be fixed. And we cannot reduce the dependencies because both of the organizational architecture, as well as the systems architecture, we've seen this over and over again. I think the key and I don't have the end answer, but I think the key is companies have to determine whether they want to optimize for product delivery or specialization and really focus on making those changes. They can utilize things like a dual operating system, maybe considering involving technical people in the design of their organizational team structure. But this has to be dealt with because it's causing our large legacy systems over and over again, to fail in the same way.

00:10:45

The next barrier we've talked about is this lack of common language, right? So the dev ops coach, uh, is really working with the executive to talk about cross-functional teams and how working together drives innovation. Um, however, what you're you're experiencing is a lack of a common language, which is reducing trust. So currently we've got, you know, program management really focused on lean, which is exceptional, but systems engineers are focused on systems thinking and design thinking, uh, software really focused on agile and, and dev ops where you're still seeing operations, um, talking about it. I L or it infrastructure library, the bottom line is all of these environments want to deliver optimized systems, but they're having the tower of Babylon issue. They are not being able to communicate, which is leading them to not be able to work towards a common vision. So things like building a common language, developing a Rosetta stone or mapping the language together so that people have something to look at, could potentially be the solution.

00:12:07

Okay. And building off of that, they have another barrier that they are facing, which is they don't understand their value stream. So also like so many organizations struggle with understanding the value stream, especially in these complex, um, environments where there's software, there's hardware, there's suppliers. Um, and, and why do we care about this? What we're working with also to help them understand that the value streams really help them understand the flow from their requirements and the customer needs to then delivery of value and my understanding our value stream, we can also measure and visualize the flow of work, the metrics, the progress we can understand where the bottlenecks are and also how to best align our teams to deliver that value. So it takes time to really understand and capture the value stream. So we are talking with them, I'm working with them to take time to hold the training, hold the workshops, and really map out that end to end flow of value.

00:13:13

Um, so where the greatest opportunity for improvements, we're going to look at the whole value stream and do some assessments. So maybe we don't have to do everything all at once, but let's think about where do we want to start that provides the greatest return on investment. Um, so some things that we are also going to do is, um, use the metrics to measure the improvements. As we make the changes, we're going to use a modeling tool to help make the value stream visible. And we're going to revisit this regularly. It's not like we do the value stream once, and we're done we'll map out the value stream. We'll look at the opportunities for improvement, for the alignment of work, um, where we can make improvements in the flow of delivery. And then we're going to measure that progress and continue to make improvements.

00:14:07

The next barrier we run into is access to patterns to design the system, especially these large complex systems. So the dev ops coach is really focused on working around building your system around products and services, right? And they're trying to support the executive in so that they understand what Conway's law is, right? Any organization that designs a system will inevitably produce a design. That's a copy of the org structure, right? Um, now frequently you can hear executives say, but if we have really good documentation or if we have really clear roles and responsibilities, but we know that that's a trap recommendations to the business is going to be decomposing your system based upon products and services, not based upon input based artifacts, shift to product teams, over project teams, architect, to reduce those handoffs, right. Dependencies are killer, and then create small cross-functional teams that share this common set of practices is very difficult, but it's going to really help all set

00:15:30

We want to make sure that people value inclusivity, but for years valuing exclusivity made people feel special, right? You were in the in-crowd the Indian club. All right. So here, the dev ops coach is saying, you really want all hands on deck. The answer to the solutions of the next generation are going to be from these diverse groups, these diverse people, um, and happier people build much better systems, right? So, um, this is difficult for the executive because they've earned their stripes. They've earned their position. This is, this is a really hard pill to swallow. And, you know, they haven't seen it in action yet. So it's, it's unnerving, right? They're, they're actually having to take a leap of faith. So recommendations for the business here are going to be applying a growth mindset. Right? Think beyond what you thought was the answer, use a model to decentralize decision-making right. We really want to distribute that out. Build safe environments, not just environments for the cool kids environments for everybody, ask questions and practice active listening. Um, we want to leverage tools that, uh, teams can brainstorm easily, especially now in the terms of COVID where we're all working, uh, separated, right? We really want to make sure that we're using our tools and our voices to value inclusivity

00:17:16

And in order for this to work, we really need to focus on creating an environment of psychological safety. Lack of psychological safety can sometimes really prove to be a barrier to your transformation, especially with the increased transparency and visibility of progress. And what is working when, when things are not going well. And how do we react to that? As leadership, um, psychological safety creates environments, where we can really thrive and where people are willing and able share their ideas, ask questions, they can raise concerns. They recognize that when they fail, that's actually a learning opportunity. And from that learning opportunity, we can continue to innovate and improve. So we fail fast, fail early, but then we learn and innovate as a result of that. Um, and the psychological safety environment management recognizes that short iterations of failure. Coupled with this rapid learning actually reduces project risk and creates an overall ability to make more, um, better improved decisions, because we have information at hand, um, and readily available based on what the teams are actually demonstrating.

00:18:29

So our coaches recognize me for transparency. They will coach the teams. They will coach our leadership to understand that failure is, can be seen in the right context as an opportunity to continue to grow and learn. And that's actually applying the growth mindset that Robin was just talking to us about. So some recommendations to our business is to ensure that we're leading by example, uh, make sure we understand our culture and how we are reacting and responding to these small increments of failure and actually building a generative culture. A generative culture is where we have, um, high collaboration, high levels of trust across the organization, up and down the hierarchy, as well as horizontal across the organization. And we continue to have studies from industry that show, um, qualitative results and quantitative results about how that actually produces better results, high performing teams. Um, and we want to also think about making sure we are aligning our incentives and our performance reviews with the behaviors and practices that we're seeking within the organization.

00:19:40

So in addition, we're going to take all this learning and we're actually going to create an attentional culture. And we do that by looking and building a roadmap of these behaviors and things that we want to change. And we'll actually group them in four different areas, these different areas aligned with all the different topics that we just covered. So we will look at mindset validation, the organizational structure, technical competency, active role modeling, and with these different areas, we'll actually think about what improvements do we want to make as an organization. We'll actually put it as part of our roadmap and have intentional, um, cultural architecture as well. And we'll get feedback how we're doing, how are we progressing so that we continuously, how our organization triumph over the barriers that we're experiencing.

00:20:35

So help we're looking for from you is feedback. Uh, the next one is what do you see working in your environment? I think that we have some ideas what we've seen working in ours, but I think there's more so we would welcome any ideas or stories from you. We're currently working on the next draft of the industrial dev ops paper. And we would welcome feedback from that draft. And then do these barriers resonate with you? Have you seen these or is it things that we're just seeing in our environment please share? Um, we're so excited to hear back from you. Thank you.