Working on DevSecOps Culture: A Team Centric View (Europe 2021)

Working on DevSecOps Culture: A Team Centric View


Patrick Debois

Director of Market Strategy, Snyk



Hi, my name is . And today we're going to talk about DevSecOps culture and how high believe we're heading towards a team centric view. Some of you might know that I've been advocating dev ops since many years now, and, uh, I've never had a real definition of what that DevOps should be or could be for that matter. Um, I kind of started settling now after so many years on the following DevOps or DevSecOps is everything you do to overcome the friction created by silos and all the rest is plan engineering. So there's definitely some technology involved, but there's a lot of things that you're actually doing to overcome that friction. And for me, that's kind of the core, uh, purpose of DevOps and DevSecOps is looking at what friction that we can overcome. Friction points have been in many places. Uh, the bottlenecks we've kind of discussed, uh, the technical bottlenecks, uh, quite often, like what different environments, the dev environment, the test environment, if, uh, this custom management style discussions about budget, which group gets, which budget, uh, personal educational, like who gets what training and kind of our answer in the past has been to put the command and control somewhere in the middle and that we are regulating like a police saying what goes in from the, what goes in to security, what comes in customer and operations, but these pressure points have been building up.


And, uh, luckily we're, we're trying to, uh, overcome them somehow. And we've seen the following pressure shifts. Uh, agile has kind of moved the pressure from the customer and cut out the middleman, kind of get feedback to the team who was implementing them. Uh, ops kind of got a pressure point by virtualization and ops that they started less focusing on infrastructure and more on the service dev ops kind of moved the application towards production and started caring a lot more on the production. And now we're kind of, in-depth SecOps where we have another friction or pressure point, which is security, and we're trying to address security early on in the value stream. So those forces or kind of, um, things are moving towards the team and are trying to cut out, uh, the management of command and control in the middle. So management actually becomes a supportive function, which is around the team supporting what they need to do.


So that's kind of where management stylists is heading in this direction. So what we actually hope the Walhalla of what we're trying to do is that that team somehow knows magically what to do. So it has the autonomy to do what he thinks it's. So we're putting the power to the people who are doing the work. So that's kind of the movement where we're going through from people who aren't doing the mom work, who kind of tried to control who does the work, uh, to kind of the team doing autonomously. So there is a transfer of authority that we're trying to, to build here, uh, that be empower people in the team to do the right thing.


What's interesting is that the company culture, um, has evolved over many years and in his book, uh, reorg, uh, reinvent the organization, Frederick Lulu kind of shows you the kind of, uh, transition the thinking has been, uh, making, uh, around collaboration culture. The first collaboration culture is kind of command and control, and it stems a little bit from, uh, the cost hulls and, and making sure nobody gets in and the sovereign who kind of dictates what the law is, this kind of power centric, command and control thinking. Uh, you can still see it as in metaphors like the firewall or the cost of coming in and coming out. But we're, we're kind of evolving, uh, from re we have evolved since that. And we started approaching things with a more modern way of automation, uh, you know, think of it as the evolution from coastals to building factories and the factories, all the automation kind of brought stability, a repeatable process, uh, of things we can do.


So it becomes a different way of controlling things. It's kind of controlling the process, moving from this automation or a factory setting. We started looking at how we get better at this process and, and, uh, the measuring became important. Uh, the scientific method, uh, method, uh, prevailed. Uh, it wasn't just like a repeatable thing, but now it was about getting better at this repeatable thing. Then we moved down to the next thing. Uh, we started asking like, what kind of, who are we doing this for? So it wasn't just about the process, but we started looking at, uh, customer centric fuel, and we really wanted to be focused on what the customer wants, uh, in a good way and not just what we could build. And then eventually he argues that we're, we're entering the phase of evolutionary. Uh, we're not just doing it for somebody.


We want it to be meaningful, what we do. And, and it seems each of the steps is still somewhere prevalent in our culture of collaboration. Uh, some will, you know, have like hints and glimmers of the power centric and some are kind of hints and glimmers at the evolutionary step to be clear, probably one is not per se better than the rest, uh, but you need to apply, uh, them differently and understand when to apply them. Um, and it, it looks to me that each of them is actually creating space for the next phase to happen. So it's very rare that you can just enter evolutionary if you don't try to get a clear, uh, safety on the parent control. Um, so each of them kind of builds on top of each other and, um, that space is where I think we can actually start thinking about changing things.


Uh, and that's where changing things, uh, are things that we can do to reduce the friction. We kind of see, uh, this command and control versus evolutionary also happening in the DevSecOps or DevOps team parents, uh, described in the book, uh, on DevOps topologies, our team topologies, um, some kind of keep this center of authorities and we're in the middle. Some are trying to overlap completely, and that's kind of a mixture of different styles. And again, one could have more impact than the other, but depending on what evolutionary step you're in, in your company, you're probably going to adopt one or the other. Uh, so seeing one is better than the other. You could see it's like another evolutionary step in each of the items. Um, but you might get adopt one, but then you might have to move to another model because you adopted a different collaboration pattern over the years, and then the one model doesn't fit anymore.


So I think there's an evolution in the topologies that is happening all the time in companies, not at the instant scale, but on the micro scale, I think that's what's happening. So the team centric versus the dicentric, the meaningful versus the control, they're all patterns still influencing how we kind of think about the team centric view, but the prevailing thing is that we've come from the command of control and we're heading to this meaningful team. Autonomous team is doing all the things and they are empowered to do the work and all the rest is trying to optimize to help them to do their best job.


How have you seen the interactions? Uh, well, we can just kind of give some, um, collaboration, uh, of working together. We can just be facilitators so less kind of driving it, or we, uh, seek, uh, we saw that with, uh, doing something as a service. So the automation, so there's different way. We try to overcome this friction that exists in the different models. Um, but the nice thing is we now have options and we are thinking about these options that people have tried, uh, over the years, but the real problem is not how you organize and group your team, or how you, in which area, your companies, uh, in, in the collaboration letter, it always boils down to whatever you do to the trust. And the trust is, uh, kind of a bi-directional thing. And, um, often people adopt different collaboration models because they have a different understanding of trust in the company.


We often put an over this emphasis on trust, uh, by looking at people's competence, if they're knowledgeable about what they do, then they often have a big plus. And that's very natural because when we are a technology centric, we look at how well people are doing, and that's our first indicator, usually of, you know, what they're doing, uh, or how well they are doing this. Uh, but there's in this, uh, kind of book they described, like for audit things. Uh, the other thing is about being sincere. So you can have all the skills set that you want, but it might not be sincere. Uh, I wouldn't go, uh, completely saying people are lying, but they might have different agendas. And that, that kind of shows, uh, in certain, um, interactions between the groups and then they might have the best, uh, uh, intentions that might have the best competencies, but that might not be reliable, baby D they're not yet good at it, or they have other priorities or there's other things that kind of block people from trusting them.


Um, and usually that would be as simple as showing up in a meeting, being there all the time, being reliable, somebody can call you, you will answer. Uh, and again, all those three, uh, are already important, uh, part of, uh, building that trust. But then the ultimate thing is that you want people to care. And I see this quite often, we can train the team in their competence. We can see that they're sincere, that, you know, they kind of are really eager. Uh, w w we, we see that they're fixing the bugs and then reliable is that they're fixing bugs all the time, but they might not just care. And that care factor is something that is very elusive in trying to build that, uh, in a team. Um, and that's why some of these transformations will just fail. Um, it's also important to note that if one group one silo tries to trust the other silo, there is also the process that they have to become trustworthy themselves.


So it's easy to say the other's groups should do a, uh, but it, you have to do things yourself. And you're trying to convince other peoples that you are actually competent. You are sincere, you're reliable, and you actually care about their problems. Uh, for example, if development group mainly cares about their features and not about the security, it will show if it's security, mainly you will care about the, uh, the security and not about the features it will show. So that clash of trust is hard to combine, but it's an instrumental, uh, thing to look at how you can get better at these things. Um, over time, I kind of distilled all the things people have started doing around, uh, DevSecOps in this case, uh, in four areas, one is a lot of activity around the secure stack, what are we building? Um, and then the next is usually about how we are building, uh, and there, that could be technical, that could be processed, but most of the narrative has been about, uh, this being a technical part or automating part.


And then there's a flip side, is that why we're building this? And I would say governance in the, in the broader sense, actually give us an ID on, uh, what that should be, uh, that why it could have various wise, but this is, uh, actually giving direction to things we do in securing the stack and securing the delivery. And then ultimately, uh, building on the process, building on the motivation, building on the stack, uh, we're hoping that we are empowering teams to do the right thing. Um, and that's kind of the fourth area. So it not just, uh, is enough to talk about the governance on why we're doing things is it's, it's very important to talk who is doing the work and how we're helping them to do the work.


Um, if I were pictured the secure stack, a lot of the DevSecOps narrative has been about what the developers can do, and that's really great. They can kind of check their co-dependencies, they can fix their code there. They can look at the container configurations and all the things that are very deaf centric. But what I'm advocating is that that is just a slice of what they can do themselves, because it's easy for them. Well, I'm not saying it's easy as such, but it's in their comfort zone in a way, but I think we're trying also to look how to expand this to how a security framework is the team responsibility, how operational things have become, uh, their, uh, uh, focus as well. And then the business focus as well. So we're putting so much stress and so much things that a team needs to do.


And that's where I think the whole dev ops person or the full stack person, or one person who does it all, you know, that's, that's clear it's, this is becoming a fallacy. You can't have all these skillsets in one person, uh, it's way better to collaborate as a team than just be like a single, so silo who does one of these areas and that balancing between DevSecOps and overcoming the friction that you're not just caring about your code, you're caring about how it will end up in production, how it is secured. Those are kind of things you do to overcome the friction between the silos. And that's what I think the narrative of DevSecOps probably is about. And same thing happens in the delivery process. So if you're thinking about the pipeline, that's things we can do in the development that we can kind of, uh, get at, but as we moved through the environments from the development environment to the test environment, the production environment and the operational environment, um, there's different things we can do.


And that broader view, um, is what I want to instill on that team to take the ownership of what they can do. Um, and this is usually again, hindered by friction of silos. So if the development group con look at the built environment and can look at a production environment, or the other way around this is friction points in each of those areas that you overcome. So another way of looking at it that this is sometimes referred to as the sonar model, we're trying to ping from one area to the other, uh, to overcome these friction points, but the, it gets lit a lot blurry, as long as we're having, uh, uh, longer feedback cycles there, ultimately we're also want the process to be taken care of. And, uh, on DevSecOps, I usually see it come up at the friction point of the incident management, which is probably the first, uh, area where kind of these people start, have to collaborate together somehow to overcome this incident.


And then later they kind of expand this to longer cycles of vulnerability management, all the way to compliance management and security, uh, continuously. So this is again, a longer cycle and a lot of the developers, they don't have the power to look at the other areas. Um, and on the flip side, not, uh, the developer groups or the developer things don't have to do everything. So again, this becomes, if you collaborate somewhere in the middle in these areas saying it's a hundred percent overlap is probably not the perfect, uh, uh, concept, uh, but somewhere between different groups, doing different things with an overlap of about 70, 30%, uh, in, in the middle, but where the balance is, is probably gonna be up to your company's culture. And ultimately we want to move from just collaboration, uh, to authority and that the actual collaboration could start, could be as simple as, Hey, we're seeing this vulnerability, Hey, have you seen about this?


And that kind of gets the conversation going, and what you're actually want to go for is the next phase. If people are saying here's something, you're something you would hope they want to act on this. Um, and sometimes the lack, the competence, sometimes they lack, uh, the focus to work on this. Uh, but you ultimately want them to get better at what you're surfacing as a security issue. And then once you see they're getting better, a lot about the educational part of there is that you want to make them accountable. So it, it, it, it puts a not a pressure point on moving the team from knowing what to do to being responsible, uh, for what they do. And then the latest area is that we start like trusting them as the authority. They have the authority to do certain stuff, uh, and that's what we hope to transfer to that team, but it doesn't happen overnight.


This is a long process that takes a lot of the pain away, but, um, ultimately by just looking at the stack and just looking at the delivery process, or, um, the, the governance will not be enough. This fourth pillar of collaboration transfer much like the circles of the organizational culture, um, is what you're trying to achieve, which will fuel actually a successful transformation. So, as I mentioned before, um, I showed you a lot of places where these friction points, uh, uh, are happening. And I think when we improve something in a stack, it might improve something in the delivery in might something improving, uh, in the governance and the apartment. And so they're all influencing each other. So I'm not saying governance is more important than the delivery or the stack of the tools versus the culture. One goes hand in hand and, um, again, it's creating space. So if we improve something there's time and, uh, budget to improve something else. Uh, and that's kind of, uh, almost like a spiral spiral spiraling thing that we want from a central owned security owns a DevSecOps, uh, mentality to a team embedded mentality where you will end up and where are your balances? Uh, that's again, up to you, uh, on, uh, to see how that fits into your corporate culture.


And a third thing that I usually think people kind of forget is the alignment of, uh, or the motivation. Um, I've given you a lot of inspiration on where the friction points can happen, uh, where you can start tackling them, how the transfer of authority from collaboration could happen, but there's still this motivation. So if there's no alignment on why we're doing all this, then again, this is going to be really hard. And in my opinion, eight saying that it's good for the business is rarely the personal motivation people seek out. Um, so this really usually starts when you're trying to get from collaboration to the learning phase, uh, in DevSecOps the it's been a, a common practice to look for a security champions and the security champions are often on the scale is that they want to kind of work around security there, the security curious, but as we move down to the line from, I want to understand security too.


I have to understand security. There's a whole spectrum. And that motivation of just saying, it's good for the business, won't be enough. Uh, you can appeal to being a professional, so quality of code, uh, but it could be all the way up to, uh, you get a reward. If you do it, I can tell you, but it's an, uh, an important thing that, uh, it's often, uh, lost in the whole transformation narrative is how do we actually, uh, not put the whole model on the team on the group, but how do we get the people who are doing the job to accept that they're doing the job? And that motivation is again the crucial, uh, in the whole transformation. So are we done yet? Um, for sure. It's never done a, and that's where transformation after transformation will happen. There's no definite, uh, transformation, so we'll, we'll be going and improving all over and over again.


Uh, our collaboration mentality might change over the years. You might see a, yet another kind of wave of how we're thinking, because this culture is actually reflected also by, uh, the, uh, the, um, the society in general, uh, how they behave and how they think about these, uh, collaboration, uh, models. And I also found that this kind of a paradox, the paradox for example, of command and control, is that the harder you squeeze kind of to get the grip of control, the more elusive the control, uh, is actually, uh, ha happening. It's like sand, when you push it too far, it will just flow out of your hand. Similar paradox I've seen with the automation is that the more you automate, uh, you think it will be all about like repeatable process, but after a while, the people who are, have done the automation, they kind of are leaving and nobody really understands anymore why we did all these automation.


And so it becomes like a legacy system and it becomes less reliable because we aren't sure any anymore why we're building this, um, on the next level of measuring we're having a similar problem. So if you start measuring, uh, to get feedback and to improve yourself, that's great. But if you're start measuring, uh, and you look at the measurement as the end goal, what we've seen quite often is that people start to game-ify and kind of just play on the metrics instead of the result. So it's a paradox about measuring to get better is when you look at them, uh, metric as the goal, then it will probably start eluding your, uh, ultimate goal on the level of collaborative approaches. Uh, when you, we want to empower people, we often have the narrative around, uh, customer centric. Um, and what's interesting on that paradox is that when we are very customer focused and we ask what the customers want, um, quite often in the starting phase of a SaaS company, they will listen and they will look at all the things that customers want, and they don't really judge on what is important.


After a while, when you get bigger, you start listening to the customers who paid the most, or the customers who are the loudest or the customers you value most in whatever your valuation is, and you're actually becoming less customer centric for the other group. So again, it's a pendulum swing from being very customer focused to some customer focused. Um, and that's a paradox that we keep getting over and over again. And then ultimately we are knowing, going for in the direction of autonomy, uh, and get the team to decide. But we already know that once we get there, there's going to be a big call when some people don't make the right decisions or the decisions we agree with, there's going to be a flip side, is that people really want to get back at controlling things, uh, and giving up on the autonomy. So autonomy is good when it suits our purpose and they're going in the direction that we want, or they give the meaning that we want.


But if we don't agree, we want to resort back to different, um, kind of, uh, approaches. And so those paradoxes will kind of keep coming in and keep going back. And even in general, uh, there's a paradox around security. So if you are safe, you probably think the least about security, and if you're not safe, you think the most about security, but you probably are. If you start doing more and more to build a secure systems, after a while, you get less interested, because it's less visible why you were doing all these things. So these paradoxes will keep us busy over and over again. And it's something we need to be aware of. And, and the balance is, as I usually say just enough, but we're just enough. This is probably going to be, uh, uh, a 10 line in your company. Uh, where are you going from one pendulum to another switch?


So I hope I kind of made it clear that, uh, DevSecOps works somewhere on that friction created by, uh, silos, uh, of the tree groups working together, um, that there's different approaches from the central approach to the more collaborative approach, um, that you can do a lot of engineering work, but if you're not trying to do it to overcome the friction, uh, of the silos, then it's like very good engineering, but it's plain engineering. So if you're doing the engineering to overcome the silos, in my opinion, that is a kind of DevSecOps or DevOps, um, as they both kind of implement, uh, influence each other from a tools and a culture perspective. Thank you very much for listening to me. Um, as always, this has been a broad view on things. Um, I'm happy to have a conversation with you anytime. I like to think with you together on how we can improve this, and what is your world view on how we are going, what direction we're going, or what improvements we can make. So thank you very much, and you feel free to reach out to me on the Twitters Patrick, well, or via email. Uh, and I love to hear your feedback, hope to see you again somewhere soon, somewhere in real life, uh, at another conference. Thank you very much.