Las Vegas 2019
Auditors' Workshop - What You've Wanted to Ask an Auditor but Were Afraid to Ask
Auditors' Workshop: What You've Wanted to Ask an Auditor but Were Afraid to Ask
MB
Matt Bonser
Director, Digital Risk Solutions, PwC
PF
Pierre Fourie
Advisory Services Senior Manager, Ernst & Young
SG
Sam Guckenheimer
Product Owner, Visual Studio Team Services, Microsoft
YL
Yosef Levine
Managing Director, Global Technology Controls, Confidentiality & Privacy, Deloitte
DO
Darren Orf
Principal, Cybersecurity & Privacy, PwC
TP
Topo Pal
DevOps Enthusiast,
CQ
Caleb Queern
Director, Advisory Practice, KPMG
JR
Jeff Roberts
Senior Manager, Advisory Services, Ernst & Young
AS
Amy Sword
Audit Senior Manager, Deloitte
MW
Michael Wolf
Managing Director Modern Delivery Lead, KPMG
Transcript
So I'm Sam Guckenheimer. Welcome. This is a, um, this is six years in the making that we have a chance to get all of the big four in one place for you to ask any question you want. Um, I'll be in the hall with a mic. Uh, uh, so that you, we we don't need to play the repeat the question. Game Topa. Why don't you go over the ground rules.
Yes. Uh, so welcome and I think, uh, we want more people, and I expect that more people will come in after John starts, <laugh> is done in the other room. But in the meantime, I think we got enough to start off, uh, for Sam and I, this is like our dream moment. And we had been trying this for about four years now. Oh, expectations <laugh>. Yes. Uh, so this is kind of, uh, in our view, the last hurdle in our DevOps transformation, our DevOps journey. Uh, we brought in business, we brought in security, we brought in ops and everybody involved in the development lifecycle for our DevOps transformation. And I think this is the last one, last hurdle that we need to cross, uh, to make it really happen. Uh, couple of years ago, we as a team in Gene Kim's DevOps Enterprise Summit Forum, we published, uh, love Letter, dear Auditor.
And I think that got, uh, uh, good traction. It was kind of, uh, uh, uh, olive branch to the auditors. Please come and help us. We really want to work with you. So the result of that dear auditor love letter movement, if you call, if you wanna call it that way, is this, that we have all the people who can actually help us on the same stage. And for us, it is again, our dream moment. So before you start, there are some ground rules, uh, for the camera. Uh, we know that they're not focused on, uh, the audience. They're focused on actually the stage here. And you are not going to introduce ourselves. Uh, all you need to know is that they are the actual people who can help you and help us. And we are just two developers, right? And we are going to ask some dumb questions, and then you are going to ask any questions that you may have in your mind in terms of audit and compliance and how the DevOps transformation is taking place. So the ground rule is you raise your hand to ask a question. Do not introduce yourself or where you are from unless you want to get into trouble.
Raise your hand ahead of time, and one of us will bring you a mic, right?
And then ask the question. Do not think, uh, about whether the question is relevant or not. If you have a question, please ask. And they're going to, uh, try to answer the right way. Uh, before we start, uh, we can do some icebreaking.
Sure.
Right? So we have some questions. Uh, you wanna ask the first one?
Yeah. So true or false, all that adds value to software delivery.
I think this is the question for you guys.
<laugh>, who thinks audit adds value to software delivery?
Don't feel stressed out that all of us are staring at you.
Okay? So for those of you who didn't raise your hand, who wants to argue that it's of no value, no dissenting opinions. For those of you that did raise your hands. Who wants to argue that it's valuable?
Yeah, thank you. Um, so I'm not saying who we are, but we're in the medical, you know, medical software. It's like our software as a medical device, uh, regulated. And the regulations basically just want to ensure that the software can be used safely and as intended. Like that's basically all of that. It comes to that, right? And, um, we find that in order to ensure that we need to have our own internal controls so we can keep our customers happy, and we care about the patients at the end. And so we're totally aligned with what really the regulations want. Of course, there's a little bit more bureaucracy when you deal with like, particular, um, regulations, but that's what we're struggling right now with. But, uh, we think definitely that, um, auditing helps us achieve the goal we are after. Anyway,
Go ahead.
So I actually raised my hand to say that no, I did not think it added value, but I wanna make sure that I understand that the, the question correctly adds value to software delivery. Was that the question? Yes. Okay. I, I think that auditing adds value. I just dunno that value is in the software delivery necessarily. That's a, a not very strong opinion held very loosely. And I'm, I'm willing to be convinced otherwise. So I, I'll take a first pass at convincing. Like, one of the largest churns in software delivery is rework, right? Like the amount of times as, as, and I said this earlier, I consider myself a recovering developer. I'm on the advisor side and help translate between the, the auditors and, and technologists. Um, the amount of times is a dev reworked, the same stuff was, was astronomical. Now, if audit is part of that internal audit, IT audit and compliance and risk controls are part of that, they're gonna catch it, right? So if I make a mistake that impacts a medical device or EMR and they can catch that compliance thing before it ever gets out of my user story into code, that really helps my software development life cycle. 'cause all of a sudden I'm not wasting efforts, things like that. 'cause I can only know so much as a developer. Uh, I'm not gonna be a compliance expert, but we have compliance experts who can help with that. That's my shot. At least
I would expand it a little to say it's smart auditing, auditing. That's understanding the methodology, understanding what it is you're trying to do, and, um, kind of adapting the audit approach to that. And so not bringing old fashioned, uh, controls and techniques, but actually working with the methodology, working with the way that you are working, um, to essentially not be part of the quality check. 'cause obviously quality need needs to con, you know, be part of the dev team, but where you've got a lot of overlap between what audit's doing and what quality's doing.
I have another concept on this that I talked to developers about. And that is, what if you as a developer, you're annual performance was based on the quality of the software that you delivered. Would you feel the same way? Would you care about quality more? Would you want to get all the help that you could possibly get in doing your job? You know, when you take it down to that level, you know, when you, all of compliance is based on quality. And if we all own quality equally, that really helps, you know, bring everything up to go quicker and faster. And that's what we all want.
I'm interested to ask the panel, what, for the, uh, audit profession, uh, and compliance professionals, what do you think is the biggest behavioral change that is you are seeing is needed or you're seeing is happening or is not happening? I'm interested around the kind of the culture and the behavior.
I'm, I'm not one of the auditors in the room and I, I want to address that. Maybe be kind of powerful since I'm on the other side of the fence. I'm a security person first and used to be a develop, also a recovering developer. What I've seen, at least from a, um, cyber side, is that kind of along the lines of what the folks were talking about here on the last question is we have a lot of people with the same skill sets, especially in internal audit, um, that are all kind of going, uh, trying to achieve the same means to an end. Yet throughout most of my 20 year career, they were kind of pitted against each other. Some, uh, you know, at, at places, particularly where there's cost pressures, I'm seeing a lot of motivation to pull resources, um, on, you know, I would say probably activities that were a little further down the, the chain.
Uh, and they're realizing that as they kind of shift left together, there's a lot of economies of scale to be had. And especially when you're talking about some, you know, things that are, uh, in the security space where there is a war for talent right now, I'm sure we're always battling each other. <laugh> on that front, um, is there's a lot to be gained there. So I see the cultural, the culture shifting to be kind of more inclusive with both groups and realize a lot of those efficiencies, uh, for a variety of different drivers. So you
Use the term internal audit. Uh, I know that a lot of people have questions around that. What is internal audit and external audit and how do they play
Together? You started to broach that
Question. Yeah, <laugh>. Sure. So, um, we're saying audit is very generically, auditors are a lot of different things. Um, the, the main distinction is internal audit and external audit. External audit are the people who are coming in and reviewing your financial statements and giving an opinion over those financial statements over the, whether, whether the information contained in those is fair and true. Um, and that is, um, presented to the market to the SEC, um, or whoever your investors are, um, to help give them some comfort about the accuracy of those financial statements. Those internal auditors are typically, well, not typically are kept at arm's length. They need to be independent. They're, they're called external for a reason. Um, and so the role that they can play in this process, especially in the front end of the process, is relatively limited. Um, however, they will audit, um, and, you know, they will audit in particular for systems that help generate those financial statements or feed into those financial statements, your internal auditors.
And I'm saying that very kind of loosely now, to bring in your compliance groups, your IT compliance, um, your security teams, your, I'm gonna kind of, anything that's internal to the company, they still may audit you. Um, however, they, they don't have those independence restrictions. They're not reporting out to the market. Um, and so they can give a lot more advice and they can be in the room a lot more with you as you are creating and designing controls or, or setting things up. So they're, they're kind of the two main distinctions. There's a lot more distinctions, but, but that's kind of the two different paths. And internal audit can play a lot more of a kind of a, a helpful role along the way, whereas your external auditors need to be kept at arm's length.
Just to kind of add to what Matt was saying there, I think to expand the external too, I think a lot of, I mean, a lot of it's obviously external auditors versus the financial statement auditors, but also a lot of, I'm sure a lot of you issue SOC two reports or SOC one reports where you potentially, these products might not be in scope for financial audit purposes for your external auditor, but you might have it in scope for the product you deliver or the, the, the device that you're creating. So a lot of times those also are also some of your audience on the external audit. But just to kind of jump on, to add to your point, like, how has the mindset changed from an audit pers from an auditor's perspective, I think where I've seen, we, it's really the, the whole DevOps model really pushed us is really, auditors have to get away from a c check checklist based audit. And it really are forced to really understand the business and the flow of transactions through the systems. I think more than ever, we need to really understand how transactions flow and how things work through the whole life cycle of the system to understand where the key risks are and working closely with the IT organization to understand how are they mitigating, how they grading re regression tests or unit tests that mitigate some of those risks that we really care about.
And just to bring that home when we start, and Amy could probably give some great sound bites on this. When we scope the audit for a product to do a level two type of internal review, the first thing as auditors, whether you're the business risk or the tech risk that we ask for, is we go to the architects and we say, show us your, your, your proposed architecture and walk us through the tech stack and all the services that go into it. That's where the risk starts.
Let me jump in and say, the question was about culture. So does anyone want to wanna do a good culture versus bad culture?
I, I can do a quick soundbite on that. So from my perspective, a a good culture is how do we get to a place of yes, there should be yes on all sides, right? The business has an objective. It you wanna get us there, you know, some of the best ways to do that. And compliance wants to enable that. So how do we get to our, our state of yes and bad culture would be you coming to me and me saying, no, I don't ever wanna have to say, no, that's not a good feeling. It's not a good feeling for you and it doesn't make sense for enabling the business and for what you guys do with technology. There are a lot of great ways to make things happen. There's a dozen different ways to perform a control and mitigate a risk. Let's do it together and figure out how.
Yeah. The, the other important piece about culture and this, I've gone gong I've been banging for the past several years and very much today is about empathy and understanding, right? So, so who people, how many people in this room would consider themselves a developer? Okay? So if I asked all of you 10 years ago what a UX person did, you would've said pretty pictures, pretty picture people. But now if I ask you what's a journey map? Yeah, that's something, you know, right? We, we absorb that into our culture. Now, if you go and ask the average internal auditor, explain to me DevOps and agile, they'll say developers have access to production, right?
And segregation
Duties and segregation duties and no documentation. It is on our onus to have empathy for what their problem is and how we communicate it. And inverse, right? That community opening up and going, tell me as you're saying through the architecture, how does this work? And you can go through and talk about all a lot of different things around culture, about what is the behavior we want when something goes wrong, right? What's the behavior we want when something goes well? Things like that. But to me, that that un common understanding and, and not seeing them as the office of no, right? Which is often the perceived perception. Yeah.
Looking at it as a partnership.
So I think I am the one that raised hand for both of it. Does it add value to software delivery? I raise hand saying yes and no, right? So here is my experience. So when we started doing continuous delivery into production, right? The first thing is we go to the, the infrastructure team and say, look, we want to do this continuous delivery. They go, you need to go talk to your, your audit and compliance people. What are they say, what, what did they tell me? They told me, you know what, you cannot do that. You need to have segregation duties. So compliance, it goes on. And I ask them, why? Why does it matter? Because all we are trying to do is make sure that wrong, malicious, whatever things does not get into production. I ask them, so why can't I just show you what is the compliance route? How did it go from dev to production? The answer was right. So we have to go from that place to a place where we are today. And that was a very, very rough journey. So does it add value to production? I think it's in context. The, it's in context of the people that you're working with in context of how much they are aware of DevOps and, and this continuous delivery, right?
You guys have beer in your office, maybe that could help <laugh>. So you gotta take down these mental silos, right? If everybody has the common goal of delivery and providing value for the business. And if leadership on the business side stands up and says, we need this technological approach to drive value for the business and to ex grow revenue and to get us to where we wanna be strategically, DevOps is now a strategic priority for, for, for the, for the business. So don't just think about the compliance people as having to the burden being on your shoulders to influence them. Look who influences the compliance people other than the regulators. Have the leaders of the business and the CEO and whoever else you need, like COO type of person, make it their priority. Yeah. It starts at the top.
Let me, let me just it say, what do you say to the gentleman's experience where you're in the situation where your internal audit is saying, no, you can't. How do you, how do you work with that?
Well, if you happen to have an external auditor, they could be an ally, right? Because when the external auditors come in, they're looking at the work that the internal auditor does as well. I spend a lot of time as all of our competitors up on a stage just talking to clients from a sheer, from a mere educational element and explaining to them what is DevOps work? What's this new type of process? How are organizations utilizing this approach to transform? So once you play that a couple times, the repetition makes it start to sink in and people ask intelligent questions, you'd be surprised. Hello?
Uh, I have a question here, but before that, uh, one last bit of it. I think the problem that you're saying is, is, is failed by many developers. Developers do not directly talk to auditors.
That's, that's exactly one point that I wanted to add is we don't know who the external auditors are. We have no ally.
So
So you got eight people right over here that you're not, that are here to help you now. Yeah.
And I think that's also just finding those folks within your organization. We talk a lot at this conference about the engineering focus, change and modernization, right? That same exercise has to happen within your organization on the compliance side as well. And so it's, it's finding those allies within your organization and then it's, it's taking the time and energy to also educate them as well. Because a lot, a lot of them unfortunately haven't in a sense to use jean's words, you know, seen the light as it as it comes to this here. And, and unfortunately those first conversations can be rough, but it's, it's sticking with it. It's helping them understand the impact to you and your business and the need for, for them to be supportive of, of that end goal.
I, I think it's actually nice that you guys went and asked, I have clients where they tell us we are going DevOps, you're the auditor. Get comfortable with it. Um, so we've sat down with the head of engineering saying, okay, and this is, comes back to my earlier comment. We really have to understand the business and really what are the, the key KPIs that you guys are committing to your clients and also from your own products, like how you recognizing revenue, all those things. So we sat down with the, the head of engineering and say, okay, if you're gonna go down this model, here are the key things you cannot get wrong. And so we sat down with them and they created unit test or regression test, a specifically test that functionality every time you push code. And that's how we get com. We, we got comfortable with it.
Yeah, I think one of the ways that, uh, I'm also a security guy by trade and we have this experience that I think has gotten much better in recent years where security is considered and we now have that DevSecOps name or whatever acronym you you feel like using around that. But in the same way that we would say, Hey, um, dev team, you should probably talk to the security folks earlier in the SDLC to make sure that you bake in the right things. Um, and I say maybe that means cutting a ticket so that every six weeks or whatever periodicity makes sense, you have a conversation. It can be that tactical, and I'm exaggerating for the sake of storytelling, but hard coding, those conversations with the people in this, um, example, IA or, or whatever auditors we're talking about, um, at least in the beginning of our journey, can enforce the right behaviors and the muscle building between those two silos.
Change the culture from the top. Those you that have influence start, you know, the way I always say it to people is, look, you need to change the mindset from it being any compliance function, whether that's kinda internal audit or otherwise change it from being a, a tax to being a competitive advantage. If you kinda live that mantra that brings together the culture, all these points that, that all these great points people are saying, that kind of seems to resonate. 'cause nobody likes to pay taxes. <laugh>
1, 2, 3,
So much of governance, uh, or at least a lot of the conversations I wind up in, uh, at some point start to mention co um, from soc. I'm wondering to what extent any of you are active in that. And in particular, co even Covid 2019 is still based on plan build run, which is essentially a statement of waterfall development. I'm developing the opinion that plan build run is no longer suitable as a basis for governance is. So this question is two parts. Is covid relevant? Should it be changed?
So we talked a little bit this morning about you have a control objective at a high level, and then how you implement that may vary depending on your delivery model. I'd say the holistic principles that you just nicely articulated, you could apply those to, to a DevOps world. It's just repeating itself. You know, in Microsoft's case, what is it? 85,000 deployments a day you guys do. So, you know, it's about the application and how you implement the, the principle, even like accounting and basic auditing standards. Their principles was with a loose guidance of how to apply them. The facts and circumstances that you apply them to will dictate, you know, if you met the objective.
Yeah, I would agree with that. I think the one thing it does is it really pushes the boundaries of the tools you're using because you can still do that same waterfall approach as long as you're using the tooling to kind of automating the whole process through that.
I will add, let's, let's talk about the straight up frameworks for a second, right? Covid has its issues from a, to your point, the somewhat dated methodology. Now people found ways to satisfy those aspects of it in a modern realm. And you can also see the progression. So like what, what is the latest? Is it, I tell V four is what they're calling it. It's basically DevOps, right? You look at that framework and you're like, yep, connect the dots. Cool. There is an industry shift going on, right? I I argue that right now we're at a, we're at a key nexus point, right? I I described it earlier as like we went from being a garage band, you know, punk band in the garage to now we're on a major label, right? We're at that sort of nexus point, um, where I think a lot of the things are starting to change. You wouldn't have us up here having this conversation unless this wasn't the moment. But when you look at that moment, there's this, this uncomfortable pivot, right? Where some of those things of like, oh, that framework that I said we're gonna follow doesn't match the thing. And I think you are at that point where it is, it is shifting and because it's being more iterative, it's much more about these inner relationships and, and understandings.
Well, and I would look at that from the perspective of that's verbiage on, on paper and it, it may function that way, but you have to look at it from the other side of what's the risk that is trying to mitigate. All those frameworks are driving towards risks. The risks haven't changed. Our verbiage that we use around them, the, the processes and procedures, the controls we put in place to mitigate those risks, that's what's changed. And so articulating that back and pushing back on them saying, here's the
Risk still, and this is how we're mitigating it. And the best way I've explained this to coders before is I think most of us probably work at organizations that have data sovereignty or regulatory challenges. When you're doing business at a global scale and you have so many different countries around the eu, you know, around the world, and they have their own data postures, their own DA data rules. If you go to certain jurisdictions, you know, specifically in Europe, they may have some archaic data rules that have been on the books since the sixties or some since World War ii, right? And how do you apply these old regulations that have never been updated to comply with the spirit of the law. So you have to think about how do you comply with the spirit. I think we had a question. So, hello. Um, I guess where my struggle is, um, is with the way that this engagement is, um, and let me explain that. The, the developer community is expected to get non-functional requirements. Very clear. This is what you do. This is the chain of custody, this is the segregation of duties. These are all the requirements that you need to fulfill.
And, and, and that's very clear, right? These are all the rules that you must follow. Um, in contrast, the auditors that we, we heard from this morning, four of you, um, said that you believed in DevOps values and you thought that DevOps was a good framework for, for audit. After that, a lot of conversations that I had, um, people were, um, saying, you know, that would be an interesting video to play to my auditor. 'cause I can bet my auditor probably will not agree with that <laugh>. So why is that internal auditor, you mean <laugh>? Why is that a, a critical mass that needs to be achieved? Why is there not a clarity there? Why is there not a standard body for auditors who could be convinced and who could say, these are our standards, these are our rules, these are the rules that we live by.
Just like the developers are supposed to live by a certain set of rules. These are the things that we will accept. These are the frameworks we will accept. These are the changes to the covid rules that we're going to make. Uh, because we understand that it's outdated rather than achieving a critical mass or, um, talking to enough people and making enough change happen or making sure that your auditor likes you or having to buy them beer. Why is that the conversation your auditor should be buying you beer <laugh>. I just wanna be clear on that. But, but, but it, you're taking a, a binary view as, as a coder, as a programmer would to give me the NFR and I'll implement them. But that's not how it works. 'cause your data sensitivities are different. Your different services are different. You may get some levels of coverage from a SOC report from your cloud provider for some services.
Others you may not. So there's all of these factors that come into play and de and depending on the type of functionality and the type of compliance mechanism that you have in place, somebody mentioned medical devices, somebody may have to comply with, you know, the, the confidentiality pr, uh, principle from a SOC report or high trusts or, or PHI, you know, how you comply with all those depending on the tech stack that you as a developer wanna use. And every tech stack and technology is uniquely different. So the approach on how to comply is never gonna be the same with all these different differences. And we want to empower you, the developer to use the best approach that you believe from a technical lens is needed. And we have to partner with you to find the right answer to mitigate these risks. So it's never gonna be gimme the NFR and I'll implement them. It's very much an art and not a binary science.
Ultimately it comes down to the risks. And, and that's, that's the shift in the audit world as well from really as mentioned earlier that that real checklist approach to what are the true risks in your organization. Some of those historical controls or checkpoints that you had may no longer apply because they're not providing any value. And, and so it's, it's really that interaction between the process owners, in this case, the engineers and your auditors to have that open and frank conversation of, look, this is our environment, this is how we understand it. These are our risks and this is how we've determined how we're going to address those risks. And so your auditor may have an opinion of whether yes or no that those risks are addressed, but now you're having a conversation around the substance of, of how you move forward to, to actually do this versus just the the no or the, the hesitation that you've gotten in the past.
I, I think, and I use this verb, um, on purpose, I, we empathize with the frustration. I think the eight of us on the stage here now are here because we want to be part of the solution with you and have our colleagues where we work be part of that solution. Um, but it will take time. But as Sam and Topo opened up this conversation with, uh, we'd like to think that this, that OPS enterprise summit is a signal and um, hopefully an inflection point that to the outcomes that we're all hoping for together.
The other thing I'd say is, let's be real. If you're a developer and you're at this conference, you're the minority, right? You're the minority of developers, you're passionate about this, you're changing this, it's then you on the advocacy to go the other way. So this year is, was a turning point for me. I presented at two isaka conferences. ISAKA is, uh, Isaka is a, a, uh, a bounding organization around, um, IT audit, right? I presented including Guy Hubert who is a, his best title in the world. He's a risk futurist at Atlassian. He's their chief head of risk, right? And we were both presenting about Agile and DevOps at an IT audit conference. And then I go and I speak at DevOps world and I go, you guys should talk to the auditors. We should bring this together. So just as much as as, as it is them educating, it's also us as an advocacy, uh, group for tearing down a silo, go into those communities, right? And, and have those conversations. 'cause it is as much about that, that learning and understanding. And they do have groups like this just as much as, as we have, you know, meetups and things like that. So maybe next time you should invite some developers to come to your
Conference. He says a question topo, he's been waiting patiently. Yes.
Alright, so my question is kind of two part. First of all, have you had success having conversations about modern DevOps practices with the United States government, particularly the OCC or FFIC on the financial side? And then secondly, um, if you have, what, what has made that successful and how have you reconciled their handbooks, which can be as much as 15 years old and very focused on waterfall methodologies. And they come in and they have very prescriptive thoughts on what your, your risk and compliance and audit focus needs to be based on those handbooks?
So I have an interesting story on this one because, you know, again, I'm a, I'm a financial auditor. I went to the technical side and now, and I'm a risk manager. But we as an audit business, and if you think about how old auditors are, like, we have a several hundred year old profession, we went through a massive transformation in terms of how we do our work, changing things using enabling technology to really shake up how we do audits, you know, from a qualitative standpoint, from a values standpoint. And that's a big transformation. So a lot of the government agencies recognizing that the audit community has gone through a lot of transformations have reached out to us. And I've found myself in multiple agencies having a discussion about transformation. What does transformation look like? How do you transform? So our consulting practices, everybody's trying to help the government transform.
And depending on the agency, they, they may have, you know, different commitments that they've made in terms of what they want to do. But that is being driven by the business. When your business drives your transformation, the risk side of the house comes along with that and has to look at themselves internally and come to a conclusion, say, well, if this is what our leadership wants, what do I have to do? Or what do we as a government IT organization have to do to modernize, to meet the needs of the business? So that should never be just on the, you know, on the lap of, you know, the IT audit and the compliance group, but it's really a business objective. And, and we have this concept internally that, you know, the cloud is not a technology strategy. The cloud is a business strategy. So when you look at it through that lens, and we talked before about the tone of the top that opens the doors and the conversation just really changes.
Um, the, but specifically to those CC and FFIC, um, those handbooks are old. Um, and they're definitely out of date. Um, and I think that's acknowledged. Um, we've been through a couple of exercises where we actually take those principles or those requirements and map them through to a methodology like safe, um, where, and we say, Hey, this is what you're trying to achieve here and this is where we achieve it over here. And it's not the same thing. It's not one for one, but we're achieving that objective. So it's a bit of a slow process you have to go through, you have to do that mapping, and then you very much have to take your regulator on that journey with you. So whoever's in the trenches with you, um, whether it's, um, remediating a finding or, or doing an audit and actually work through and say, Hey, it's not that we've just ignored it, we just think it's got covered in a different way and just kind of go one for one, um, through, but yeah, they, they, it doesn't cover them off neatly. And so it's kind of on you, um, either as the technology organization, it probably has the technology organization to actually say how you are achieving those objectives. So you kind of pull it back to, rather than the specific activity, what's the objective that activity's trying to achieve? And here's where, where we achieve it, even though it looks and feels different.
I'll just add too, I'm, I'm definitely not in our federal practice, but at least once a month I am pulled into a pursuit with our federal practice because they're asking for our approach for agile DevOps, right? So I can't, who's the design arm that the federal government spun up at some point, something 20, I can't remember, but they, they had built up F at TF, right? They built up an agile standard methodology that they rolled out from design thinking through CICD. And every single proposal I get from Fed at this point is talk to us about your DevOps principles and your agile methodology, and what is the tools that you're using that is becoming standard expectations from Fed. Now, you brought up an interesting point, which is, look, we say DevOps like it's a single thing, right? And it's got a single definition. And what I'm gonna do to implement DevOps in an organization that is a mainframe based legacy application is inherently different than a functional programming, uh, go cloud native Kubernetes thing, right?
But all of my tasks are gonna be in a task management system like, like Azure, DevOps or Jira, right? I'm gonna follow a standard process process and even when I'm doing a legacy Waterfall thing, I can still use those tools, right? We've been referring to this as, uh, as as, uh, waterfall with modern tools, right? To enable those sort of scenarios. So it's, it's really, we need to be really conscious of, of what are the delivery models within DevOps, not just going, I put it on the DevOps. So now I'm the Agiles. I, I just wanna share real quick that it's not just the civilian
Side that's asking for support and how to transform into, into this way of working, but we see requests coming in, in, in the military side as well, which is exciting. We do a lot of labs where we'll bring in the folks from specific government organizations and talk through it, white space it do the mapping. Again, you're gonna try to take a substance over form approach, like was articulated in terms of the objectives. And again, it's about getting all the people together, but that requires tone from the top.
I, so, um, I was wondering if the community as a whole, the audit community, the external audit community to be precise, has a community of practice developing as well around either DevOps and or agile and or lean. And I'm asking because to put a shout out to both KPMG and PWC quite a few years ago when we started our first implementation of an application that came with a tool set that did, um, implement continuous deployment capabilities, though our processes didn't allow for it. They were the ones that actually helped us teach our internal auditors, right? So that was great. Um, and so now as we're really adopting it wholeheartedly at the enterprise, our internal auditor at least, there was one small group that actually understood what we were trying to do, and they've been really helpful. But I'm wondering if there is some community of practice or some organization, um, informal organization or formal that we can point other people to, because it's great to say, yeah, go to your external auditor, but as other people have pointed out, number one, not all the teams have visibility to their external auditors. I did, I was lucky. Um, also, we're heavily regulated, like that gentleman over there, various government bodies who I, I I was really happy to hear you say you've been mapping it to safe, but our arm of the Fed clearly doesn't know that based on the fi the questions we've got this year. So it would be really awesome if, if we could sort of point them.
So I think, I think my colleague over here, Yosef has an answer, but I want to repeat. She said KPMG and PWC. Just to be clear, that's the words I heard. Okay, your turn buddy
<laugh>. I heard Arthur Anderson. Oh,
Oh,
Just kidding. We gotta have a sense of humor here. But, you know, there is a consortium of internal auditors. It's called the I A A and they do talk about cutting edge technology. Now, I know if Jean were, were here right now, he would stand up, grab the mic out of one of her hands and say, Hey, when Sarbanes Oxley came around, we actually helped one of write one of the chapters in the internal audit approach to testing controls under a COSO perspective. And you know, gene, and, you know, a lot of his colleagues helped create that for the Institute of Internal Auditors for compliance. You know, maybe there's another chapter that has to be added to evolve it. But, you know, I would say cloud is definitely talked about within the internal audit space and within the industry.
Uh, goes I have a question over here. Yes. Uh, he was raising hand. Uh, you go first and then you come.
What guidance do you have? If, um, there's an IT team wanting to go to compliance for the first time, make an assumption that a couple of times it didn't go well, they just shut everything down. What would you recommend or a 90 day period to bring the compliance team on the journey? What should we do? We'll buy them beer, we'll do all of that stuff, but what else can we do? Can we bring them into a Dojo setting? Can we, um, can we bring them to the development teams? What would your advice
Show them this YouTube video? I mean, we're all internal auditors. We we're external auditors or security professionals that deal with DevOps and compliance every day. So it works. So that's the first thing. You have to make sure people understand that it's something that can be done. Then it has to be a strategic priority. In other words, it's something that they must do for a business reason. Once you overcome those two obstacles, then you're into, okay, well now how do we get it done? You gotta break through the barrier and then get into the implementation.
I, I don't mean to sound, um, maybe patronizing <laugh>. I hope this doesn't sound that way, but I would broker those conversations with an alignment to the broader mission, right? If both groups have the shared mission and we can at least agree on that, that's a starting point for how do we get it done together, which Joseph was saying, um, it's oversimplifying it, but I find that that's helpful.
Where we seen to be most successful is when those compliance folks are part of your liver and deployment team of, of rolling this out, right? That they're in the planning sessions from the start, that they're, you know, brainstorming with you around those risks and how you guys are gonna kinda define those risks, right? They're, they're part of the process from the start it, where it becomes more challenging is you've, you've developed your pipeline, you've developed your process, and now you bring the compliance folks in, you say, certify this, tell me that it's good so we can go forward. It's much easier to, to kind of guide, guide the path with them there from the start.
Yeah. We have a, a couple of examples where we've worked with them kind of from those early stages and you get the internal auditors or whomever in the room as early as possible. And probably our big, our biggest success story is they gave multiple feedback points along the way. Um, and so the, when we first asked them in, they were like, really? You're asking us in? Or, okay. Um, and then it wasn't like, Hey, we've done this. Give us feedback. It was like, Hey, we've, we've done a little bit, what do you think? We've done a little bit more. What do you think? We've done a little bit more. What do you think? And so we weren't getting to the end and saying, Hey, isn't this great? Can you sign off on it? We were giving them multiple, um, kind of iterations along the way to, to give that feedback back and give their input.
I, I think too, like very few places,
Except for the military, is truly a hierarchical organization, right? Like, raise of hands, are you in a matrixed organization? Like I know that I am, I know that this guy is right. Finding those advocates, right? And, and, and, and an interest, uh, person who can be your advocate that is aligned into that organization that you can build a relationship with that may not be the lead of the organization, but isn't an influencer and, and have them bring it along, right? It's kind of like bringing your friend to a party who's then gonna spread the word to the other friends to come to the party as opposed to, well, we went to the head of of internal audit and they said no. Right? Finding those, those advocate
Groups and we're everything we just talked about. It's cultural and soft skills and change management. It's, it's not technical in nature. It's how do you get buy-in, you know, to be an enthusiast and then go to the next level of being a, you know, a person of change and that, not, not just a change advocate, but being a driver of change within your organization.
I think one thing that the audience may have gotten from us is that we keep repeating, you know, bring them in early, bring them in often. Um, that sounds, um, you know, a little bit blunt. There might be a strategy where you say, Hey, you know, we've got a few experimental projects over here. We're gonna just see what happens. You know, invite these guys to these three small things, very low impact, uh, you know, requests from the business, not the big one, you know, with massive strategic implications. You wouldn't for the first time broker the conversation there, but, you know, build the muscle small. Like it's just like going to the gym. You're not gonna go bench three 50 your first time there, you know, work your way up to it on smaller gigs
And just engage them as, as partners. I mean, have that kind of attitude. I think the fact that we've got folks up here that are a mix of auditors and non auditors, and we obviously work very closely together, speaks to that and speaks to the success of that.
And I, I would just add making sure that you can ask them what are they really concerned about? You're going down this path. Help us understand what, are you guys real? Why is this a roadblock for you guys? What are the, some of the, the requirements you are stuck on? And let us work together to figure out what those are. So
Let's try to close this out with a great story, right? You have to be able to, to effectively drive change to be an unbelievable salesman or storyteller. So internal auditors don't always want to do a lot of work. They're, they tend to be, if it's the old fashioned way, their checklist space, they may be doing the same thing every year unless the risks profile changes in the organization. Paint a picture. What if internal auditor, we create a paradigm where you will see in a dashboard all of the exceptions for whatever the specific control is. And you could actually see not just the exceptions, but how they've been resolved by clicking through a screen in front of you from your desk. Is that a future that you think would help protect the business and drive efficiencies? Everybody's gonna say, where do I sign? So paint a picture of what the future life of an internal auditor or an internal compliance specialist is gonna look like in the, in the data heavy DevOps world where you could drive compliance through automation.
Uh, my question's around, uh, PCI, so, uh, I know you guys brought up the, the having the segregation between dev and the production and DevOps, we're doing the opposite, right? We're having developers ac giving access to production. Um, do we expect that PCI requirement to change? And then also, um, if we're being audited now, um, is having the controls around the deployments and then also having logs, uh, going to an external system sufficient to pass an external audit.
Yeah, so I'll, I'll take a stab at that. Uh, I'll start with the first, the segregation of duties at thing. No, dev actually wants access to production. That's terrifying. It's horrible, right? The actual value is can you put controls in place? So I've got a guardrails to work within, right? And then that system creates the ability to have tracking of all of those things, right? So if all of a sudden I have the guardrails in place, the system will control whether I have multiple people approving pieces, right? So I don't see that going away. I actually see the guardrails improving. I've told this story multiple times today, but it makes sense to me. Like it wasn't the FBI that stopped people from stealing music. It was iTunes. It wasn't the FBI's compliance and governance and scaring everybody into it. It was Spotify right? Now, if you apply that to our DevOps model, well yeah, if I'm using a connected tool chain that's enforcing my compliance policies about, you know, segregation of duties and uh, tracking of that, uh, end-to-end piece of it, then it's, I'm living the compliance because it's supported by my, my tool set.
I don't see that going away. I actually think it gets more so because infrastructure's code should not be developed by the same people writing software to
Run before we come. Uh, quickly, uh, you mentioned, uh, fully automated systemic change of production systems and then you also said about multiple people approving it. So can you kind of,
Lemme try to talk about this. So if you think about like your chain of command on a deployment, if security has to sign off on a deployment, if DevOps have to sign off on the deployment, all the different stakeholders involved have to systematically say, yes, this is really ready to deploy in multiple environments. You've just made your world better than it is in waterfall by design, by definition. And all of that has complete audit trail. 'cause it's all systematic. That's number one. Number two, when you think about fundamental access to production in the old world, there would be a few individuals that would always, always have access to going and maybe their access was logged, hopefully, you know, and somebody was monitoring it. Maybe today's world, the individuals have the potentiality to have access to production where they will go and request a temporary, you know, 24 hour or six hour time span, you know, code that gives them access to it so that it's not standing access, it's potentiality to get access, or they have to open a service ticket and state why they're going in and then it's removed. That is a new world, you know, on-prem, you don't necessarily have that unless you've invested in those capabilities
And, and even better, right? It's actually nobody who has access to it. I'm using a GI ops type system where I commit a thing, it gives me just in time tokens that opens up for the automation only. And you govern the automation like a human right, as who has access, et cetera. But the key is being deliberate about that. So like one of your other questions was like, hey, if I've got all this, what I call like data exhaust from the system that gives me the logs and the traceability, yeah, but this is a human issue. I'm not probably gonna trust that you have the data exhaust. So I'm gonna want to trace bullet through that whole thing. And that's the gaining the trust of that, that organization.
I'm still confused about, uh, multiple people approving production deployment and a fully automated, how do they line up? Where do the people come in?
So I'll give it and then I, I'm, I'm I'm mic hogging. Um, I think there's a couple things there. One, which is I had a customer is like, I want, I want my dog to be able to approve builds and there's rationales that should be fully automated, right? The reality is that is not the case for most things. There are many things that be fully automated based off of a risk threshold, right? There's other ones which may be notification based, and someone, let's just say in ServiceNow gets something that somebody has to approve that then kicks it back to another system, actually build it. There are still some human controls you might decide to be in that system. Some of them might be automated controls that it passes all of the tests, right? I, I ran through Sonar Cube and it got a low enough threshold. It passed all a certain level of threshold. My unit test, I'm not impacting more than X systems low. It passes through the workflow and says, yeah, go ahead. But that was something that this guy approved the workflow of that this was an agreed upon setup
About
The guardrails. I think it's what's important here in that conversation is organizations defining their process and their risks, right? Different organizations have different risks, different thresholds, and there, there very well could be cases, right? That require no approval up to, to multiple levels of approval. But it's, it's how you define and understand your environment, how you lay that out and say, look, this is how we look at our world. This is what's critical. This is what's important. And these are things that are, are just kind of day-to-day operations that, that are, are lower risk for us, right? Each of those defined kind of thresholds are gonna have potentially different approval requirements. And so it's, it's really back to how do you understand your own environment? How do you determine those risks within your environment? And then how do you explain that thought process to your internal and external auditors so that they can weigh in on, yes, we agree with this or we don't, but now you're having a conversation on risk, on process versus, no, you can't do this.
And I mean, to add to that, I mean you really to, to understand the environment. What can the developer do in production if the code's already compiled and sitting in GitHub, like in make configuration changes in production, but you have a configuration management tool that runs every 15 minutes. Chef Atrides, everything is a production anyway. So worst case, there might be a configurations out of date for 15 minutes in production. So again, it comes back to understanding the risk and understanding the environment. And
I think that too, this is a mind shift scenario. You heard, um, at UHG the other day, on the main stage it said they went from doing a quarterly release to two releases a day. That also means that my risk blast radius became so much smaller. And so, so on on the the risk and internal audit side, they're willing to accept a different risk because what's the big deal? I've got a bluegreen deployment, I hit a certain threshold, I rolled it back. The risk of that and the blast radius of that became much smaller than Big bang release we do every quarter. That infect that, that impacts big bang. Every five systems is higher
Risk. If you have a release once a year, your risk profile is through the roof, the incremental releases, the more releases you have, the less risk you are.
Hi. So, uh, it's a hypothetical situation, right? Uh, so we, we heard about some of the questions were around covid, some of the questions were around the higher level. What do we do about it, right? If you think about, uh, let's say some, uh, large organizations, right? Where all the policy standards controls were written in an era where there were waterfall, where there wasn't automation, right? And uh, all of a sudden they're doing projects to product, right? They're product based, now they're waterfall to agile, they're now DevOps, right? They're adopting cloud. What would be your suggestion? Let's get, say we've been through your two steps and we are trying to figure out how to implement the controls. Would you look at that individually or would you step back and say, do we just need to, because it changes your risk profile changes, your responsibility model changes everything, right? So would you step back and say, are those policy standards and controls and implementations the right ones? Or would you do it one by one? And,
Um, I mean, you really wanna look at it as a, as a whole. Um, and you want to, it's kind of everything you just described. 'cause if you just, if you look at things in silos or in isolation, you may not be covering off that risk. So you've gotta say, Hey, okay, we're moving to this new process. Here's what that new process looks like. Here's where the risks exist in that process. Here's old controls that we had. Are any of those still good or not? If they, if there are some that are still good, pull them forward. Um, then look at the remaining risk exposure, um, controls to, to close any gaps. And then, um, going around and making sure you're updating those policies and procedures. 'cause as boring as it is when an auditor comes in and policies, um, and procedures aren't updated, they're gonna d you on that straight away. Doesn't matter how good the controls are, if the policies and procedures aren't updated, you've gotta issue.
Yeah. Sorry, one more question I can add on. Who do you think in, in your mind is, uh, this approach, who owns recommending this approach in terms of owning, saying this is how we should approach co compliance? Because we are doing all those four things. So there,
There's the spiritual law, but somebody has to own compliance at the end of the day. So when we came out with COSO 2.0, which came out three years ago, four years ago, I forget already when it became effective, it was an update to the Sarbanes <inaudible>, you know, initial framework. So what all public companies around the world had to go through and do is they dusted off all the initial controls documentation that somebody did five years before and most likely hadn't changed unless there was a new process. And they had to now look at all their control mappings and all their policies in light of COSO 2.0. And what people learned when they went through that exercise is that why did we not go back and change this? And why were we forced into a way of operating just because at a point of time, this is how we did the mapping. There's nothing that says you cannot adapt and change how you comply with the control objective from a compliance lens. So somebody has to, you know, take it upon themselves to transform within the organization and challenge the way we've done things to make it better. And that doesn't mean that you're not complying, it means you're redefining how you're complying. And there's nothing against that.
And like in terms of ownership, that's, that's, that can be challenging. Um, it may be an IT compliance group who kind of quarterbacks the process, um, but it, there's no easy answer on that one. Um, but it's a case of finding someone and kind of pinning them to it, um, especially in this transformation state and saying, Hey, you know, maybe it's your VP of business agility or whoever's driving that, and now they're not gonna do it, obviously, but having someone who's on the hook for it. And then to your safe's point, looking at that at the ongoing, the ongoing process and making sure you don't just blindly follow these policies that were written down and keep iterating on those as you change and as you grow.
We have a question. Yeah.
Hi, good evening. Um, so as an engineer, I practice failure. I deliberately break things in test. I practice major outages. How do I practice failing an audit? And, you know, is there any kind of techniques or am I really stepping into a hornet's nest?
So there's a concept that we refer to as a defendable position. So if you want to take an approach for something, and it's not exactly how, you know, maybe your process is laid out, but you still believe that it complies with the spirit of the law. Ha make sure you could defend it. So if you wanna go through an audit and have an, an approach that's set up and I'm audited and you know, and she's audited because all the 54 applications we support go through internal audit, go through ISO readiness audits. So we're audited and we're, we're sort of the auditors and we have audit applications, so we deal with this literally off boat all the time. But if you take a position, you gotta have a story to tell. So when you go through that life cycle, why are you willing to deploy code if you know that there's known frailties in it? Because nothing's perfect. There's always frailties. We know that, you know, there could be tons of different things that fail from a functional perspective, but you take a risk-based approach as to why you're willing to go to production. You know, with all of that, that's a defendable position.
I think one way to look at it as well is maybe in almost building on your question as well, when you ask who owns compliance in this space, right? Who owns the DevOps process, right? Those are the owners of those controls of that process. And, and just like you're trying to constantly improve your product, right? Similar from a compliance standpoint, right? It's, it's another thing to regularly look at and say, how are we doing this right? Is this the right way? Is this the best way? Is there a better way that we can do this? And unfortunately from a compliance lens, or Yosef mentioned it, right? Controls stay static until they're forced to change, right? You look at an organization, they say, we, we've been doing this, you know, since we implemented for the last 10 years. And that's the mind shift that also has to happen, is how do we constantly improve the way we're, we're meeting our compliance objectives?
And, and it, so to take, to take the very engineering response to that. So, so we created, uh, you know, chaos engineering and, and all of that stuff to kind of break our own stuff, right? So the same rationale I've been trying to push is compliance as code, right? Not policy as code, that's, you know, configuration everything but compliance as code. So pairing a developer with a risk person to say, yeah, try and write a test that commits code and then approves your own pull request, right? Try and break the system and then you can continually run that on every single build, right? Um, and, and that really means just like a content expert, uh, would, would be helping the developer with the functional requirements around a, a user story. This would be a risk or compliance person who's helping you write the test that proves that the control works.
And tactically, if you're bringing in these risk and compliance folks early on in the process, you can have 'em go ham on an a QA environment on a lower environment. If you wanna fail an audit and fail it fast, have 'em look at one of your lower environments and say, here, come in and take a look. Tell me what I'm doing wrong.
Sorry, follow up. Uh, I know if you can hear me quick follow up to that, uh, compliance has coded, right? If you think about securities, uh, sorry, it's related. So think, think about security, right? You have secure coding, you have threat modeling that we're starting to teach develop developers, right? Is there something like that from a compliance perspective that you guys, you said compliance as code, but is there something like that? Well, from
An audit perspective, you know, actually we, we try and get developers to push bad code because they always pun to these downstream monitoring controls that will catch anything. So we're saying, okay, well if you can push this code, let's see if it falls out on the back end. So I mean, happy to talk to you. You know, I work in the ad tech space a lot and so, you know, one of the things are they'll never charge for duplicate clicks. I'm like, okay, now push code that all of a sudden now you're gonna start to recognizing revenue on duplicate clicks. It's so hard to push code. And so some of these things we definitely try and see if we can test that monitoring that they have on the back end that would detect some of these things that we really care about in the bad code. So,
So also just following up into that, where reliance on compensating controls can go wrong in a major way. If you always point to the same compensating controls in addition to where those controls serve as a primary code, you have so much weight now and so much dependency on it. And if that breaks, you're failing a lot. So pushing things and being reliant on other compensating controls, you have to really be methodical in terms of the frequency that you do it. And think about what we call aggregation risk. What's the risk if this control now breaks to everything else in the system?
Hello?
Is
The three lines of defense model dead? 'cause I hear a lot talk about IA being involved in consulting and doing things that, you know, where I come from, you don't do as internal audit. What's your perspective on that?
Uh, I've got one and I mean, I think the three lines of defense are working much more collaboratively with each other. Uh, it doesn't mean that it's necessarily dead. I mean, I, I, in fact, I think it's actually strengthening it. Uh, we're in, you know, there's, I I'm seeing projects in the last 12 months, not just even with the pipeline, but just in the whole security portfolio where we've got projects that are chartered paid for by the security organization and the internal audit function, and they're working on it together. And I'll give you one one specific example. We're doing, um, a security assessment where both parties are paying for it within that same enterprise. And we're, you know, we're doing a kinda a data driven assessment. Uh, we're doing a traditional kind of subjective type of, of, uh, interviews and things on top of it, and then we're taking that same body of work and we're gonna go and have, write an audit report over here with it and a security roadmap over here for the security organization. They've saved a ton of money. They haven't wandered out of their own boundaries in their three lines, and everybody's benefiting because they're collaborating and they've adopted that partnership type approach that we've all kind of been intimating. So
The three lines, you know, for many, many years, if not decades, they were siloed. Every single one was siloed. They didn't talk to each other. It was a checklist based approach. And that was it. Organizations today with transformation, with the speed of, of technology, the collaboration that's needed, the knowledge here that's needed, the technical skills that are needed, you're still maintaining that objectivity in terms of being able to, you know, to, to apply those principles. But it's much more flat. And it's, and those silos are really just from a budget lens,
It's about speed. I mean, you, if you, if you don't move faster, you die in this culture, right? And so they're doing it out of survival and you don't need to break down kind of your fundamental role in the organization to move faster. You just have to move more collaboratively with each other.
Yeah. I have a question. Uh, so once a year, we have external auditors who come in and audit us for, uh, ISO 9,001 and TL 9,000. And they come and audit various products that use different, you know, styles, whether it's waterfall, agile, and, and the, the DevOps styles. Uh, and then they fi they, they find they have some findings and then through some kind of a magical formula, they come up with major nonconformances, minor nonconformances, and sometimes, uh, things that we need to improve on. My question to you is, and specifically on the DevOps style, in your thoughts, what is that criteria that they use to say, oh, okay, this is a major nonconformance versus maybe a minor nonconformance?
So, so every firm has, has their own approach in terms of classifying errors and, and, and findings and translating them. So the language you just used, you know, it's probably specific to a particular audit firm, but in a nutshell, right, there's different criteria that's typically risk-based that leads you to a level of severity, however it happens to be defined. And there's different factors. There's the frequency of, of an error, there's the severity of an error. There's should there have been a compensating control? Was there bells and whistles going off? And you actually had the logging, you actually had the monitoring, but nobody did anything. So it really depends on the, on really the facts and circumstances of the findings that potentially could lead to different buckets of severity.
And especially with iso, they have this specific domains, I'm more familiar with ISO 27,001. So if a major non-conformity is you, you cannot meet one of the criteria, right? And so that, that could potentially, they can pull your certificate. So there's some of those things where they're looking at and then, uh, minor nonconformity might not not been in, in compliance with your own policy. And then, um, areas for improvement, typically it's just like, you know, we're adding value to your business, but
If you do have an intern audit department, they should come in before the auditors do a readiness exercise.
So
Last question, go ahead
To add, add on to that. So I understand the determination of risk is gonna be very specific to the, to the company, to the application, to the environment, uh, but are there any specific controls that would be common that will determine risk? I mean, we, we seem to be throwing some examples around suppression of duty. You talked about, uh, uh, you know, vulnerabilities, you talked about test automation. Are there any like, uh, if you do these, these 10 are the most common that everyone in their pipeline should include to judge the risk and then everything else will be kind of context.
So I got this standard answer I give to this question. It's called background checks, right? Because background checks are in every single one of these rules. And typically there's a, you know, standard approach or that all the background check agencies utilize and it is really a joke, but it's facts and circumstances, the spirit is the same. The application is gonna be different depending on the type of cloud you're using, depending on your data, depending on your application. So I'd love for there to be like a unicorn way that it's always gonna be a checklist. But we talked a little bit before the checklist is sort of gone now and we take a risk-based approach and it may vary. Come find me after and we can double click on that. Yeah.
But there, there are gonna be some things that are always gonna be common. So especially around access. Um, and so if you are being controlled by the tool, um, you can't have access to go in and change the configurations of that tool. Um, and so regardless of what those configurations are, that's gonna remain, remain true across the secure pipeline. Similarly, controls around who can change those configurations and making sure that changes to the tools themselves. So more on the ITGC side of the tools. Um, there's definitely commonality in those controls. Now what exactly what they look like is gonna change, but around security access, locking down those pipelines, that's gonna be consistent.
We've got a little bit over one minute left. Um, Sam,
Caleb, you can take a last word if you want. I was actually. Okay. So for those of you in the room, we have a session tomorrow where you can go deep privately and ask anyone here specific questions and you can say, my goodness, ours case is so messed up, what should I do? And for those of you who aren't in the room, but are watching this on YouTube, if your situation is so messed up that you're not doing compliant DevOps, and you think that audit is blocking you, we would like you to reach out to anyone on the panel to say, how do we fix it? So let's give a hand to our panel.