00:00:13

Welcome to dev ops and internal audit a great partnership before we dive into the material rusty. And I will tell you a little, little bit about ourselves, what we do and where we work. My name is Clarissa Lucas, and I'm an it audit director at nationwide insurance, which is located in Columbus, Ohio. I focus on auditing our technology organization and providing it audit support to our business auditors. I'm not your typical it auditor though. Uh, in fact, up until the end of 2018, I wasn't an it auditor at all. I've been an operational and business auditor as well as a risk management leader. And recently got the pleasure of joining the it world as an auditor. This non-traditional background and journey into it. Auditing has enabled me to bring a fresh perspective and challenged our customary way of doing it. Audits. This has resulted in more meaningful views of risk and sustainable remediation of gaps, and I'm not getting all geeked up on risks and controls and audit procedures. I enjoy spending time with my family and staying active by running and lifting weights rusty

00:01:21

Good morning, or good afternoon, everyone, depending on where you are. My name is rusty Lewis and I am an it staff auditor, also a nationwide working with Clarissa in our internal audit department. I joined nationwide nearly two years ago, but prior to that, I worked for PricewaterhouseCoopers for two years in their, ah, it audit practice similar to Clarissa when I'm not equally excited about it, audit risks and controls. I enjoy spending time with my wife and our dog Scooby. I also enjoyed recording a podcast with my brother-in-law, where we discuss our passion for video games, technology, and pop culture. Now I'll turn it back to Clarissa to give a brief introduction about nationwide.

00:01:58

Thanks, rusty. Let me get my screen

00:02:10

Right

00:02:19

Rusty and I both work at nationwide insurance, which is located in Columbus, Ohio nationwide exists to protect people, businesses, businesses, and futures. With extraordinary care. We accomplish this through a diverse portfolio of products, including financial services and property and casualty insurance. Our financial services business includes offerings such as life insurance, mutual funds and retirement plans. The property and casualty business includes commercial lines, such as agribusiness and standard commercial insurance, as well as personal lines like auto and homeowners insurance. We also offer some less obvious products such as life insurance for your pets and insurance for your sports cars. We are here for our members and the things that matter most to them. In fact, nationwide is number one and 4 57 retirement plans. And as a number one pet insurer and writer of farms and ranches, extraordinary care doesn't end with our members. It also extends to our associates partners and communities nationwide has been named a fortune 100 best company to work for and best workplace for diversity. Now that you know who we are and what we're about, let's talk about the objectives for today's session.

00:03:40

As internal auditors, we work closely with our clients and often receive questions. As our clients are looking to make a change or implement a new process or pool, they want to make sure they get audits stamp of approval. Before proceeding. We often hear questions like how can we pass an audit when we moved to a DevOps operating model or what our audits requirements for segregation of duties when implementing DevOps practices, incorporating dev ops practices into the delivery pipeline can have significant impacts across the three lines of defense from developers to internal auditors. At the end of this session, you will have learned to change the way you think about risks and controls, understanding why the controls are performed and selecting the best control or controls for a specific risk. In a given situation, you will also learn how dev ops can positively impact your experience.

00:04:33

During an internal audit, we will accomplish the learning objectives by defining audits role, exploring the impact of dev ops on our way of thinking, walking through examples of controls that may look and feel different in a dev ops environment, from what you're used to and taking a focus, look at key risk and control considerations. In our examples, we'll focus on the risks mitigated by the controls. Then you can apply this thought process to other risks and controls and use it to mitigate those risks in a way that makes sense for your organization and operating model.

00:05:07

So understand how dev ops can impact your experience during an audit. You first need to understand audit's role in risk management and how that impacts you in your role. Do that. Let's first introduce the three lines of defense. The first line of defense owns the risk and executes controls to manage the risk. An example here would be the development teams. These are the individuals who are writing code, testing the code and reviewing and approving any changes. The second line of defense is responsible for policy creation, defining a risk tolerance and monitoring adherence to those policies and those tolerances and example of a second line of defense using that same example, using that same example, building off of the development teams, a second line of defense would be information, risk management. And the third line of defense is internal audit. Internal audit provides to the audit committee of the board, as well as to senior management through independent assessments of risks and controls.

00:06:03

We accomplish this through seeking to understand what could prevent the organization from achieving its objectives. And we consider a multitude of risks. We then evaluate the actions that management is taking to mitigate those risks within established tolerances, the tightest together, we conduct integrated audits to evaluate business risks and controls and the systems or applications that support those business processes. Examples of it controls evaluated could include change management, user access and interface integrity. And we'll also look at adherence to that information security policy and any supporting standards. So part of understanding who we are as internal audit is also understanding who we are not, we are not it risk management, we're not compliance. We're also not the external auditors or regulatory body. We're not the development teams or operations. It's important to note that internal audit does not tell management how to manage a risk. Internal audit does partner though with the first and second lines, as well as external parties, as we provide assurance on the organization's ability to meet its objectives.

00:07:09

I think that DevOps is a transformation, both thought and practice, and it requires partnership and collaboration between the three lines of defense or example. If your policies require segregation of duties between developing code and promoting it to production, the three lines of defense need to work together to understand whether this works with a dev ops model, if it doesn't and the organization truly wants to move to dev ops, the policy may need to be revised to allow for a different method of controlling the risk that's currently mitigated by segregating those duties. A little context about Nationwide's internal audit team. We have about 70 internal auditors, uh, with a wide variety of experiences, degrees and certifications. We focus a lot of our professional development on increasing our business acumen and better understanding the areas that we audit rusty and I it's technology. Our technology audit team provides assurance over the technology supporting key business processes at nationwide. So I've touched on the audit part of the dev ops and audit partnership. Now rusty will get us into the dev ops side of the partnership. Rusty,

00:08:15

Thanks Clarissa. This picture is a visual representation of how some practitioners may see the DevOps movement. There's a lot of items at stake and things can go very badly really quickly while modifying the devil development process. Incorporating dev ops into the delivery pipeline can also have significant impacts on assurance providers and security practitioners. And this may cause people to think of dev ops as a bull in a China shop, as this image here, depicts continuous delivery challenges, the enterprise to balance control execution with speed and maximizing automation can change how traditional controls are performed and assessed. As an example, this gives all of us an opportunity to think about segregation of duties differently than we have in the past. Typically segregation of duties is thought of as two separate people in a single process, one person developing the change and another promoting it to production.

00:09:04

We may begin to think of duties being segregated through automation and with the potential higher reliance on other controls, moving to dev ops has a strong value proposition. And while it may change the way we think about risks and controls at a baseline level, the risk and control environment are relatively similar to the control expectations of today. I'll walk through some specific examples in a few slides, but a few quick examples for now would be the expectation that access is restricted as appropriate and changes are tested prior to being migrated to production. Regardless of whether today's change management process is followed or DevOps processes are followed as nationwide adapts, the development process, internal audit is also on the same journey to adapt the way we assess controls.

00:09:47

Now we've talked a little bit about risk, so what truly can go wrong using the example of change management, the primary risk associated with change processes is compromised systems or data availability, integrity, or confidentiality, which can have adverse impact on business operations. Having a strong control environment in place helps you prevent this from happening by preventing the items listed here on the slide, including unauthorized changes, introduced in production and introduction of security vulnerabilities, internal audit performs audits to evaluate controls and determine whether these items are well-managed next. We'll walk you through how internal audit currently evaluates these risks and how that may change as we adopt DevOps practices.

00:10:34

Now, currently our testing focuses on these three key controls. The objective of the approvals and authorizations control is to ensure that all changes related to an application being audited are logged, prioritized, authorized, tested, and approved prior to release into production, depending on the total number of release changes during our scoping period, we typically select a sample between one and 30 changes for detailed testing we'll then request doc documentation, evidencing approvals for sample changes at various stages of the change management process. Our second control segregation of duties focuses on verifying that appropriate separation of duties exists within the change management process to prevent individuals from writing, testing, and promoting their own code to production without independent checkpoints. And lastly, our third control metrics reporting really focuses on monitoring and reporting activities to determine effects of changes on business operations. Now, let's take a closer look at the second control segregation of duties. So we have to ask the question, why are duties typically required to be separated? Let's look at an example. And the old ways of thinking are traditionally a developer isn't allowed to promote their own code to production by themselves. They would need to have someone else from say operations or quality assurance promoted for them. And so then the question again becomes why, why do we need to separate these actions from one another?

00:12:00

Well it's to make sure the code isn't malicious or fraudulent nature, or to make sure the code doesn't create disruption to systems or data issues when migrating and production. And so what does the person promoting the code do to ensure that these things don't happen? They reviewed the code and this helps in detecting and fixing defects early in the process. It also helps to maintain a level of consistency and design and implementation of the code. They review also allows for uniformity and understanding, which will help the interchangeability of team members in the case of non-availability of any team member.

00:12:38

So let's say the organization's risk appetite allows for detective controls or controls that focus on finding and fixing after an event or incident has occurred rather than preventative controls or controls that focus on checking before an incident or event has occurred, what sort of actions or controls could mitigate the same risk here. Now, there are a number of examples here on the slide, but I want to focus on automated reviews or tests. So what if we wrote a script for an automated test that would review the developer's code and promote it to production? Once it's been tested, reviewed and approved, the automated test will then enable developers to more efficiently and quickly review code and on demand. Now the duties are separated, not between two humans, but between a human or developer and a digital worker or bot or code. We also have a mechanism now for providing immediate feedback and learning to the developers and from a promoting code without extended wait times. Now I'll turn it back to Clarissa to take another look at a previous slide outlining are outlining our three key controls of focus.

00:13:46

Thanks rusty. A few minutes ago, rusty, walk us through these traditional change management controls. He just showed us how segregation of duties can be achieved using dev ops principles. Take a look at this slide again, this time focusing at the approvals, focusing on the approvals control at the top of the side, as a reminder, the objective of the approval control is to ensure that all changes perform as expected and that they do not cause problems when they're introduced into production. Now, let's look at an example of how change approval controls might differ in a dev ops environment. We ran into this recently and one of our infrastructure platform audits in this instance, individuals have access to both develop a change and move it into production. We're currently working with management. Who's mitigating the risk that changes can be made without going through the enterprise change management process.

00:14:40

One way to mitigate this risk is to work with the change management governance team, to evaluate the types of changes, uh, to determine which require approvals and might be part, uh, approvals for each change. And which could be part of a change pre-approval process at that pre-approval process is where you would gain pre-approval for the type of change. And these are going to be typically your low risk routine changes and all changes that fall into that category can be made and logged without obtaining a separate approval for changes that are routine. The changes are logged and management monitors, the logs to ensure that only routine low-risk pre-approved changes are made without gaining that separate approval and then changes that are not routine are also going to be logged and management can monitor those logs to ensure that these changes have a corresponding approval.

00:15:39

Now, rusty and I have walked through detailed examples of controls that look and feel different in a dev ops world. This slide shows additional risk and control considerations specific to the dev ops environment based on industry best practices. And this may be different from the controls that are currently in place in a non, non dev ops operating model. In the middle column, you see the risks, these are the things that we're all worried about, not just as internal auditors, because they represent what can go wrong controls on the right hand column are the activities that are performed by management. These are what internal audit assesses to determine whether those risks in the middle column are managed within established tolerances. Now, I won't read each of these bullets instead. I want to walk through an example of how a control mitigates a specific risks and how we would evaluate that control in an internal audit.

00:16:27

I use the example of the risk of implementing and pass it around. I authorized code, which could lead to increased operational failures, poor transaction processing, uh, or unreliability of the system itself control that can be used to help mitigate this risk and manage this risk is automated production deployment. In this instance, the code is automatically moved to production as long as it meets predefined requirements, such as positive test results completion of a peer review or authorization or completion of some other control or suite of controls that mitigates the risk that the code will not perform as expected or that it'll introduce vulnerabilities. Once it's moved into production, the automated deployment control will also reject code that does not meet these predefined requirements, thus reducing the risk of untested or unauthorized code getting deployed into production. How would we as internal auditors test this since this is an automated control, we would check the configurations of the control to determine whether it's designed and configured to mitigate the risk, to say that differently in less audit, heavy terms, uh, we'd review the configurations of the deployment mechanism to see whether it's set up to only move code into production, if it meets the requirements and that it is also set up to reject any code that does not meet those requirements.

00:17:47

We'd also test through or review of evidence after the control has run one instance where the code meets the requirements to see that it does get promoted as expected. And we'd also look at one instance where the code does not meet requirements to make sure that it gets rejected as we expect. This might sound a little different from the audit experience that you're used to. Uh, you might be used to your auditors requesting pages and pages of documentation for sample of 30 or more transactions such as code deployments or changes on the next slide. We'll dig a little bit deeper into the impact to your audit experience.

00:18:26

As organizations are changing their software development processes, internal audit is responding by adapting our control assessment practices to better fit the world of DevOps. We now have a better understanding of internal audits role in the organization and their objective, which as a reminder, uh, audit's objective is to provide assurance to key stakeholders through evaluations of risks and the controls mitigating those risks. We've also shifted our mindset to focus on the risks and how best to address that risk in a given situation, rather than looking at a checklist of controls to implement. Now, let's look at how dev ops can change your experience during an audit and why dev ops and audit are truly a great partnership. When you have automated controls, we can look at a smaller sample for our testing because the automation removes the human error element. If it's set up correctly, the automated process will execute as designed each and every time.

00:19:24

This means that you spend less time being audited. I know we're not in the same room right now as we enjoy this virtual experience. Uh, but I know somewhere somebody is just cheering right now. And as if that wasn't enough as, as if that wasn't enough of an advantage, there's more audit can also extract evidence directly from source systems resulting in fewer requests for evidence, which means decreased interruption to your daily work. And with greater reliance on automated controls with audit trails comes greater assurance through full population testing. This is great news and I'll bet you even like your auditors a little bit more. Now Rusty's going to bring it home with a question on everyone's mind.

00:20:06

So the question Clarissa's referring to in one that we're actually frequently asked is when we begin leveraging DevOps practices, or we have been for quite some time, how do we actually pass the audit? And the short answer is there's really no magic formula or one size fits all approach over the last 30 minutes. You've learned that internal audit purpose is to provide assurance on the organization's ability to achieve its objectives. We do this through understanding the risks to achieving the objectives and evaluating controls in place to mitigate those risks in alignment with the organization's risk tolerance, as you're transforming your delivery pipeline and implementing DevOps practices, bring your internal audit friends along for the ride, help them understand how you're mitigating risks. And don't be afraid to challenge them if they're stuck in an old way of thinking. For example, thinking that segregation of duties is the only way to mitigate a particular risk, show them how the controls you have in place also mitigate those risks and how they can see evidence that the controls are being performed appropriately.

00:21:04

And we've walked you through a few examples of suites of controls that can mitigate risks differently than what we're used to. And earlier, we also encourage you to bring your internal audit friends along for the ride. I can only imagine, especially in our virtual presentation environment, that some of you were scratching your head questioning how auditor and friends could ever be used in the same sentence, but the sooner internal audit can connect with operations and development teams. The sooner that relationship becomes less about confrontation and opposition, instead, it can be founded on trust with a focus on collaboration, creating camaraderie with your peers and risk mitigation.

00:21:42

Now on behalf of both Clarissa and I, we want to extend a very special thanks to each and every one of you that joined us today for our presentation and a special thanks to gene Kim and everyone at it revolution for allowing us to present this year at the dev ops summit. We also don't want the conversation around DevOps and its impact across the three lines of defense to end here. Our contact information is listed on the slide deck, and we would encourage you to reach out to us directly via email with any questions you might have. Thanks again, and enjoy the rest of the DevOps summit.