00:00:07

Welcome to DevOps and Internal Audit, a great partnership. Before we dive into the material, rusty and I will tell you a little, little bit about ourselves, what we do, and where we work. My name is Clarissa Lucas and I'm an IT audit Director at Nationwide Insurance, which is located in Columbus, Ohio. I focus on auditing our technology organization. I'm providing IT audit support to our business auditors. I'm not your typical IT auditor, though. Uh, in fact up until the end of 2018, I wasn't an IT auditor at all. I've been an operational and business auditor as well as a risk management leader, and recently got the pleasure of joining the IT world as an auditor. This non-traditional background and journey into IT auditing has enabled me to bring a fresh perspective and challenge our customary way of doing it. Audits, this has resulted in, uh, more meaningful views of risk and sustainable remediation of gaps, and I'm not getting all geeked up on risks and controls and audit procedures. I enjoy spending time with my family and staying active by running and lifting weights. Rusty,

00:01:15

Good morning or good afternoon, everyone, depending on where you are. My name is Rusty Lewis and I'm an IT staff auditor. Also a nationwide working with Clarissa in our internal audit department. I joined Nationwide nearly two years ago, but prior to that I worked for PricewaterhouseCoopers for two years in their in, uh, IT audit practice. Similar to Clarissa, when I'm not equally excited about it, audit risks and controls, I enjoy spending time with my wife and our dog, Scooby. I also enjoy recording a podcast with my brother-in-Law, where we discuss our passion for video games, technology and pop culture. Now I'll turn it back to Clarissa to give a brief introduction about Nationwide.

00:01:52

Thanks, rusty show my screen right.

00:02:14

Rusty and I both work at Nationwide Insurance, which is located in Columbus, Ohio. Nationwide exists to protect people, businesses, businesses and futures with extraordinary care. We accomplish this through a diverse portfolio of products including financial services and property and casualty insurance. Our financial services business includes offerings such as life insurance, mutual funds and retirement plans. The property and casualty business includes commercial lines such as agribusiness and standard commercial insurance, as well as personal lines like auto and homeowners insurance. We also offer some less obvious products such as life insurance for your pets and insurance for your sports cars. We are here for our members and the things that matter most to them.

00:03:03

In fact, nationwide is number one in 4 57 retirement plans and is a number one pet insurer and writer of farms and ranches. Extraordinary care doesn't end with our members. It also extends to our associates, partners, and communities. Nationwide has been named a Fortune 100 best company to work for and best workplace for diversity. Now that you know who we are and what we're about, let's talk about the objectives for today's session. As internal auditors, we work closely with our clients and often receive questions. As our clients are looking to make a change or implement a new processor tool, they want to make sure they get audits stamp of approval before proceeding. We often hear questions like, how can we pass an audit when we move to a DevOps operating model? Or what are audits requirements for segregation of duties when, um, implementing DevOps practices, incorporating DevOps practices into the delivery pipeline can have significant impacts across the three lines of defense from developers to internal auditors.

00:04:11

At the end of this session, you will have learned to change the way you think about risks and controls, understanding why the controls are performed, and selecting the best control or controls for a specific risk in a given situation. Also, learn how DevOps can positively impact your experience during an internal audit. We'll accomplish the learning objectives by defining audit's, role exploring the impact of DevOps on our way of thinking, walking through examples of controls that may look and feel different in a DevOps environment from what you're used to and taking a focus look at key risk and control considerations. In our examples, we'll focus on the risks mitigated by the controls. Then you can apply this thought process to other risks and controls and use it to mitigate those risks in a way that makes sense for your organization and operating model. So understand how DevOps can impact your experience during an audit.

00:05:04

You first need to understand audit's role in risk management and how that impacts you in your role. Do that. Let's first introduce the three lines of defense. The first line of defense owns the risk and executes controls to manage the risk. An example here would be the development teams. These are the individuals who are writing code, testing the code and reviewing and approving any changes. The second line of defense is responsible for policy creation, defining a risk tolerance and monitoring adherence to those policies and those tolerances. An example of a second line of defense using that same ex, using that same example, building up of the development teams. A second line of defense would be information risk management, and the third line of defense is internal audit. Internal audit provides assurance to the audit committee of the board as well as to senior management through independent assessments of risks and controls.

00:05:58

We accomplish this through seeking to understand what could prevent the organization from achieving its objectives, and we consider a multitude of risks. We then evaluate the actions that management is taking to mitigate those risks within established tolerances. To tie this together, we conduct integrated audits to evaluate business risks and controls and the systems or applications that support those business processes. Examples of IT controls evaluated could include change management, user access and interface integrity, and we'll also look at adherence to that information security policy and any supporting standards. So part of understanding who we are as internal audit is also understanding who we are not. We are not IT risk management, we're not compliance. We're also not the external auditors or a regulatory body. We're not the development teams or operations. It's important to note that internal audit does not tell management how to manage a risk.

00:06:54

Internal audit does partner though with the first and second lines as well as external parties as we provide assurance on the organization's ability to meet its objectives. Shifting to DevOps is a transformation both thought and practice, and it requires partnership and collaboration between the three lines of defense. For example, if your policies require segregation of duties between developing code and promoting it to production, the three lines of defense need to work together to understand whether this works with a DevOps model. If it doesn't and the organization truly wants to move to DevOps, the policy may need to be revised to allow for a different method of controlling the risk that's currently mitigated by segregating those duties. A little context, uh, about Nationwide's internal audit team, we have about 70 internal auditors, uh, with a wide variety of experiences, degrees and certifications. We focus a lot of our professional development on increasing our business acumen and better understanding the areas that we audit For Rusty and I, it's technology. Our technology audit team provides assurance over the technology supporting key business processes at Nationwide. So I've touched on the audit part of the DevOps and audit partnership. Now Rusty will get us into the DevOps side of the partnership. Rusty,

00:08:09

Thanks Clarissa. This picture is a visual representation of how some practitioners may see the DevOps movement. There's a lot of items at stake and things can go very badly really quickly while modifying the dev development process. Incorporating DevOps into the delivery pipeline can also have significant impacts on assurance providers and security practitioners. And this may cause people to think of DevOps as a bone, a China shop. As this image here depicts continuous delivery challenges the enterprise to balance control execution with speed and maximizing automation can change how traditional controls are performed and assessed as an example. This gives all of us an opportunity to think about segregation of duties differently than we have in the past. Typically, segregation of duties is thought of as two separate people in a single process, one person developing the change and another promoting it to production. We may begin to think of duties being segregated through automation and with a potential higher reliance on other controls.

00:09:04

Moving to DevOps has a strong value proposition, and while it may change the way we think about risks and controls at a baseline level, the risks and control environment are relatively similar to the control expectations of today. I'll walk through some specific examples in a few slides, but a few quick examples for now would be the expectation that access is restricted as appropriate and changes are tested prior to being migrated to production regardless of whether today's change management process is followed or DevOps processes are followed as nationwide adapts the development process. Internal audit is also on the same journey to adapt the way we assess controls.

00:09:42

Now we've talked a little bit about risk. So what truly can go wrong using the example of change management? The primary risk associated with change processes is compromised systems or data availability, integrity or confidentiality, which can have adverse impact on business operations. Having a strong control environment in place helps you prevent this from happening by preventing the items listed here on this slide, including unauthorized changes introduced in production, an introduction of security vulnerabilities, internal audit performs audits to evaluate controls and determine whether these items are well managed. Next, we'll walk you through how internal audit currently evaluates these risks and how that may change as we adopt DevOps practices.

00:10:28

Now currently our testing focuses on these three key controls. The objective of the approvals and authorizations control is to ensure that all changes related to an application being audited are logged, prioritized, authorized, tested, and improved prior to release into production. Depending on the total number of release changes during our scoping period, we typically select a sample between one and 30 changes for detailed testing, and we'll then request doc documentation evidencing approvals for sample changes at various stages of the change management process. Our second control segregation of duties focuses on verifying that appropriate separation of duties exists within the change management process to prevent individuals from writing, testing and promoting their own code to production without independent checkpoints. And lastly, our third control metrics reporting really focuses on monitoring and reporting activities to determine effects of changes on business operations. Now let's take a closer look at the second control segregation of duties. So we have to ask the question, why are duties typically required to be separated? Let's look at at an example. And the old ways of thinking are traditionally a developer isn't allowed to promote their own code to production by themselves. They would need to have someone else from say, operations or quality assurance, promote it for them. And so then the question again becomes why? Why do we need to separate these actions from one another?

00:11:54

<silence> Well, it's to make sure the code isn't malicious or fraudulent in nature or to make sure the code doesn't create disruption to systems or data issues when migrated in production. And so what does the person promoting the code do to ensure that these things don't happen? They reviewed the code and this helps in detecting and fixing defects early in the process. It also helps to maintain a level of consistency and design and implementation of the code. The review also allows for uniformity and understanding, which will help the interchangeability of team members in the case of non-availability of any team member.

00:12:32

So let's say the organization's risk appetite allows for detective controls or controls that focus on finding and fixing after an event or incident has occurred, rather than preventative controls or controls that focus on checking before an incident or or event has occurred. What sort of actions or controls could mitigate the same risk here? Now, there are a number of examples here on the slide slide, but I wanna focus on automated reviews or tests. So what if we wrote a script for an automated test that would review the developer's code and promote it to production once it's been tested, reviewed and approved? The automated test would then enable developers to more efficiently and quickly review code and on demand. Now the duties are separated not between two humans, but between a human or developer and a digital worker or botter code. We also have a mechanism now for providing immediate feedback and learning to the developers and from promoting code without extend wait times. Now I'll turn it back to Clarissa to take another look at our previous slide out loaning our outlining our three key controls of focus.

00:13:40

Thanks Rusty. A few minutes ago, rusty walked us through these traditional change management controls. He just showed us how segregation of duties can be achieved using DevOps principles. Take a look at this slide again, this time focusing at the approvals for focusing on the approvals control at the top of the slide. As a reminder, the objective of the approval control is to ensure that all changes perform as expected and that they do not cause problems when they're introduced into production. Now let's look at an example of how change approval controls might differ in a DevOps environment. We ran into this recently in one of our infrastructure platform audits. In this instance, individuals have access to both develop a change and move it into production. We're currently working with management who's mitigating the risk that changes can be made without going through the enterprise change management process.

00:14:34

One way to mitigate this risk is to work with the change management governance team to evaluate the types of changes, uh, to determine which require approvals and which might be part, uh, approvals for each change and which could be part of a change preapproval process. Now, that pre-approval process is where you would gain pre-approval for the type of change. And these are gonna be typically your, um, low risk routine changes and all changes that fall into that category can be made and logged without obtaining a separate approval. For changes that are routine, the changes are logged and management monitors the logs to ensure that only routine low risk pre-approved changes are made without gaining that separate approval. And then changes that are not routine are also going to be logged, and management can monitor those logs to ensure that these changes ha have a corresponding approval.

00:15:33

Now, rusty and I have walked through detailed examples of controls that look and feel different in a DevOps world. This slide shows additional risk and control considerations specific to the DevOps environment based on industry best practices, and this may be different from the controls that are currently in place in a non DevOps operating model. In the middle column, you see the risks. These are the things that we're all worried about, not just as internal auditors because they represent what can go wrong. Controls on the right hand column are the activities that are performed by management. These are what internal audit assesses to determine whether those risks in the middle column are managed within established tolerances. Now, I won't read each of these bullets. Uh, instead I wanna walk through an example of how a control mitigates a specific risk and how we would evaluate that control in an internal audit.

00:16:21

I use the example of the risk of implementing untested or unauthorized code, which could lead to increased operational failures or transaction processing, uh, or unreliability of the system itself. Control that can be used to help mitigate this risk and manage this risk is automated production deployment. In this instance, the code is automatically moved to production as long as it meets predefined requirements such as positive test results completion of a peer review or authorization or completion of some other control or suite of controls that mitigates the risk that the code will not perform as expected or that it'll introduce vulnerabilities once it's moved into production.

00:17:03

The automated deployment control will also reject code that does not meet these predefined requirements, thus reducing the risk of untested or unauthorized code getting deployed into production. How would we as internal auditors test this? Since this is an automated control, we would check the configurations of the control to determine whether it's designed and configured to mitigate the risk to say that differently in less audit heavy terms. Uh, we'd review the configurations of the deployment mechanism to see whether it's set up to only move code into production if it meets the requirements, and that it is also set up to reject any code that does not meet those requirements. We'd also test through observation or review of evidence after the control has run one instance where the code meets the requirements to see that it does get promoted as expected. And we'd also look at one instance where the code does not meet requirements to make sure that it gets rejected as we expect. This might sound a little different from the audit experience that you're used to. Uh, you might be used to your auditors requesting pages and pages of documentation for sample of 30 or more transactions such as code deployments or changes. On the next slide, we'll dig a little bit deeper into the impact to your audit experience.

00:18:21

As organizations are changing their software development processes, internal audit is responding by adapting our control assessment practices to better fit the world of DevOps. We now have a better understanding of internal audit's role in the organization and their objective, which as a reminder, uh, audit's objective is to provide assurance to key stakeholders through evaluations of risks and the controls mitigating those risks. We've also shifted our mindset to focus on the risk and how best to address that risk in a given situation rather than looking at a checklist of controls to implement. Now, let's look at how DevOps can change your experience during an audit and why DevOps and audit are truly a great partnership. When you have automated controls, we can look at a smaller sample for our testing because the automation removes the human error element. If it's set up correctly, the automated process will execute, is designed each and every time.

00:19:18

This means that you spend less time being audited. I know we're not in the same room right now as we enjoy this virtual experience, uh, but I know somewhere somebody is just cheering right now. And as if that wasn't enough of as if, as if that wasn't enough of an advantage, there's more. Audit can also extract evidence directly from source systems resulting in fewer requests for evidence, which means decreased interruption to your daily work and with greater reliance on automated controls. With audit trails comes greater assurance through full population testing. This is great news and I'll bet you even like your auditors a little bit more. Now, Rusty's gonna bring it home with a question on everyone's mind.

00:20:00

So the question Clarissa's referring to, and one that we're actually frequently asked is, when we begin leveraging DevOps practices, or we have been for quite some time, how do we actually pass the audit? And the short answer is there's really no magic formula or one size fits all approach. Over the last 30 minutes, you've learned that internal audit's purpose is to provide assurance on the organization's ability to achieve its objectives. We do this through understanding the risks to achieving the objectives and evaluating controls in place to mitigate those risks. Risks in alignment with the organization's risk tolerance. As you are transforming your delivery pipeline and implementing DevOps practices, bring your internal audit friends along for the ride. Help them understand how you're mitigating risks and don't be afraid to challenge them if they're stuck in an old way of thinking. For example, thinking that segregation of duties is the only way to mitigate a particular risk.

00:20:50

Show them how the controls you have in place also mitigate those risks and how they can see evidence that the controls are being performed appropriately. Now, we've walked you through a few examples of suites of controls that can mitigate risks differently than what we're used to and earlier. We also encourage you to bring your internal audit friends along for the ride. I can only imagine, especially in our virtual presentation environment, that some of you were scratching your head questioning how auditor and friends could ever be used in the same sentence. But the sooner internal audit can connect with operations and development teams, the sooner that relationship becomes less about confrontation and opposition. Instead, it can be founded on trust with a focus on collaboration, creating comradery with your peers and risk mitigation.

00:21:36

Now, on behalf of both Clarissa and I, we wanna extend a very special thanks to each and every one of you that joined us today for our presentation. And a special thanks to Jean Kim and everyone at IT revolution for allowing us to present this year at the DevOps Summit. We also don't want the conversation around DevOps and its impact across the three lines of defense. To end here, our contact information is listed on the slide deck. We would encourage you to reach out to us directly via email with any questions you might have. Thanks again and enjoy the rest of the DevOps summit.