00:00:14

Hello. And welcome back to the day two general session talks. So over the years we ask the community about their top obstacles to things that they want to achieve. And almost every year it is audit that strikes the most fear, dread and frustration, probably because of the special power they have to generate findings that are seen at the highest levels of the organization and their use of sometimes decades, old audit practices. I am personally grateful for all the work that Clarissa Lucas has done. She is director of technology audit at nationwide insurance based in the United States and is one of the largest mutual insurance companies. So over the past two years, for reasons, I don't quite fully understand she is presented at this conference with incredibly specific and useful advice to people who work with audit and amazingly she and her team have shared very specific sneaks on overcoming audit issues, concerning separation of duties and change approvals.

00:01:12

So in years past, she presented with auditors on her team. But this year to my utter surprise and shock sheep had planned on co-presenting with someone in technology leadership at nationwide, which to put it lightly is just simply not normally done. She is presenting with Todd. Bickley an associate vice president of information, risk management, responsible for the identity and access management systems. This is a shared service, which is such an important security control because so many major applications rely upon it. They describe one of the most startling audit engagement models I have ever seen. <laugh> in fact, it's one of the most startling presentations I've ever seen. Period. I genuinely believe that they are on the frontier of revolutionizing internal audit practices for the entire profession. And I don't say this slightly as I've seen decades of audit talks from the Isaka and IIA communities. So here is Clarissa and Todd to talk about what they did, why they did it and the value they created.

00:02:16

What comes to your mind when you find out that the auditors are coming to do a, a review of your processes, go ahead and put those thoughts into the chat. Now I imagine most of you, aren't jumping for joy and, uh, a lot of you probably aren't typing things into the chat like, or having a great time or enjoyable. Now, what if I told you that it's not only possible to enjoy a highly collaborative audit, but that you also have the tools to get there. I'm Clarissa Lucas, and I'm here with Todd Bigley together. We'll guide you through Nationwide's journey to agile auditing, where you'll learn about the benefits you can experience in an audit performed with agility. You'll also learn valuable insights on how you can work together with your auditors towards a collective value added outcome and have fun along the way. I currently lead Nationwide's technology audit team, my team, and I help nationwide achieve its objectives by providing assurance on key risks and controls. We also help management see around the corner to provide advice on emerging risks. I've been with nationwide for about nine years and in my current role for a little over three, before taking on this adventure, leading the technology audit team, I've been in various roles in audit risk management and compliance and outside of work, I show for my eight year old son to hockey practices and I enjoy pretty much everything star wars with my husband and my son. Todd,

00:03:37

Thank you, Clarissa. Hi, my name is Todd Bickley. I'm currently the identity and excess management product owner, uh, for nationwide and was formerly the identity and excess management product manager. Uh, we'll talk about that in a few minutes. Uh, I've been at nationwide for a little over 20 years now, uh, and I've covered a spectrum of technologies here. I did start off an IAM team, uh, actually as an engineer, uh, and I've sort of done full circle here over the last few years and came back to lead the IAM team, uh, because I love the IAM technologies and, and the reason I love IAM technologies is because, uh, as, as the world moves to a zero trust model, um, to, to secure our associates data and our members' data, um, making sure people have the right access to the right things is critically important. And that's what IAM does.

00:04:24

Uh, and given that it's critically important. Um, the controls that we use to manage that are also critically important, uh, and are audit partners like Clarissa, um, also help us manage our, our control hygiene, uh, to make sure that we are doing things the right way. So having a good beneficial relationship, make them, making them feel part of our team, uh, when they come in to do these audits is, is, is, is really critical to our success. So I appreciate Clarisa, uh, invited me to this session and, uh, look forward to talking to everybody about, uh, how we made this happen.

00:04:59

Um, so who is nationwide? Uh, Nationwide's a us based company. Um, we don't sell any products or services outside of the USA. Uh, historically we were, uh, an insurance company, uh, with a, with, with, with a financial background and now really, uh, we're, we're actually a pretty large financial services company. Um, you can see a couple of stats up there, uh, but we're number one, uh, in state sponsored 4 57 plans. Uh, we're number one in, in selling agriculture insurance to farm, uh, in ranches, number two in corporate life. And overall we're, we're the number eight, uh, overall property and casualty insurer, uh, in the us. Uh, we have about 28,000, uh, us based associates. Um, and in terms of kind of where, where we are with, with rankings, uh, across we are, uh, number 25, uh, out of 100 for fortunes, best places to work.

00:05:49

Uh, we're number 50 out of 100, uh, for our best workplaces for, uh, diversity, uh, and community involvement, uh, is a, is a very big priority for nationwide. Um, we're, we're a big sponsor of children's hospital in Columbus, Ohio. In fact, it's nationwide children's hospital, uh, volunteering, giving back to our, our local communities, whether it's in Columbus or a lot of the other satellite cities that we're in is very important. And it's actually one of one, one of the things that, uh, is one of our big objectives every year. And it's, it's one of the reasons that attracts, uh, a lot of people to this company is cuz not only do you do good work to protect people's futures, uh, you're doing work in your community to help everybody, whether you're a nationwide consumer or not. So let's talk a little bit about, um, what we've done, uh, in terms of our, our product model journey for IAM.

00:06:37

Uh, so identity nexus management, uh, kind of encompasses, uh, all of, all of your standard IAM technologies. We've got single sign on and multifactor, we house all the authentication, repositories, all the authorization, repositories, um, all those standard things. Um, we started a product model journey, uh, really late in 2019, uh, with, with kind of a focus on implementing all those great agile DevOps product practices that our application development teams have been using for years. Uh, you know, we were trying to really solve this classic it problem of, um, getting work to flow efficient, efficiently to the teams, getting work to flow efficiently through the teams and getting work to actually execute efficiently, to deliver outcomes and values, uh, for our consumers, we wanted to understand our unit costs. We wanted to have people in standard roles. We wanted to be able to use standups, uh, you know, at minimal times.

00:07:30

So people could focus on doing work, uh, during the day, just kind of all, all those great things, deliver things in sprints. Uh, breakup works in the consumable chunks. Think about MVPs, just all the great things that agile and DevOps have, have done for our development communities forever. We wanted to start to inject those practices, uh, into our, um, infrastructure teams. So when we looked at identity and access management, uh, and our, and our product areas, we, we really, there there's really two specific functions we broke in. We've got our technology product management function, which is really our, our infrastructure teams. And they focus a lot, uh, on the hands on work. They do the hands on keyboard work, they're configuring and programming and supporting and operating all the systems. Uh, and they, they, they, they really keep everything sort of the heartbeat of the systems going every day on the risk side.

00:08:17

Um, we have the product ownership. So the product owners, um, work closely with the product management team, uh, to help, um, set the work, to help groom backlogs, to help bring work into the backlogs, to help the product management teams understand what's important. So they can focus on actually doing the hands on keyboard work. We do a lot of the strategy work, um, and we sort of set that forward, uh, for everybody to execute. Um, so, so we really started taking these, these product principles, uh, in late 20, 19 and early 2020, uh, and defining them within the infrastructure teams, uh, and embracing all these agile methodologies. Um, so about five or so months into the journey, uh, we have a biannual, uh, identity and access management audit, uh, that Clarissa and, and, and her team, uh, bring forward. Um, and, you know, Clarissa came and said, Hey, it's time.

00:09:08

We said, great Clarissa. You know, we love working with you. We love the way you guys look at things differently than we do, but we're really in the middle of this journey. Uh, if you're gonna come in and work with us, you're gonna have to work with us through our new agile processes. You can't, you know, we've got the team really excited about, uh, the sprints and the backlogs and the fact that we've got single front doors and we're using JIRA to, to manage all of our demand. Um, we can't come in and do a waterfall delivery of all of, all of our agile work, because, you know, as most of you know, when audit teams come in, they're really just generating demand into your team. We're producing evidence, we're having meetings, we're reviewing things. It's really just like a development request or a request from other teams.

00:09:47

So he said, you know, you have to do this. And, and Clarissa being a great partner said, you know what, you know, we've been talking about agile audit, uh, within the industry. And really with here in nationwide, this is an excellent chance for me to sort of rally our, our, our team around doing this. Um, so, so, you know, we, we brought them into the team and, and really, they just kind of became part of our IAM team for the three or months or so that, that we did the audit. Um, they attended our standups. Uh, they came to our sprint reviews. Um, they attended some backlog grooming sessions. Um, they used our, our flow processes that, that, that we, that we sort of defined for our other, uh, folks who were bringing demand in the team. And they really just sort of functioned like every, like everything else.

00:10:31

Um, you know, and then I think it changed a little bit about the audit outputs, uh, because what we didn't do is we didn't get a big chunk of work or a big chunk of, uh, uh, what happened with your audit at the end, what we saw was we saw our audit now not only was the work broken up, but our audit results were also delivered in incremental pieces. And it was much easier for us to consume those things instead of just a big hun of work that was, that was dropped on our plate at the end. Uh, so, um, you know, it worked really well and, and appreciate them being able to engage,

00:11:03

But wait taught it. It wasn't quite that simple. Um, so where we typically do our work, yes, we were totally on board, but this is, this is a huge change for how we do our work. Um, you know, the way that audits have been performed for years has been pretty consistent, uh, using that waterfall approach to, to do an audit. Um, you know, we, we plan out the entire audit, figure out what are all those key controls and risks that we wanna cover. We go through our approval gates and get everything approved before we move to testing and field work. Once we move to field work, then we test all of our controls. And once we finalize the testing on that last control and in scope, then we move to the reporting phase and then we can communicate our key results to, to you and your team. Um, and this, this has worked really well, but we know that there's always room for improvement and with your, your team's encouragement and guidance, uh, we were definitely able to at leap and make that drastic change in the way that we were doing our work.

00:12:07

So some of the things that were really helpful, um, that we, that we implemented in this were adopting some agile concepts. Um, we anchored throughout the engagement to the four key agile values, and we made one slight modification along the way. So we modified the value on the bottom left of the screen screen that typically emphasizes working software over a comprehensive documentation, valuing working software. Doesn't really speak to the team. Didn't really speak to the audit team or Todd's team in this instance because software wasn't our deliverable. Instead, our deliverable for the audit was actionable insights. And those include assurance that key controls are designed in operating effectively findings articulating a risk or control gap and insights on how to improve the effectiveness or efficiency in that process under review. Now, please don't get me wrong. We auditors, including myself, still love documentation. Uh, we value both documentation and those actionable insights, but the key difference when we changed our way of working with Todd and his team, is that, you know, we made sure we didn't lose sight of that collective outcome of delivering actionable insights for the sake of dotting all of our eyes and crossing all of our T's in our work papers.

00:13:17

Nobody sees those work papers except the auditors and the people who audit the auditors. They are incredibly important to support those conclusions that we reach, but that audience is really limited on the other hand, those results that we communicate the assurance over the controls, the control gaps, and those opportunities for enhancement has a much larger audience and a greater impact. So that's where we focus. Most of our attention, we anchor back to these, to these agile values, as well as the principles outlined in the agile manifesto and successfully implemented a number of those agile concepts like self-organizing teams prioritizing our customer's needs, fostering a collaborative environment and delivering results frequently. So one of the things that we did with that self organizing teams was we further expanded the team beyond just Todd's team and my team to include someone who was really well versed in agile auditing standards.

00:14:16

Uh, and they helped provide those insights along the way to keep us within those guardrails while we adapted agility and were still able to comply with auditing standards, we also intentionally prioritized our customers' needs and fostered that collaborative environment by planning the engagement scope together. So I just explained the traditional approach to an audit where, you know, we would talk to Todd's team, get a high level understanding of what their, um, what they did, what they were accountable for. Go back to our desks, develop our scope, and then come back and present it to Todd and his team. Here's what we're going to do in the audit. We didn't take that approach this time. So together we worked with Todd's team to identify those key risks and controls relevant to the area that we were reviewing. And then collectively, we came up with the most effective way to test each of those controls together.

00:15:10

So what, who better to tell us how to do a con, how to perform a control and where that documentation is and what it looks like than Todd's team who lives and breathes identity and access management every day. Another element of the agile manifesto that we incorporated was delivering results frequently. We were, we delivered our results every 30 days. So instead of waiting until the end of the audit, we issued interim audit reports at the end of every 30 day sprint this enabled Todd to begin addressing any findings earlier in the process, which reduced risk, risk exposures, much sooner

00:15:49

At last year's DevOps enterprise summit. I joked about taking a page out of your book this year. I took it straight out of the DevOps handbook. We incorporated the three ways of DevOps into this audit as well. The first way was, uh, Todd described this when he talked about how we became one team, we also incorporated the second way, which is feedback loops throughout the audit by intentionally soliciting feedback from the entire team. Again, not just Todd's team, not my team, but all of us, including executive leadership. We held retrospective reviews at the end of each sprint to identify what went well and what we wanted to focus on improving in that next sprint. At the end of the entire audit, we sent surveys to Todd and his team to collect additional feedback. And most importantly, because we worked so closely together during the audit, we exchanged realtime feedback too.

00:16:39

So rather than waiting until the end of the sprint or the end of the audit, we got realtime feedback. Now we learned the third way, the hard way. Um, as we neared the end of the audit, we faced a really significant challenge when we reverted to our old ways of working for that final reporting stage, we reverted back to the auditor and the auditee structure when we compiled that final audit report and determined the overall rating, instead of experiencing this part of the audit together, like we had up until that point, reverting back to our old ways of working left Todd and his team on the opposite side of the table from me and my team, we were briefly adversaries. Now, the partnership that we built and strengthened during the audit, as well as that commitment to the collective goal that we all had as one team is really what brought us through this challenge stronger than before. Todd gave us grace. As we navigated through this hurdle, he created a culture that fostered taking risks and learning from failure, which is consistent with the third way of DevOps, continuous learning and experimentation.

00:17:42

We also implement a practices like making work visible using the pull versus the push method of a method of assigning work and daily standups. We use the planner functionality within teams to make work visible. Each task for the audit was represented on a task card and information like the accountable party and the status were always available for the audit team to see and act upon. I didn't have to go to the project manager like I would in other engagements to ask them for a status update. I could just quickly view the planner board and see where everything stood in the traditional audit approach. We use the push method of assigning work, where the project manager assigns work out to the staff. So staff number one, you take these four controls staff. Number two, you test these five controls in this audit. We, um, we use the pool method of assigning work.

00:18:33

So the staff self-assigned their work and this facilitated a much better matching between capacity and assignments. So I'm on vacation next week. I don't have a whole lot of capacity. So I'm gonna assign myself work that doesn't require a lot of, a lot of effort or a big time commitment. It also facilitated a better matching between, um, those assignments and the team's interest and development goals. Another way we implemented these newer ways of working was to attend daily standups as a collective team. This was Todd's team idea Todd's team's idea. And it was one of the improvement opportunities that we identified in the sprint retrospectives. So it was definitely met, met with some apprehension from the auditors at first, but it was absolutely an incredible success. The time that we spent setting a request and following up on it while Todd and his team fulfilled, it was drastically reduced using this new way of working and using these standups.

00:19:30

So in most instances, using our old way of working, we would send a question or a, a request for follow up documentation, and we'd send it through email to our point of contact who would try to figure out who actually was the right contact, try to understand what it was we were asking for, pick something, send it back to us, probably not the right thing. So it would take us days or weeks to get what we requested using these standups. We often received whatever we were asking for during that standup or by the end of that same day, it was, it was awesome.

00:20:06

So, you know, although we were in this really great new processes, there, there, there were several challenges that, that we faced going through this. And first, and I think kind of the biggest one was, you know, although we used the sprint approach very effectively and we got, uh, we were able to understand what the issues were per sprint, uh, based on the scope that we broke up, we sort of lost sight of, of, of how it all came together in totality. So when we got the final audit, uh, um, you know, document with everything we had discussed and accepted per sprint, it really looked a whole lot bigger than, than, than we thought, uh, it was gonna be. So, you know, one piece of advice I would say, or, or the thing that we're gonna do different next time is, is we're gonna make sure that we're really making that we're tracking and understanding and aligned on what those issues look like.

00:20:52

Not only per, per sprint, but, but as they, uh, continue to grow, uh, through, through the multiple sprints, um, the, the, the second one, which is honestly just a little bit of a, uh, circumstance of our environment, uh, we didn't do a whole lot of upfront planning, uh, at, at the time, you know, the, the, the, the team was really going with its agile journey. The auditors came in and we said, you know, you you're gonna have to play in our processes. Um, and so we didn't have a lot of time to do a lot of upfront planning to see how that would actually work. So next time, you know, I, I think what we're gonna do is we're gonna take some time, a few hours, maybe a half a day, and really figure out and sort of plan out how we're gonna execute our, our sprints and, and the scope of these sprints.

00:21:31

Uh, so we can be a little bit more organized delivering. And then the third piece, um, was, you know, the auditors didn't have a ton of, of, of knowledge of the agile processes. Um, obviously with DevOps handbook and, and great books like the Phoenix project, um, you know, they, they, they understood concepts, but they'd never worked in inside, um, those DevOps practices before. So, um, you know, even though the auditors are now more, uh, engaged and knowledgeable of, of agile as, as well as the IM teams, I think we are gonna take a couple hours, but before we kick off the, uh, next audit, and we're just gonna level set on terminology on approaches. Um, so we're all using the same language and sort of, uh, working off the same sheet of paper going forward

00:22:15

And Todd and his team, weren't the only ones who had some challenges, uh, as we tried this new way of working, the first was a fear that we would violate our professional auditing standards. Um, you know, we're switching this new way of working and can we still do this and, and comply with the standards that we're held to, um, as a, as a profession. And the answer is absolutely yes. You know, one of the most common questions that, that the team had, and the biggest fear I think that we had specific to staying in compliance with our auditing standards was the independence and objectivity that internal audit that's so important to internal audit. You know, how can we maintain that level of objectivity and maintain our independence when we're working so closely with our clients? Um, the key to that is really just maintaining decision rights with the audit team.

00:23:04

So while we're collaborating with Todd and his team to identify which risks and controls do we want to include in scope, and do we wanna spend a most of our time in we're leveraging the knowledge that they're providing us and we're making those, those key decisions ourselves. So a lot of great input, uh, I, I truly believe that we made better decisions, more well informed decisions. It's just Todd didn't have the decision and his team didn't have the, the decision on what we would audit and what we would exclude from scope that really still lot stayed with my team, uh, and myself. So that's how we really maintained that independence, which was probably the, the biggest hurdle when it came to that fear of the violating, the auditing standards. We also, as Todd mentioned, we lacked experience with agile and DevOps practices, and we had significant cultural and procedural changes that we needed to support this new way of working that fear, uh, on violating auditing standards, the, um, lack of experience with agile and DevOps, DevOps practices really didn't help either. But as we, as we talked through, we made it through these challenges. We, we learned a lot throughout the way, and we really had a lot of benefits. So, you know, Todd and I walked through what it, what it means to audit with agility and how you might do that. What, what are some concepts that you want to include when you're audited next? I think it's really important now to shift to the why, why would you wanna partner so closely with your auditors and invest all of that time?

00:24:39

Here are some of the benefits that our team. So again, not just my team, not just Todd's team, but that collective team working on this engagement here are the benefits that we enjoyed greater collaboration and engagement. We focused on areas of the greatest value and highest priority to the organization. We successfully adapted to change. We had much greater buy-in, more timely communication of results. And my personal favorite is a reduction in the, the amount of time that was wasted during the audit. I think it's one of Todd's favorites too.

00:25:14

And here you can see some measurable results that we achieved during the audit as compared to the last time we were in this space in 2019. So the length of the engagement went down was reduced by 10.5%, the amount of coverage that we got increased by 77%, the number of days from when we identified an issue to when we opened it and got it in Todd's hands. So he could do something with, it was reduced by 48%. The percentage of issues with progress made by report issuance had a significant increase. So this is where in 2019, we followed that waterfall approach. So we finished all of our testing before we delivered any results to Todd. So the story there to our key stakeholders. So our audit committee and executive leadership was we did some work in this space and here are some gaps that Todd and his team need to start working on fast forward to 2021, implementing agility into our work.

00:26:11

Um, the story there was because we delivered iteratively Todd and his team got those findings and got those control gaps into their hands much earlier and made progress on those by the time we got to that final report. So there were instances where they already had a plan in place. They knew how they were going to address these gaps, or they made progress on we've we're working on this plan, and it's not just a plan. It's things that we're doing to mitigate this gap. Or my favorite was, yes, it was a gap we've already put a plan in place we've already mitigated mitigated it. And then Claris, and her team have validated that that indeed is no longer a gap. So completely different story to the audit committee and to executive leadership from 19. It was, here's a bunch of stuff that Todd and his team have to do in 21.

00:27:02

It was here are, here are the results from the work that we did in this space, and look at all this progress that Todd and his team have made so far, like much better story. And then the last measurable benefit is the client survey results. So I mentioned before that, we send out a survey at the end, end of the audit to get feedback from Todd and his team. And part of that is an overall satisfaction rating that improved by two rating levels, which is awesome. So what does this all mean? It means that we provided more assurance and results were communicated and addressed sooner than in prior years. All well, spending less calendar time auditing Todd's team, and based on the survey results, they were pretty happy about it.

00:27:47

As a matter of fact, these are direct quotes from Todd and his team taken from our client surveys. Things like exceptional, great time, very positive, worked hard and enjoyable to work with. I'd wanna put these on my fridge and read them every day because this is exactly what I strive for and what my team strives for in working with our clients. And while we didn't have a real time metric to measure every benefit that we experienced, like that greater team, collaboration and engagement, we still experienced them. Our daily standups were a lot of fun. We built a lot of rapport with each other, and when the audit was over, we missed working with each other. I missed working with Todd so much that I was just dying to work with him again on this presentation and was thrilled when he agreed to do this with me, the engagement and collaboration on the audit led to a number of lasting professional relationships that are still going strong today, about six months after the engagement was completed.

00:28:46

So how can you get there? Uh, you know, when your auditors come to you and, and you wanna, you wanna bring them into your DevOps and your agile practices just offer to coach them, uh, through, through that learning curve. I've, I, I, I think what you're gonna find is that, you know, your auditors are gonna wanna learn how to, how to work with you, just, just as well as you wanna be able to work with them, to make these things as easy as possible. Um, second is how to, how to demonstrate on how to run effective standup. So we know how important and effective standup is, uh, to the agile end dev off experience, uh, help them understand your Coban boards, help them understand your J boards, help them understand how you're doing, uh, your flow of work and how work flows through your systems.

00:29:25

So they know how to better take the work that they're going to give to you as demand and be able to put it and utilize those systems. Then keep an open mind, um, always think about, uh, how things may work now to how things may work better, uh, when you're bringing auditors into the practice, uh, and encourage collaboration with your team members, um, you know, your, your auditors are really there to help protect you from, from the things that are going on. Uh, they're, they're, they're not there to find, uh, deficiencies in, in your systems to make you look bad. Uh, they're there to, to, to find issues in your systems to help you fix them. So we all look good and then really encourage your teams to buy in. Uh, that's that that's really the most important because once you've got the teams buy in, once you've got the people to be able, uh, to do it and work effectively, uh, I think you're really gonna see success.

00:30:12

So, so wrapping up, um, it was, it was really a great experience. Uh, and I would even say we're, we're even looking forward to the next one. Um, you know, a couple things that I would ask this group is, you know, as, as you are going through agile and DevOps experiences, um, how are you seeing those evolve in your infrastructure areas? Do you see people adopting product practices? How are, how are you bringing external teams like your auditors into your agile practices to make sure they're adopting those things? And are you seeing the same sort of, uh, advantages and sort of up up upticks and the flow of work, uh, that we've seen over the last year.

00:30:47

And then I've also got a request of the group as well. So for those of you who, in the beginning, you typed into the chat that you had great experiences with your auditors. I'd love to know what are some of those things that you've done, or that your auditors have done to help make that a better relationship. And then for those of you who haven't typed in great, enjoyable experie experiment, experience love hanging out with my auditor friends. Um, I'd love to know what else is driving. Some of that, um, those challenging relationships, because this is one way that my team and Todd's team have collaborated together to create a better experience, but I know there's more ways and I would love to learn what are, what are some of those other things that you're, you're challenged with with your auditors and on behalf of both Todd and myself, thank you to Jean Kim and the selection committee for giving us a stage here. Uh, we love sharing our story and learning from each of you. Thank you. Thank you.