00:00:05

It's great to be here with everybody so we can share experiences and learn how to go faster. Um, you'll notice a lot of common themes in this conference and across your own experience, which is why it's so important to get together so we can all get better. Um, one you've already heard about that Kevin's going to touch on quite a bit as well, is the need to span the business and technology divide. It's not just about dev and it's not just about ops and it's not about automation. It's really about giving everyone a seat at the table from the CIO to the release manager, to the compliance team, to the line of business owner. Everybody needs the visibility and everybody needs the information to truly make the business successful. Another common theme is you need to design your infrastructure for the growth. As you move beyond DevOps pilots, you really need to look for the future and think about the platforms, the technologies and the processes.

00:00:56

So how many of you saw the BMWI eight outside, if you haven't already? It's a really sweet car and I'm a crazy BMW fan. A few years ago I took their performance driving class where race car drivers teach you how to drive fast around a track and control the car while you do things like pull out of a skid. And as you're spinning in circles, they say to come out of it, look where you want the car to go and the car will follow. And sure enough, I could never remember turning into the skid, which way is turning into the skid. But when you looked where you wanted to go, the car did in fact follow. And it's the same for everything in life, including DevOps. You really need to set clear goals and set benchmarks along the way to make sure you're tracking them and capture the right metrics to get to your goals. Not just any metrics. Capture the right metrics, keep iterating, get feedback, and then make sure you're still going in the direction you wanna go in.

00:01:50

Okay, so let's talk about Lincoln Financial Group. They are a grand old dame of US financial institutions. They're an insurance company that has more than 9,000 employees and they're 113 years old. They have hundreds of billions of dollars in assets under management and they do hundreds of releases every month. Their business is to enhance people's lifestyle through financial stability. They are in the business of mitigating risk for their customers. So as you can imagine, it's a pretty challenging environment in which you need to make changes, right? Risk mitigation means don't break anything. How many people here work in highly regulated industries? Quite a few. So you know, it's hard, right? Lincoln also has the complicated mix of legacy and modern technologies that you would expect in a company that's been around for this long and everything has to come together to release these applications. On top of that, they have various processes across various business units and the speeds of processes very widely, and it actually creates a synchronization problem, surprisingly enough. Oh, and by the way, they're undergoing an enterprise wide agile transformation. They're still using some waterfall, they're moving towards agile, but there's a lot of change going on here on top of that, a DevOps transformation too. So how do you succeed implementing DevOps in such a time challenging environment? How do you lay the foundation for success? I'm gonna turn it over to Kevin to tell you.

00:03:23

Thanks Lisa, for the warm introduction. Opening remarks. Can everybody hear me okay? Okay, great. I'm excited to be here today. This is a great opportunity to learn from all of you and be able to present and speak and talk about our DevOps journey at Lincoln Financial Group, the elephant in the room. DevOps. What are some key concepts that come to mind when you hear DevOps? Don't be shy. Shout 'em out. What happens in Vegas stays in Vegas?

00:03:53

Automation culture

00:03:58

Change. Automation, culture change. Velocity. Velocity. One more.

00:04:05

Tools.

00:04:06

Tools, perfect. The gentleman over here mentioned it. You must have seen my deck. DevOps, we feel like a financial group is a culture change where we want to change culture to deliver high quality working software faster. Another way we can talk about it is how can we increase the quality of our product and bring it to the market faster? Furthermore, it's a philosophy where teams are accountable for everything to get their code developed, tested, and deployed to production while shared services teams provide the automation and tools to enable them. And at the bottom of the screen here, you can see it starts with agile. So we really wanna make sure as we embark on our DevOps journey, that we're keeping agile in mind and we're shifting from a waterfall to an agile mindset because that's really key. You're gonna struggle and we did early on if we're not agile. So keeping culture in mind, we realized we needed a sound strategy and execution approach. One which focused on being agile, but then automation enables DevOps across our business and our enterprise. How do we do that? I'm not gonna go into all the bullets, I'm just gonna touch upon one for the sake of time. They say you have 15 minutes of fame in your lifetime. I got 25, so I gotta stick to the 25. So we're gonna go deep and broad. We really wanna pilot, learn from that and scale out.

00:05:44

The other area where we focus on is mainframe and DevOps. So we're an insurance carrier, insurance carrier, financial organization. We have a large distributed application portfolio and footprint, but we also have have a lot of footprint and presence on the mainframe. So when we started embarking on our journey, the mainframe folks were saying, great, everything you're showing us is around distributed applications, java.net. Does DevOps even work with the mainframe? Yes. So what we've done is we partner with vendors, we created pilots, we created a working group specifically with our mainframe partners within our company. And we're arriving at enterprise patterns for mainframe that can be leveraged by the other teams that are beyond the pilot. So the key here is you really gotta know your audience when you embark on this DevOps journey. And we're doing that by pulling in these mainframe folks. Next up we have center of excellence.

00:06:44

How many have COEs within your organization? You probably have 'EM for DevOp or uh, application development. PMO Business analyst. We have those too. So to keep up with the Joneses, if you will, we had a DevOps, but what does that do? Just like we saw the benefits in these other center of excellences, we're seeing that for DevOps too. And we're coming up with best practices, collaboration and helping drive that culture across our business units. Speaking of business units, it comes to really business unit specific adoption from a IT perspective and in my role as enterprise release management lead the team and I, we track nine business units. We have life annuity group protection, retirement plan services. We also have a enterprise data management automation team, omnichannel and our retail arms. So nine business units, it's a lot to manage from a release perspective. It's also a lot to to onboard from a DevOps perspective.

00:07:46

So how are we doing that? We've identified partnering with our senior leadership team in each of these business units to identify a DevOps leader and that individual, he or she is really responsible to partner with the enterprise team and drive the adoption within their business unit. So we work with them to develop a backlog for teams to onboard. They're ultimately accountable for the success of DevOps within an organization. We also have a run organization within Lincoln. This team really focuses on production support and maintenance. So in in general terms, DevOps, if you build it, you own it, we need to figure out how we're gonna adopt this model and flush things out to make sure that we can accommodate not only new development but also break fix and all that into DevOps. So like our mainframe model, by having a working group, we have a run organization where we focus and we bring in representatives from the various vendors that we partner with for our, our maintenance and support work.

00:08:49

And we are arriving at pilots and patterns to identify opportunities to improve DevOps within this space. So again, know your audience and bring them to the table. Lastly, steering committee. So the steering committee is really the go-to people and team when we have blockers and impediments to help flush us, flush things out from a governance prioritization, help us with some decisions, and again, assist with the overall DevOps roadmap and to help bridge the gap between technology and business. We have at the center of all our DevOps enterprise practice. So that inner gear, if you will, spans out to these six areas. Visualization. We really focus on metrics, reporting and dashboards. We want to be able to show the progress of our success along the way. Next up, we have collaboration. I just mentioned in the prior slide about mainframe working group, the run organization, working group.

00:09:50

At the core of it all though, it starts with our DevOps working group. We have the nine business units that I mentioned. We have representatives from every business unit. Furthermore, we have representation of all functional areas within those business units. So what do I mean by functional area? Application development, qa, architecture, security, infrastructure. Every release management, everybody has a seat at the table, if you will. They have the the table stakes so to speak, and that's real important too. So that helps drive culture. If someone feels left out, they're not gonna adopt what you're trying to do. They're not gonna see the value and others are gonna be on the board and heading down that journey with you and they won't. So know your audience, keep them engaged, improve your culture along the way. We also have innovation enablers. So we carve out time within our technology areas to spend time on innovation. And that was a hard sell at first. We wanted to make sure the business understand the value, but they were first coming back saying, well, our projects are gonna suffer delay a little bit, but we walked through those processes, explain the benefits, and in the long run they're gonna gain efficiencies, we'll eliminate waste. They were able to see the big picture. So we're slowly spending more time innovating and we need to do that when we embark on our DevOps transformations.

00:11:14

The continuous delivery platform really focuses on CI and cd where we were gonna automate our build or deployment. We focus on automating release management infrastructure as code as well. Next up we have our continuous testing area where we focus on test automation, stating the obvious there, but then also test data management. And it's also not just for our distributed apps, it's for our mainframe and our other areas too. And then lastly, to tie it back to visualization with our continuous feedback. This is where we monitor all the metrics, the dashboards and reporting that we've identified in visualization. So we kind of come full circle with our continuous feedback.

00:11:57

So in addition to our DevOps practice, enterprise release management, as I alluded earlier, has representation across all business units at Lincoln Financial Group. That's real important. 'cause what we're trying to do is we're trying to drive out a release management practice to get standardization not only from process perspective, but also tools per tools perspective. And we can't just roll something out top down if we don't have all buy-in across every line of business or, or business unit rather. So some of the key steps in our journey from pilot to enterprise adoption have been these, and some I'm gonna dig deeper in in future slides, but first we solidified an engineering team. We have a small engineering team at Lincoln Financial Group, but they're powerful and they're skilled and they do great work and they're building out the enterprise patterns for which all DevOps teams will be able to leverage.

00:12:51

Right now they're in the weeds. Eventually they're gonna become consultants, if you will. And if you remember the old game show where, uh, I think it was let's make a deal or, or something like that, right? And you got stuck, one of the options was pick up the phone call a friend. So that model is really where they're transitioning from, doing the hands-on coding, integrating the tools, et cetera, to now being more of the consultant for the DevOps business leader and their teams as they look to embark on their journey within their business unit. So it's an exciting role for them. Hands-on very much into it. Then they're gonna transition. The goal is to, uh, a little bit more consultant, um, perspective, establishing a support model. Just like our enterprise team, we don't have a big army. We we have a lot of tools. We have a lot a large user base for all these tools.

00:13:41

How do we support them? How do we get innovative? How do we get creative? So we created in the, in the mindset of being agile and, and DevOps, where there's open communities and DevOps days, et cetera. We have office hours or user support meetings, user community support meetings rather, and allows people to dial in on a weekly basis basis at a defined time. And engineers are on the call release, managers are on the call. And we have this for our build tools, our testing tools, our release tools, et cetera. And it's just a unique way to be able to support our large user community. 'cause again, we don't have the capacity of dedicated resources. We don't have a call center for, um, GitLab or for, uh, ZB Lab's Excel release product. We have office hours. And it's a, and it's really an innovative way to be able to support user community.

00:14:31

And in fact people dial in just to learn even if they don't have a question or they're not stuck. So we know it's a successful model. So really challenge yourself and come up with innovative and creative ways to allow you to be more successful with your DevOps journey. Uh, metrics and dashboards and transition from manual tasks to automated pipeline. I'm gonna, like I mentioned, I'm gonna go a little bit deeper into those, but the key with the transition from manual tasks to automated tasks. Don't just automate everything. Automate the tasks that bring the most business value.

00:15:05

And then lastly, communicate your success. So in IT this morning we were having a, a conversation at our table and someone, uh, lady had mentioned that it doesn't seem to do a good job expressing the work we do. Exactly. We struggle with that at times. So we, we've really improved that at Lincoln Financial and with our DevOps journey, we do that as well. We wanna communicate our success. Obviously we track and have our visualization metrics, et cetera. So that helps support the case too. But it's really important to keep people involved because again, it all ties back to culture. And if you're communicating your success, they feel part of the team, they're gonna be more successful. And then lastly, keeping agile in mind. We want to document our improvements. So lessons learned, we all conduct those when we have after post-release. Not all times. Maybe we implement those. So how do we improve that experience as we embark on DevOps? So document improvement areas, fix and move forward. So now we've talked about our strategy execution approach and our DevOps practice. I want to pivot and talk about release management and how Lincoln Financial Group decided to utilize ZB Labs XL release products to release or to manage its release pipeline.

00:16:29

And working with these nine business units across our IT organization, we came up with hundreds of release individuals and these release in individuals were responsible for submitting their own change records and managing their own release. And the majority of it, as you can see, hopefully you can see was like in a spreadsheet. So we had spreadsheets flying around, emails going back and forth. We didn't have enterprise visibility. We lacked the transparency. We didn't have enterprise calendar. Things were in network drive, SharePoints, emails, et cetera. So we're getting that. And then we implemented two phase strategy approach. One, which is manual workflow. So we said get off your spreadsheets and get into a tool. It's gonna give you the visibility. Yes, you're gonna put in the same tasks that you had in that spreadsheet. And when the spreadsheet was blank, guess what? You have to start from scratch and you fill, you filled out the tasks, who was involved, how long it took, what time, et cetera.

00:17:24

You're gonna do that in Excel release. So now we have for the past year have this momentum of all of our releases for production releases in the tool. Well guess what? We got that transparency. Now we also furthermore, build up a user community who is excited about the tool and we'll see that in a second. And more importantly, there keeps them tied back to culture. Again, that's the theme is keeps 'em jazz and excited with our movement and keeps them involved in our journey. This is a multi-year journey. So we wanted to establish our production baseline. That's what we're getting at a manual workflow. Phase one, if your release takes 10 hours, perfect, at some point it's gonna be less. But when it is, we'll never know what that true efficiency gain is if we never tracked what it is. Now, phase two, we start to slowly integrate automation and we talk about the full pipeline.

00:18:22

Some of our applications, our dev, UAT prod, some are dev test, UAT, pre-prod and prod, et cetera. It doesn't matter. The point is that we're expanding tools, capabilities, integrating with other tools and going beyond our production release. And this is where it really gets fun because now we can start piecemealing opportunities for them to get excited about automation while we're building out the enterprise patterns. So the challenge we have is from a release management perspective, we have all these folks doing their manual workflow production releases. How do we what their appetite and evolve their user experience? So we introduced notifications. And I really love telling this story because I always see a lot of heads nodding and it resonates with people. Think about releases that occur on a particular weekend and how many emails fly around. So someone has to do work for me, I send 'em an email, do work, they do it, they email me back.

00:19:22

Pretty simple. But now you've done that maybe 10, 15, a couple dozen times for just your one release because you're emailing a ton of people that are involved in your release. But now your business unit's got maybe 20 releases going on that weekend. And then, oh, by the way, that's just one business unit. Well, Lincoln Financial Group, we got nine, so now we have eight other business units. So now those minutes become hours and now those hours become multiple hours. And that's a nice efficiency gain. You can hang your hat on by just leveraging a tool to do communication. So think of that. Next up we try to expand it further with integrating deployment tools such as XL Deploy and other tools. And what that allows is shift left. We don't have to submit a ticket to a development team or to an operations team, rather to push a button and do the work.

00:20:11

We can do it ourselves. Again, simple stuff. We're not at an enterprise release pipeline for a whole organization yet, but what we're showing is this is our evolution. How are we gonna get there? And this is what our release community is saying. Tools easy to use, tasks are more organized, right? They're not, they, they have more transparency and visibility than ever before. I like the one where it says one of the tasks was missed that showed the release progress less than a hundred percent. So we knew it was pending. You didn't get that visibility when emails are flying around. Things weren't SharePoint spreadsheets, et cetera.

00:20:51

And then lastly, the tools enabling more animation. I just kind of spoke around that, spoke to that around the, uh, notifications and deployments. So as we've evolved our user experience from phase one workflow to automation, we're also behind the scenes with our DevOps practice, building out enterprise patterns so that ultimately people transition from their manual workflow for production releases and take leverage the enterprise pattern to get into their automated workflow. But again, that's taking time, that's gonna take time. We have to work with all these business units to identify all the teams in our organization and come up with a backlog to onboard them. So what can we do in the interim? So we've identified our enterprise patterns with integrated and customized tools, develop processes. And in this pictorial here you can see ZB Lab's XL release tool, orchestrating plane, air traffic controller if you will, up top sitting over top from a visualization collaboration automation perspective.

00:21:56

So it's giving us that visibility across the full pipeline from storage code, build, install, test, validate and release. We never had that before, so we're getting that visibility. The other area I want to touch upon real quick is our value stream analysis. This is a theme that is part of this conference. You'll hear, uh, other presenters talking about it. It's based on lean principles and, and agile as well. We conduct working with that DevOps business unit leader value stream workshops with their organization. And we identify the work streams in their space and we arrive at ways that we can eliminate waste and gain efficiencies. So we have them bring in their federate architects, they're senior developers, they're qa, they're release manager system analysts. We have the, the DevOps, um, business unit leader who's usually an officer in the room too. It's a multi-day workshop.

00:22:51

And what do we get out of it? It's really eye-opening for us from a DevOps release management perspective, but also the individuals that do the work and are participating. So what do we mean and what are some of the outcomes? So we dig into, once you get business requirements, what are all the steps from A to Z till you release in production? How do you code? How do you branch, do you trunk? What tools do you use? Do you unit testing? Do you functional testing? Are you leveraging any automation for, for QA performance testing? Do your applications require security scanning? If so, what are they, how often do you release? What's your release cadence? Where are you with your Excel release journey? Are you still doing just production releases?

00:23:33

And we do this whole exercise to again, really arrive at opportunities to help them on their journey. And we find that so much has nothing to do with DevOps. It's just I submit all these tickets to people to do other work and then, or I have to send an email, then I have to wait 45 minutes to do testing. So these opportunities come up, then we create a backlog and then we work with their team to implement these strategies. So one might be let's, um, transition off of a legacy, um, version control tool to, to GitLab. So we're embarking on that journey. We onboard folks with that. That's foundational. Let's just get some quick wins early on before we even start leveraging our automated release pipeline. 'cause if we don't do that stuff to the left, we're gonna struggle onward as we go.

00:24:25

When I talked about culture change earlier, I said the the the goal is to increase the quality of our product and bring it to the market faster. So how are we gonna define our delivery performance? So the science behind DevOps, this was uh, I think 2016, but the Dora report just came out and I know there's a, a presentation around that and the material still lines up with that. And what's really important is delivery performance predicts higher organ organizational performance measured in terms of business productivity, profitability and market share. So while we're tracking metrics on our manual workflow, we look at change volume and are they completing, uh, production releases through the tool? But we know ultimately we want to get to what's your lead time? How long does it take from code commit to code running and production?

00:25:18

We're not one of the unicorns, but that's okay. We have teams that Sprint that aren't doing DevOps and they release every two weeks. That's okay. That's great. There's other areas that do quarterly. So what can we do collectively to raise our bar and increase our lead time, deploy frequency? How often is code deployed meantime to restoration and change failed percentage? We got a couple minutes. I got one more slide so I'm not gonna dig further in. But these are really four of the key metrics that leaderships look for and you'll be able to produce to show your progress along the way.

00:26:00

So in closing, in the spirit of Vegas, here's my lucky seven, or Kevin seven, that's corny, but seven key takeaways. So address culture change first before doing anything else. Make sure you have strong leadership support within your organization At Lincoln Financial Group, we do, we have it for our agile transformation. We have it for our DevOps journey. In fact, we have individual contributor roles that roll up to the team goals, department goals, the organization, business unit goals to overall IT goals it if you will. It's that stake in the ground that makes it easy. We're fortunate. Some teams will struggle or some companies will struggle where they scale out just within their business unit and they can't figure out how they can go beyond that from an enterprise perspective. It's because those other business units don't provide the same support that that one did. So again, strong leadership support.

00:26:54

Agile DevOps aren't separate. Bring the processes together into one. If your waterfall start agile first. You won't be successful at DevOps without being agile. Number three, equip your release management team to build a bridge from dev to prod. Focusing on culture tools and training. They know who's involved in a release the best. They know who all the key players are, what tasks they do, how long things take leverage them. They will help you be able to scale out with your DevOps journey across your enterprise run pilots to see what does and doesn't work. Bless you. What's important here is that it's gonna help you. A pilot will help you discover and solidify your requirements. So what does this mean? Test and learn, then expand. Think enterprise, not the car rental company. One-offs won't line to scale. So think big, don't be siloed.

00:27:52

Take your success from a pilot and scale it out. Get that leadership support across your enterprise. Next up you want to create that enterprise release pipeline. This is the pattern which all teams will leverage. So in our case, we have those business units, we have the DevOps leaders. They're accountable for the success for DevOps within an organization. And they're leveraging the enterprise pattern. It's the guardrails that keep people in line. You're driving down the road, you know where you gotta stay least here in the state's, right on the right side because there's lanes, okay? The guardrails from an enterprise persp, uh, perspective to keep you in line and in check. And then lastly, define relevant metrics for delivery performance and track them and track 'em not only from your pilot to your enterprise adoption and communicate your success along the way. So with that, I think we have a couple minutes for questions. I guess we don't. Well, hey, I re I appreciate everybody, great audience. Lisa and I will be here the rest of the week. Um, they have a booth in the, uh, in the, uh, showroom. And feel free to hit us up on Slack or just find us. We'll be here too. So appreciate it. Thank you.