The Delta Air Lines Dojo Journey

My name is Keanen Wold, and I'm here today to talk about Delta's Dojo journey.

I realize it's the last day of the conference. Sponges are kind of full. You're excited to probably get home to your families, and better yet, get into the office and implement or get working on some of the great things that you've learned here in this wonderful community. So I thank you very much for your time today, and thanks so much for showing up.

Let's talk about Delta Air Lines. We generate $40 billion in annual revenue. We turn 90 years old next year. We've been around for a while. We have over 80,000 employees. We've used technology pretty much since it existed, in various ways. And technology is recognized for our company as a top priority. In everything that we do, we try to be thoughtful, innovative, and reliable all throughout. And you can see how those themes really resonate with what we do collectively in our technology space.

This is an interesting slide, or data point, that came out this year. Forbes announced the top, or most relevant, companies in America. And when you look at this list, Delta's number 11, between Netflix and Spotify.

What I find that's very fascinating about this is general day-to-day conversation. What Tesla's doing to the automotive industry. "Hey, I'll send you my Spotify playlist. I got some great tunes." "Did you binge this weekend on Netflix?" Just water cooler talk and the brand presence that these companies have. And to be there, I think we're very humbled and honored.

But what really sticks out when you look at this list is how young many of these companies are. There's only two companies of significant age here. Disney's the oldest. Disney Brothers started in 1923, and Delta started flying customers in 1929. We were crop dusting before that. So if you check Wikipedia, it'll show a little different, but like I said, we're hitting our 90-year anniversary next year. The next closest is Nike, started in 1964.

So comparatively, the age of these companies, and for us to be on this list and be relevant, I believe because of what we do in technology at Delta, along with our customer experience and a lot of things we pride ourselves in, is just very fascinating. So we're very proud of that.

We know to stay on this list, we're going to have to continue to push on this stuff. And we've had conversations this week about how companies have fallen off because they haven't had good technology strategies. So we know that we have to keep pushing forward.

About myself. I'm coming up on my 18-year anniversary this time around. I started at NWA as an intern. No, I wasn't hanging around with Dr. Dre and Ice Cube. It's Northwest Airlines. Northwest Airlines, for those of you that remember us. We're actually coming up on our 10-year anniversary since the merger.

I've played various roles. I started out as a COBOL developer. Had to hide that on my resume because you're getting like 18 calls a day in the late '90s. Moved into distributed development, moved into an infrastructure engineering role as a web engineer. Did some WebSphere work. Moved back into the portfolios on the app dev side as an HR architect. Went back into infrastructure as a leader and found myself in this role that I have today.

Much like many other people at this conference and many of you, we find ourselves in a position of having empathy because of our experience. And no different than so many other people that have spoken at this conference, I feel that purposely bouncing between the sides of the wall, if you will, within my organization has helped me position to be in this role.

So, my team. I manage the DevOps Center of Excellence and Developer Practice. I'm also the Dojo program sponsor. My team started to form in late '16. I still had my infrastructure day job as a leader there, and I started pulling in resources, and we were trying to figure out what we were going to do to get going.

One of the first people I pulled in is Caleb. He's in the back of the room there, Caleb Wolf from Delta. Great guy. He's a wonderful guy to talk with. And I started pulling in like-minded individuals who were going to help change the way that we deliver software at Delta.

We started by selecting some tools. We needed a baseline foundation to what we were going to do to launch the behavior that we were really intending. So my team has three responsibilities.

There's developer tools: anything the developers touch, IDEs. We had legacy source control. We launched GitLab. And by the way, thanks to GitLab for their encouragement to actually do this. I wouldn't be up here today if it wasn't for them pushing me, so I appreciate that. We implemented Jenkins, Sonatype Nexus, SonarQube, and Slack. Just basic foundational pipeline functionality that would get people enabled to do the work in the way that we desired.

The second thing we're responsible for is the developer practice. So I've got this advisor, architect Ankur, on my team, and he's doing a great job. Our CTO, KK, has enabled us to do some bits and bytes sessions. We buy lunch on Wednesday. We buy pizza. We're bringing people in. It's building a community of practice where they can show off the things that they're doing. We can show them where we're trying to take things, and he does a great job there.

And then the last one, the third one, is the DevOps cultural adoption, which is carried through in the Dojo and some of the other engagements that we do, and of course, is the hardest one of them all.

So, technology footprint. You name it, we have it at Delta. Operating systems, databases. We're a heavy Java shop. We have a lot. Here's a couple pictures from our museum of our first mainframe implementation in the '60s. Apparently, you had to wear short sleeves and a black tie in order to be around that equipment.

So yeah, whenever you guys use delta.com or Fly Delta and you're booking a ticket, you're changing your PNR, there are service layers that go down and talk to our z/TPF system, Transaction Processing Facility. It's based on C, Assembler, and SabreTalk, and we have many layers beyond that. And again, this is what you use today, talks to the mainframes. But we have to deal with that when you talk about the cross-functional nature of the work. It's vast for us because there are so many layers of technology at Delta.

All right, so why did we need a Dojo at Delta? Rahul Samant, our CIO, joined us in February of 2016, and he kicked off some transformational efforts. He wanted an API COE. He wanted an Agile COE, my DevOps team. We already had quality and security, and we're introducing a new private cloud platform.

So we have all these teams, and they're engaging with delivery teams. If somebody was using a pipeline, maybe even in survival mode before my team had it available, they stood up their own Jenkins instance and what have you. They were off doing things. Maybe they felt good, like, "Okay, I've got the pipeline thing down. I'm doing the DevOps." And, "Hey, maybe I need some help with my velocity, so I'm going to engage the Agile team," or, "I need some API design help, so I'll work with the API team."

It was disjointed. So it's kind of a la carte for these teams, and it didn't create consistent delivery or behavior for the teams.

So our journey begins last year in London. We had Jasmine, who's one of my many rockstar engineers, and Cassandra on the right. She's my peer that runs the Agile COE. We got to meet the trio here. Love these guys. They've all been very helpful and resourceful, and started to build this network of people who are very willing to help. And I hope that you guys are really embracing that here in this conference if this is your first time. There's a lot of people here who are willing to help and exchange stories. Make sure to leverage that. Reach out, and please, I'm happy to help out as well.

So we had that conference, and it was very motivating and enlightening, too. Other people are going through the same struggles. We're not much different from other companies, and I think that you guys find here is like a good therapy session. It's very interesting how alike all of us are.

So on a timeline, like I said, Agile COE, they were engaging, they were coaching teams. They were the first one that was stood up. I was still building my team in late 2017, really started hitting the ground in early '18. I'm sorry, in early '17.

And then we started to go talk with other companies. We met with Ford. We met with Home Depot. We learned about how they approach software delivery. We talked with them about their private cloud platform. And then, like many others, we went to Target, and they were gracious hosts, my neighbors up in the Twin Cities. Got to talk with those folks. Beautiful facility. They taught us about how they're succeeding in their Dojo delivery, what their Dojo is to them, and that really got our minds going around the time that we went to the DevOps Enterprise Summit conference.

So in the fall, we started to strategize around how we were going to do this. I pulled in a partner called Point B. They've been extremely helpful, based out of Seattle, and we started to drive our strategy for a launch in June of this year. And you can see that we continue to look for input. We recently met with Honeywell. They're doing some really cool things in the Atlanta area. It was very exciting to see some of the things they're doing.

So we get our strategy kind of put together. We needed three things.

We needed a physical space. We had this large effort going on in another area of the IT organization, and they had this huge room about a quarter mile away from our headquarters, more like in an operational building. And space isn't easy to find. I was really jealous of the Target trip because, man, they have a lot of space. So I see this room. It's surrounded by kind of five smaller rooms, and I'm like, "Hey, we'll take it."

And we take that room and we line up tables. We set up tables. There's six of them to support six two-pizza teams. We made it our own. We put some color in there, presentation capabilities, better collaboration capabilities between the teams. So we've got our space. That's number one.

Here's a classroom that we use. We use this to promote behaviors, the benefits received when following through with test-driven development, continuous improvement, quality mindset, and secure coding practices, et cetera. We'll do workshops, we do formal training, you name it, but it's nice having that in our facility.

So we've had the physical space now, too. Number two is a set time. And what we decided on, which is a little bit different than some of the other Dojos, is some of the other ones are doing more accelerated muscle memory building, which I think is wonderful. What we decided on is four two-week sprints. So we have teams in for eight weeks, and we help them.

We get them in there, we teach them about the cadences and everything, but it's more real-time. And when they're there, they're not there for a training. They're giving up some of their time, and it's an immersive experience. We are working along with them while they're delivering their product.

Number three is the coaching staff. So we've got, again, all these teams, and you can see now the arrows are kind of turned around and we're working together with the delivery teams, and we've got some extra support. Rahul often talks about fingers to a fist. So we had all these disjointed efforts that I talked about earlier. How are we going to bring that together? And that sounds kind of violent, not intended to be. It's a gentle fist.

But the coaches, again, they inject themselves into conversations and ceremonies. They may offer some help or guidance in line. They might pull somebody aside afterwards. "Would you consider this approach? Here's another way to do things." And we are there for them when they need us. They come and physically grab us.

We had an incidence in the first wave where it was a Tuesday, and a project manager scheduled a meeting for Thursday, and I was kind of baffled. I'm like, "We're right here. Just grab us." So again, that muscle memory or that behavior and changing that and teaching people that you're accessible and you're there, and you're there for them, it's very enlightening to them, and it's fun to see as they start to grasp that.

So again, the tools are easy. We're promoting the behavior: just continuous improvement, automation, debt reduction within sprints, empowering the teams. They all come in with varying levels of maturity. They might be waterfall, and we teach them how to get into a sprint cadence and conduct those ceremonies. They may or may not have ever done API design. They may be on our pipeline, they may not. They might be extremely manual in their processes. We'll help them with the automation.

So again, we're just pulling on all these levers. We don't have any qualification for, "Well, you have to be up to this bar in order to move forward." It's wherever you're at, we're going to help you get better.

Another thing that we had to do with that eight weeks of time is we have what we call a concierge service for some of the infrastructure. For some of the legacy processes, we have ServiceNow requests, and then things fall into a black hole. Being that we only have eight weeks, for some of those longer requests, that might take two weeks to get turned around. We don't have two weeks.

So what we do in these intake sessions when we're getting to know the teams, we go and grease skids with infrastructure teams, and I talk with my peers, and we get resources allocated. So during that eight weeks, we still follow the process to do the request, but then I have a bat phone, and in some cases, we just have the people come live with us in the Dojo for infrastructure support.

So here's coaching staff. We're missing one guy, Chris, from my Agile team. He was out sick when we were taking these pictures, but these guys are the heart and soul of the Dojo. They do an awesome job, and I'm blessed to be surrounded by such talented and passionate people. I love working with these guys, and they're the real ones that deserve the praise from the teams and our leadership.

So we got those three things: physical space, set time, and our coaches aligned. Now we had to go and put our marketing hats on and go sell. So we went and talked with... We have three delivery VPs. They all happen to be named Matt. So yeah, Matt times three. We went and spoke with the Matts and kind of sold them on the concept. And some of them were kind of more aligned with where we want to take things. Some of their organizations were not.

But we started to build our own backlog. We conducted intake sessions, and we still do. That builds our backlog. So every week, we invite teams that are interested to come in, and we have conversations, and we find out where they're at in their journey. And then that helps us kind of prepare for what we're going to be helping them with.

We ask that they're long-lived, dedicated teams. Two-pizza teams is what we're looking for, and that they're interested genuinely in improving the way that they work.

So we get that list and lift off. Now, this was an experiment. While we started looking at this in the fall and then carrying up till the summer, we had plenty of things that we were working on and getting some formality around the program. But as June 4th was coming, people were getting nervous.

It's middle of May, and they're like, "Man, can we push the date? I'm getting worried about this. I don't think we're ready." It's like, "No. Damn it, we're going to move forward." And we figured it out as we went. The facility wasn't complete. You guys saw the contrast in those pictures.

And for eight weeks, we worked with three teams. So we started out small. Everything we read says, "Start small, don't go crazy." And it was a lot of fun. Eight weeks later, we graduated those three teams.

That was on a Thursday at the end of July. Monday morning in August, we bring in six more teams, and we graduated them at the end of September. We're now in our third wave of our Dojo, and we brought in six teams to Atlanta on the 8th of October. And in addition, we actually brought in a Minneapolis team, so we're very excited about that. We're starting our Dojo in Minneapolis.

All right, so victories. We had respectable demos after only eight weeks. We've had production launches from the wave one and two teams since they've gotten out. That's awesome. High fives in the hallway. They're so excited. "Hey, man, we pulled this off. Customers are happy."

Through that concierge service, we're starting to bridge and find low-hanging fruit with the infrastructure organization. "Hey, what opportunities do we have to improve this process? What opportunities do we have to automate some of the things we're doing there?"

And seeing the mindset of the teams questioning how things are done or why they are that way is so rewarding. They're constantly challenging things around them that maybe in the past they would've been complacent with. I tell them when we launch the thing every time, "We are the anti-'This is how we've done it for 10 years.' Let's get beyond that. Let's make everything better."

So we have a question that we ask, a survey. It's like an NPS thing. We say, "Would you recommend the Dojo program to your peers at Delta?" Wave one, 87% said yes. Wave two, 89% of the graduating team said yes. So that was very exciting for us. I really wasn't sure what to expect.

Thank you.

Struggles. You guys ever done a jet pull? It's pretty fun. Once you get it going, it's going.

So there was a couple times I wasn't sure that we were going to move forward. If there's anything that you guys take away from this particular presentation, if you're preparing to take this journey on your own, just recognize you're going to get punched in the kidneys just when you think you're off and running.

So I was asked at one point to put things on pause and, we'll assemble the band again in a couple of months. And we ended up pulling through. We didn't pause it. But I was just disgusted about it, and I had my head in my hands and one of my buddies walks by, he goes, "It can't be that bad."

And I'm kind of an emotional guy. I think I was probably welling up a little bit and trying to hold it together. And I have a call with my friend Nick Whittaker at Virgin Atlantic every once in a while, and it just so happened that that day, we had a call. So I get on the phone with him, I'm like, "Man, I can't believe this. I thought we were golden."

And he's like, "Hey, my mentor at another Virgin division told me this is going to happen. It'll happen to you, it'll happen to me, it happens to everybody. When you're trying to drive this kind of cultural change, just when you think you've got it, you find out you don't."

And so that really helped me because I realized, well, okay, I was being really hard on myself. How could I have avoided this? How could I have done this better? Or is there something wrong with Delta? And it's like, no. This is just something that happens.

So again, if you learn anything from this presentation, if you're going to take this journey, be ready for the fight and keep fighting. It's worth it. Walking into the Dojo and seeing the buzz and the energy and the excitement from these teams puts the biggest smile on my face. There's nothing more rewarding than seeing something like that. So, like I said, keep fighting. It's worth it.

Lessons learned. Three main ones.

All teams are different. Be prepared to adapt to their needs. We've heard some presentations this week, if you're overly prescriptive, you're going to box yourself in. You can't expect people to form you. We're coaching, right? You make the best coaches play to the strengths of their players. You don't make your players play to your playbook. So be ready to adapt.

Number two, be careful of hard deadlines. We learned this in wave two. We had some teams that came in. Again, this is an immersion experience. We need time to coach, and if you're stiff-arming us the whole time because you can't take advantage of the coaching time, it's not working for anybody. And it wasn't the fault of these teams. They were told that they had a hard deadline, and they were trying to deliver on that. But it's something now that we look out for when we do these intake sessions, that perhaps they're not a good fit at this time. The idea of slowing down to speed up, that exists in this scenario.

Number three, we have a lot of work to do in the housekeeping of our eight weeks with our teams. So we continue to iterate through that. But between wave two and three, we gave ourselves a week. So that wave one to wave two is like bye-bye on Thursday, and then welcome the new teams on Monday. It was exhausting. We had a week to go between this time, and it was awesome. Coaches, we got a lot of stuff done that we hadn't had time because we were so immersed in the teams. And we were just better prepared. But we have a lot more to do there.

Again, when we set that date for June 4th, hell, we probably could've planned for a year to try to get some of this stuff done. That wasn't going to happen. We needed to go.

Testimonials. Things that they told us about their experience.

One of the scrum masters came in. She hadn't ever done it before, and she's just extremely happy on how she's able to help her team execute ceremonies and really help run the team.

One person said, "We thought we knew what we were doing, and the Dojo helped us polish those behaviors." And again, those are kind of tweaking those levers and helping them mature.

"We now function as a DevOps team."

There were some other comments, like in what would you say to other teams coming in? And it was, be ready to learn. You have to have the right mindset. And that's, again, part of the criteria. You have to see a sincerity about wanting to learn and wanting to get better.

What's next for us? So like I said, Minneapolis, we just had a team start there. We need to build it out. Planning to have two to four teams, if possible, there in January.

And post-Dojo support. If you think about it, as we have more and more graduates, alumni, whatever you want to call them, the snowball as it rolls down the hill, it gets bigger and bigger. How are we going to handle that? We need to scale. We need to bring on some more coaches and help with some of that stuff.

Product owner-specific training. In our retro that we do with the teams as they're leaving, that's something that, while the Agile coaches are working with scrum masters and product owners, we really need to hone in on that particular competency and help raise awareness to the business side executives as well.

Gene asked me, if I could wave a magic wand, what would I do? I would increase our capacity within the physical locations by two- to three-fold. And the reason for that is I don't necessarily think that we need 18 teams at any given time because I have concerns about how we would drive consistency between the coaches. It's more about after the eight weeks, these teams really feel like they're having their umbilical cord cut. And they get concerned like, "Hey, we have you here. We're able to grab you. What do I do next week?"

And so it would be nice to transition them through the Dojo and have a more graceful exit instead of just the chop at the end of the eight weeks. There's teams that need more help as they're leaving, and that's okay. We need to be able to address that.

I want to add more coaches, and the last one would be, if I could wave the magic wand, is to bring more teams through and bypass funding-related constraints.

Yeah. Magic.

Okay. Where we can use your help. So just because I'm up here, I don't proclaim to be any kind of expert on this. Like I said, this is an experiment. We continue to iterate. For those of you who are already doing it, how have your orgs scaled up in the ability to coach and continue to deliver that consistency that we are looking for? Do you see anything missing from our program offering? Is there anything glaring to you? And I'm all ears for any feedback that you guys have.

We all learn from each other. It's been fun to compare notes throughout the week, talking more with Ross's team at Verizon and hearing some of the things that they're going through. Again, it's all very therapeutic.

We're hiring, so if anybody's interested in flight benefits. Yeah, we are hiring. You should see job postings. We're going to have a blitz out there of positions later this year and into January. So if you're interested in working for Delta, please reach out to me or watch deltajobs.net.

And I thank you guys very much for the time today, and have a safe trip home.