Las Vegas 2020
The Air Force's Digital Journey in 12 Parsecs or Less
Learn from Adam Furtado, Chief of Platform for Kessel Run, and Lauren Knausenberger, newly appointed Deputy Chief Information Officer for the United States Air Force.
AF
Adam Furtado
Chief of Platform, Kessel Run, United States Air Force
LK
Lauren Knausenberger
Deputy Chief Information Officer, United States Air Force
Transcript
For years, I've heard about Kessel run an amazing effort that was enabling the us air force to build software in a manner very different than how they've typically done it for the last 30 years. They're deploying multiple times per day to support all the people who depended on them last year, thanks to Twitter. And some friends at pivotal. I managed to meet two of the founders of this group, and I was blown away by what I learned. I am so honored and delighted that Adam Patato who is chief of platform for Kessel run will be sharing their origin story and the all the amazing thing they've helped enable since then, I had a chance to visit them along with some friends in this community, uh, earlier this year, which incidentally was the last trip I made before the COVID travel lockdowns. Adam will be co-presenting with the amazing Lauren Nelson Berger, who was recently elevated to become deputy CIO for the U S air force, a sign of all the amazing contributions she has made to date helping change what she describes as one of the world's largest bureaucracies. I finally got to meet her in person at an amazing event. She helped organize called spark tank designed to enable airman to solve other Amira's problems across the entire us air force. It is not an understatement to say that meeting the teams who were pitching a judging panel that included some of the top levels of leadership within the air force was one of the highlights of my career is just another example of how the work that this community is doing matters to people who matter here is Adam and Lauren.
All right. I just want to start by thanking Jean for inviting us to speak today. It's humbling and exciting to be here. My name is Adam Furtado and I'm the chief of platform at castle run a us air force, digital transformation effort, helping normalized DevOps within the world's largest bureaucracy.
Good afternoon. I'm Lauren can Alison Berger, the newly appointed deputy CIO of the air force and really just joined the air force three years ago because I became very passionate about this problem space that Adam is now working in at Kessel run, and just really wanted to speed adoption of a commercial technologies and bust some of the bureaucracy that these war fighters find them in every day and really excited to chat today with my friend, Adam Patato and to be invited by another, another friend, gene, Kim, who has made such a great impact on our airman as well. Uh, so Adam, over to you for where our story begins.
Yeah. We're going to talk a lot about, uh, genes influence on some of the work that we've done. But first I want to talk about outcomes. So on October 3rd, 2015, the U S military struck a doctors without borders, hospital Afghanistan, believing it to be an enemy strongholds, Afghan commandos are under fire, and we had to respond fast to support our partners. Unclassified analysis later showed a number of fillers that contributed here. Uh, there was no time to fully brief the crew, their aircraft didn't have the latest data ID to ID the hospital. Basically our failed it ecosystem caused an AC one 30 gunship to attack the wrong building. Now it wasn't the crew members fault. We failed them with ancient software and stove, pipe systems. And unfortunately for us ancient software and stovepipe systems are ubiquitous in the military right now. They exist everywhere from enterprise business, it weapons systems.
And what happened here is not some kind of black Swan event. It was predictable. It's going to happen again. And while what happened here was tragic. It doesn't even begin to compare to the what ifs of major theater war, uh, with a peer adversary, uh, with the same type of effects. Now, at this point, you're probably thinking, did I click on the wrong URL may in the wrong room? Uh, you didn't, you're not, I'm just talking about business outcomes. And this conference is about the relationship between technology and how it enables business outcomes. And these are the business outcomes of the U S air force. How do we amass air power to the level needed to achieve air superiority leading to our objective of protecting America's place in the world and defending democracy. We have all these conversations about business outcomes, and we can't forget our reality that failure to us, isn't a loss of revenue or loss of subscribers federally to us is a loss of lives.
So Lauren and I are going to talk a lot about how the us air force is enabling better outcomes. Kessel run as an entity. It wasn't started to do DevOps Kessel run started because it had really hard business problems to solve. And we recognized that the traditional way of doing defense, it wasn't working. We wanted to test that using modern software practices, processes principles would enable better outcomes. So a small coalition of presumed rebels formed and with a complete focus on mission, a collective disregard for the status quo and a willingness to just work harder than everybody else. The Kessel run Mo movement began now Kessel run named after Han Solo's famous smuggling route was named that because we knew we had to smuggle this way of working into the DOD, the world's largest bureaucracy. And it wasn't, we wouldn't be enough to share a couple of case studies and make some folks read the Phoenix project.
You have to understand at the time the DOD was a virtual case study itself and what not to do from a software perspective, I spent eight years active duty and I can vividly remember going into work and my phone scrolling, Twitter, checking my bank balance. As I walked in, then you had to put your phone and electronics and this little cubby on the way next to the door. As you walked into the office, you were suddenly transported into 1974. All of a sudden everything was analog. It was like walking into a time machine, whereas hardest. And an email and collaboration was nearly impossible. And a decade later it's still roughly feels the same, but the worst part is the Stockholm syndrome. This doesn't even feel abnormal to anyone anymore. The men and women were relying on to protect democracy, unable to collaborate on a Google doc or have access to a real chat tool, basic communication capabilities that we all take for granted in our regular lives.
You shouldn't have to go back in time to go to work. And it wasn't just business tools. The DOD was struggling with, according to us, digital service across all of the federal government, 94% of the federal it projects were behind schedule or over budget. And 40% of them never delivered a single thing. Eric Schmidt once testified to Congress, the DOD violates every rule of modern product development. Meanwhile, elsewhere, in reality, the software is eating. The world movement was happening all around us. Target Adidas, Walmart turning into software companies right under our nose, but the small coalition was paying attention. They wanted to turn the air force into a software company that can win wars to begin, to turn their attention to the key business outcome. That was right in front of them. And it was modernizing the air forces air operation center. The air force plans, their wars from places called air operation centers or AOC.
There were a couple dozen physical locations around the world where the air force strategizes plans executes the air campaign of wars and due to our archaic it infrastructure and our historic and ability to take advantage of modern technology. We have to do all this work with specific humans in specific buildings located in specific locations to access specific data on specific hardware. This brick and mortar approach has been in place for decades. With the only real improvements coming. We can get Microsoft office updates into the field. You might think I'm lying, but a recent search search sowed over 2.8 million Excel and PowerPoint files on one of the servers. In one of the locations, we have completely failed to get our users, the tools they need to be successful for far too long. Despite the leaps made in technology in the real world, the AOC continues to operate much in the same way as it had for decades.
There are specific humans with computers looking at maps, but none of this data is available to be shared with the outside world at a reasonable speed. Computer's getting more powerful, but no way to take advantage of them in the national defense strategy. It states we cannot expect success fighting tomorrow's conflicts with yesterday's weapons or equipment. This process needed a complete overhaul from infrastructure to networking to the software and enables. And that's what we set out to do. Gall's law tells us that any complex system and variably needs to start with a simple system. We utilize the strangler pattern to slowly chip away and iterate this behemoth out of the field. 22 physical locations, each with their own hardware software that slowly drifted away from each other over the years, did a unique needs and personal preference, impossible to update, maintain, improve. Our started chipping away at this was that a specific function within the larger process.
We had added pressure here, too, that we were simultaneously trying to prove that a shift to a dev ops structure and mindset would yield better results than the ingrained bureaucracy that folks were comfortable with. There there've been three failed modernization efforts at this point, over a billion taxpayer dollars spent on this problem. So the air force and Congress provided us some room to try something new. Now, unlike commercial airliners, the air force refused their aircraft in mid-air at very high speeds, very close to each other. It's one of the cooler sort of underrated things that the air force does, but it takes a massive amount of coordination. There are hundreds of aircraft that need fuel and only a fuel refuelers or tankers to provide it, ensuring that the tankers will be where they need to be at the exact time that the other aircraft need them at the right altitude with the right hardware.
It's really difficult. If we don't plant efficiently enough missions, can't get completed. It's basically a symphony in the sky. There's a group of pilots to do this every day. They plan this work and as they work through the day's mission, the way that they used to do this, where they move these cultural putts around, you can see here signaling where, and when tinkers would meet up with the Jetson needed fuel, there was an Excel macro involved, somewhere in a bunch of manual data entry. Now, while there's team that is processed out to a science, they can only be as efficient as their brains would allow, and they could not easily react to change. If there were significant changes, they'd have to erase the entire whiteboard. Essentially start over again from the beginning, if there were moderate changes, they made it decided to send one more tanker into the air to ensure that no missions were uncovered.
This led to a massive amount of inefficiency, as you can imagine. So this is the first problem that we set out to solve. We started with a small team of active duty air force software engineers. Another group of rebels who were frustrated about the air forces, inability to utilize our unique software talent. We partnered with industry started working in balanced team models, using extreme programming, dev ops principles, and built a piece of software that at first just digitize their process. We utilize a partner organization's infrastructure and platform services, and we got a product out to our users in a matter of weeks, the initial minimum viable product led to enough efficiency that it kept one aircraft and its associated crew from flying every day, a fuel savings of $214,000 a day. But as we were all aware, software's never done. We kept iterating. And about a year and 30 iterations later, we doubled those savings.
Keeping two aircraft and crews on the ground today, uh, fuel savings are $425,000 a day, almost $13 million a month, just in the fuel savings alone. They were also able to cut the planning team in half meaning. More people were didn't have to be deployed away from their families in order to do this work. This is a big enough win that we got noticed where we needed to. We were able to get so many moments and momentum that working in this model can lead to better outcomes, but then we quickly hit some roadblocks after all we existed within the world's largest bureaucracy. And that's where Lauren costumer came in and joins the story.
Thanks, Adam. It's always great to hear that story. It never gets old for me. Um, so, so here I was coming in probably three months into my role in the, in the air force. And I met these incredible guys at this place called Kessel run. Most of the other folks that I met were at some sort of acronym that I had to ask about a bunch of times to figure out what they really did. And here is Kessel run the smuggling route out of salt, out of star wars, um, and all of these passionate airmen. Um, usually more junior actually than the folks that I was running into at the Pentagon. And they could deploy code to the warfighter five times a day and in an environment where most of the folks around me were complaining that after years they still had not gotten their first, uh, deployment of whatever it was that they were looking for.
And at the time, you know, I was coming up to speed and looking at all of the different problems that were in people's way and cybersecurity accreditation was one, uh, color of money. Um, just the, the way that funds are given to programs was another, the way that we did testing was another, um, and really focus on user was that was another issue that I saw across the air force. And here are these guys that had knocked out all of the other noise and they could deploy five times a day, but they were still waiting months for their software to be accredited before they could push the button. And they had already adopted all of these modern software practices. They had a CIC pipeline that automatically checked all of their code. They had built in scans. They had some really great continuous monitoring set up in their production environments.
All of these things were things that I saw immediately, um, that they were not getting credit for, for the outstanding work that they were doing. And as I jumped in with the team more and more, I realized that this was probably one of the most secure deployments of software in the DOD. Um, and, uh, you know, I, I, at the time asked if anyone could show me a more as secure software process and no one could. So we'll say that you were, um, and so really I was looking at the time for a way to test new cybersecurity accreditation methods, and we found each other. And actually, Adam, I don't remember if I found you guys or if you found me, um, but the second that we met, I just, I really wanted to be able to help you. And so we worked together on what became the first continuous ATO process authority to operate.
And what that basically means is that instead of going through the very static gated approach that we have had in the past, where you develop software, it's tested over a long period of time. You have some security assessors come in and also assess it. Um, and then at some point in time that software, that again, you were able to deploy five times a day, uh, six months later is deployed even the best teams in the air force, got it down to maybe six weeks, but that wasn't fast enough for our war fighters. And so we fully automated it and we accredited the process by which they developed code. So that by the time they pushed to production, it was automatically accredited. They no longer had to have a gate in their way. And that was game changing for the air force. Um, the officially signed that memo, I think it was April of 2018 and that just set off, um, you know, it really redefined what the standards were for software development, because in the past, actually a lot of our most innovative efforts would hide.
They didn't want someone to come in and tell them that they were doing something wrong or that they should be using different processes. And all of a sudden there was this lightning fast process that we used and everyone was coming to me, um, because they had signed this memo and saying, Hey, how do I get a continuous ATO to, and that actually sparked, um, just some incredible collaboration across all of these different groups that we're doing DevSecOps and Kessel run even more so was seen as an organization that even the biggest bureau bureaucrats, um, in the CIO's office, um, could get on board with. Um, and so that was just an incredible moment, um, that we were able to get this team actually deploying at the speed that they could deploy at, um, embracing the hacker community to kind of check our work. And we were able to come up with a process that both the innovation community and the more hardened bureaucrats could get on board with and give these guys credit for all of that security that they were baked in. So, but Adam, now we had United the, uh, the DevSecOps community you guys were deploying at rocket speed. What happened next?
Yeah. Thanks Lauren. Uh, so after the earliest success with injera fueling, we follow that by standing up a few more teams, getting some more decent size winds from them as well. At this point, we're pretty sure that we found a way to solve major enterprise problems with new ways of thinking and new ways of working armed with the initial successes and some executive champions like Lauren. We started to grow our team between April of 2017 and April of 2020. We got more than a dozen additional user facing products into production and built our own platform. We've got the organization to a place where we were teetering on the edge of elite software delivery, performance metrics, cycle times, counting hours, that of months and years there were magazine articles and awards and accolades and offshoot innovation organizations popping up everywhere. We had visits from top general senators, congressmen and the seemingly daily basis and great user feedback for the stuff that we were building.
Uh, in fact, we got some, some fan fiction, um, okay. That one was a little weird, uh, but it was an incredible time of growth and success, uh, for the organization. Uh, initially in 2017, this initial band of five members trying to solve some problems in about a year growing to somewhere in the 50 range a year later, we're upwards of 300. Uh, and then now in 2020, the organization's grown to over 1300 people. Um, and that scale has been, uh, great and exciting, but also it comes with its challenges as well. Um, an initial, a five person coalition grew into a 1300 person organization. And in that we had over 50 product service and platform teams and with our success came more work of increasing complexity, both technically and politically. And that scaling has been super fast and you may have guessed it basically caused everything to break the things that used to work for us.
Didn't it doesn't don't work anymore. Uh, we slowly over time started to see our practice, start to atrophy, our values and principles as an organization. We're not really as clear anymore different parts of the organization develop vastly different cultures. And we struggled to find the mid tier leaders. We needed to really handle that growth. And at the time I was leading product and application development and I was struggling to build the bridges that I needed to within our platform organization. In other words, we went from being a beacon of DevOps hope for the air force to struggling, to successfully even be a dev ops organization internally. It took a visit from none other than gene Kim and his friends, Nick Kirsten, Jeff Gilmore, and Tom Longstaff. And they gave us what we needed to really start turning things around. Jean came in and gave a great talk to the team.
And it really changed the way I thought about our challenges. He walked us through the five ideals and I looked around the room to a few hundred of our staff. And you could see folks feeling a little bit uncomfortable with how close to home some of the points and examples were hitting. Following the visit. We started to rethink where we were as an organization, starting with our organizational design. We pulled an inverse Conway, maneuver, restructured our organization in the way that we wanted to, to deliver that we want our delivery, uh, to reflect. Now, at that time I was leading application development as an organization's first product leader and grew with the organization over the years, taking on increasing responsibility in various product roles. It was about this time where I was asked to move from product to leading our platform organization. The thought was that in an effort to make the organizational statement around what we want it to be and grow the empathy across the DevOps divide, I could have lead that platform organization improve our user focus and our relationships.
So this new role gave me some new found energy, and I started to transition and settle into it. And I was reminded of gene and company's visit and knew that I needed to focus on some key specific improvements, uh, to move things forward. So I settled into these four key focus areas. I'm going to walk through, uh, the eventually became the basis of our OKR, but without realizing it until thinking about this talk, uh, these organizational objectives tie very neatly to the five ideals that gene, uh, brought to us with the unicorn project. I'm not sure how they affected our organization. The first objective that we focused on here was to increase customer centricity. Um, even though the platform team is built around services and kind of back end work, I think it's a responsibility of any service or platform team to really understand the larger business objectives that they're here to support.
One of the first learnings that I had though, when I joined with our team did care about their customers. They just said yes to everything and dropped whatever else they were doing. This led to an unconscionable amount of unplanned work and scores of work in progress across the organization, all in the guise of trying to make their customers happy. But customer centricity doesn't always mean you react and do whatever your customer is yelling at you to do. And it might mean just means that you have to know enough about their position to know why they're yelling at you. Sometimes it's being more customer centric is to say no to a request in order to deliver, you know, a more sustainable longer-term solution. Our second objective is to increase flow. I mentioned our work in progress, a work in progress earlier. We've compounded that with like an inability to accurately plan capacity, we were capacity planning up to 90, 95% in most cases, and then getting the unplanned work dropped on top of that leading to more work in progress, less delivery and massive flow problems. Now we're tracking all of our unplanned work better so we can better understand the actual capacity for work that we have and ensuring the place that we can automate some of that away by putting it in a backlog and prioritizing it into our work. We're also doing an analysis across our organization for places where our bus count for a service or technology or task is less than or equal to one.
Our third objective here is to increase developer productivity here. We're focusing on all the places where we've inadvertently replaced human communication with ticket systems or took away developer access developers, both internal to the platform organization, and our customers should be able to complete and deliver all of their work on their own without manual asks to outside entities and our fourth objective increase employee engagement. And this is the one that can often get forgotten about. And the biggest takeaway I had coming into this new team was really the lack of psychological safety that existed. Uh, and in many ways still exists. And I noticed that there were all of a sudden, a lot more DNS happening during meetings. It was hard to get a accurate picture of things and conversations as people weren't as comfortable to challenge each other or provide different perspectives. I started seeing this tangible negative effects of not having the requisite psychological safety to be successful.
And this is an interesting one to talk about for a military organization. I would say that the military hierarchal and command and control structured, or completely antithetical to a psychologically safe environment, but it is imperative for success on the battlefield. If you're in a trench, having a single NC to in-charge kind of saves lives to solve complex problems on software teams. I need everyone to feel comfortable and safe to provide their opinion, without fear of fear of dismissal or ridicule. And we've worked hard at this internally, but we still have a ways to go. One of the things that you would notice if you walked in our office, if we ever get back to it is that none of our military members wear uniforms. And that isn't because we are proud of the work these service members are doing. We just don't want a military hierarchy to come in and play, uh, come into play while we're coding.
Now, these four objectives only exist in service of Kessel runs, larger business outcomes, being prepared to win and inevitable war against a peer adversary. That is why we came up with a new vision statement for our platform. And it's a simple one, build a platform to win the war general. Maddis said that America has no preordained right to victory on the battlefield. So we have a lot of work to do, but let's think about what this outcome actually means. In a lot of ways, we have an impossible product problem. We're trying to build a platform and associated products to win this future war. It's a user scenario that's never happened before nobody's ever experienced. Nobody knows what will it look like or how things will go. We're all just doing our best guests to be as prepared as possible for when that day comes. For those of you who are in retail.
Imagine having to be prepared for a black Friday event that would likely go on for weeks, months, or years, but never knowing when it's coming. This obviously makes ensuring that our system is reliable and modular and resilient that much more important. And these objectives help us ensure that we do that. Now, a Kessel run success has been based on rapid, has been rapid and led out to the hyper-growth. We talked about earlier five to 1300. Our scale is really a drop in the bucket to what Lauren is working on the air force level about 600,000 active employees. But I do think customer has played a hand in building some while we were, when we were smaller. We often used to say that, you know, this is not actually a rebellion. Uh, it's a revolution. And I do think that's proven to be true, but in the, pass it back to Lawrence and you can talk about how she's trying to take these lessons learned and scale them across the larger air force.
Absolutely. Thanks Adam. So Adam and his team and the other folks in our growing DevSecOps community across the air force really have put on the blasters and, you know, kind of launch the rockets of innovation, so to speak, but they still have to get into their DeLorean and go back to 1985 when they deploy. Um, and oftentimes, you know, we have this incredible modern software, um, but they have to go to 10 different deployment environments. Um, and, and sometimes more, um, just because there's not consistency across the board. Um, and so now that I am the, uh, the deputy CIO for the air force, I'm really looking at some of those really boring problems down the stack that make a huge impact all the way up. Um, so that Adam and his team at castle run when they deploy or Nick Shalane and his team over at platform one when they deploy that we're getting all of that efficiency all the way down the stack and consistently.
And so some of the things that I'm looking at in the new CIO role are deploying a really rock solid digital foundation. Um, recognizing that we have these modern tools that we can use in the war fighting environments, making sure that we have the rock solid network on which they ride, making sure that we can automatically push across all levels of found of, of classification and that we have constantly the connect speed that we need to get to the cloud. And, you know, we, we kind of joke in the military sometimes that we're in the process of moving from dial up to fires because, um, you know, we have been, uh, behind in our cloud journey in the air force, I'd say we're leading the DOD, we've done some really great things, but it's only been over the past three to five years that, that we've gotten there and we have to catch up on our connect now.
And in the rest of the world, you know, that you're going to be able to connect you kind of assume in a way, um, we assume in a way in our military conversations too, but we can't, uh, because the paradigm has shifted and we have to actually focus on that level of really, really boring, foundational, often physical investment to make sure that we're going to have that consistent connection to the cloud, uh, and the speed to see all of the data that we need to make decisions at all levels. The second part that I'll be focusing on over the next year is enabling our digital talent. And that means enabling people with the education that they need through things like digital university, where folks can go in and they can be assessed regardless of where they are working today. You know, whether their job is to be a software developer or whether their job is to be the cook.
We can see that they have incredible skills in whatever areas they have assessed in. And we have democratize the playing field so that the has every bit as much the opportunity as the DevSecOps engineer to go in and take all of the training that they want to, and to rise in the leaderboard based on their work and their passion and their aptitude, and to be called into really important missions based on that. And I kind of joke it's the under siege principle, um, the movie with Steven Seagal on the submarine, you know, when, when the best fighters, the cook, you know, it's awesome that he saved the day, but, um, maybe if they had something like digital, you, they would have known that before the terrorist or the sub. Um, and so really we want to use every airmen and the place where they are going to be the most powerful.
And the second part of that is making sure that people have the tools they need to do their job. You heard Adam talking about the guys on the whiteboard and boy, they could do amazing things with that whiteboard. And, uh, and that is just a microcosm of things that happen across the military every day. You have incredibly smart, dedicated people that are solving problems in ridiculously complex ways. And it's taking much more time and effort and sweat and blood to get things done than it should. And so giving people the tools that they need to do their job, whether that is developing tools and pushing it to the war fighter, whether that is just buying a SAS offering and making sure that everyone has the opportunity to connect to that, we have to do a better job of serving our war fighters. And that is a major area of focus.
And then finally, um, we're going to ruthlessly attack, manual process and policy and legacy infrastructure. And part of that is because it is in the way it's slowing us down, it's affecting our user experience. And part of that is because we have to, because our budget is getting tighter and the more that we can get rid of anything that is kind of old and in the way, the more that we have trade space for those new investments that we have to make. And so, um, Adam, it's been so exciting in the past as the chief transformation officer for the air force to help enable things further up the stack and to be able to see those immediate results. Um, now I get to kind of go into the trenches, the pipework, so to speak, um, to make sure that you have all of that digital foundation, that digital infrastructure that you guys needed to just rock it every time for our warfighter.
That's great. Thanks Lauren. Um, and I, I think, uh, Jean asked us to, uh, end on a place where we can ask for help from the community, uh, that's here. And, uh, I guess the place that that comes to mind for me is I want the community to hold us accountable as the air force, to being a digital leader in this space. I think the air force and the DOD writ large as we make these kinds of changes in this way of working should be a place that top talent wants to come and really has a chance to make a global impact. I want to make sure that our efforts in this space are not just good for government. They're just good. Um, so I'm asking for, uh, for some to be held accountable to the same standards everybody else does, um, to make that impact Lauren.
Absolutely. And you know, when we say we want to bring in new talent, we really do mean it. And we mean that in a, in a bunch of different ways, you could be like me and jump, uh, quite accidentally actually from the entrepreneurial or the venture capital space, or maybe the large tech space come in and just, you know, jump into the trenches with these guys. It is the most rewarding thing that I've done in my career. There are so many amazingly complex problems that you cannot solve anywhere else. And some of that commercial talent, you have the skills to solve it so easily because you've been, you've done it before and we're going through it for the first time. Um, so if there is a, you know, if you're, if you're watching this and you're thinking, I want to be part of solving, some of these problems reach out to us, we could use you.
And we are very serious about expanding our defense industrial base. You know, we need the guys that have been here for a long time. We rely on them, um, for building ships and airplanes and helping us do mission critical software, but they can't do it all. You know, we need new talent in those areas too. And I'm so thankful for Elon Musk and companies like space X showing us how to do space and satellites completely differently. Um, companies like Tesla to, um, you know, just showing us that, that we don't have to solve problems the way that we always have. And so, um, really, if you're excited about this problem, space come and work with us, um, there are lots of different ways to get involved. Um, there, the super program is there. We have OTAs working through DIU, and then you can also partner with our more traditional primes, but we're trying to do things to lower the bar for entry and to really talk to you.
And even the big guys, when you're doing business with us, as you have for years, um, try to help us to get to where we need to go. Um, try to help us look in the mirror when we're asking you to do things in a way that's not going to help us. Um, kind of like Adam, uh, gave the example earlier, you know, we ask for something and it doesn't make sense, help us to see what would really help solve the problem rather than the thing that we asked for and, uh, you know, and negotiate with us because at the end of the day, you know, we want to move our country forward. We want to be smart with our investments. We want to adopt awesome tech, not just tech. That was good enough for government. And, um, you know, you've probably heard me say this before, but, you know, we have some exquisite capabilities, you know, we can hit the back end of a fly from halfway around the world because, you know, we've been focused on those problems forever.
The easy stuff is what's really hard for us sometimes. So as much as you can kind of help us cut through the easy stuff and to, um, get a really good picture of where we are getting in our own way, um, repeatedly, that always helps. So Adam, you guys have achieved so much and we've been able to push forward some things together that no one has done before. And I'm wondering if you could have one wish right now that I can help with, on what could we do to just really unkink the works and get you guys to the next level of success. What would it be
Putting it on the spot? Um, I think the thing that I would love to see some focus on is how we are measuring the success of these various transformation and kind of innovation efforts. Um, I think, uh, we run the risk of some of these things becoming, um, focused on vanity metrics and kind of outputs and kind of forgetting that we're all doing this to solve a larger business outcome. That's really hard to measure, but I think a focus on value streams and how, uh, dev ops as an example is a tool for us to achieve a larger outcome is a place to air force really needs to start kind of focusing on in order for us to like really be successful. Long-term
Well, I love it. And that's right in line. So I'll just go like this wish granted. Um, so we'll definitely work together and make that happen. Um, and I, I think our corporate structure will love that as well. So thanks as always great to talk to you and thanks Jean and team for having us.
Thanks. Ready.