Las Vegas 2019

Our Journey to 100% Agile and a BizDevOps Product Portfolio

This talk will describe the "why" and the "way" to 100% Agile @ BMW Group IT - a holistic approach with 4 focus areas: Process, Structure, Technology and People&Culture. Ralf will give a deep dive into the transformation from "Projects" to "Products" defined last year.


Frank Ramsak started his career within the BMW Group in 2003 and is presently responsible for Architecture, Innovation and Technology in the BMW Group IT Governance. With his team and the community of architects, he defines and drives the innovative, competitive IT solution space for the feature teams.


Before his time in the IT-Governance, he managed international IT projects and was responsible for enterprise architecture management for the R&D, quality and production areas.


Frank studied computer science at the Technical University of Munich (TUM) and the University of Illinois at Urbana-Champaign. He received his PhD from TUM for his work on multi-dimensional indexing in database systems.


Ralf Waltram has been with the BMW Group since 1996 and is responsible for IT systems in vehicle development since 2015. He and his team focus on the possibilities of digitalization in the R&D process, with an agile collaboration model and a focus on a BizDevOps structure. Prior to this, he managed international IT projects, e.g. in China, in the area of R&D, sales and marketing and was responsible in different line functions. Ralf Waltram studied computer science at the Munich University of Applied Sciences, specializing in computer vision and neural networks.

DF

Dr. Frank Ramsak

IT Governance, BMW Group

RW

Ralf Waltram

Head of IT Systems Research & Development, BMW Group

Transcript

00:00:02

The first presentation of the day is from the amazing team at BMW. Ralph Walstrom used to be the VP of R and D uh, VP of it for R and D, but he's recently taken over VP of it operation for a VP of it for production. So he's no longer responsible for the seven year design process. He's now responsible for almost every aspect that enables to production of millions of cars per year. He will be co-presenting with doctor Dr. Frank rom SOC head of architecture and it governance over the last three years, they have been on this tireless journey to modernize the ways of working across all of BMW. I met them last year at DevOps enterprise London. And when they told me about what they've been able to achieve, I know I knew immediately that this was a talk that needed to be presented here. I think the quote that best sets the stage for what they've done is this, it is this work that we are doing is responsible for the biggest change to how BMW does business in the last 20 years with that team BMW.

00:01:46

The future of driving, nothing to be afraid of Because we are working on safer cars for a safer tomorrow with large scale data-driven development At the BMW autonomous driving campus.

00:02:12

Thank you. And good morning, everybody. We bows are so excited being here at the third and the final day of this grade, Def ops enterprise summit here in Las Vegas. And we are so glad to share with you our story about getting dev ops biz dev ops and our attorney to 100% agile. And as most of you know, dev ops can be also very spooky sometimes, but it's definitely nothing to be afraid of. So my name is Ralph, and as Jean mentioned, I'm responsible for my team for the it dev ops in the area of logistics and production. And I'm together here with Frank. Yeah, good morning. He also, from my side,

00:02:54

I'm head of architecture, innovation and technology within their T governance. And what we want to share with you is I think really one of the greatest opportunities that I had over my career and BMW is really to drive this transformation. We want to share with you. So, but to start this, we set some context where we hope that you are aware of the BMW brand. So the ultimate driving machine or sheer driving pleasure, we are the world's leading premium car manufacturer and provider of premium mobility services. We have 135,000 employees worldwide. Last year, we sold short of 2.5 million vehicles. And in the times about electrification, I think it's important to know that by the end of this year, we will have 500,000 electrified cars made, be either fully electrified, like the or one of our many plug-in hybrids on the streets. And with this, we are also leading in the automotive industry.

00:03:51

So, but what is probably a, not that familiar to use that we already have more than 12 million connected cars out there because we were the first automotive manufacturer, shipping SIM cards per standard. And this gave us a good chance to really provide connected services to our customers. So in, when we're talking about the automotive industry and you talk about it, you always have two sides, right? The one thing is the it within the product and around the product, for example, the control units for the engines, entertainment, navigation systems up to the back ends, right? For the connected services. And this is what is in the real of our development division, right? And we call it connected company. So rather than I am from the corporate it or the group, it, and we are responsible for all the process from the early phase of research development, purchasing production sales, and after sets.

00:04:45

And we also have a regulated area with our BMW bank. So all supporting all these processes, this is the responsibility of the BMW group. It, and we are 5,500 employees, worldwide headquarters in Munich, but all major plant sites and national sales companies, we have location there. So we have a global footprint. We are, I would say, a grownup startup, right? So we have a landscape of almost 5,000 active applications. We have 30,000 servers, they are 100 petabyte of data, almost doubling every one I have years. So as the head of architecture, I can tell you, this is an asset. And sometime it's a burden because we have all technologies over the last 40 years in use somewhere. The only thing I can guarantee you, we don't use punch cards anymore. So that's a good thing. So, um, so this sets the scene, but when we want to talk about our journey, I think it's important to understand the starting point.

00:05:46

So let's do a time wrap back to before 2016, and you have to understand that BMW by heart is a project driven company, all the great products you see in the streets, they are the results of projects, right? And there's the holy milestone in our company called start of production SOP. And as consequence, all our corporate, it was also project driven. And we had a very sophisticated ISO certified process model, project oriented. And as Taylorism is big in automotive industry, we had 40 plus roles, right? Most of them ending by manager because we internally did not the real work. We just managing externals to do the real work. So waterfall was dominant, right. Project driven, even. So we did some very good experiences with applying agile methods to our projects. Mostly it was waterfall with all the issues I think you are familiar with. And our heroes were the project leads, right?

00:06:43

Always fighting the never ending war about solving the magic triangle content time and budget. And in the times of uncertainty and volatility and user requirements, this kept became changed, challenging and challenging. Our release weekends turned from fun to drama because they started sometimes already on Wednesday to be finished by Monday morning was our major two times a year, SAP releases and last but not least governance, good old direct and control, right? Throwing out regulations that hardly anybody read and even fewer adhere to. So the question was rough in such a certain, certain things. How can we really challenge the challenges of the automotive industry, autonomous driving connectivity, electrification shared? So,

00:07:32

So definitely in 2016 small group, he said, we have to change something because with the digitalization, with autonomous driving, with all the new technology coming up to the market, we saw that we have to tackle three things, being an it more flexible, bringing more speed to, to the it products and to the business and put again, the focus to customer and Ms. Customer mean I, the internal customer, the real users and our external customers. And in 2016, we have been a bi-modal it a two-speed it, we had waterfall and agile. We hit 80%, uh, teams working in a waterfall world and 20% working already in an agile way. And they had a lot of fun, but they were blocked by the waterfall people, two releases like from Frank mentioned before per year, and the agile teams could release much faster. So we had, we said, we need to change something.

00:08:29

We want to go away from the, bi-modal it to our answer. 100% agile. We know it is a wish. And at that time it was clear vision. And we said, it's starting a churning and we will not reach it over a weekend. So it's not that you buy a book about agile or scrum, give it to the team, read it over the weekend. And Monday, everybody is 100% agile. No, we started a big transformation process and we, it tackled it in four dimensions. First one, clearly finding out what is the right agile process we are using at BMW. But the process, the methodology is not all in a 100% agile world. It's more, it's the structure, how you work. That's why we decided to reorganize and bringing Def and ops people together at the lowest possible, a possibility it's the feature team. And also, and we covered this later.

00:09:23

We decided to go away from projects to a clear it product portfolio. Third, you need the right technology. You use, you have to use the right technology to become 100% agile, which means we strive more with the help of the team of Frank in the architecture, architecture, microservices, but also cloud enabled 10 call it technologies. And last but not least a very important point for me. It's we have to change the culture because the culture in an agile world is totally different. As in a waterfall world in agile, we focus more, an openness and transparency with the daily scrum events and the artifacts, you know, all in agile. So that's the way we started our transformation. And I can say to you, it's not always a straight pass. And I like this quote, if the past before you is clear, you probably on someone else's and you should really see what is the next step, where we have to put a bit more pressure on it, or you have to really say and think about what's what's next. And we would tell you four milestones from our journey so far, we are now three-year on our journey and we will cover biz dev ops product portfolio, the agile working model, back to code and user experience. And Frank, can you start with BIS Stef ops product portfolio?

00:10:49

Yeah. This is an awkward term, right? This dev ops product portfolio. What is it? Well, this, this guys is the greatest change that we were experiences and BMW group at T over the last 30 years, because it's effectively the end of projects and telling you before that BMW by heart is a product driven company. We probably don't have to lecture you. Why product is the right way to go, but to tell our stakeholders, to tell our management right, to tell our people that it's good to let loose of project. This was one of the hard things to cover. And of course we had to tell our business it's not worse having the contract game at the beginning of his project, right. Fighting for the best position about content time and budget all the time. So this is not the common ground, right? To solve the problems.

00:11:37

And even the boundaries, the ball of contusion within it, between dev and operations. It's so ridiculous that at the point where the project really becomes effective, when you have the results, when the users first can really benefit from the results of your project, you're handing over responsibility of somebody else. How could you ever implement a fruitful inspect and adapt cycle, right? If you're changing responsibilities, how should this work? And this was our utmost belief that really in our BMW culture, we had to change from projects to products. And we did this. So what we did is basically bringing the business and the development and operation teams in one end to end responsibility for a product, for the creation, for the maintenance and for the operation of a product. So what is the product for us while we started with the value streams over the company, for example, production, right?

00:12:29

And then we used good old enterprise architecture techniques like business capabilities, right? To refine the room. So we have a logistics domain to be stay in the real morphologic of production. And within this logistic domain, we have an, a product, for example, for the in-house logistics of a yard management. And this covers all of the, it, all the applications that need to be that this business capabilities fulfilled. And this is the end to end responsibility of this cross-functional business ops product team. So we started this two and a half years ago. We now have a product portfolio out of 10 value streams, 50 domains, 270 products covering the whole space. As I mentioned before, from the early phase of research up to a BMW bank captured in this one product portfolio, and this is effective from the 1st of January of this year, we really stopped doing projects with the start of 2019.

00:13:25

And you have to change everything in it. You have to change the way you finance, you allocate resources to it. So this was the hardware to do so, but you get this end to end relation responsibility. And this is so important. And even more, you get potential because the first time you have a chance to get what we call it clear 360 degree view on what is going on in it beforehand. You had, you were measured at T projects. You had operation KPIs, maintenance, KPIs. How could you ever tell if all the risks is a mitigation, you have all the resources to mitigate a certain risk. Some was done in project. Some is in maintenance now linking everything within our it to a product finance, head counts, operational KPIs, security, right, risk technical depth. You have all the numbers now to really do a really decision on where to spend your resources where it's best for the business. So this is how we changed the structure. We really went to products, but the question is now, how do we work in this new structure?

00:14:31

Yeah, Frank, that's a good point. I mean, we, when we started, we decided how is the way we want to work in an agile area? And we define it in three layers first. And a very important point is to feature team level and Derby focused on using at child bandwidth. So we did not decide to work in model for the whole it, we are using the complete bandwidth. It's fine that the product starts with Kanban. And if it's mature, it go, it goes to a scrum and even higher to a scaled framework. Second level is the product level. And here we want to use synergies between the products, uh, from the principles, from the methodologies, from the processes and how they are using in the HL tool chain and cross-product product between the domains between the value of value streams. Frank mentioned, we try as much of decoupling and if we need, we use lean synchronization between these value streams.

00:15:31

We put focus on dead. We have one common structure and one common language. Every team speaks the same, but it's not a textbook. It's really a living framework where the HR master contribute. Um, every week in every cycle, they get good things in bad things out. So we shared the good and bad practice practices, um, because there's a car manufacturer for cars, we are ISO certified. So it is very important that our framework is compliant and we try to build it in to our framework. So our framework should be right away compliant, and we support this by an agile tool chain. So the one thing is giving the teams a working model, but also to assist them with the right tool tool chain. And this is really a success story. We started two and a half years with establishing AHL tool chain, uh, with gyro confluence and the CITC CD pipelines.

00:16:28

And we focused on making a great offer, a great experience to the teams. And we started with 100 in the first months after one month, we had already 8,000 users on the HL Tunechi to a chain. And now two and a half years later, we have 70,000 users on our HR tool chain. Not only it guys, because we are doing biz dev ops. Also the best business colleagues are using our ATC very heavily. And I'm really proud about this success story. But one big thing is if you have an HR working model, what is about the skills you need for being agile? Oh, yes.

00:17:09

This is really a huge challenge, as I mentioned before. Right. Um, we were good in steering, right. Uh, in 2016. And I think steering in agile is not really much good. Right. In, in, in Adrian, it's about getting things done, not about steering. So what we learned is that we have to increase our internal software engineering rate, right? So we were starting about low, be below 30%. And it's not our goal to, to match the digital champions more than 80% internal software engineering is probably not that feasible for us. And even it's probably not. That makes sense. But what we are aiming for is to achieve about 50 to 60% internal software engineering. So how do we get from the starting point here? So we basically started with two initiatives. So the first one is back to coat, right? We encourage our people to lay Ms.

00:17:59

Project aside and go back to coding, right? So this is initiative. We gave them training methods. We gave them as software engineering principles. And for example, we have a back to code campus that is a really through three months, full time training campus, where we teach them to do enterprise style, Java, cloud, native development. We're taking them from their current jobs, doing this three months training, and then going back, not to the old job, because this would waste of time. We bring them to feature teams where they really can code. We had a really good pilot. The spring currently the first 30 colleagues are going through this campus and we are planned to have 200 plus colleagues a year going through this campus. And as you can see above there, we also doing coding dojo. So building up the software engineering community within our organization, which is very important.

00:18:50

However, this is just one step. We need the soft engineering skills, very urgently. So we did a second step and we found it a joint venture last September, critical tech works in protocol with two locations in Lisbon and Porto. And in the last 13 months, we hired almost 600 software engineers, half of them working for the connected company. So for the engineering and development department, the other half working for us, corporate it, and they are doing really full stack software engineering in an Azure with an end. It's such a great success story. They are very fast, high quality. They often outperform our late traditional suppliers and we will keep up investing in them. And this is a really good experience that we have in booths, this crucifix, and what we eventually want to generate is really, I heard this term also here in one talk, this developer experience. We really want to make BMW cool place for software engineers, but it's not only about developing experience. Ralph. It's also about

00:19:53

The user experience as it be mentioned. All right. As I mentioned before, we want to put more focus on the customer to internal and external. And that's why we also started an initiative called UX matters, user experience matters. And I have two great examples with me from our legacy world. As you can see one, one mainframe application, how many of you have still mainframe in their legacy landscape? Raise your hands, please. So a lot of you, so, you know, the real dark mode, the mother of the dark mode,

00:20:27

But the problem is Dartmouth becomes fancy again, but it's still the old user experience, the UX from hell. And we decided that this is not the way how we want to treat user experience in the future because good user interface is like a choke. If you have to explain it, it's probably not a good one. So we focused, we focused really to bring more UX skills into our teams, into the product ownerships and so on. And here you see new modern way, uh, user interfaces and workflows based on mobile application. And why are we doing that? The old style was really from expert for experts needed a long training and still off the training. It was not fun to use it. The new UX is very intuitive. It's from the real users or with the real users. And it makes fun to use it. And probably you would ask how are doing that?

00:21:29

Do you have a great power point presentation? Do you go to the product owner? No, we did the other way. We created a space, a real space to experience user centered design on this. You see here, the layout, it starts with understanding the user, the real user, not the expert, the understand the needs or define the needs design and prototype it and get the user feedback instantly. And here it's an action. It's a more than 500 square meter space. It's an open door full of Sophie. Every team member can go in, walk in and get help. And it's an easy access to professional UX support. And believe me, you skilled is different to a coding skill. And I'm really glad we had the one year in university, um, this beginning of this year and already 80 product teams used our UX life center. And they were really happy and delivering good and really great user experience in our new it products to our, um, to our business partners. These are the four cornerstones or milestones we want to share with you. Um, for us, it's still a journey it's not ended so far. It's a very successful journey, sometimes spooky, as I mentioned, because not everything is fine at the beginning. Um, but very important is what is the business telling us about, um, our Vista Phelps journey, Frank,

00:22:52

It's not that we tell about our success or not. It's about the business. We are not as mature, to be honest, as some of the companies I've seen here the last two days, where which are measuring really dev ops, let's say performance, but we have feedback from our users and what we can see where we really achieved a certain degree of our transformation. We addressed clean down with time to market, right? Even in the SAP environment, we are down from two releases a year to both, almost two releases a month. And this is very beneficial for our business units. We also improve in software quality, bringing down incidents, which is also highly appreciated by our business parts. And user satisfaction goes up as we are investing as just meant in the user experiences. But there's one thing that I really want to share with you is it's about the value about flexibility.

00:23:44

So in the automotive industry, we had this very challenging new CO2 legislation coming, uh, effective September last year w LTP. And, uh, if, if you want to hear to this and to apply to this, you really have to change systems in R and D up to the car configurator that you are using as a customer, because we want to show you the individual CO2 footprint of the car you have configurated. So, but if the legal requirements keep on changing, changing all over the time, we would not have been able to mark the deadline if you would not have our AGL working model. And this was a great success. And to a matter of fact, we were the only premium car manufacturer that had all our models, WLT P approved by the 1st of September of last year. And this is also due to our work, to our contribution to the company. So, um, but how could we achieve this Ralph?

00:24:35

So coming to the success factors, we picked out five of them. So first of all, this transformation was self-organized. We didn't hire a high paid consultant showing us fancy slides. And we tried something out afterwards and fail. No, it was self about organized from the management team, from the feature teams, a small group started, and this was really focused on a small group of the village people, the right people. And then you start defining HR, working model, find your lighthouse products and show them to the business and show them to the other feature teams. And they will really get into this transformation. We also started a small group of eight shell transformation agent, internal colleagues, helping the teams in the transformation and stick to your vision. We defined 100% agile and in the beginning in sounded strange, but it's a vision stick to it, and we saw it or we'll see it in our it strategy version two, two dot, oh, we have still our vision 100% agile in there. And finally, in the beginning, I didn't focus so much on this point. Start with biz dev ops, take already at the beginning, your business departments within the transformation, because otherwise they think the crazy it guys found out a new buzzword and they're doing agile now. No, you have to really bring them into your journey from the right, from the beginning and do this dev ops together. These are the five success factors, but it's attorney Frank and we are not at the end so far. We'll so what's up next?

00:26:23

No, not finished at all. And goodness, um, as Russ just mentioned just four weeks ago, our board approved the next it strategy to that. Oh. And we was basically laid out the rails, uh, the, the vision for, for the next three years to really support the BMW group, to become an and stay the leading, let's say premium car manufacturer and service provider. And we just want to highlight some of the challenges to you because where we seek it wise and counseling and exchange with you guys, of course, starting on the upper, right. Cloud-first yes. This is architecture strategy that we have laid out, but it's not about the new thing. It's about, about the legacy landscape. So how do we transform this? What is our strategic cloud footprint to find this out? This is a great challenge ahead of us and this, by the same time with minimum downtime, you have to imagine we have to produce cars every minute, right?

00:27:21

If the plants are broken, right, we will lose profits. So we have to do this in a very concise and in a careful way. So this is a challenge ahead of us. We mentioned back to code. I said, we are in the beginning, we have to further ramp up. Let's say the engineering skills with our organization, 50 releases per day, oh, we get so much feedback from our own people. Why is this? We don't need this. It's about really changing the way we do dev ops to the professional level. We talked here over the last two days here, and to get to this level with automation, automation in the deployment processes, automation, the test, this is the challenge. You really have to bring this further in our organization, of course, API first, too. How do you do this and legacy, right? And to do the right decoupling of the architecture API, getting APIs in there is a very crucial part for us.

00:28:18

And to do, to define this, to design this, this is a challenge ahead of us. So it's a very important journey is still ahead of us. It's will be as the old one a bit. It will be fun, but there's also will be hard times. So, but we're looking forward to it. So coming to a conclusion for you, if you want to take just five points out of this talk, um, let's, uh, some advice we want to share with you. So first of all is be bold when you start your journey, or even if you're in journey, stick to your vision. If you cannot be too ambitious to a, to name the goal, like our 100% AGL, well name it, and then stick to it.

00:28:58

Second one, it will not be a five-star wellness cruise. There will be rough water in front of you, but it's worse to start the journey,

00:29:08

Right. And do so, um, stick to the HR method in your own journey. There is no such a master plan, right? You can follow, you have to apply, inspect and adapt and tell your management, tell your stakeholders in the business. And for example, very important, tell your employees that this is an inspect and adapt a style doing transformation because otherwise they will get afraid.

00:29:31

And the fourth one learn from other attorneys and the Def up summit here is a great place to exchange to journeys. I was glad that I met the colleagues from Adidas last year in London, and we had a great exchange to get over with them about the dev ops platform and how to handle a dev ops platform. So use to opportunity to exchange with others about their journey and learn from them. And in the end always have a good motto and our motto is enjoy it. And now enjoy it and watch this, how this looks like at BMW group. It

00:31:16

Thank you very much for thank you.