The DevOps Transformation Toolbox -10 Tools that Drive RBI's Transformation to Modern SW Engineering (Europe 2022)

Transformations are complex processes. Moving an IT organization to a modern way of working is no exception. RBI understood the complexity of this journey and applied a trusted and tried set of tools along the way. This talk explains the 10 most important drivers that helped RBI to shape the transformation to product teams applying DevOps methods and practices and summarizes the current level of progress that has been made. The selected set of tools has shown to be appropriate to help transforming the IT organization and considerable progress has been achieved.

breakout2022europe
MF

Matthias Fleischmann

Group Product Leader of Fraud Transaction Monitoring, RBI

RR

Robert Ruzitschka

DevOps Guild Lead, RBI

TRANSCRIPT

00:00:14

Hello everybody. We are OT and Mattias. Today. We want to talk about how our company RBI has been approaching the journey to modern software engineering. This is obviously not a straight path, but a might face and complex process. And we want to show you a pretty high level overview about, uh, the tools that we used to guide us on this journey. We call this our DevOps transformation toolbox.

00:00:46

We are not able to talk in great details about all of the tools given the time available, but try to give a bit more context about the ones that are a bit specific to our company. This is also not a talk about the glorious success, but the glimpse into a journey that started a while ago. It is, it is also not our journey, but one of many people and teams excited about getting better in what they do. But let me shortly explain where we are coming from. J bank international is a medium sized banking group headquartered in Vienna, Austria. It is subsidiaries in is operating in 14 countries in central and Eastern Europe. It has, it has around 19 million customers and approximately 46,000 employees across the group RBI as part of the S and family that was found based on cooperative concepts has always been a rather traditional organization.

00:01:50

Considering the pace of change in business. It was clear that also RBI is required to consider adopting way new ways of working and operating. A couple of years ago, a substantial transformation process was started. That was kicked off with the definition of a vision and mission for the year 2025. I won't read it out, but the idea is obvious enable the organization to deliver continuous innovation to drive customer value. This is a very high level statement, of course, but our specific approach to fulfill the mission is moving to agility on all different levels. What we wanna make sure is that we deliver the right stuff quickly and it's equally important, deliver it in a way that is good for our people. At the end. What we want to achieve is a continuous flow of value to our customers.

00:02:51

Getting there requires a lot of changes on all levels. Organizational changes, changes in collaboration and budgeting. Of course, it also requires changes in software engineering. RBI always had a quite strong it, but clearly some things needed to change. Let's talk about how we started our journey to new way of working for our respective teams, right from the start. It was clear that we needed and still need to catch up in some areas regarding our engineering capabilities. There is a lot of knowledge out there how to improve from an engineering perspective. And we took this knowledge and tried to apply it in an RBI specific version. This is what the DevOps toolbox is all about. We took the proven tools out there and applied them in our specific context. But first of course, first of course, we needed to have a clear target picture.

00:03:54

Having this clear idea of a target of our journey is the first tool that we pulled out of our transformation toolbox. Our target operational model is autonomous product teams using DevOps method and practices. Just to be clear what that means. We are talking about teams owning the complete software delivery life cycle from requirements, implementation, testing, to operation and maintenance. We are not applying this approach to each and every team though in the bank, but for our most important software products, products that define our position in the market, our target operating model is autonomous teams working in DevOps mode. We have defined that target operating model as well in our it strategy.

00:04:49

What we have done together with management is to define objectives and respective key results. You know, the pattern, the approach is called OKRs. And we started to apply this to almost all areas across our bank. So OKRs are the second tool that we pulled out of our toolbox. They help us to focus and align our efforts and priorities across the bank. I won't go into details as this is quite popular, and you can get a lot of information about O OKRs by doing a quick search on Google. But I just want to mention here very briefly that we used this approach to create the common understanding of our organizational goals. And it's also bold statement about priorities together with objectives comes to definition of the respective key results. And we see this as crucial to achieve the objectives. It is super important to understand that how the key results are achieved is up to the teams. They need to find their own way to contribute to the overall key results. The teams are the ones that know their context and problems based best micromanagement is horrible. And we really try hard to avoid it

00:06:05

Discussing thers with the teams we experienced that many of them really needed. Some kind of inspiration were to start and here, uh, tool number three comes handy. It is our agile engineering maturity review. This review is basically a questionnaire that is filled out by the team in a joint session and gives the team some framework to figure out whether it is room for improvement in the areas of C I C D DevOps testing and test automation. So how does it work? It's basically just an Excel sheet with many questions and the team puts itself for each of the questions into a maturity pocket. You see it here on the slides, like pretty simple crawl, walk around these other levers. There's some, some guidance to explain what the questions are about, but the evaluation is strictly done by the team members themselves. I already mentioned that we are convinced the teams know best how to improve.

00:07:06

So it's up to them to evaluate their strengths and weaknesses. This review process gives us a baseline. It helps the team to create common understanding, and it also helps them to figure out where to start any improvement activities. These reviews are done by the teams together with engineering coaches that guide them through the questionnaire. We have a team of these coaches here at RBI. We call them agile engineering coaches. Their role is to coach the teams on C I C D practices and tools, test strategy and test automation. I'm actually one of these coaches. Um, and our role is to be with the teams in their daily work and enable them to solve their problems. So we as a team really see ourselves as enablers. This means that we don't do the work for the teams, but we help them with our knowledge and experience to solve the problems on their own.

00:08:06

That's actually pretty different from the old concept of having a pool of specialists that join the teams, solve a problem, and then move on to the next team. In other words, this means there is a silo of highly specialized knowledge. Something that we just want to avoid is it only creates dependencies. Our is our goal is to help the teams solve the problems by themselves. So practically this means that we stay with the teams only for a limited amount of time. Ideally, this is like around three to six months, and of course there need to be sufficient people in the team to handle the load, even when the coaches leave again. So our team of engineering coaches that enabled the product teams in all things related to engineering is our tool. Number four, I mentioned that coaches work with the teams on C I C D and testing and test automation and the appropriate tooling, autonomous teams have to handle a lot of topics. So it's important to reduce the cognitive loads they need to handle by taking care about the basic stuff. Yeah, this means that we offer teams central C I C D platforms, and also testing tooling.

00:09:25

This state of the art centralized developer platform is our tool. Number five, Just be also very clear here. Teams are not obliged to use the central development platform, but our experience is that the benefits are obvious and they're hardly any downsides. So we see great acceptance by the teams. We do mandate putting the code into our central version control system though. Yeah. Code for us is like a means of sharing knowledge. So this is mandatory

00:09:58

Now to be very specific. What we have done is we rolled out the Kitab enterprise platform to the RBI RBI group. This gives teams access to one of the most popular codes in the market and also access to a powerful, continuous integration solution. As a service. In addition, they will get the central developer portal that is based on backstage. You may have heard about it and a managed artifact repo. So at the end teams don't need to worry about repos that the codes, scanner security, tooling, pipeline tooling. It is just there for them. And working This centralized developer platform allows us to offer great tool chain. But in addition, the shared and openly accessible code repo is also used as the base for our inner source activities. This brings us to tool number six and tool number six is our inner source initiative. As you probably know, InnerSource is based on the same ideas as open source, but just inside of an enterprise organization, I guess you've realized that RBI is a fairly big company. And if we consider all it people across the group, I guess these are probably a couple of thousands. There is a lot of knowledge and experience available.

00:11:23

This knowledge manifests itself also in software code. So what we want to do is to spread this knowledge via a code that is accessible to everybody.

00:11:34

And as you also can imagine, I guess there must be some kind of lightweight set of rules to govern this inner source process, but we really try to make sure that the preconditions for this kind of collaboration are as few as possible sharing knowledge and experience by our inner source initiative is tool number six. And this immediately guides us to tool number seven, our communities sharing knowledge and experience not only in code, but also in the community is a powerful way to learn and improve as an organization, but it also needs to be structured. This is why we have built up more formal and more informal communities around specific topics. So, um, approach here is that we have on the one side established guilt as the more formal approach to a community. And on the other hand, communities of practice more informal guilt have a dedicated Guild lead who facilitates discussions and knowledge sharing sessions and Guild also require contributions by their members. These communities also provide guidance on good practices and help in shaping processes. Something that as you can imagine is quite important for a company working in the financial sector that is heavily regulated here in RBI. We have guilds for DevOps and for testing and test automation and more informal communities of practice for many other topics.

00:13:03

But sometimes it is important to discuss topics that go beyond the scope of a specific community and share accomplishments with a vital audience. And for this purpose, we've established tool, number eight, our internal engineering conferences. We call them major engineering days and they usually take place twice or three times a year. They are the perfect format to showcase interesting accomplishments for teams and allow us also to bring in the perspective of external experts. The conferences are typically planned around a main topic like DevOps testing or cloud as the major theme, but we are quite flexible with contributions and also of course, try to cover cultural aspects. There are also a great opportunity for top level, top level management to engage with the engineering community.

00:13:58

Agile engineering days have been hugely pop. We had to do them online due to the pandemic during the last years. Um, just as an example, the last conference in February attracted around 300 participants. So this is really quite a huge number. Uh, the online format helped us tremendously to increase the reach, but what we are missing is the social interaction. Yeah. So, but that's, uh, something that, uh, all of the conferences have been experiencing. And, um, I'm pretty sure that also some of you are, you know, really missing the social interaction that is provided by these kind of gatherings

00:14:40

Coming back to sharing knowledge, which is important, but in order to be able to share knowledge, people also need to have knowledge working on individual skills is therefore also an important part of staying on top of our game tool. Number nine is to support. We give people for individual learning here. We have basically separated into two major initiatives. The first one on the one hand side is our internal it academy where we offer a more classic approach. Engineers can choose from comprehensive multi-modal programs that help them building up the foundations of their respective skillset programs that we offer here cover a lot of areas from channel introduction, into digital engineering practices, up to software architecture from test automation to public cloud infrastructure. The second initiative are our highly individualized learning journeys that should help people to get very specific knowledge, which is needed right now in their respective teams.

00:15:50

These learning journeys are self-paced. This means people decide on their own how fast they want to move on. And the goal is that every piece of knowledge that is collected there on their respective journey can be immediately applied in the daily work. These learning initiatives are very popular and there is a lot more that could be said about this approach. Um, but I guess we could dedicate the separate talk to this topic. Now we told you a lot about what we do and how we try to improve, but it's clear that we also need to know if all our initiatives are really moving us in the right direction as we engineers, it is clear that we need to measure sound simple, but it isn't. Of course, we know that key performance indicators are problematic. In many cases, they measure only the output instead of the outcome. So what we did is we settled for a combined approach between qualitative and quantitative measures. The qualitative part is done via our engineering maturity review that are already mentioned earlier as tool number three teams do this self assessment at least once a year, better every six months results show, if progress was made in the areas identified as the most important,

00:17:14

What you can see on this slide is a as an, is an example of an obviously very motivated and successful team. It's a real world result just in case, uh, there are an adults. So what you see here is that the team, uh, in the review from 2020 had a lot of fields that were yellow or red. Yeah. And if you look at the results from 2021, a lot of these fields have moved to green. So what we see is progress basically in all dimensions. So very motivated team, very good results. We're very happy about this, but besides the qualitative approach, we also would like to have some hard numbers. And, and for this, we have decided defined a set of, uh, key performance indicators that we call the engineering KPIs. And here we have tool number 10. Again, we didn't need to come up with our own KPIs because there is a set of well understood metrics that can be considered as a proxy for good engineering, uh, good overall engineering maturity. And these are the dome.

00:18:22

You may have heard of them as part of the state of DevOps report. Um, I guess most of you know, this it's really widely known and popular, if not, and I just can recommend to check it out. It's really mass reading for anybody in a trade. Um, and besides these KPIs, there's good research available that exactly these key metrics give a very reliable picture about the overall engineering capabilities of teams. These metrics are also related to good business outcomes. Yeah. So what we do is we collect the KPIs on a quarterly base and they allow us to see if we are improving also based on hard numbers with the collection of these numbers, we have closed the feedback loop and can assure that we as an organization and the individual teams are moving into the right direction, knowledge to a quick recap about our tools. We started our journey with tool. Number one, the definition of a clear target picture.

00:19:27

This guided us during the introduction of OKRs tool. Number three, the team engineering self assessment gives teams inspiration and a baseline that they use to identify improvement areas. Our engineering coaches enable teams on their improvement journey. Cognitive load for product teams is reduced by tool number five, our central developer platform that also is used as the base for our inner source initiative, knowledge and experience sharing is driven by tools seven and eight or engineering communities in the internal conferences support for individual learning assures that the skills required for our transformation can be built up by all. And finally, tool number 10, the engineering KPIs, close the feedback loop and provide the quantity feedback that we need

00:20:23

After you've heard all of this. Now, I guess one of the most important questions that you have in your mind is right now, did it work? Have you made progress? Our answer to this would be a sound. Yes. Are we already where we wanna be? Definitely not, but we have achieved a lot. Let me just share a couple of achievements that we made as an it organization. We have created a lot of awareness about good engineering practices in all areas of our it organization. The know-how about C I C D DevOps and testing has improved, improved tremendously. From a technical perspective, we have moved dozens of all the products to pipeline tooling and enabled them to be built and deployed automatically. The number of deployments for each Dell teams has nearly doubled in the last two years. And consequently lead times have gone down significantly. We have increased our footprint in the public cloud from basically zero to thousands of products. We have established lively communities across the group, And hundreds of people have taken part in our trainings and collected knowledge that will help their teams. We think it is safe to say that we are clearly on the right path. And the reason for this is that we as an overall organization have understood that transformation processes are complex and must be taken from many, many different angles.

00:22:05

We have very powerful tools in our toolbox that helped us to come to the place where we are now, but we are also convinced that it will help us to continue to improve and to become better. Over time,

00:22:21

We are aware that our tools must be maintained on the way as it is required for all tools. And we also need to replace one or the other, but we definitely have built a solid foundation to continue our journey to fulfill our mission. So I hope we were able to give you a good overview about our approach and we would be really happy if some of the things that we do may also be useful for you. So we are at the end. The only thing that is left is to say, thanks for listening and for having us, if you have any questions, just get in touch. You also see, uh, our, uh, the, the links to our LinkedIn profiles, um, and to, uh, Twitter profiles. So, um, have a good day and, uh, deploy confidently.