Las Vegas 2020

Powering Better Journeys for Our Developers

Product organizations are highly demanding for their internal support teams. Their focus is always high quality deliverables and faster time to market. Internal enabling services have to provide top notch tools, products and experience to their developers so that they can code better, release faster and sleep better (monitor better)!


This talk is about how Amadeus has been helping improve the Developers (& Ops journey) in transforming the way they design, code, test, build, deploy, monitor and support day to day life at work. Every company focuses on customer delight - this talk focuses on Developers delight!

AN

Arun Narayanaswamy

Director, Engineering, Amadeus Labs

DU

Dr. Udo Seidel

Digital Evangelist and Enterprise Architect, Amadeus

Transcript

00:00:12

Good morning, good afternoon, or good evening, depending on where you are. Uh, it's, it's a pleasure for us to be here. Uh, I'm going to present the story of a medias in the transformation that we have seen into making our journey of a developer being better, right? This transformation is really important for us, and it's really helped in how our developers spend their time on a day-to-day basis in the organization and how they build the best products for our customers. So let's look at what we were in the nineties, right? And most of us, we were developers in those times and our job used to be, to know everything in the software world, know the infrastructure, know the networks, uh, be a good coder and eventually deliver the best to our customers. Right. But what really changed in this, in these years, uh, while we transformed from being a small company to a large enterprise, we basically moved into silos.

00:01:08

And I think this is not just a media, as most of us did in their own organizations. Um, we, uh, we started having developers who were, say specialists, a C plus plus developer, a front end developer, a backend developer, et cetera. We also created roles which were like siloed in terms of infrastructure specialists. We also had on the other extreme, like a front end specialist or a front-end expert, then we also created a developers or database experts, et cetera. Right. And all of these became large and large. There was nothing wrong in it. It was more about adapting or adopting to the situation on those times. Right. What's really changed again, right now we are in a mode where we expect the developer to know everything is expected to be a full stack developer of what it also means. He has to understand the networks.

00:01:55

He has to understand the infrastructure and moving to the cloud. Obviously he has access, he or she has access to do whatever he wants to do from an infrastructure angle and obviously put things into production or put things into action fairly quickly. Right? So this is sort of a journey that we wanted to cover, uh, at a very high level, and also give you some case studies on how MDs has transformed, but let's look at what I'm ideas is so that you get a feel of where or what our foundation is and where we are coming from

00:02:32

Technology is constantly transforming travel For more than 30 years, Amadeus has met that change, Always asking how will global travel look tomorrow with our open source projects, we're bringing the world's best minds together to accelerate our industry's progress. We're building new tools that turn big data into better decisions and smarter actions. We're pioneering the way with cloud-based architectures that deliver unparalleled flexibility and agility. We're creating API APIs that opened the way for new partnerships, new ideas, and new growth. We aim to develop rich platforms that deliver reliable and secure performance. And we're innovating at scale, delivering new value to the industry And developing the new technologies that will shape the future of travel. So if you want to know what travel technology will look like tomorrow, just ask Amadeus.

00:03:53

Now you saw what I made is brings out, brings it to our customers, right? So, uh, I also call upon Dr. Udo, uh, who has been with Amadeus for a long time. And we would like to hear the history of ideas through him. And then, uh, we would see how our developers are transitioning into this journey, uh, within a medius. Well, Komodo,

00:04:15

Thanks everyone. And also welcome come from my side. So assets, my name is, uh, and, uh, as Ron said, I'm with the company for quite some time, I have several roles in this presentations. First of all, I can give you some insights wherever you're coming from. Since we are not a company who started business yesterday, but also my background is more operational background. So our own represents more the developer side of DevOps. I'm more on the ops side of DevOps and, and we work together. So to compliment a little bit for Schwab. So shown in the video is okay, what is our ideal is doing? Uh, so we are leading service provider on the travel industry, and we do have houses of developers who do develop our own applications are, and you can see some, some, uh, business figures here, which gives you an idea of how big we are and how many transactions, how many bookings we are, uh, processing.

00:05:11

So we, uh, normally count our business are in presidential sport at and, and bookings, but of course the services and the software, they cannot run on air. They need to have a solid foundation, which is the platform. And we do have quite a few technical challenges they have, which comes on top, uh, to the, uh, transformations we have come through in the past. So you see that we do process a lot of data. We do store a lot of data. So we talk about 55 plus petabytes of storage to use, uh, 20 bookings per second is what we are processing and as assets. So we have quite some challenges to master on the technology side. That's why we have to have the right skilled people. We have to have the right tools, the right approaches to collaborate, to work together, to write those services. On the previous slide, it was mentioned that via are present in 190 countries with 7,000 employees.

00:06:08

And two of them are talking to you, which you see on the next slide, uh, our own quickly introduced himself. So he is a director of the dev of spectrum engineering, working out of our site in India and myself, working as an enterprise architect are in the platform unit as well, but based in Germany. So you can already see that this is a virtual conference and we are working virtual teams, uh, and thanks to tools and the dev ops journey, which we are going through. Uh, this is of, uh, like working together, sitting in the same office or sitting on this on the same floor. Next slide. So Amadeus is, uh, in the business for, for quite some years, uh, 30 plus years actually. And we have gone through several transformations technology wise, but also skillset or, uh, working together wise in, in, in the past, when I turned on my ideas more than 15 years ago, uh, and part of the company was using a green screen terminal.

00:07:07

It's connected to two mainframes, and this was the foundation we had to provide the services, but we had already started our journey going out of those legacy or traditional, uh, central approaches to disappear it, uh, open systems. And we have replaced, uh, what's inside our data center, speak time, going from those mainframes over to Rex of servers, pizza boxes, our storage arrays, replacing a tape property and things like that. Of course, along with changes on the, on the hardware side, there was also changes on the software side, new coding languages, new approaches, how to develop stuff, new paradigms to provide services, uh, to coach reviews and things like that. So you have also explored those paths. Alrighty. And I said, so we went through several transformations. Then internet came along, uh, became available for everybody, became a reliable infrastructure, even for us to transfer messages from a to B in a securely manner manner.

00:08:09

And more recently, of course, things like new ways of application packaging, uh, deploying, uh, using containers and adapting the cloud and the different service you can get from the cloud. So in a nutshell, uh, with those technology transformations of ideas, we did of course as well, uh, transformations on how we work together, how we separate and distribute eulogies, uh, our, we make, uh, a good service, uh, for our customers and also a good service for our developers ourselves, because what we do externally, we also have to do, uh, an internally and the technology journey has just started. Uh, so cloud and containers, not the end of the story, it's just the beginning. Uh, you see a lot of more things coming up, emerging technologies, uh, with lots of opportunities, lots of promises. And we want to be able to explore, to find out, to decide what is best for us internally and also externally or vice versa. And it dev ops journey is definitely the right approach to do so and has its own demands and desires and needs. So despite the fact that we have already mastered quite a few transformations, our dev ops journey is something, uh, not new, but the something with its own challenges and desires and needs and our own Vil, uh, give you more details on what they have done here in the past already, and how this has enabled us to master this, this path back to you. I wrote

00:09:38

Thanks for giving the history of Amedia and where the developers have evolved from. So I'll take it over and basically give them a view of what has really changed from a technical or a process angle. So for sure, this is something that you would have all seen and enterprise journey is complex. And once you get there, it brings in a lot of challenges, right? When, when the environments get big, when the number of people in your organizations become big, uh, there are things like change control approval kits, uh, the large environments, uh, environment shifting from pre-prod to prod or a dev to QA, to product, et cetera, right? What really enables developers to move in this journey to be fast, right? Uh, so basically one of the core competence is the quality of tools or the new age tools that we bring in. So I'll give you some case studies in terms of what tools that we use, but we also look at what the transformations have been.

00:10:35

And with more people in there are more tools, right? Uh, we also enable, uh, the platform to be available for developers to develop things fast, which means we need to provide them the best quality tools. Right. So what has been the disruptions? What has been the disruption for every developer within the organization? We've seen changes from waterfall to agile. Uh, we've seen source to InnerSource. I cover most of these topics in individual's lives, but if you notice there has, these are large scale transformations. I mean, it's, it's not something small, which a developer just adopts in a day or so. Uh, many of them goes on for months. Some of them like a legacy to a microservices model probably goes on for years in some cases, right? So these developers, um, who are developing products for the customer, has to be given with tools and services, which enable them to do fast and not really bog them down with technology challenges.

00:11:35

Right? So let's look at the first one, which is waterfall to agile. This has been going on for 15 odd years now. Uh, we've also transitioned from a agile to say, a scrum of scrums or slash safe. Uh, then, uh, the developer life cycle has radically changed, right? So what really needs to be enabled from a tooling standpoint, as we have to build in better collaboration tools, uh, automation is a month. You would probably see it on every single slide. Uh, the more the automation, the better or more agile we are, right. And flexibility and transparency is something that's really important. And the platforms slash tools that we provide, I, I sort of mentioned the platform and tools, uh, quite often already, but what it really means is a platform which our developer can hop on to start developing onto it. It's more like sort of a cloud service or a tooling platform, which people can consume.

00:12:33

They have their own systems to build on and have flexibility to make changes to it. And, but governed by something central. So which enables them to move fast, but also give them the best of the tools available within the industry for them to go faster. Right? So what this all means is to be provided with a set of integrated tools, which work seamlessly between each other. So there are cloud services, uh, let's say, as your DevOps, we have Google cloud tools, et cetera, but how are they integrated with each other? Right. So the platform provides them these, uh, in a way where they can develop wherein they don't have to bother about what's sitting behind the scenes, this sort of foster Foster's DevOps, because they transition between one and we're on to the other, or one system to the other, or one person to the other in a more seamless fashion what's happening on the DevOps to the no ops cycle, right?

00:13:26

So we've almost transitioned from ops to a DevOps model, a no ops or chat ops, kind of a thing is something that's we are transitioning to. And what it really means behind the scenes for a developer is he has to focus on automation. He has to focus on cloud first because the sort of no ops server, less auto healing, all those kinds of things, which are provided by cloud, uh, both public and private is something that we are providing to our developers. Microservices transition is also happening during this journey, and which is very important because the more, uh, connected tools or connected products we have, but even more disconnected from a point of view, whether how they interact with each other, or do they have contract based, uh, approaches, et cetera, very important when we have to go to a new ops model, uh, because then we can build an auto healing, recovery chat, ops models over and above that.

00:14:21

Right. But what happens on a tooling ecosystem around it is the process of building federated tools, right? What federated tools mean is very similar to a public cloud provider model, where there is a central set of tools or systems, uh, which has governed, has certain set of practices and policies or security, uh, aspects put in place. But then we give control to a developer where he can make changes, uh, give administrative access, remote access for people and esteem make changes, very plugins, remote plugins, pull out a tool itself, plugin a tool that he prefers, et cetera, right? This enables them to be innovative. This enables them to move fast. And this also enables them to have a feeling that they have control on whatever aspect of development they need to be. It's not there's the core part, but also the tools that are provided for it.

00:15:14

Moving on to our open source, to an inner source model. I mean, this is not a transition, but this is sort of a scale up or an increase in maturity. I would put it as right. So most of our products are built on open source languages, but what is the next phase of it? We have, we have to enable people to contribute within each of our own products, right? Uh, uh, the, the move away from silos is certainly enabled by this. So, uh, we do multiple things here. We enable, uh, tools which are open. Uh, we enable access, uh, to all tools and products, uh, which is globally accessed by every developer across the globe. Uh, the code could be reused. So because all, most of our get hub or rather the good repositories are public, most of our JIRA ticketing systems are public. Most of our confluence pages are public.

00:16:07

So when I say public it's public within the organization where people can see what others are developing and contribute accordingly, right? So the core component on this is providing tools or platform, which are open in nature, uh, which enables our developer to move fast and improve that experience on a day-to-day basis. The bigger thing, uh, which is with all, uh, is the move from a data center to cloud or to server less, right? Um, the data center and cloud, I would still say more closer to each other than on a server, less, uh, kind of an application here. We are looking at transforming our developer to become full stack developers. We enable trainings. Uh, we enable them to understand other aspects of working together. So if we first moved them into a virtual kind of a team, which, where they understand operations and development well, and then eventually transition them to a full stack developer where they focus on automation, both on the infrastructure side, the cloud side, and eventually the application side, uh, what it means is we have to bring in new age tools.

00:17:10

Uh, the, the thing that I mentioned in the past is we have been a software, um, software first company. What it really means is any tools that we built were initially built in-house, I mean, beat our ITL system, uh, incident response system, everything were homegrown and tools that were built for buyers and for our developers. So what we have transitioned is transitioned to new age tools, which is commercial in nature. Uh, we try to customize it less. We use the full functionality and adopt our processes and policies to work with the tools which are available publicly to other customers. But I would also give you another angle to this, which I sort of, uh, mentioned it at the end, but bringing a new age tools, helping large scale adoption within the organization, we have 16,000 plus developers. So we need to ensure most of these guys are single ways of working, but flexible ways of working.

00:18:10

The other aspect of it is the tooling team slide, providing the tools as a platform, right? So because our products are scalable, our products are reliable. What it really means the developer needs to get the same kind of an experience when he or she is consuming tools and products from the tooling ecosystem. Right. When I mentioned about tooling ecosystem, these are, uh, DevOps tools, right? So, uh, the bit here is very important is the tooling ecosystem or the platform or platform as a service that we provide has to be scalable and reliable. So these are connected tools where people, um, our developers work with each other, they contribute. Uh, and this is a simple example. Let's say, if I have to upgrade the bit bucket and spins from version extroversion, why I don't expect the developer to wait silently and wait for this upgrade to happen, our platform has to be, um, resilient.

00:19:06

It has to be sort of a zero downtime. Like our products are, uh, I mean, that's, that's a bit that I would want to pick up from the history is most of our products that we build have been zero downtime for our customers, because any person who gets onto a plane gets on to use our platform, uh, the, uh, be the airlines, be it, uh, the airports can't really see a downtime because the world keeps moving. Um, and they need to see products which are really up most of the time and the same we need to provide for our developers. So looking at some of the case studies, which is important, I'm basically going to tell about what tools we have in place and how these are enabling these aspects of being open, being connected, uh, being integrated, right? Let's look at JIRA. I mean, this is not a new evolution.

00:19:55

This is something that we transition a few years ago. Uh, the JIRA ecosystem is helping us adopt agile in a better way. Uh, it's helping us transition from homegrown ITI systems into JIRA based systems. And obviously we create thousands and thousands of tickets. We have one of the largest JIRA instances within the organization, and it helps build integrations and connectivity. I mean, if you look at our pull request model, how do we connect our tickets into, um, the code that we are building? How do we automatically close tickets, et cetera, are all enabled through JIRA, similarly on a low ops or a no ops model, uh, or the things that we are building from bots within teams and within the prior, uh, sort of collaboration tools. We have 50,000 plus jobs on our Jenkins instance, we do 25,000 belts. Uh, most of it, I mean, I would say 95 to 96% of them are a hundred percent automated.

00:20:52

We still have about four to 5%, which has to stay manual because of some processes we have in place, but we are moving faster and faster to a hundred percent automation on this, right? But notice, this is not small scale. Uh, all this needs to be enabled. We have to move fast on it. Uh, collaboration on all these tools are very important. A lot of incidents or, uh, things that happen on Jenkins is automatically published to teams and tools, uh, and teams within teams, uh, so that they can react fast. Uh, bots is something that we have evolved. We do tons of hackathons within the organization, which enables developers to build bots and move faster and provide things to our customers faster as well. Bitbucket is, uh, is one of the core competence for our platform, which we have, uh, we have put in federated systems around it where people can give access, grant access, revoke access within their own systems.

00:21:49

But one of the core functions that it enables is being InnerSource, uh, or enabling people to be or product to be in their source. Uh, we do about 2000 pull requests a day. We have 7,000 developers on it. And, uh, what it enables is, um, people contributing to my product or one product from need not be within the same team, but coming or originating from some other teams, right. Uh, it's still not massive. I would put it as 10% of it comes from external 90% comes from within, but it's still significant when we look at it from a point of view of how much we, uh, get from other sources within the organization, collaboration, most important aspect. Uh, we have about 12,000 users. We generate about 8,000 pages a day, which basically means most of our communications happen through confluence and everything is public. I did mention this to you earlier.

00:22:44

Uh, people can access this it's we remove dependency on a particular team or a developer. We enable it to be public within the organization so that everybody can learn from what others have done and other mistakes that other have performed properly. So one core aspect that I want to touch upon is that we are unique, right? And every company, even you, your organization is unique. Uh, we adopt majorly, uh, tools and products from the market, but we also build tools which cater to our own systems, right. Um, there is, there is always that thin line saying for us to be ahead of the curve ahead of our customers, do we provide the experience to our developers so that we don't restrict to what the product alone has, but also build in some kind of tools and integrations, which help them move faster. Right? This is very important.

00:23:35

The only thing that we put about that is a centralized governance and processes, which is sort of a principle, not control to say how you can be better, how you learn from others and build, uh, build the products which you, uh, with the customers love. So last but not the least to wind it up, I would say we are providing new ways tools to become more DevOpsy, uh, we enable developers to be happy chatty. So we contribute through tools, uh, collaboration, tools like teams, uh, Yammer. We have, uh, products like, uh, Bitbucket, which enables people to contribute or chat geeky Lee in some ways, and also have confluence where, uh, where they can share, uh, for long running documentation. Uh, all this is very important and most important aspect is being transparent, keeping the whole environment, uh, safe from a point of view where they can make mistakes, learn from it, evolve, uh, which the core aspect of it is tools can be plugged out and plugged in into the platform, which enables them to adopt faster, but not just that.

00:24:44

We also innovative in a way where they, if they feel they have a brilliant tool coming into the market and they want to plug it into the platform, they're enabled to do it. So all this helps them, uh, in this whole journey and to keep the whole environment fun. So thank you so much. As I say, the future is now, it's not something that we looking at the future. Uh, we are all already built in tools and practices, which enables this currently for our developers. But again, the core mantra of DevOps is continuous evolution. We continue to evolve and continue to improve. Thank you so much. It's been a pleasure to present to you. Uh, thank you, Odo as well, who contributed to this, uh, from the history and from, from the learnings that you have at Amadeus. Thank you so much.

00:25:28

Thank you.