Las Vegas 2019
Dawning of a New Era
AutoTrader is currently in the middle of a journey migrating all apps and services from two dedicated datacentres to the Public cloud, this talk will cover a brief synopsis of what we do, the need to move to the public cloud, and will give a 'warts n all' coverage of what worked and what didn't.
AH
Andrew Humphrey
Head of Customer Operations, AutoTrader
DW
Dave Whyte
Operations Lead, AutoTrader
Transcript
It's on honor, come on here. Basically mean Landy come from Autotrader, Autotrader, UK. There's no collection toward.com, but it's quite a similar company and we've flown all the way from not so sunny Manchester. So it's a really honor to be stood there, talking introductions. My name's Dave Weitz, I'm an operations lead or trader I've been there 15 years. I'm also a co-organizer of our dev ops Manchester meetup group. So quite proud of that.
And I'm Andy Humphrey. I work in customer operations that will say trader, uh, I also co organize, uh, DevOps Manchester, but I thought it'd be weird if we came in matching outfits. So I let Dave have the t-shirt, but, um, but yeah, we worked together a lot, worked together for maybe 10 years later. And, uh, and yeah, just really nice to be able to tell the story to you guys about how we work.
So talks dawning of new air Autotrader. Essentially. It's a migration to public cloud journey that we're on that set the scene. We've currently got two data centers, uh, physical VMware servers and the sort of setup we want to kill everything and move everything to the appropriate public cloud. We're talking more around GCP, but it's quite ambitious for us to move lock stock and barrel to public cloud, but be taken on board the benefits when they get from that.
So if you understand about autotrader.com, you probably understand about our business as well. We run the biggest automotive marketplace online in the UK. Um, it's a really, really busy platform. We connect vehicle buyers with vehicle sellers as you'd expect, and we've got tens of millions of consumers coming to our platform every every month. Um, we've got a really big influence over the Autotrader, the automotive industry in the UK, uh, because so many car dealers use our platform in so many consumers know our brand. Um, so we're really lucky from that point of view. Uh, from the point of our organization, we floated on the London stock exchange in 2015. Uh, we've doubled our valleys since then. Uh, and we're one of the top hundred companies by valuation in, uh, in the UK. So we do that with, we've got 800 people, mostly based in Manchester in the UK and, um, got over 200 developers. Uh, and then we've got a centralized infrastructure operations team of 27 people.
So errors are Autotrader. Uh, we've gone through a few areas. Our first era was what as a magazine company. So from 1977 to 2013, we were very successful magazine company. Um, in 1996, sorry, we launched our first website and very quickly became apparent that the internet was deep to think of the future. So we built out services utilizing the internet. In 2013, we published our last magazine and made a really successful transition to be a hundred percent digital. So that meant start Nat periods, a traditional website, essential database. You come to our website, you want to search a vehicle it's all done. And our infrastructure, I'm saying from 2019 to now, I believe you're more of a technology company. I think I say next, because we've gone from traditional website setup to be utilizing our data insights, to enhance and extend our platform capabilities. That's making APIs available. That's taken on board like technology from GCP. That's building really, really quite cool products for our customers that help them with selling vehicles.
So just talk to you a bit for about our applications and our services. Um, as you can imagine, one of the things that we're really most known for is connecting vehicle buyers with vehicle sellers. And Dave tried to find a nice appropriate kind of search for vehicles, uh, for an American audience. He thinks you'll drive around in six liter Dodge charges. Uh, it wasn't the case when I picked up my heart car from the, uh, from the airport. But, um, uh, so this is what we're famous for over 90% of people in the UK. Uh, no auto trader does, and this is what you think of us. Um, but actually we, we have loads more services than that. It's not just one application. Um, when we talk about the different challenges that we face, um, we've got, uh, from a consumer side, people want ever more choice.
We're really, uh, we're really well-named for selling used cars, but people want to go to one place and see a new car lease option, different finance options, or used car on the same place. So we need to offer people more choice and we're continually trying to evolve our applications to deliver that. Um, the car buying process is really stressful. There's only been one sport at Nucar recently, but it's stressful. You have to argue over the price. You have to fill out thousands of forms to get financed. It takes a long time, and we just want to make that, make that really convenient. So the next car you buy might be an online purchase. Uh, this is, uh, this is a, um, uh, a vending machine for an electric vehicle in London that will take you out of a set up. So you can walk up to there with your debit card and the keys will pop out for your new car.
So, um, so that might not be how it materializes. That's more of a PR stunt, but, but online transactions are coming and, uh, and already in the U S Amazon and other players are trying to do this as well. So from our customer's point of view, most of our customers are car dealers. And, um, and they are going through this digital transformation themselves. They are faced with more competition needing to use data more and more to drive their businesses. So we see all the information about the automotive marketplace in the UK, and we provide them with data-driven products to allow them to know what's what kind of vehicles to buy in their area. What's in demand, what price to buy things for what price to sell things for and help them run their business really efficiently. And as we, as we progress and build our platform out, we're now interacting with different kinds of customers like banks, insurance companies who need our data and our insight and our services.
And they're connecting to us through API. So this is a whole new world. And you can imagine that this means we're not just, um, we're not just a single application company. We've got hundreds of applications that are continually evolving and the challenges to move quicker and getting more and more every year. So this is where we are. Um, we still have two physical data centers, which is traditionally how we've hosted all our applications, um, uh, but more and more starting to use different cloud, um, cloud platforms for different workloads. We, we tend to use different cloud platforms depending on what, what, what we'll see our workloads most. Uh, but at the moment, we're in the middle of a all out migration of all our applications from data centers to GCP. Uh, so Google platform, we've, we've managed to migrate over 300 applications in the last year, and we're hoping in the next few months I'll be complete. So part of what we're going to talk about is our story of how we're doing that.
So the question is why public cloud now? Yeah. Pub club quite cool. There can be quite a good thing to do. You don't have to move if don't really need to. And he's mentioned that performance is pretty good. Previous slides should really go stability, but for us, it's all about, uh, increase in org ELT and increase in velocity and releases. So first step was built our private cloud platform. That's built on a cloud stack platform, uh, but very quickly it became quite hard for us to maintain that and look after the underlying infrastructure. And also it was open-source. So it's quite hard to women had issues to get the right support for that. Lastly, it was around more around GDPR. So we had a customer who wanted to be Antwon encryption for an application. And I think we spiked up a bit of work for six weeks, three months later, we couldn't build a nut.
That platform. One of our colleagues spiked up a bit work in GCP in two days, you had to work in solution for us. So for us, it was very clear, but it's moving forward to the public cloud last week. Actually we've just completely killed off a private cloud, all servers powered off. So that was quite a proud moment. Um, so around our migration, I believe it's Vista three important areas, which is really helping us with a really good migration. So the first area is around culture. Obviously, hold here a lot about culture, a second ever origin GLT, and a third area. People we're going to go through each of these and little bit more detail.
So a culture is really, really important to us. Our culture is our brand that also trader and a, and we really believe that a great organizational culture should give you highly weighted motivated teams who are really energized, delivering great value. A great organizational culture should give you a place of work where you can be yourself and do your best work. And, um, yeah, a great, a great culture can drive an awful lot of things for your organization. What we were finding maybe five, six years ago was that wasn't the case. You've seen the evolution of our company from a print organization to go through, to be a digital organization. And that was a big change for us. So some of the things that we didn't heritage from being a prince organization was we were really fragmented. We had 15 different offices all over our country, which is probably about the size of Nevada.
Um, and so it was difficult to communicate between, uh, between different teams and, uh, and all our work areas were quite siloed. Uh, we had, um, so we had, uh, we built up quite a lot of different product teams who were competing with each other. We had no real ongoing vision or mission about what that we could align behind. Um, we had, uh, quite hierarchical layers of management. So, uh, lots of senior managers would sit in their offices away from all of their teams and the communication between different layers of that hierarchy was quite, it was quite poor. So we knew we had to make a lots of changes.
So at that time, about five, six years ago, we got a new CEO. Who'd had experience running a digital, um, consultancy. And, uh, one of the first things that he did was to, um, with his leadership team agree, uh, a mission for auto trader. So this is the first time that we had an ongoing mission that was around the digital objective, make it a clear demarcation between a print organization and a digital one. This is still our mission to lead the digital future of the UK automated marketplace. Uh, and that's how we align all of our work towards that goal. Um, the other thing that we did was to try and introduce, uh, some principles about how we work some values and, uh, and these are our, our six values today, which we still talk about a lot. Uh, these, these values are not something that you just see in a presentation, but this is, uh, this is how we measure performance.
Uh, this is how we, uh, this is how we construct interview questions when we're hiring people. This is how we construct our in induction. When you're looking for a promotion, this is the kind of evidence that you have to give around how you behave and, uh, and, and what kind of, what kind of person you are. Um, so these values are really important to us just to pick out a couple of being really crucial. I think we're quite, we have a massive market share in the UK, uh, in terms of our, our industry. And, and that means that it can be easy to be complacent or arrogant. We have to continually strive to, um, to make sure that we're delivering customer value and making our services better and better. So being humble is one of those things. That's really important to me to, to remind us of that and, and maybe being courageous as well.
Uh, if we're going to lead the digital feature of anything, then, uh, it's really important. We make bold decisions, we do new things. We disrupt our industry and take it in a new direction. And that makes that it's really important that we challenge each other to do that, and don't be too safe. So this is the start of our change in terms of defining our purpose and values. One other thing we did was, uh, to talk about how we were going to work, what kinds of, um, w how, what, what work we value. And we, uh, agreed as a company at organizational level, these operating principles, uh, I don't know if you can read that right to the back, but it's a similar kind of format to your agile manifesto. We've got, um, principles on the left-hand side that override the ones on the right hand side.
That doesn't mean we don't do the things on the right hand side. It's just clearly stating we value the things on the left. So this is sort of five, six years ago. You just putting a line in the sand to say, this is how we're going to work, and just to help us make decisions and, and try and work together in a really clear way. So two things that might be really useful in this context is, uh, we've talked a lot in this conference so far about products versus projects. So product evolution has been a focus of ours for five, six years. Now. We used to have everything was done as a project, a short-lived project. We bring in lots and lots of, um, contractors from outside the organization deliver a new product in three to six months on the day of going live. All those contractors with leave and me working in operations will be left to clear up whatever happened next.
So, um, so five, six years ago, we decided that we would have long lived teams. Multi-disciplinary multi-disciplinary teams that owned applications, uh, and evolve those continually. Another one would be people development. So, as I said before, we used to hire lots of people from outside the organization, but we made a conscious decision. If, um, technology is our core competency, then we should be focusing on developing the skills and experience of our own people. So he called that out clearly to say that people development is one of our operating principles. So, uh, other things that, uh, we've we've done to try and, uh, promote a really healthy culture would be around our working environment. So, as I say, we were really fragmented in lots of different offices. We brought everyone together into two offices, a smaller office in London, and a bigger office in Manchester, in the UK and a small office in Dublin.
So that means that collaboration is a lot easier. These, these offices are built for collaboration. Um, we hot desk, even the, the legs underneath the desks and recessed slightly. So the, when you're sitting next to your colleague, or if you want to go and pair with someone, you don't bash your legs on the, on the desk. So the design, so that you can be fluid, flexible in your team structure, you can go and sit with who you need to that day to get your work done. And that's a totally, totally new thing for us. Other aspects of how we change were around reporting lines. We used to have an it department, which seems crazy for a technology company. I don't believe it departments should exist anymore. Um, and that was causing lots of friction between product owners who reported to a different executive versus a technical lead, who report into the CIO. So we don't have that anymore. Our product and technical teams were brought together into multidisciplinary teams with a mission that aligns with our overall organization goal. Uh, and that's how we work now. And it means that technical, uh, voices are heard alongside product voices. When we make any decision around our products.
Plus the first era covered is culture. Second area around all the agility. So for us org agility is a capability of a company to rapidly adapt to change. So bear in mind, we're talking about the foreshore company, moving all those stuff into public cloud. That's a massive change, which we got to rapidly adapt to, uh, GDPR, uh, issues with competitors. The competitor brings a product in line. We have to adapt to that and bring out something that's better at night. So we're constantly adapting to change, and we need to build infrastructure for that. So to do that, you'd really have to, uh, or to embrace org agility. You've got to be an enabler and not a blocker. Uh, I believe in, we definitely were B not me. Definitely. And there was massive blocker and a Paso, uh, operationally, just stuff like, um, having to build servers, physical servers or VMware servers, they take awhile. So it might be, it might be six weeks to build a service, seven weeks raising tickets, go and ticket black hole, uh, cap process. We start the cup process don't anymore. And it used to be like two or three cabs a week taken two or three hours at a time. Whereas for us now that that's not a thing. So basically embracing new processes or flexible processes, embrace new technology. We're now more enablers. So operationally we aren't called blockers.
So the story of our organizational agility is probably best told with this graph. Um, this is going back about seven or eight years, uh, the story of a number of releases that we've done to our live environment. So, uh, this is something that me and David worked on quite a lot. Lots of other people at auto trader have worked on this because to get these things right, takes people from, uh, infrastructure, product teams, all kinds of skills to, to try and work together. But where we were five, six, sorry, seven or eight years ago was that we had people doing manual deployments, logging onto servers all day everyday to try and get releases out of the door. They'd be operations staff and the development teams would have to plan the releases a month, six weeks in advance. Um, then we, as we move to being a digital company, we started to focus on release automation a bit more.
So that meant extending our continuous integration pipelines out to the live environments. And it meant that we could start to automate things and get rid of manual errors and start to increase the speed. So our releases came down from a few hours down to maybe, uh, maybe an hour each at this point, uh, and we were getting better repeatability squad. So at this point, starting to learn how to get into that cadence of regular releases. Don't save up all your work for a big release every month. Let's try and get, you know, as, as frequently as you can and your team let's try and get that value out of the door. So we've moved on through the years. We've, um, instigated a private cloud environment, which again, doubled the amount of releases that we were doing each year, because now where we were, um, using infrastructures code and, um, and releasing infrastructure changes, tested in a way just like we were through a continuous integration pipeline for, for applications.
So suddenly all of our infrastructure and applications were being promoted at the same time. And then, um, as of last year, we moved to public cloud environments and you can see that the release numbers have traveled again. So, so now we really believe we're in that continuous delivery, uh, kind of mindset where people don't save at work. Um, the, our product teams have worked to get, get really slick at releasing value early. Um, and, and our platform allows that. So the point where we this year, we think we're going to do 40,000 releases. Our biggest, um, our biggest day, this year was 455 releases. So we really feel that we've made massive progress on there. So the reason why it's important is that we think it means that any threats that come along, we can react quickly and the opportunities we can really, uh, we can really react to that quicker than anyone else. Uh, and in terms of the chaos, this might cause if you look at the graph and the rights, then, uh, you can see that year on year. The number of our failed releases is reducing as we release things and more components, smaller packages, they're easier to test, uh, quicker to get out there, but also they fail less. And even if they do fail, we can back them up well quickly.
So you can see from the left-hand graph, massive, massive increase in release velocity, which is great, but operationally it's like, oh my God, how do I know what's broken? What's caused issues and stuff like that. Historically, we used to have a outlook calendar, and as our force schedule change calendar, you had to manually put in, I'm going to do a release. The issue is, was velocity. Uh, it's is, it is hard to manage it hard to do that. Secondly, if you're trying to see three, four releases, you might a day did that release actually tying with the outlook calendar as nightmare. So we've got some quite clever people or trailer, and we do like build in some own products. This is in the house product called lighthouse effectively. It's our release dashboard factory. Now filter release that pipeline. When we start the release, it was a timestamp to lighthouse.
And at timestamp, once a release is finished. So appreciate, it's hard to see if you fall back, but to zoom in a little bit, you can see when this screenshot was taken, we complete 185 releases at that point. And you can see this blue that the consumer gateway was one of the applications. You can see the time it started and you see how long it took. So operationally, if I get, uh, monitors or, uh, monitoring telling me there's a, a problem related to consumer gateway, I can work out, ah, this was due to release because it started, everyone asked that release went in. So it gives us more information. That's a tool we built ourselves taking on board. Uh, the mentioned when we moved to Google cloud idea was to embrace technology. So we, our main digital apps are on GCP, utilizing Kubernetes and Istio service mesh.
So this dashboard is from a node that we built, taken on board, all that information, most of it from from code. So this is the first platform dashboard from that from here, we can see to my offer Docker containers, and this is live the amount of CPU, memory being used. Again, live data, uh, the amount offer applications that are in this environment. I can do the same for prod, uh, non prod and dev. It's very easy to see. So this is the initial bit from that I can dig into more, uh, screenshots and data. So the next area we've got from that is our app directory apologize. You can't see it clearly at the back. Um, but basically this is a service catalog of all our applications. So every single application that's live and being used is available to anyone to see and get the data from you.
Haven't got a search for a Wiki. You haven't got to look for other areas. Information is all in one for dev and ops, which is truly a dev ops dashboard. We built. This is not a vendor or anything, uh, to zoom in a bit more closer to in Kansas. In more detail, you can see there that the application here is AB task allocator. It's consumer experience is a squad or responsible as a product squad. You've got the owner, Andy Riley, he's a developer owner for the application. It ties in. So if he leaves the company, he left company tomorrow, he'll get took RVD. We'll get an alert, telling us that the owner for the app has left the company. Please go find new owner. So we, we do that, uh, to tier one, we've got various BCP and Dr. Strategy to tier one is like zone and region flight fail over to, I know what the, our strategy this app is.
I've got a buddy it's a buddy is a operations engineer is a squad buddy. So operationally, even though we're centralized operations team, one operations person will get a squad and they go to vest stand-ups and the meeting, this beats them, and basically liaising with them. They've got a person they can speak to with an operations and the bottom. You've got one line on what the application is on the right-hand side, there's a drop down box. And that dropdown box is basically bookmarks. So it had the URL flee up. I can veer off for an admin for the app. I've got a link straight through to the pipelines. So deployment pipelines, the source code, cabana logs, or trade or log or shared developer operation. It's all seeing the same logs. It shouldn't be in the head. And, uh, various metrics service mesh would go into a bit more detail and there's one therefor tracing.
So I got flicked straight through to that. So inbuilt within the Istio service mesh mesh, you can load a trace in Bishop Jacob's racing, so I can go really detailed to investigate any slowness in the application. It's really helpful trying to troubleshoot issues. We want to next bit. So this is around app stuff. We've had a problem for a long time where it's, how does everything fit together? So we used to have very much a big wall and someone drew on the wall, how all that fit together, which it gets outdated very quickly, even though it's quite good for troubleshooting. So we built out a dependency graph, and this is utilized, um, using, uh, usually in D three and Cuban that is network policies and Istio service mesh. This is not live, live, not close to it, but pretty much this is our live environment as an zooming in and zooming into one application particular, which has source and the API, the size represents how many things, how many apps connect to that app or that, that API, and you can see flow. So you can see if the arrow is going to an app that's, that's where it's connecting to or connecting. And the color is the squad that's responsible for those applications. If any of these were in a state of alerts, it also tells me that there's an alert there. So you get a true, um, like blast radius, if there's an issue with an application.
Excellent. Um, as part of SOSS mash, we have, uh, like dashboards. This dashboard is a pretty cool dashboard while I'm at work back at, back in the office. This is on my monitor. This is like, we used to have like millions of, of like a foreigner graph, shown everything. This is all in one place. And the platform. If I've got an app in that whole new, uh, new environment, that's got an issue. It very clearly shows up in graphs and we can then visualize and see the issue. It helps us investigate. So there's a lot of really rich data there. And also you can see on the hope for something to see and right inside, I can also see timestamps. So when releases are going for a system, it's important to know if something changes, it's easy to see it has a release occurred because is he just tracked down then what the issue might be?
The next one is around cost management. So I think it's to mess with the public cloud. One is that if you go to public cloud, you lose control of all of visibility of applications and all goes out of control. So we showed that isn't the case, but cost-wise again, cost can spiral fed horror, lots of horror stories about cost pirates, spiraling. So the idea of this is that we've got true cost management. I can tell a squad how, how cost it is for their monitoring. And if they just turned a debug off their nurse saved the company X amount of money, I can see how much, uh, so login monitoring. Uh, there's a whole lot of data we get from this. Secondly, is when we build containers, we don't build it like, right. It has to be a three CPU's and six gig of memory.
We, we change it depending on the application. So the application is underutilized. Dan, we would change that in code that'd be operations or dev can, can change that. So it quite a lot of graphs, obviously not going to be viewing grass all the time. So you need some way of telling you when issues occur. So we've got skipper, Skipper's a bot and it plugs into what basis, a web book, sorry for Prometheus's data and basically prettified previous data. And this one is example. So if I zoom in a bit, you can see at the top, this apps for forecourt court service and returning a load of 500 errors. So you see very clearly the, what that might mean. I can see clearly what containers causing the issue. This is a critical alert. So the ops engineers and the, uh, this is Paul who's, a developer is get alert in slack telling him that there's an issue and a bottom we've got a link to run.
Book is very important that any alert you get to have already cleared run runbook. So yours aim for the engineer at 3m that might get alerts to be knowing what to do to investigate the issue. I can see this owned by Paul supports, a developer who took, who took ownership of this issue. So not two or three people investigating the same issue and if needed to do it often, if needed, we can silence be alert below that there's two replies. It's the first reply is from skipper. The bot itself, it's realized through intelligence that this app has been deployed to the last half an hour. And it's thinking, you know, what I reckon is probably due to this. So it gives us a link straight through to the pipeline and straight through telling us what the last bit of code was, was pushed out. So that's quite good.
I'm the last response was from Paul Paul's put in down slack, basically he's rolling back the issue. So I think it's quite cool. A nutshell, really what trying to say is that a developer has pushed out a release. We did mention that before where developers or tried to push it, or at least ops don't push that release. Uh, within minutes and minutes, we've known the systems notice that there's an increase in 500 hours. It sends an alert ops and dev that developer has picked up that and rolled it back orphans seven minutes, which I think is quite a small timeline for them.
So it just talks about culture and organizational agility. The last of the three things we wanted to touch on is how people can enable our cloud platform migration. We're really lucky to have some great, talented, highly motivated people at let's say, trader, uh, we've worked there for a long time. And we were like, our colleagues were not always as happy as this. It's not always like that on a Monday morning, but you know, if there was a good face. Um, so, uh, but the problem was, we've got a big cloud migration, which we're deciding we need to embark on and we don't have the skills and experience to do that. So, um, so really the next bit is how we can kind of try and, um, try and deal with that. So we had some decisions, how are we going to bridge that gap? We've got great people.
We've got a desire to learn these skills in our engineering community, but no experience of doing it. And this is a big knee problem. So what we did was, um, we wanted to use our existing teams. So we went out to different conferences, different vendors, spoke to different companies to try and understand how they've dealt with this problem. And most of the feedback was quite depressing. Lots of people outsource this problem to a different company, or they hire a new engineering teams or build a new cloud platform. And then all the old teams sit there looking off the legacy kit until that dwindles away and they leave the organization. So we wanted to think about how we could do that differently. So the way that we did approach it was to have really open conversations with our teams, um, to be really clear about the reasons that we're moving to the cloud, really clear about what we thought it would mean for people's roles and to be open about the fact that we might not have all the answers.
Um, the reaction from quite a few people was a lot of concern and a lot of skeptic skepticism. Um, it's really difficult when we're planning these great big moves to trust that an organization is going to support you through that. And, and coming out of some of those meetings we had in network engineers, one of the guys that we worked with called Chris, who'd worked at auto trader, most of his life saying, have I got a role here anymore? Um, so what we did was we looked outside of the organization to hire a couple of people who had experienced of building cloud infrastructure, public cloud infrastructure, outside of Autotrade. I've done this before. And we integrated those and those people into our operations and infrastructure teams and using, um, yeah, brown bag sessions, uh, using knowledge sharing sessions, using boot camps. We started to embed that knowledge and expertise into our teams, and people could see how their roles were transitioned through this period and how they could see that their roles growing and the impact on the team has been really good, you know, rather than being negative, people can now see that they, um, they can see how the roles changing and rather than being negative about it, uh, almost the opposite.
Like they can feel part of something really special. So these are a couple of quotes that we put in here. This is Chris, our network engineer, who, um, was really worried about his career and also trader his, his feedback was I was really worried about it, but actually this move has enabled me to massively expand my skills and, and be a more effective network engineer. Another quote we've got is from Sean. He's one of our customers of TAC lead a leader of lots of engineers who needs to deliver new products and services really quickly, uh, and, uh, all the time. And he was happy ish before, but now he says he can get new products and services delivered in minutes and all the instrumentation that the, um, information he needs, the monitoring, it comes out of the box. So he knows how his applications are performing.
So in summary,
The three areas, so culture, you hear it a million times on both presentation does conference culture, culture, culture, the right culture is really, really important to key to success. We think as well, it makes he bought in to all levels, right from top down. Otherwise it's not going to work or agility go from being blockers to enablers, be bold and courageous around your quest for continuous improvement. And lastly, people, your employees, most important assets in company that empower them to be part of the journey. The last line that Jean wants to do. So we need your help with, so I think we say it's a hibernation. So this conference is awesome, really awesome, but there's other little conferences and meetups that occur all over the world. We're saying, get involved with them, support your local dev ops meetups. It's amazing might stuff you can learn from those, those meetups. And lastly is we're here telling a story. Everyone here has got a story. So we're asking you to go to these meetups and share your stories.
Thank you. Thanks.