In Pursuit of Transformation: How Paysafe is Supercharging Their Software Development (Europe 2021)

Low efficiency in scaling development environments, manual processes and the need for better visibility were just some of the challenges that Paysafe faced with their software development practices. Paysafe is a pioneer in the digital commerce space and to continue to maintain leadership and competitive advantage, Paysafe needed to transform its software development to achieve faster time to market, improve operational efficiencies, while reducing risk. In this talk, Sujit Unni - Chief Technology Officer of Paysafe, walks us through the process of achieving this transformation - challenges to be addressed, goals to be achieved and the process of selecting the right DevOps platform to supercharge this transformation. This session is presented by GitLab.

londoneuropebreakout2021
SU

Sujit Unni

Chief Technology Officer, Paysafe

TRANSCRIPT

00:00:14

Hi everyone. Thank you for checking out the video. My name is . I am the chief technology officer here at Paysafe and over the next 30 minutes, I'd like to talk you through basic as an organization, um, how we are pivoting our business, um, the critical role that ops and tooling plays in our transformation and our joys of get lab as a partner in our tooling journey. Um, I'll start with an overview of Paysafe. Uh, basic is a payments platform. We, in a sense provide both payments services and payment acceptance services to both the B2B and the B2C side of the segment. Right? Our, uh, B2B ecosystem, um, has about close to 400,000 merchants largely in the us, Europe and Canada. Uh, and we process about close to a little, a hundred billion dollars in transactions annually, um, within the B2B part of our ecosystem on the B2C side of the ecosystem.

00:01:14

Um, we've got a digital wallets offering under the brands of Skrill and Neteller, um, and we've got a gastro digital ecosystem, um, which we kind of run under the brand of Paysafe cash, uh, between the wallets and the pig pay safe cash ecosystem. Um, we bought close to what 15 million active customers, um, in terms of our geographical footprint. Um, like I said, you know, a B2B side of it is largely in the us Europe and Canada. Um, but our B2C ecosystem actually supports close to 120, um, countries all over the world, um, which is where these 15 million odd customers come from. Um, so all in all jump, totally global business, a wide geographical spread. Um, and as a result, uh, as the saying goes, it's five o'clock somewhere, um, for Paysafe at any given point, right? So we are by definition, um, required to be a very highly available ecosystem and just given the nature of our business out of any data intensive, um, ecosystem, um, as we start to look at the business going forward, um, what I'd like to do is give you a view of what powers our business today, um, just so that we kind of ground this conversation and, um, in some kind of a baseline.

00:02:32

Um, so Abe at the core of our business is our deaf community, right? So we are people from our close 2,200 developers, uh, globally spread, uh, very large hubs in north America, Sofia, Bulgaria, India, and we are in Austria, right? And it's these thousand 200 developers that largely provide, uh, most of the, uh, functional features that we roll out across our ecosystem. Um, in terms of majority, we, as an organization are relatively kind of mature to the extent that we are in a dialogue and have been so for several years, right. Um, um, very heavily focused towards, um, you know, a metric or metrics driven, um, measurement, um, ecosystems, so that we are constantly improving. Um, capabilities have been fairly mature for awhile. There are parts of our ecosystem where we do close to about eight yard, new feature releases every month. Right. Um, and we, we kind of did a lot of the early kind of growing in terms of moving our testing left and stuff like that.

00:03:39

Um, a couple of years ago, right. Um, now this theme of 1,200 developers largely works on a set of core platforms about five of them, um, which we largely leveraged to provide cross-business capabilities for the, um, for the three businesses that we work in. Right. Um, they much a product mindset. So you're kind of building out product features in the construct of three business units that I stopped about, right. What features you cash features or payment acceptance features, which we then roll out. Right. Um, and security of course, is at the core of our business, right? We are at the end of the day, a payments platform. Um, and we are only as good as a security is. Um, so we we've had a fairly mature ecosystem around managing SAS desks CA kind of, uh, scenarios and, um, for our business, um, as we start to look at the next few years, right.

00:04:35

Um, our desire is likely to start to continue to mature this platform primarily around business aspects. The first is increasingly we want to kind of move the organization from a product mindset to a platform mindset, um, largely with the intent of kind of focusing on customer outcomes and not specific products, right? So a classic example, he'll be, um, if I was a customer, right. Uh, and I want to pay a merchant rather than kind of give them an offering that says, Hey, pay through your wallet, or kind of say, you know, use the eCommerce product to pay for it, or use your credit card. What we want to be able to do is give the customer the optionality that he can choose whatever he wants, um, based on the channel that he is in, in the moment that he's at, right? So if he's in the store, you know, you can choose to use a card or he could choose to use his wallet.

00:05:25

Um, but our intent is very much to provide that omni-channel experience over the next several years. The other big kind of opportunity that we see within our business is that as you start to build this integrated platform out, um, we think that bringing the data and the risk aspects of our business together actually allows us to start to kind of recognize alternate revenue streams outside of the more traditional revenue streams that we have within our businesses today. Um, and then of course underpinning this, uh, business outcome very much is our technology, right? And we've got looked at our technology organization and said for us to be successful, there are three pillars, right? Um, the first pillar of course, is, uh, making sure that we have the right architecture, right. Um, which is pretty much the classic story that most organizations are going through, where we are taking heavily siloed, monolithic, um, technology assets, and then starting to decompose them into microservices that are more modular and can be offered out independently.

00:06:25

Right. The second aspect of, uh, our transformation journey is very much the automation and tooling, right? To the extent that we can actually build out the right pipelines and the right degree of automation. Um, we definitely are more rigid in the market. Um, and the last pillar and probably the most critical pillar, um, is a bunch of right at the end of the day, culture makes or breaks an organization. And we actually see the, the technology and the tooling, the tooling also how to be very critical to defining that culture of the organization. Right. So the most classic example that I can give you, um, is that the risk appetite of an organization that has the right kind of tooling, where when a change is made and put into production, um, should something go wrong, um, you know, correcting, it takes a little under two minutes to do.

00:07:17

It's very different from the risk appetite of an organization where the technology, as well as the tooling is at a point where the change we made and rolled out and shouldn't were wrong. Um, you could potentially put the business out for several hours or, or even worse several days, right. But he, obviously the risk tends to be very different than these two, um, organizations. And we believe that getting the right tooling will very much enable us to have the right degree of kind of appetite for both, uh, taking risks as well as dealing with change. Right. And then of course, uh, um, security is never too far behind in any of these conversations. Um, I out approach to, um, making a business and resilient as well as, uh, secure, um, has largely been based around two kind of pillars, right? The first element of it is that we very much want to get to a place where we have a mighty cloud strategy, right?

00:08:12

So really as much as we got parts of our businesses in, you know, within two kinds of, uh, public cloud probe providers, um, the long-term intent is to very much have a mighty cloud strategy just to make sure that you're also protecting ourselves against the exposure from, you know, with a particular provider. Um, and the second element of it is very much to graduate our security offering, right? So pretty high security of offering is still very much, um, uh, end of the cycle. So to say, right, but the, the desire very much is that we would like to get to a place where, um, we think of security more from a preventative measure. I identify protect early in the cycle as opposed to kind of, you know, detecting and resolving later in the cycle, which kind of makes you more reactionary. Um, so the thought process very much as a doctor's testing is that over time we want to take all the security aspects of our business and continue to shift left and who them as close to the design and build aspects as possible.

00:09:13

And the hunt very much was to me for such a tool. So as we started to look at our kind of tooling, um, we kind of looked at it largely for us, a couple of areas, right? One, like I said, is that we wanted to make sure that we bought a tool that supports our multicloud strategy. We wanted a tool set that we knew could kind of also support our long-term aspirations in terms of where we want to take our, uh, uh, security capabilities. And then of course, um, to my point earlier, um, very heavy focus on making sure that we choose a tool that drives the right kind of automation that enables a French culture that we aspire for. Um, now given that, um, I, I would say, you know, um, a long list of items and, uh, the tall order, um, we started our journey largely with the position where, okay, what is it that we are struggling with today?

00:10:10

Um, and again, I think this would kind of resonate with, um, pretty much all of you. Um, I think most organizations have seen no, um, some of these limitations or opportunities no different than we have. Right. So, um, as with most organizations, I think the lack of standardization within our very complex landscape, um, was a cause of concern, right? I mean, I really wasn't ineffective. Um, I just think it wasn't as effective as right. So we, we do have tooling and we do have the kind of automation that allows us to be reasonably a giant, but for us to kind of really break ground and fit that Jacob, um, we think it required a fair degree of uplift, right? So I think standardization was one area where, when we, when we did our in wintry, we had about close to almost 2000 pipelines with varying flavors of majority.

00:11:09

Um, so there was very much a need to say, look, if we do, if you want to run an integrated platform, we've got to standardize the tooling, which means that we've got to take these 2000 odd, um, flavors of, uh, pipelines that we have, um, and whittled them down to a code set that would actually allow us to scale, improve the fungibility and the cross-functional, um, assignment of our resources. And to that extent, that became a pretty critical kind of, uh, that we went with. Right. Um, the second element of it, that, again, that was very, very, uh, Pete was just given the fact that we are a very metrics driven organization, right? So we, we were pretty ruthless. We don't measure and we constantly improve. Um, we wanted a tool set that were actually give us the kind of insights that would allow us to measure our productivity, our effect effectiveness.

00:12:00

And then of course allow us to mature over time. Um, again, we had some of that, but a lot of it was massively manual, right? A lot of work going around. Um, there wasn't, there was a little cottage industry that we want in place that naturally takes all these metrics from these various tool sets that we have puts it together and then starts to draw out insight. Um, I didn't tend to us to actually make that as into, on the platform, to the extent as possible that teams can start to consume it on their own and then start to improve, um, almost create that self generated, um, desire to improve. So that kind of was another key pillar. Um, the third pillar, which again, kind of, uh, was pretty critical to us was just the ability to be able to automate our auditing and compliance, uh, uh, ask for it.

00:12:51

So like, as it, most organizations we bought developed software development policies, the word compliance asks that we need to adhere to. Um, and for most parts, a lot of that control and check ecosystem is either manual or happens through, you know, after the fact kind of sweep or audits and stuff like that. The intent again, was to kind of almost embed a lot of these policy ideas and compliance asks into our tooling to the extent that if teams find out early they're using it, and you're almost kind of running the entire ecosystem more through a, uh, a lens of preventing, uh, um, preventing variances, more so than, you know, uh, the current culture of having to go in and do an audit, find the outliers, found, find do the findings come back and do remediation work. So that was a pretty key element of Audra.

00:13:47

I decided to kind of find the right tool set. And then of course, uh, like I said earlier, right over time, the intent very much was to start to shift left. A lot of our security asks. So those are the broad asks again. I, I think a lot of these are very prominent in a lot of organizations. So, um, I, I don't think we were unique to that extent. Right. Um, given that ask, how did we go about it? Right. Um, so we actually got off started at the top of the house where we were very clear in terms of what are the outcomes that we wanted our target to set to drive. Right. Um, kind of combination of both. Um, I would say, you know, the development team insights and asks as well as, uh, the, and some, um, some industry views from non kind of repositories, like black nerd and Forester to actually put together a scoring scoring template.

00:14:41

We identified five kind of potential choices in the market. And then of course ran a pretty extensive proof of value across these five tool sets before we decided take up pig. Uh, we eventually went with, uh, get lab like you into the fact that it had the most functional, functionally enabled value set that we could, uh, that we wanted. Um, but more importantly, right. I mean, at least to me a large part of our decision with good lab, also with a strong sense of partnership that being brought, right? Because as you can imagine, um, any one of these changes could take, um, you know, migrating from one tool set to another tune set takes anywhere between six months and a year, right. Um, and as you start to do this, as much as the teams are of putting in the effort to migrate away from a particular tool set, there's an element of a bubble cost that a lot of organizations need to deal with.

00:15:37

Um, in our case, uh, Goodlatte actually came to the table, understood our, um, our current ecosystem, the kind of contracts and commitments that we had with the incumbents, and actually worked with us in a very collaborative manner to make sure that we had a commercial value proposition that worked for both sides, to the extent that it didn't overly penalize my budget, but at the same time also kind of gave get lab enough, uh, enough value in the deal to the point where, um, you know, they, they engage with us and we made sure that we had a very pragmatic, uh, implementation plan, um, as we started this journey. So I think the partnership element probably stands out more to me than anything else. Uh, Bexar the tool is definitely well, well positioned functionally, and in terms of its roadmap also, it's kind of, I would say fairly future proof.

00:16:29

Um, but what is the scale and get labs favor for us was also the sense of partnership. So I just want to make sure that, uh, we don't dilute that element of, uh, or the relevance of that element in our decision making process. Um, so as we started this journey, what is the, what does the first phase look like? Uh, first phase of course, is very much what we did with the low hanging fruit, right? Um, the benefits view that largely underpinned our business case. Um, and I've talked with some of these things earlier, right? Um, and the very left end of the spectrum, right? We are looking to kind of standardize our CI CD pipelines. Um, really what close to about 2000 billion of varying flavors of our pipelines. They are looking to actually consolidate that onto a core set. Uh, we expect anywhere between an eight and a 13% improvement improvement in productivity from existing themes, just to that standardization.

00:17:30

Um, right. Part of this is very much as we are starting to do this, we are also templatizing our CIC pipeline, right. So when you've got to go back and look at an organization like ours, that's kind of in a, hyper-growth more, um, we have constantly been in one and growing the team, right. And one critical element like growing a team is easy, right? In the sense that it's not so much about getting people in, whereas the element of the new people that you bring in, how many of them are truly productive and how soon can they get productive. And one of the things that we found out is that our ability to make a team productive, um, is more enhanced when we start to templatize a lot of the tooling. Um, and to that extent, I think that's kind of the second value proposition that we are hoping it lab will kind of, uh, put in place for us.

00:18:19

Um, like I said, the real, the use of the pipelines are the box combination of the toolset and the, uh, um, you know, and out approach itself. But, um, but we think that this, the core set of templates that we're going to land at will largely future proof us for the next several years in terms of how we continue to kind of, um, roll our platform out. Um, we S we spoke about the automation, right? So again, we are in the midst of our move to the cloud. We bought a very large part of our business in the cloud. We bought smaller legacy pieces that are still sitting on prem, that we are looking to move. Um, they didn't the cloud construct we've actually got, uh, both, uh, um, maybe one or two public cloud providers that we partner with, uh, for our business services.

00:19:06

Um, again, the hunt was very much for a tool set that would actually spend this scenario, right, support us in not diminishing on-prem footprint, but also then continue to effectively support us across our, um, two cloud providers being much with the intent saying that at some point we will be multicloud. So, you know, we w we, weren't looking to kind of put the organizations that truly know the transformation of having to move across that tooling at the same time. So I think the lab kind of insulated us from that exposure. Um, and then of course, um, I got spoken about the other two, right, which is as we started to do this very heavy focus on ensuring that we want the right controls within the tool set, that makes sure that teams are ready, ready to our broader software development policies and procedures. So I think getting those five items kind of started out over the next, uh, next two months or three months, um, gives us that base camp in terms of the core benefits that underpin our business case.

00:20:11

And then I think comes the extra lift that we are hoping to bring in with, um, with good lab, um, again, in no particular order, but starting on the left. I think once we've kind of got to base camp with our automation in terms of standardizing the toolsets, um, making sure that, you know, we've got the right, um, um, automation around controls and adherence to policies. Um, we intend to pick up the shift left element of our security features, right. Which is what we want to do is start to bring security increasingly closer to the design and build phase. That would be, I would say phase two for us. Right. And then of course, finally, phase three, um, which is where we are hoping that the kind of, um, metrics gathering that we are hoping to do out of the schools that really powers our measure and improve ecosystem.

00:20:59

Right. Um, V as an organization, I've traditionally been very focused around constantly measuring and improving. Um, but the overhead in terms of hoping that getting that measure today is pretty high. What we are hoping is this becomes a lot more intuitive and, um, centric to the tune set, right? So to that extent, there isn't this body sitting out there, that's taking all this data, drying the insight, giving it back to the teams for them to then start to kind of put together the improvement measures. Right. Um, we want this to be served up to the teams directly through the tool set. And to that extent, every team kind of owns its own destiny, right. It's constantly measuring itself and improving itself, um, both in construct of what they are doing as well as in comparison to their peer set. Um, and that in my mind kind of brings us to a place where we want a tool that's humming.

00:21:48

Um, I of move on in terms of what the benefits are. Um, definitely short-term benefits, right? And I've kind of touched on bits and pieces of this, uh, over the past several slides, right. Um, at the very left end of the spectrum, I think getting the right tool set put in place just increases the sense of ownership and that feeds right. Um, put in an ecosystem, we bought a tool set that while working just doesn't give developers the kind of visibility that they have, that they would like within the process. Um, doesn't give them the kind of insight, like they're waiting on somebody else to give them the insight. Um, here, I think since it's all being served up in the tool through the teams, um, it'll drive a much stronger sense of, um, um, ownership as well as a stronger desire to continuously improve.

00:22:40

Right. So, so to that extent, I think it definitely builds through a more, uh, um, self-sufficient, uh, organization, right. Um, kind of, uh, I would say it's an actually bad byproduct, but you know, the better that our tooling and our automation gets, especially around the audit elements there adherence to our SDLC, the quality of code obviously starts to improve. And to that extent, we start to see metrics like, um, you know, elements around rework and things like that start to pile up substantially improve, um, increase in audit and compliance checks. Like I said, you know, like most organizations, we've got a little ecosystem that actually almost supports our development ecosystem. That's very focused around the right audits and the compliance checks to the extent that we can stack to bring this and make it more integrated to the tool that overhead was a bit improving, both agility, as well as, you know, there's an efficiency element for the organization in terms of not having to do that manually.

00:23:42

Um, and then of course the last one of course is very much a, almost a byproduct of the other items, which is that to the extent that we get the first three items started, um, we definitely become a more efficient ecosystem, right. And as part of a business case, we have underpinned our investment, um, with a certain committed product, uh, productivity improvement that we are hoping to do the work. Um, these, I think are the most short-term investments, um, as everything else, you know, that is a long-term element, right? Which is that this in conjunction with our, uh, wider architectural, uh, uh, um, the transformation that's happening, um, in essence puts us in a better place, right? A I think a combination of our microservices architecture, the right tooling, you know, um, we will be able to deal with stuff to a business faster, and we already starting to see it right.

00:24:37

Um, over the last several weeks, we've kind of seen multiple instances where we've kind of taken the sheer breadth of our releases from, you know, 10 or 15 apps, every release to about 30 apps, every release. So we will launch products to market faster. We think the sheer number of releases that we are going to do will, uh, will go up. We will see that have marked improvement in our meantime to recovery. Um, so everything that suggests that an organization that that can with pace, right. Um, secondly, I think ultimately we get to be in a better place too, right? I mean, at the end of the day, one of the biggest, uh, I would say friction points in the organization today, um, particularly the dev teams is the share overhead of having to manage this heavily fragmented ecosystem and all the inefficiencies that come with it, right.

00:25:30

To the extent that we are able to streamline it during an automation. Um, we definitely kind of provide a better work environment for our teams to be more effective, right. And to that extent, um, it can really help the culture of the organization. Um, and the last settlement is better. Um, you know, again, as security gets more and more credited to our ecosystem are inherently building out a safer value offering for our customers as another employees. Um, and more importantly, you know, just given the fact that we are making the, uh, the entire development process much, much more integrated around a single stone set. We are also hoping to be an efficient, um, organization. Um, I, I stopped briefly that I don't think I had anything else. Um, but hopefully that kind of gave you some kind of view into just what we as an organization do, how particularly getting the right tooling and is too, especially with regards to the right culture. Um, and then of course, a high-level view of what kind of benefits we are hoping to see, um, a lot more to cover, but, um, I, you know, feel free to reach out to, uh, me or anyone on my team. Um, nobody happy to answer questions. Um, if nothing is, uh, share war stories in terms of what's worked for us and what's not, .