Las Vegas 2020

Fast Product Development in Digital Banking Without Sacrificing Security

In this session, you will learn how Bancolombia, one of the largest banks in Latin America faces the challenge of improving the lead time and security standards of its applications to keep competitive with the new products emerging from the new 100% digital banks and the emerging Fintechs.


Get to know how a 30,000 people bank serving 14 million customers, moved from 6 months lead time in its applications to under a week, through the implementation of DevOps and security practices.

RA

Rafael Alvarez

CTO and Co-Founder, Fluid Attacks

CP

Camilo Piedrahita

IT Manager - DevOps & Software Engineering, Bancolombia

Transcript

00:00:13

Hello, everyone. I hope you and your families are ready with. Despite the current situation today, we want to tell you a little bit about how you can achieve speed in an increasingly competitive financial system without losing security. I am coming DevOps and software engineering manager at bank Columbia. I'm currently responsible for defining the stats on best practice for the software development of the one Columbia group. In the past, I left the box information in large company. the leading insurance company in Colombia. I am here with Rafa Emilio. Thank you for this great introduction and hello everyone. My name is . I'm the CEO and co-founder of fluid tax. I live at one of their largest civical half and teams in the Americans focus on plastics rotation and will never leave. I look forward to sharing with you a week lever to use by Columbia and your adoption of this. It costs in some takeaways, you kind of like through your business bank. Colombia is have been with one kind of 40 years of history and one of the largest in Latin America, we are present in nine countries and in the world. We will have more than 30,000 employed until you throws them on there. It, by surprising, during 2020 or planes are related to digital capacity of an 87% of all conduction have been totally digital in the way we serve in more that 14 million client with an RJ, Bonnie proposed feature in debates or development practices.

00:02:01

And I feel like the attacks we have to send over 240 millions line of code in 50% of the application with test, we find at least one critical high , which put the businesses risk. We have guarantee the security of more than 28 million clients compilations and it lightens our false positive rate is super saying, you know, our, our false negative rate is 4% competition in Latin America. It becoming more demanding. We no longer have to worry about other lane, America banks, much less other banks in Columbia or competition. They've really been affected by all the FinTech that entered the financial market. Of course, by the proximity of other giants in our sector, such as apple pay, I'm paid by us by the enters of JP Morgan in Colombian market. This is why we have to accelerate the processes. We cannot continue with the old traditional approaches allies implementation of core banking body delivers to our mobile application every month by user, neither, neither more digital on a straight forward experience for the reasons a few years ago, we decided to as part two, I started the part of DevOps, agile and cloud.

00:03:22

So whole class or journey been six years ago, wound to the project called scene in which created more than 200 work cells and decided to be 100% agile. However, we're at less changes. The methodologist wasn't enough. We also needed new processes and new tools. That's why after learning about this obsessed studies of capital one, Walmart, but Iceland nationwide, another companies will choose to embark on a journey with the both practices in 2017. Sorry. So do they have more than 700 applications? We continue the delivery process that allowed us to make several Bali's deliveries in a single day, but what was happening with the security during this time, we were still very of low security and the end of the process, my deploy to production environment, impossible. We had on the day to deliberate production. That's where the SecOps came in. We went into ship less the security into the security process of bank Colombia.

00:04:33

So now we have DevSecOps one of the big trends worldwide, but what next we were definitely not as fast as we wanted to be. You cannot compare this fit to change our deploy a microservice. We deploy a complete cobalt program on a mainframe architecture, moderates in Mullins alone, therefore wonderful next challenge. It was to be 100% cloud. So we created a project called all in with which we've migrated or created more than 100 applications using any of the ways while a low end in us to be much more cost effective elastic and resilient. Our journey is inspired by nationwide. When we started the HubSpot in 2017, we looked for a certain consultant and suppliers to help us through the fine, the tolls unprocessed at the time we're embarking on our referral journey. We here in the box in our price as well. A world leader, such as capital one, Microsoft Disney, another mature DevOps companies.

00:05:42

We form a team of Sherpas condition of the box engineering with experience. In other companies, we are run. We arrived. The spare thing counter to that. The whole organization could understand their way of going to play the highest peak possible. In the base camp. We defined tools, change processes and created many training sessions called there. The Dodger we found early adopter or advances quiet to accelerate the organization, cultural process and huddle first automatic deployment with many hours, for those who adopted the best practices, then we started to promote the strategy and define a role called mobilizer responsible for introducing the practice to each team is stain had a mobilizer seeking to about decentralization of knowledge in DevOps team. Later when we arrived and not camp, we decided to identify the role of developer and tester. Following the war late of appalling 2017, you build it, you test it, you own it always starting mentioning four key metrics.

00:06:51

Subsequently Holly, with the super of the vice-president or expedition leader, darter was given that the only way to make changes in production environment was through the box. The boat came on that sorry, policy on practice in the organization today, we're able to perform over 3,600 changes to production on a monthly basis. But what is the next challenges? Then it's big. We wanted to claim a 16 continuous case under both for database, but the big challenge is to be 100% cloud and open for in the organization. Wi-Fis in bank Colombia with definitely have medium performance and low performance application, but we, with joy, we can say, we also already have a lead performance application here. They have a sample of different application that have perceived the benefit from the boxing bank, Colombia for an old people lab, which has had 50% improvement lead time to inclusive application that have more than seven deployments per day with an entity of less than an hour and chain with failure of 1%, we have a love to improve by the process as quarterly allowed us to deploy a new feature in a couple of days, the hub generated transactional growth of up to 30% for those applications.

00:08:13

If you look back two, three years ago, that was completely impossible, but who do we go for an application with a lifetime of months to application with seven releases per day, DevOps in mainframe. We know that we want to have the same practices as a microservice, but we need to improve another Nitel processes while we kill our monolith. Here, we can remarry continuous delivery process and regression testing. We are also in the process of defining or unit testing strategy. Also with our Grella leaky base, we automate that away deployment process for you. We many and ruining the non production. We've integrated the Cuba with gates and created an extension of Azure. There was such a to the real engine the Liquibase has. Then when bugged the hammer stuck the leaky base in order to execute the promotion of this changes. Now we guarantee homogenous environment in database.

00:09:18

When you are ready to go, we will take us a whole day to perform the database deployment projects, including ticket sign work, use guide construction, another site tasks. Now we need 20 minutes to carry out the continuous delivery products for, uh, for the database changes. And we are working hard to send that strategy with appropriated continuous testing for database changes. We've also indirect cares in generic strategies in Bancolombia. Usually this kind of the strategy is difficult to carry out in highly regulated companies, but how can we get this speed you've or business continuity exercise, our conduct every six months, can we really guaranteed business continuity? What are using case Tolkien or pilots? We are longer validate or resilience at the end of the semester, we do it in the pipeline. Every time we have a now for the production, we are able to guarantee you're resilient to failure.

00:10:18

This is the way to train or for protection problems. We muggy on there. 100% that works. The strategy is supported by several pillars, including continuous testing because their work quality is to have first struck four for in production a month. Columbia, we have a largely Romania, all tests all with having computers in the rider. We decided to include robotics in our 18 states and integrated these into a pilot in the way we have gone from taking 18 days to run a complete recreation to take in only two days, but hope can we guarantee that really our past as Guinea cane, Jennifer coronation, why, when he could say the PGR to go, Ethan is Itali to have for the workforce metrics compass of course, having a tool like has allowed us to have a real lighthouse of information for the organization. We can see the six CS of deployment lead time in more than 2000 barks today, all the readers have the visualization of their application and developer are able to ask them the relevant information or they component in non single place tanks.

00:11:36

The popular one for releasing Hungarian. Speaking of capital one in 2017, they had a station called better governance. What did they emphasize? The important of giving their countries densities peace of mind through governance and not compliant. They presented concept of clean room and advanced Colombia. We've decided to adopt this concept and we have all gates uncontrolled in the same pipeline, the politics bellyache topics, such as cover us 600 bill code review and different strategies that allow us to have a real quality control in our pilots. In addition, we have deeply engaged in the pilot that may automatic palliation. So Chaz, what are the performance and regulation tests successfully executed either technical them not increasing, and we've even created an integration with the API standard of pipelines, repositories and deployment. We are a freeway strategy, viewers' approval, more governance, but if we have governance, what did the acuity automation isn't enough?

00:12:54

I've done Colombia or go to take security to the level of the process. I think mentioned before they develop her, have to have all the tools available to find the vulnerabilities, as soon as possible, we could summarize it or as an RQ profile and use putting McCloud to Lea security in or containers and waste whatever we integrated right. And monitor the Docker images and open source libraries used by your developer where we have success. Definitely not. We have tools, but , didn't diminish in the last report. We have 2 million vulnerabilities to resolve with tools. And over 50,000, we continued hockey. When we're going to resolve that vulnerabilities, we were able to kind of boat infrastructure, container and so on. But what was next that what we found fluid attacks continued parking is reality, which allowed us to reduce the gap between open and closed vulnerabilities within our application.

00:14:01

But it wrap up, can you show the secret sauce? Sure. Let's have a look at how we implement that. The cough of Bancolombia and what you can apply to your company. Let's now understand the magic. We call it packets of the center accelerated like AI. So illustrate this. We will use a simple diagram that shows the interaction sequence of the different actors involved in our continuous hockey process. For this accounts. The first actor is the development and operations team responsible for creating or maintaining neater application system or server. This thing independent of the methodology and follows for the products. Architecture will main commit to a long term branch in a, in a good, these good repository is the only prerequisite for any continuous hacking approach and will allow full traceability of the changes made by the T using the code store in get input, continuous tracking, cold thrills from now on we'll be in performing attacks to two types of techniques, studying applications.

00:15:18

If you're testing and soccer composition analysis, the first one allow us to reveal the security of the application even without having any functionality deployed. And the second one allow us to determine if the third-party components use are secure. The fact that hackers carry out this process, make it possible first to realize that systems have a false positive it's reported by the tools. Second to find vulnerabilities, not detected by them, also called false negatives. And third to relieve the developer of the hard work of this carbon and understanding vulnerabilities is related to Publix attack techniques that evolve on a daily basis. Then thrills themes or factors we've reported a comfortable everybody's providing detail evidence of the attack, beat animated video screenshots, structured records, or the media by attack surface manager call integrates integrates prevents. Anyone from removing existed will then obedience on throws when each vulnerability is viewed and by whom and in general, meaning mindset, meaning biases or prevents zero day vulnerability management from becoming a spreadsheet or, or email based Brooks user rights also allow us I'm depending on hacking team to mark the bullying disclosed after a technical reattached non documentary on the target system integrates, therefore we'll be responsible for adequately nothing fine stakeholders of our new vulnerabilities fund, as well as confirming their stuff's open or close after real attacks by hackers integrated web interface also allow us direct written communication between developers and hackers, which is especially important at the beginning when developers required explanation about the nature of their risk and will then the hackers book was on explaining the problem.

00:17:34

It never did solution so they can guarantee their independence for future Rhea attacks, the strategic positioning of the attack surface manager as well. Then obviously the storage gold means that senior management, where they are CEOs, CTOs, brother owners, scrum masters auditors. What do you mean customers can't always know the security status of each system. And however ideation process is evolving.

00:18:08

Once the peer environment is available corresponding to the code in the good branch being monitored, more advanced techniques, such as dynamic application security testing and interactive application security testing or use scenes by Columbia has continuous integration. Three aunts hackers, an agent to the pilot called forces, which is say well to break the build avoiding going into production. If the software is a steel boat narrow, these also likes the team to make explicit risk acceptance sensation that are documented in interprets. You do the formality of this approach, the applications theme briefers remedy ation, Rotterdam signing off of this acceptance of the race cars associated with Navy.

00:19:12

As you can see, continuous hacking is focused on the hacker, but to be passed into support teams with many daily predictions, there has to be more to it. Let's look at the behind the scenes of difference. The first component of our hacking team artillery is called schemes. It is an internal tool that allows us to locate low critical . Then we have an artificial intelligence engine called SERP SERPs learns daily from the vulnerability is found and does a hacker, which files or areas of the application shall be checked first as they are more similar to files or areas that have had, we'll never lose in the bus after work, they hacker out the center, does his magic and report comfortable now really so that the government team only has to focus on the final remediation. What were the results of Bancolombia with this approach? That's remember that the bank had 3 million report vulnerabilities by our tools will be the attacks under the continuous hacking model report over 50,000 configurable men abilities over at 18, 18 month, period by Columbia has been able to reach actual rate of 83% and the no in numbers, kid improving.

00:20:52

So we get an idea of what the attack surface manager looks like. Here's a screenshot of integrates. You need, we can see the evolution over time of the shirt search for vulnerabilities and better yet there, the mobile bylines are crazy saying that management. So it focused on high level indicators that allowed them to understand the system current security stats. The first indicator is the rate of effective remediation, which is basically calculated by considering how many of the vulnerabilities have been effectively remediated below. We can see the weaker progress on this indicator. That is where the past week has been better in terms of recommendation that the week before finally, we can see that this organization in its entire continuous hacking portfolio has only two systems on their attack. Therefore management can understand the real context of the sense of the recommendation and not have a false sense of security with this simple data and the hands-off document time, the themes writes off or remedies your rises to the incredible labels as progress indicators become daily, automating transparent and understand the whole matter.

00:22:16

There are. These leads to an increase in security, priorities and improvement in resource allocation or alternatively visit acceptance of risk by the responsible Lexi. I mean, do you want to share something with, I want to share with you some lesson during the journey architectural remarks, you can include a level of practice in your monitoring, but you don't have the same spirit. You need to worry Proctor migrate and creating modern application. If you want to fit, you need to have a lean growth of this and simple typo. You can have governance. And then if you include the traditional compliance that are loose in DevOps, you can pay for now, but you can't create a new site with 10 20 fed. And the knowledge aren't enough. You need to interpret in security team in the project.

00:23:19

Probably do you have any takeaways? I think so. So to compute computers, we want to share with you. We have learned year after year. So we implemented this methodology in backlog and for many other types and which we consider kind of maximize the security of the system. The first is of course, most it's about food. The hacker sat in the center of diction only. They come find what it's grading only they can connect one vulnerability to another, to achieve a higher level attack like those would read about in the news. And they kind of minimize false negatives. Why not? They must serve all flaws with the tools that prompt speed, but which usually is not discuss.

00:24:10

Packard's also allow us to implement another great of the bread, confessions and stages, discarding photo philosophy, this security contest, even by the hackers isn't imperative. Therefore think to this process of this tax, we allow developers to save valuable inexpensive time. That can be mined telogen invested in ready to pitch. Since human the center, we must accelerate the process. And for this purpose, artificial intelligence can be used to Helen Rosner, not replacing our precisely trained and invited AI engine helps us feed out the hacker's job by allowing them to focus on the obligation of higher risk theories. Feedback is one of the three key principles of DevOps feedback is given precisely. If I break individually the red indicator, that was the developer that something is wrong and that they must repair it before trying to go into production. Again, one of the OxiClean DevOps, and especially all the indexing, that's the crops increasingly violate this principle.

00:25:25

They have all, how did they chroma Bazemore for a manufacturer's viewing? Give me a priority to speed in that permission. At all costs, we see countless companies investing millions of dollars. high speed tools with high rates of false positives that don't get timely feedback. Don't break a build and simply the car out of bad report generators to show you that due diligence has been done. Therefore their conclusion is to bring the bill with my only comfortable than only these two for recreation and resume their principle of feed. This shool supersede the approach of something that we consider going the wrong direction at a faster pace. Finally, we have to pay because computers have evolved in dove bar here. Our suggestion is concrete, but in your hands application has single indicator, their organization recreation rate with this level of development teams accelerate their fees. And not only does management allocates resources, suite B can, it will be an act of negligence to see data on the touch of a button that they were going to say, Joe has 50% of comparing open vulnerabilities and not do anything about you might belong to this case. A big corporation that exceeded the BP has some visibility of the global radiation rate. And that is the key to what makes this proof of war more than anything else. Now we just need the CEO to solve the application through these remainder. You do that formality level of this approach. The application teams prepare recreation, rather sign enough on their set down some of their risks as we'll see, even without bullying.

00:27:30

Thank you all very much for attending our tolls. And please do not hesitate to contact us if you wish to discuss any details, sharing experience, or neat ending Tyrone.