A Case Study: Open Source and Commercial Vulnerability Scanning (US 2021)

Open source and commercial solutions are often portrayed as head to head competitors in a market where open source is the protagonist saving developer teams from commercial villains who want to steal their money. But nothing of value is truly free, and while open source does not have an official price tag, its real price tag manifests in other ways, like management overhead. In the end, the true difference between Commercial and open source is somewhere in between; each has its own ideal use-case. In this talk, we will compare Aqua Trivy, the popular open source scanning tool, to Aqua Enterprise, demonstrating a cloud native security case study of the difference between open source and commercial. This session is presented by Aqua Security.

usbreakoutlas vegasvegas2021

Story Tweedie-Yates

Sr. Director, Product, Aqua Security



Hi, this is story 20 eights. And today we're going to be talking about open source and commercial in the context of vulnerability scanning. So, um, we'll use vulnerability scanning as a case study taking, uh, two specific solutions. Um, but the overall goal here is really to help you and your decisions to, to go either way, right, to, to go with open source or to go with, uh, a commercial solution for, for your specific needs. Um, so with that, let's talk about how we're going to, uh, help demonstrate this. So, um, we've got two, uh, solutions we have , which is a vulnerability scanner. Also includes infrastructure as code capabilities, and turvy has more than 3000 stars on GitHub. It's the most popular cloud, native security vulnerability scanner available today on the market. Um, it's used as a default by get lab Harbor and more, um, so it's a popular, popular, open source tool, um, and more like a dev ops tool, right? Um, on the, the, uh, commercial side, we have Aqua security. So they have the ACO platform Aqua owns Trivi and it's a commercial scanner. I would say that, uh, is a key part of a larger platform. So other capabilities, other solutions as part of the same cloud security platform.


And again, because the point of all of this really is, uh, decision points, uh, figuring out what are the decision points for your own decision moving forward. That that makes sense to keep into, uh, keep in mind. And really the key points are the ones that have a personal impact for you and team. Um, there's no right or wrong. Uh, what you'll see is that the decision can flip back and forth. They're very established companies using open source. There's established companies using commercial, um, and there are students using commercial as well. So, I mean, it goes, it goes both ways. Um, but the things to keep in mind and the things you want to make a plan for, um, are some of the following. So thinking about whether a solution will fit with your longer term needs, right? You need X right now, our vulnerability scanning.


Now, what will you need in the future? That's part of, uh, your roadmap, um, how fits a purpose? Do you need it? Do you just need one capability? Do you need three capabilities? Uh, for example, do you need vulnerability scanning? And do you need infrastructure as code scanning or do you need vulnerability scanning or scanning? Right. So w how fit to purpose, uh, with the solution fee, um, management overhead. So this means different things to different people is money more important than time in your particular, um, that will look different from open source to commercial. And this is probably one of the more, um, the decision points that's nearly guaranteed to be similar across every open source to commercial spectrum, and then open sources, less on price, more on time. Commercial is the opposite, more on price, less on time. So you don't have to spend as much time using it, other things to keep in mind.


So vendor guarantees, right, uh, in, in here comes in the purpose of the applications to begin with. Do you need a cell aids? Do you need support? What kind of support do you need around education of a team? Um, and then how much do you want to get into efficacy, accuracy best in breed? Sometimes not always, but sometimes there are differences on the open source and commercial sides, uh, between, uh, yeah, in terms of best in breed, accuracy, et cetera. Um, and then I would say the first one here, as well as last time to value generally source you get up and running much, much quicker. Um, and so something to, to keep in mind. So, um, what we're talking about today in the case study of Aqua and Aqua Trivi is vulnerability scanning. So how will we apply some of these decision factors to vulnerability scanning is, uh, as such, so when we're talking about time to value, can you integrate the scanning solution into the CICB pipeline?


Are there integrations with the integrated developer, um, in terms of longer term needs, there are a lot of ancillary capabilities in the vulnerability scanning space, because when you're scanning for vulnerabilities, you're scanning applications and application security is quite a broad space. So, uh, and then you have the cloud native element as well, which is, um, also varied. So CSEM, for example, scanning, uh, the different, uh, configurations of your cloud service accounts and making sure that there are no misconfigurations there that could expose your application to attack and runtime protection. So if you're scanning in the bills, are you also protecting and having other gates and runtime to stop something that's gotten through and where there might be an attack in progress in terms of management, overhead and vulnerability scanning. Um, this could potentially be a list that applies to maybe other types of security, but, um, how you manage multiple tools across teams that comes to vulnerability management.


Um, how do you take your remediation results and actually feed that back into the process to get a bit of a circular motion going and making sure that you're continually improving, um, how many nodes, clusters images registries you need to, uh, be protecting and you need to be looking over, um, that obviously has an impact with management overhead and then vulnerability management itself, the overall holistic process, um, even amongst people and teams. Um, so if you have, uh, vulnerabilities scanning your, your scanning business critical applications, uh, and you maybe need certain commercial terms, that's also something where vendor guarantees will come into play. Uh, some open source solutions don't have the same commercial terms as a commercial solution. Uh, and that's not something that's often brought up, but it's very true. Um, and then lastly, in terms of vulnerability scanning best in breed, for example, uh, with infrastructure as code, um, that's so I will say here, when we talk about decision points, they're not mutually exclusive.


So we were talking before about fitting as longer-term needs, fitting to purpose that also has to do with best in breed, efficacy, accuracy. Um, so they're not mutually exclusive by any means, but, um, let's say you're looking for, um, uh, malware protection protection from supply chain attacks, uh, as well as the vulnerability scanning. Um, that's something where your decision point around best in breed accuracy definitely comes into play in terms of vulnerability, scanning of open source versus commercial, uh, solutions. So with that, let's go into the actual case study, uh, and talk about these two solutions and, uh, what they, what they would look like in, in reality, uh, to give you a sense for your, your later decisions, maybe in other areas. So at the beginning, with the end in mind, I mean, basically you're going to be using Tribby for vulnerability scanning when you're just looking to get started.


Uh, let's say you're, you're doing a course you're in school, or your applications are not business critical, or you simply have less complex, less distributed architectures. And one of the thing that's not listed here is if you absolutely have to get up and running something today or tomorrow combines perspective, uh, Treme is going to be your friend. And, um, it still has, um, really it benefits from wide usage and inputs from really customers and organizations and projects across the board. So, uh, it's the default scanner for Harbor GitLab artifact hub. Um, so you're still going to be getting a best in class solution. Absolutely. Um, and one point about Trevi getting up and running quickly, uh, no database dependency, you just install binary specify at target. So again, that use case of a quick time to value. It's absolutely true, uh, with Trivi. Now, if we get into the commercial product and some of the elements on that side, again, beginning with the end in mind here, you're going to be looking a little bit in more depth.


So some of the decision points here, just to make that clear, we've got management overhead, right? And this first point we've got efficacy and accuracy and best in breed when you're looking at broadest security coverage, right. In terms of what it scans and what it doesn't. You're talking about vendor guarantees and commercial licensing limitations on you're also talking about longer term needs for continuous protection into runtime. Um, so this is, this is just an overview and there's one more overview point on the next slide. But, um, again, beginning with the ends in mind, this is what you're looking at between a commercial and the open source in terms of vulnerability management and this case study. So diving in, uh, management, overhead, vulnerability management, actionable results, and a feedback loop on the commercial side you've got, um, well actually let's choose, let's choose a different one.


So we'll be getting into that a little bit later. So I would just say, um, on the trivia side, so there's no default aggregation of data from trivia into a UI, uh, and on the commercial side, you have one place across all your systems with a dropdown menu, a way to see which are in are skinned. Um, and so that's, that's really helpful now, um, you can filter vulnerabilities by criticality with trivia, um, but in order to see in a visualization tool results outside of the command line, um, you really need to export the, the result. And I'm going to show you what, what trivia actually looks like. So you can understand it here. So, uh, share that really quickly. So press play. So this is what it looks like to actually set up and install trivia. It's all it's all coming in. And then I'll show you what it looks like in the commercial product as well.


So here you've just installed it, it's downloading. And then this is the end result of what you see, right? So you have the library, you have the vulnerability ideas, severity installed version, fixed version, and this is how you get it. And again, this can be filtered, but this is going to be very different from what you see in, for example, um, you go here, all right, so this is the vulnerability management screen of the commercial duct. But if I go to images even to see in general, all the images that have been scanned and there's drop-downs, and you can get a little bit more detail in here like this. I mean, obviously there was no dropdown, there's no extra detail in any dropdown command line, right. So it's very different. So it was just a little quick view into exactly what we're looking at here. So let's get back to the slides.


Okay. And I believe that sharing. So, um, let's talk about as well, uh, coverage. So sometimes developers will copy files directly into an image instead of installing files via a package manager. So if a file is not installed by a package manager, Aqua Trivi will not be analyzed. It's not the case on the enterprise side, because it also will scan or a standalone binary. Um, and then moving down to the next side, there, there are commercial licensing limitations with trivial. It's not for resell. Whereas on the commercial side, you can absolutely repackage the apple platform and it can be sold, uh, to MSP providers. Um, the big difference there, and a key decision point in general for any open source commercial solution. And then in terms of the last point here. So this is really a mixed, like I said, before, decision points are not mutually exclusive.


So this is both a, I would say a longer-term need decision point as well as potential decision find around best in breed and accuracy and scope. So, um, basically the commercial solution has multiple gates, multiple different points of protection along the pipeline, including in runtime. So when your vulnerability you're scanning for vulnerabilities, you're generally doing this before production trying to get the cleanest, uh, image possible. And in the case of cloud native, um, and on the open source side, on the Treme side, that's that's furnace, right? That's where it stays. It's a vulnerability scanner. Um, on the commercial side, you have an option for a follow-up protection in case there's an attack in progress. Uh, and something got past the standards, which, you know, it happens, uh, and we've seen that happen, right? Um, so moving on, we take a look at another decision point around management, overhead care, we're getting into much more depth around vulnerability management and what that would actually look like, uh, across, uh, both solutions in this case study.


So, um, holistic vulnerability management is something where you following Gardner's models and Gardner's models you assess mitigate, and then you improve over time. Um, now in Treme, as we saw you can, well, I didn't show how you can filter vulnerabilities, but you saw the vulnerabilities and how they're displayed. And if you need integrations, visualization tool, you'd have to do this externally. Um, now what I didn't show in the commercial product is actually how the vulnerabilities can be visualized in relation to their exploitability, uh, in relation to their impact on the actual environment. So I show that in a second, I'll show that in the, um, uh, in the dashboard itself, one other key thing around vulnerability management here is the option to mitigate. Sometimes you cannot actually fix a vulnerability. You have to find a way to mitigate without fixing or patching. And there's multiple reasons for that, which we won't get into.


Um, the shield is an option to do that. It's an option to mitigate without fixing or patching. It's a runtime policy, um, and gives you a bit more of a holistic kick, right? Uh, which can be very helpful if that's what you're looking for. Now, I will say, and I've shown them the bottom. There is a, uh, this, the customer is a customer of Aqua Treme who has built their own UI on top of trivia. So they went the more time, less money route, and they built their own way, and it works for them from a compliance perspective. So they're exporting data to that UI, uploading it and getting the compliance view that they need to, to satisfy the auditors. Um, that's also very much possible. Again, this is all about decision points and just an example of what you might see in either perspective. So with that, let's go ahead and look at what we said we would. So we'll look at these shields and we'll look at how that's actually taken into account with the risk based insights view that actually prioritizes which friends you should be looking at, or which vulnerabilities you should do that. So let me share again, B screen.


Okay. And that should be sharing now. So let's go to the risk insights view. So as promised, what will show up here is a, not a timeline, but aligned of relevance. Um, the threats are the vulnerabilities that on this side are relevant to your workloads as they stand today. So these are actual exploitable workloads. So from a prioritization perspective, this is the stuff that needs to be prioritized, right. Um, and now if you have these shield turned on, um, and you're blocking certain parts of certain things that might be exploited, uh, given the vulnerabilities in your workloads that would be taken into account in this, so that remediation or shields, cause it's not really, it's a shield that shield would be taken into account when this view is being shown. Um, and so that's just, you know, the UI, uh, benefits management overhead benefit that you're just not going to be finding in an open source solution in, in general. Right. Um, so if we go back to second, the demo takes a little while to get back sharing again, science. Okay. So that should be sharing.


Okay. So getting back to the flow and the decision. So, um, another decision point around management overhead, uh, and here, we're talking about the need for a central view. And, um, this is really only relevant if you have disparate teams that want to see data all in one place. So here, we're talking about data aggregation, we're talking about the ability even to set a policy across lots of different teams, right. Um, and then also at the very end, we'll be talking to you about, um, vendor guarantees, those decision points that might help with business critical applications. Um, and you'll also those to be talking about the best in class decision points for how broad coverage needs to be. As we talked about before with front-end policies. So starting at the top, um, there is no way to set like one policy with trivia into and have it spread across multiple parts of your architecture.


That's generally not going to be possible. Whereas on the enterprise side, you're going to be able to set policies called assurance policies across all of your systems in one place with a dropdown menu. Um, and you'll be able to see, um, which artifacts have been scanned all in one place, um, and set policies on that. Um, and more than that, you'll also be having the capability for our back, which is going to limit what certain people can see when they actually get into the UI. So again, if you have a large team and you want some people to have a compliance visibility, you want other people to have, um, just scanning capability, um, that's where this capability to shut people off from certain parts of the system really helps. Um, and I will show that in the demo coming up here, uh, lastly, if we're looking at business critical applications, there's a few decision points here, right?


There's, there's your longer term vision. And then there's also the vendor, uh, commitment and what the vendor will provide. So just in this example again, um, if you want to have, um, if your longer term vision is to have something tangential to vulnerability, scanning like a runtime policy for cloud native applications on the Treme side, you're going to have to write a Rigo script from scratch. And then that would be enforced, uh, with open, open policy agent and some way, uh, OPA policy. Um, on the commercial side, there are literally runtime policies from the dropdown menu. So it's just two very, very different, uh, capabilities for something that is tangential to vulnerability scanning and the context of app development. So if we get to the demo, what I'm going to show you is the policies number one. Uh, and then I will show the permission sets and our back number two.


So again, with this sprint teams trying to get a sense of consistency across the board and what people can and can't do. So if we go to a strain again, and we go to policies, so let's just see what the results are to add a new assurance policy. So you can see here all the different controls that are offered for any finance team to come through and have access to, unless, you know, they're controlled by our backs. So, um, role-based access control. So you have, for example, you could say, okay, an image will not be compliant if it's not built off an approved based image where they would help supply to the tax, right. Uh, we want to enable our scanning. We also want to make sure that sensitive data scan and want to control any super user, uh, capabilities or privileges, right? Just as an example, um, we just, uh, this is the benefit of, uh, in this case and this case study, this is the benefit of the commercial solution for a larger disparate team, right?


And if we go to the access management section and just look at permission sets, you see some really nice examples of the permission sets that can be granted. So the administrator obviously has full access to the management console. Um, this person though is having the permissions of the vulnerability operator and they can add runtime policies, audit events, but they don't have access to the management console. And this one has even less. So, um, only permissions for scan results and to acknowledge vulnerabilities, but no, um, runtime policy capabilities, et cetera, et cetera. So these things can be managed. And in general, what this will look like is the developer from team would maybe have vulnerable, uh, visibility into vulnerabilities and issues and teammates images, but wouldn't be able to see your alter security policies, um, or a global compliance officer might see reports and audit events across all, but can't change runtime policies, et cetera, et cetera, et cetera.


So with that, let's go back again. Slides just one second. Okay. So let's be balanced for a second. Um, get lab is a fantastic example of a great company solution that has decided to standardize on open source in many ways. Um, so they take best in class open source solutions and use them in, uh, this example is their auto dev ops capabilities. So for Treme, um, that's where they use it. Um, and the benefit to them is the fact that there's, long-term roadmap input, that they are guaranteed on the open source side. Um, and so when we talk about, uh, business deans and long-term plans, that's not something we brought up before, but this is a huge benefit of having open source, right. And they've successfully managed the challenges with this, and they have a real partnership, um, with the trivia team. Um, so just to, to show that yes, open source can, and absolutely is, is, um, on a, you know, on a, in a production scenario from very, uh, you know, large established company and moving on here, we see the opposite view, right.


Um, again, trying to make it balanced. So this is an article about a developer who, or an engineer who really tried an experiment. He wanted to see what it was really like using open source across multiple registries teams, et cetera. And, you know, this is the quote that, that he came up with. Right. Um, so it's, it goes both ways. There's no one truth and you just have to make the decision that works for you. So this is the last real kind of detail of this particular session. Um, and then we'll get into, uh, we we'll get into some, a summary. So if we look at some more decision points, primarily these around best in breed and maybe future plans for a security, uh, uh, security, well future plans for the app and the security solution that's needed for it. Um, here in, we're talking about accuracy from a vulnerability management perspective.


So, um, you've got the ABD, uh, and I'll just show, show that again. So if you were to take one of the vulnerabilities from, if you were to take this vulnerability from Trevi and plug it in, or it needs to be plugged in, in Aqua AVD to see the detail about the vulnerability that's, this is what you would get, right. So you would go from one screen to another. Now, if you were to go from Aqua, uh, the commercial product and you were to get into the, let's go to, sorry, let's go to the sometimes slow. That's the joy of doing a live demo. Yeah. So for her to just get into this, you'd see, I mean, it's in the same screen, right? The information about the CBE is all, it's all on the same screen. So, you know, copy paste versus just writing the same screen.


It can add up over time. Right. Um, and the depth of the information, uh, it's the difference between a dedicated security and threat team who is, um, curating constantly, uh, the threat data versus a more publicly available, uh, feed, I would say. Um, and then on the scanning side, there's an option to scan for more malware, for serverless assumptions. This is all on the commercial side, but again, um, Treme and I, um, actually should have put there as well, infrastructure is code scanning. So it's vulnerability scanning and infrastructure's code scanning. Um, so just choosing, you know, picking your poison, what, what do you do you want to get really in depth into one area, have a broader view, um, and which solution open source or commercials then provide that for you now, lastly, um, here more on the, um, decision tree spectrum of, I would say best in breed and ancillary areas of protection.


So, um, the, if you want to take your, uh, the data, like do some kind of forensics analysis on the Linux kernel and what's happening there, um, you can do that with another open source solution called Tracy. What it'll give you is data, a lot of data, right? Um, now if you wanted to potentially use the data about what's happening to Belinda's criminal and then figure out from there, how image would be then behaving at run time without running it in run time. So basically container sandbox to identify supply chain attacks. You can do that with a commercial solution. So technically you can do the same thing on both, but the management overhead is creating an entire new product basically to take that forensics data from the Colonel and create value is going to be what you're getting on the commercial side versus the hardcore data, which is also great and interesting, um, on the open source side. Um, so let's do final view into the dashboard, or I'm gonna just show you all the different scanning options on the commercial side, give you a sense of the difference.


So fair to go to images. What you'll see when I actually choose one, as you'll see a lot of different options at the top for what's actually analyzed so vulnerabilities yes, different layers, um, in terms of base image, et cetera, sensitive data malware. And this is the dynamic product threat analysis sandbox that I was referring to before. So with that, let's summarize, and I think the primary points of view from my perspective, um, given this case study is that when you're looking at open source versus commercial, the point is not necessarily exactly, exactly that thing, all of your decision points and the things that matter to you to each capability of the product, because to be Frank, they're not mutually exclusive, exclusive. We saw a lot of decision points that were around best in breed that were like melding into decision points around longterm plans for your application and what you need from a solution.


So, um, it's not explicit, but that's not the point. The point is just doing a decision point, exercise around understanding the difference in what you need. Um, in general, I think the only truth that I've seen is that you're definitely going to have more management overhead, but quicker time to value with open source. That generally is true is I've seen, um, I would also say that just for vulnerability scanning in particular, um, definitely does not exist in a vacuum and neither will, uh, I bet in open source solution for any other technologies. So just an example here. So as a result, trivia also has infrastructure's code capabilities that it's added on as a result of the other things that you generally want to be checking if you're scanning an application. Um, but that's so keep that in mind for other open source solutions, things, technologies and capabilities don't necessarily exist in a vacuum in terms of the value that you can to be getting out of them. And then last point I would make is, um, open source is not just for fun. It's not just for students. Um, it is being used in production regularly, um, by companies, uh, every day. So it could be the solution to your needs. Uh, it's up to you to decide. All right. Thank you so much. I hope this has been helpful for some of your future decisions.