The End of an Era: Modernizing Vulnerability Management

This interview has been edited for length and clarity.

Matt Alderman: Welcome to Black Hat 2025. It’s day two. We’re here live recording from the CyberRisk TV studio at Mandalay Bay here in Las Vegas, Nevada. Still a little smoky outside, but it’s getting a little better. I’m your host, Matt Alderman, and joining me for this interview is J.J. Guy, CEO and Co-Founder at Sevco Security. Welcome, J.J.

J.J. Guy: Thanks, Matt. It’s good to see you again.

Matt Alderman: I’ve been looking forward to this interview. As I said a little bit, I have a lot of experience in the vulnerability space, and this whole concept of CTEM. It’s one of these areas that we’ve kind of failed at as an industry for about, I don’t know, 20, 25 years, something like that. And so, to see this becoming a very interesting topic again is kind of exciting because I think the primary three vulnerability vendors have kind of failed at this a little bit. What made you want to go down this path? Because it’s been this, like I said, a 20-year evolution.

J.J. Guy: Yeah, so we didn’t start here. Sevco started with what we now call CAASM—cyber asset attack surface management. And when that was first labeled back in ’21 or so, it was separate and independent of anything vulnerability management. And at least our take on it, was inventory of all your devices and assets, largely so that you could make sure that your key controls to manage your entire security program, your entire IT program, are actually in place. This was a frustration of mine from Carbon Black. I was a Carbon Black guy, and trust me, I got lots of feedback from our clients on the occasions in which the endpoint agent was missing from one of the devices at a key time. We got a lot of questions and asks on how to better manage endpoint agent deployments. And as we had the opportunity to get a new startup up and going, I picked that problem to solve.

Matt Alderman: Well, CIS Control 1 and 2, know your inventory, right?

J.J. Guy: That’s it.

Matt Alderman: We’re all horrible at it, so not a bad place to start, by the way.

J.J. Guy: Yeah, and it’s like we’ve always said the CMDB is supposed to be that control. How many of us have signed off on the audit record and given our audits saying, “Yep, we got that inventory.” Meanwhile, in the back of your head, you know that it’s not quite as clean as you’re presenting to the auditor. But what’s happened over the years since then and with CTEM is CAASM and vulnerability management are starting to converge because we’ve recognized that the quality of any vulnerability assessment program is limited by your ability to make sure that you’ve got a visibility and vulnerability assessment, whatever tool you’re using to scan, is actually covering that device. And then more importantly, that whatever tool you’re managing to patch those devices is actually on that device. 

And I mean, the foundation of everything, CIS Control 1 and 2, is that inventory. So, it provides that foundational data fabric, if you will, upon which you can layer all the rest of your vulnerabilities.

Matt Alderman: Which is one of the key areas that a lot of the vuln management vendors have not been able to solve for, which is the broader asset management problem. They have a lot of asset data, right, because they’re out scanning or they have agents running. But where are the unknowns, right? Where are the assets that I don’t know anything about? And that was part of the context problem that has been kind of elusive in that space for a while. One of them started to go down that path a little bit and did some really interesting work there. I’m not going to go into names, but it hasn’t been solved holistically. 

So, you guys start with inventory. Once you have inventory, you can pivot a few different places. You can go to configuration, which is a very big component. It’s kind of near and dear to my heart, having spent a lot of time on the compliance and GRC side, and also building some policy compliance stuff. You can go that direction pretty quickly. You also went to the vulnerability side.

How are you pulling all this together? Because I think what’s interesting for people to understand is how do you understand the overall posture of an asset? Because it’s not just vulnerability data. It is a combination of vuln and configuration data, et cetera, right?

J.J. Guy: And business context. The users that are on that device and use it on a day-to-day basis, applications that are… The list of the potential business context that goes in to assessing the risk on any one platform, and thereby helps you prioritize your vulnerabilities, is significant. With everything we do in vulnerability management today, and the core tech that all the existing platforms are built around, is a network scanner… which was great back in ’99, 2000.

Matt Alderman: I know.

J.J. Guy: When those companies were first established and the technology was built because the cloud didn’t exist, mobile devices didn’t exist. I mean, keep going down the list. But the world changed significantly since then, and our tool change for vulnerability assessment didn’t keep up. And I don’t fault any of the existing vendors for failing to keep up because it’s been a very dynamic space. But what started to happen is those platforms used to be horizontally integrated. You would buy the one product that was your vulnerability assessment product, you would scan your network and use it everywhere. 

But what’s happened as our networks have gotten significantly more complex, we’ve got more devices in more places, and you’ve got these different, the vertically integrated vulnerability assessment tools, you’ve got three different ones for core CVE assessment depending on whether they’re in cloud, on-prem, Linux or Windows, and then you’ve got the different kinds of attack surfaces, each have their own kind of problems. 

Matt Alderman: Right.

J.J. Guy: And the poor vulnerability analyst is still trying to run a process back from the day when he had one single platform, and the situation is untenable.

Matt Alderman: Yeah, untenable. It actually gets a little broader because now we bring in application vulnerabilities because it’s not just the device anymore.

J.J. Guy: That’s right.

Matt Alderman: Now I got a bunch of third-party applications running on there, those have vulnerabilities. So now I got to look at software composition analysis, static analysis, dynamic analysis tools, et cetera. There’s a whole space on the application security side, but there’s a bunch of vulnerability weakness data over there that could also influence the overall security and posture of an endpoint, which is not just a computer anymore. It could be a container running on Kubernetes or in the cloud and far… The landscape has exploded.

So, in 2015, I wrote a thesis on how the cloud would change the world. In that, at some point, a company would not, they wouldn’t need a network, they wouldn’t need any endpoints. Everything would be in the cloud. We’d all connect to it from the cloud. What do you scan when you don’t have a network anymore? Right?

J.J. Guy: Yeah, so that’s the core challenge in getting an accurate device inventory. It used to be that a network scanner was sufficient because all our assets were on-prem in everybody’s network.

Matt Alderman: They were all on-prem, they were behind a set of firewalls, and I knew where they were.

J.J. Guy: Yeah, right. And that was exactly, and they didn’t move. But these days with our networks being so much more complicated, the problem is not that nobody has an asset inventory – it’s that they’ve got a dozen and they all say something different. And the challenge is that every one of those uses a different kind of technology to measure inventory and that sees a different subset of the whole. And the only way to get a complete and accurate inventory is to aggregate multiple different kinds of technologies together into one picture. 

Back in my intel days, we would’ve called that the fusion of different multispectral imagery, multi-sensor imagery. It’s the same basic problem. But today, we try to insist on using one particular kind of measurement and call it good enough, and it’s not.

Matt Alderman: So, at the heart of Sevco, is it really about integrating and understanding all the asset data first from all the different repositories? You’ve got stuff running in Azure and AWS and GCP and on-prem and remote and mobile. So, the first problem to solve is get all that?

J.J. Guy: That’s right. That’s the foundation. In my view of the world and the strong opinionated approach, and we got our clients through this in the overall process. Step one is the inventory and making sure you’ve got a comprehensive inventory from all your sources. And that takes time. You’ve got to iterate through it.

On top of that is then making sure that all your critical controls are actually deployed where they should be. And no one, in general today, very few organizations, are actively managing agent deployments. So, there are agents for vulnerability assessment, patch management.

Matt Alderman: EDRs, yeah.

J.J. Guy: … endpoint security. Yeah, you keep going down the list. In general, the number of what is typical for the fire-and-forget kind of deployment that is typical on the IT world and the software distribution mechanism is 80%. So, 20% of any given endpoint agent is missing from 20% of machines that it should be installed on. 

Now, that metric, but keep going, because recognize, I mean, from the position of an attacker and understanding the kind of risk, that means on 20% of the machines, you’re missing the key visibility of even having an inventory of vulnerabilities or being able to push patches to that machine altogether…

Matt Alderman: Or knowing if it’s misconfigured or whatever.  

J.J. Guy: Right, depending on what capability it is. But when you go through the conditional probabilities, like the likelihood that, if you’ve got four endpoint agents, that at least one of those is missing from a box now becomes 30%. And you look at it from an attacker’s perspective of how many different doors does he need to go shake the handle on in order to find a machine that is missing one of those critical controls and the number is like three. And by the time you hit three machines, you are going to hit one that is missing one of those critical controls. I mean, we spend all this time patching the CVE of the day, and meanwhile, we’re totally ignoring these critical gaps.

Now, that becomes phase two, and it’s just basic visibility and command and control. You can’t even begin to manage your program unless the tools you use to manage that are actually in place and fully deployed. And then on top of that, you can start to import and aggregate vulnerabilities, and call it just CVEs first because you’re going to have multiple sources of CVEs.

Matt Alderman: Yeah, of course.

J.J. Guy: It’s not just one. Consolidate all those together, deduplicate them because there’s going to be overlap in the reporting, and with a strong asset inventory, you can understand where those gaps are and not have a noisy dataset that’s got duplicates.

But then what’s so amazing is business context prioritization of those. Because it’s built on top of the asset inventory and there’s such a strong tie between the rich data available on the assets about the device, the users, any other attribute in any one of the systems that you have configured, the quality of the business context prioritization that you’re able to do in managing your list of vulnerabilities is off the charts. Absolutely incredible and way more sophisticated and capable than what’s in NIST 800 whatever it is…

Matt Alderman: 830 

J.J. Guy: 830. Thank you. Then there’s four criteria and state-of-the-art these days in most products is to, on a per-device basis, go to the drop-down and select high, medium, and low. Who manages that at scale?

Matt Alderman: Yeah. I know that. I live it in my day life. 

I do want to talk about prioritization a little bit because you started to bring the components together. So, you think about business context, you’ve got the inventory, you’ve got the vuln, the configuration data, et cetera. Where I’ve seen a challenge is putting that into a risk model that you can take to an executive team, not to an operational team.

I think operationally, if you can prioritize vulnerabilities and give them a stack ranking list, I think that helps the operational guys tremendously to understand where they should focus patching. But I’m talking about this from an executive level. I’m thinking CFO, CEO, board. Their risk scores, I’ve seen challenges with, in that, yes, it’s my normalized score. What’s an 87 versus a 92? What’s it going to cost me?

So, I’m a big believer in risk quantification. Because if I can quantify the potential risk of those devices, those assets, in dollars, and then I can talk about what it’s going to cost to mitigate that, now I actually have a chance of getting funding. Are you going there? When you think about the prioritization stuff, are you heading that way and how are you thinking through that? Because I think that’s a really… it’s still a challenging part in the industry.

J.J. Guy: Yep. So, we are blessed and cursed because for this particular problem, we’ve got, I’ll say, more data than anybody else.

Matt Alderman: That’s not a data problem. It’s context problem. I told you that earlier.

J.J. Guy: Yeah, so we’ve got more context than anybody else.

Matt Alderman: Right, that’s good.

J.J. Guy: And the kind of contextualization that we can provide for any given vulnerability is absolutely incredible. 

And I generally approach this with our clients and the executives, and in CTEM, like Gartner in their framework has put scoping right up number one of defining very carefully what’s in scope for your program. And we make, as part of that, the prioritization framework. Let’s just talk on a whiteboard, “What do you care about? What are the most important vulnerabilities for you? And then the subset of devices in your network, applications in your network, what are the most important ones for you?” 

And just have a conversation, stay away from anything technical mumbo-jumbo, blah, blah, blah, and be able to express that, I want to say, at the high altitudes that the CEOs and CFOs work with. So then there’s good alignment between the security executives and the rest of the board on strategic objectives. 

Once those strategic objectives are set, then it becomes a data problem. And from my optic, it’s our responsibility as a company to have the data first available, understand the existing systems throughout the enterprise that have got an inventory of all that various data, and then we’ll aggregate it, collect it, normalize it, get into the right spot, and then be able to have a query engine over the top of that to be able to express those strategic objectives in what ends up being effectively just a query.

Matt Alderman: Got it.

J.J. Guy: Once you turn that in, once you make the connecting flight between those strategic objectives, that’s how you want to prioritize, and something technical, if there’s any gap, then it becomes an engineering challenge and very focused, very narrow, very clear and acute, and tied to the strategic objective. And once all that….

Matt Alderman: So, then the CISO’s talking about whether they’re meeting the strategic objective, not about the vulnerabilities and number of CVE counts and all the other stuff we get wrapped around the axle.

J.J. Guy: Right. And then once you’ve got, it’s like any other challenge, you’ve got the objective set, the clear strategic objectives, and you derive from that all the various projects, once all that’s up in place, you can have a dashboard where all the data’s rolling up. You understand and have confidence in every layer of the stack because you know you have a solid asset inventory, you know you’ve got all the controls in place. You manage those to whatever your threshold is, 90%, 95%. You’ve got all the vulnerabilities, and then the prioritization of those vulnerabilities align with the program’s strategic objectives. We do that every day.

Matt Alderman: Awesome. Now, just one quick thing, what’s next in this space?

J.J. Guy: It’s going to be interesting to watch. I mean, Gartner is publishing a new Magic Quadrant on exposure assessment platforms.

Matt Alderman: Got it.

J.J. Guy: That’ll be coming out in the next several weeks. So that’s going to put a stake in the ground for the set of products in market today, and at least Gartner’s opinion on how they align with the objectives of the CTEM framework.

Matt Alderman: Got it.

J.J. Guy: Now, there’s a lot happening in terms of all the rest of us working with Gartner to try to get through what’s right, and all of this explosion that we’ve had over the last 15, 20 years to try to make up for the gaps in legacy VA products and how all those fit. 

We’ve got our own path, of course. I mean, we’re taking a data-first approach, bottoms-up. And you hear me, I’m an engineer by trade. I like to have high confidence in every brick in the wall, as you build it up with a strong foundation one at a time. So, we’ll continue to take that approach of being proud and confident and high-quality data that we can give confidence in, and continue to expand the dataset to be able to provide even more contextualization and more ability to tie into a different set of considerations and those strategic objectives.

Matt Alderman: Awesome. Thank you, J.J., for joining.

J.J. Guy: Matt, as always, it’s a pleasure. Thank you.

Matt Alderman: If you want to learn more, please visit securityweekly.com/sevcobh. For all your Black Hat viewing needs, you can visit securityweekly.com/blackhat.

Thanks again, J.J.

J.J. Guy: Thanks, Matt. It was fun.

Matt Alderman: Stay tuned. We’ve got more interviews live from Black Hat 2025.