NopSec Analyzes the NVD for Their Annual Risk and Vulnerability Report

This is Recorded Future, inside threat intelligence for cybersecurity.

Dave Bittner:

Hello everyone, and welcome to episode 91 of the Recorded Future podcast. I’m Dave Bittner from the CyberWire.

Each year, security firm NopSec publishes their annual State of Vulnerability Risk Management Report, analyzing all of the vulnerabilities listed in the National Vulnerability Database, along with those uploaded to their own platform by their clients.

They consider a number of factors, including CVSS score, description, type, and vendor affected, to see which factors contribute to vulnerabilities being incorporated into malware and exploited in the wild. For this year’s report, NopSec invited Recorded Future to contribute their unique insights into how geopolitics affect government-run vulnerability databases.

Joining us today to take us through the research are Sanja Nedic, data scientist at NopSec, and Adrian Sanabria, VP of strategy and product marketing at NopSec. Stay with us.

Sanja Nedic:

State of Vulnerability Risk Management is NopSec’s annual report, where we analyze recent and historical vulnerability data, and we do this every year. The idea is to discover trends that may be useful for those working in vulnerability remediation.

Adrian Sanabria:

And every year we bring on a partner. This year it was Recorded Future. That adds a nice touch to the report. It’s not just NopSec saying these things. We find someone else to partner with, to add a little bit to whatever the topic is.

Dave Bittner:

What did Recorded Future bring to the table, in this case?

Adrian Sanabria:

I think context is a good thing to have. We’ve got all this data and we’re analyzing the data, but it’s really good to have some examples, some stuff from the real world. I think that’s really what they brought into this — some really interesting views into stuff that’s actually going on out there. How some of what we’re describing here, by looking at this data, is relevant in the real world.

Dave Bittner:

So let’s dig in. Before we get into some of the specific information, let’s just start with an overview here. What did 2018 look like compared to previous years?

Sanja Nedic:

So we found that recently, there has been an increase in the number of vulnerabilities published in the National Vulnerability Database. For example, 2017 had twice the number of vulnerabilities published relative to 2016. Now in 2018, we are also seeing vulnerabilities being discovered and published at about the same rate. So, we are getting around 40 new vulnerabilities each day, which is a lot of vulnerabilities to deal with. We are seeing some of the same old problems, the same types of vulnerabilities we’ve had before.

Dave Bittner:

Let’s dig in here some. The report does start off by describing what the National Vulnerability Databases are, and taking us through a narrative of how they vary from country to country.

Sanja Nedic:

Right. That was actually the Recorded Future contribution to this report. They basically warned us that exposure and risk may vary depending on where you live. It is not something that we commonly think about, but it turns out that ransomware and malware can be run differently depending on user location, or even on the keyboard layout on the computer you’re using. They talk about the vulnerability databases also being different in the United States versus China and Russia.

They also show that the delays in vulnerability reporting are different between the different databases. So, they find that the United States’ National Vulnerability Database usually lags around 27 days between the initial vulnerability reporting and the publishing of the vulnerability in the NVD, while in China, this delay is only around 12 days. On the other hand, Russia seems to be taking much longer. But the interesting thing is, the exposure risk may actually vary depending on where you live.

Dave Bittner:

Yeah, I thought that was an interesting point. It’s almost tongue in cheek, this notion that the best protection against malware might be having a Cyrillic keyboard.

Sanja Nedic:

Yes, even though it sounds funny.

Dave Bittner:

Take us through why that would be.

Sanja Nedic:

They give some examples of ransomware and malware, such as the Sigrun ransomware. That was specifically disabled if it runs on computers that have a Russian keyboard layout. One possible reason is that criminals face more risk and harsher sentences if operating in Russian IP space. So, some ransomware or malware makers, hackers, have even gone as far as to provide the decryption codes to victims who have such keyboards and who can prove that they are Russian.

Dave Bittner:

So this isn’t a matter of them being kinder to their countrymen. This is a case of, perhaps, there being a stricter law enforcement. I guess it would be easier to track them down if they’re local.

Sanja Nedic:

Right, I believe that’s what Recorded Future’s take on this was.

Adrian Sanabria:

Yeah and I’ve heard that said too, that they take a much more lenient stance against hacking activities toward other countries. Just as long as you’re not targeting the homeland.

Dave Bittner:

Well, let’s dive in here. One of the sections that looks at the overall vulnerability landscape — what did we find here?

Sanja Nedic:

So the overall vulnerability landscape, we are seeing about 8,000 new vulnerabilities being discovered each month since the beginning of 2017, which amounts to about 40 vulnerabilities per day. Some of the reasons for this could be that there are many more vulnerable devices nowadays. It could also be that the NVD is getting faster at publishing vulnerabilities, and that they’re dealing with the backlogs more. So, some of the vulnerabilities we saw published in 2017 had CVE years dating to previous years — in fact, around 26 percent of them did. So far in 2018, around 30 percent of the vulnerabilities published can actually be attributed to previous years. So, we are seeing a trend of an increasing number of vulnerabilities being published.

In terms of vulnerability types, we are seeing that about 70 percent of all vulnerabilities published since the beginning of 2017 fall into just 10 CVE categories. There are very few surprises here. This list includes buffer overflows, cross-site scripting, SQL injections, lack of input validation, and so on. We looked at all of these vulnerabilities in terms of their CVSS score, in terms of the vendors and products that they affect, and also in terms of their descriptions. And we try to see if these attributes correlate at all with their use and exploit code, and in malware and targeted attacks.

Dave Bittner:

So really, it’s seeing the usual suspects when it comes to the types of things that you would expect to see here. One of the things you point out here that’s interesting is, you pointed out that 38 percent of all CVEs are marked as high severity, but only two percent have reached the most dangerous state of being used in malicious code and commoditized in exploit kits. Can you take us through, unpack those statistics for us?

Sanja Nedic:

Sure. So, what we find is that there are a lot of vulnerabilities being discovered and published, but only a very small portion of them are actually dangerous in the sense that they have been weaponized, used in malware, ransomware, trojans, and exploits kits, and using targeted attacks. We find that, from our threat intelligence sources, we find that this portion is about two percent of all vulnerabilities so far. While the CVSS, the common vulnerability scoring system, base core would tell us that 38 percent of all vulnerabilities are high or critical.

So, what this means in practice, when coming to remediation, is that if you remediate based on the CVSS and you say you’re going to patch all the high and critical vulnerabilities, you will be looking at many more vulnerabilities than you really need to look at. It is 38 versus 2 percent. We also find that about a quarter of all vulnerabilities have some sort of exploit code written for them, so in places like the exploit database, but that this is also not sufficient to predict whether they will be used in malware or targeted attacks. There is a big discrepancy between the number of vulnerabilities that have some sort of proof-of-concept exploit code, versus those that are actually used in targeted attacks.

Dave Bittner:

Now, is there any sort of correlation of the types of vulnerabilities that most often get turned into malware?

Sanja Nedic:

This is why we looked at other attributes besides the CVSS. So, we found that certain vendors, for example, and certain products… Vulnerabilities related to those vendors and products are more likely to be used in malware and targeted attacks. We also found that certain descriptions can be indicative of these vulnerabilities being weaponized.

Dave Bittner:

Yeah it’s interesting, you did some natural language processing in here to sort of take apart these descriptions. Can you take us through what was that process like, and what did you discover?

Sanja Nedic:

Right, so we looked at the descriptions of all the historical vulnerabilities so far to see if the language used in descriptions can be used to predict whether a vulnerability, an annually released vulnerability, will be used in malware or targeted attacks. What we found is that overall, as I mentioned, it’s under two percent that get weaponized. But if you focus on certain descriptions, this percentage gets much higher. So vulnerabilities that have certain words, or combinations of words, are more likely to be targeted.

For example, vulnerabilities that have server 2008, or Microsoft Windows, or Internet Explorer, or Adobe Flash in their description are between four to five times more likely to be targeted than a randomly chosen vulnerability. We also looked at combinations of more than two words in the descriptions. So we find that vulnerabilities that involve remote attackers executing arbitrary code are much more likely to be used in targeted attacks. So we think that incorporating descriptions as a factor when determining the score, the risk of the vulnerability, is important.

Dave Bittner:

I guess I’m trying to understand what the cause and effect is here. When you’re looking at the words and the frequency of words, is it that the bad guys are looking for these words to decide what they want to exploit? Or is it that the stuff that they’re exploiting happens to have these words?

Adrian Sanabria:

I think they actually work from it backwards from how we do it. I forget the guy’s name, but a guy working at Microsoft wrote a blog post once and said, “Defenders think in lists and attackers think in graphs.” Basically, what he’s saying is that our perspectives are different, and we have to shift those and look at things from the attacker’s point of view. They’re looking for something … They have some … It’s like looking for a house. They know exactly what they’re looking for. They have criteria based on what they want to attack. They’re looking for a bang for their buck, so, something that’s going to be very common. So, Windows Server 2008, a very common sever, tends to have a lot of security vulnerabilities, and sometimes people don’t patch them. So that’s a juicy target for them.

We haven’t done as good a job codifying what looks attractive to the attacker. The way that we came at the scoring system doesn’t really take that into account. One of those is revealed in that word “analysis.” The fact that remote code execution is very attractive … Because in most of these cases, they don’t want to just attack servers or take them down, they want to control them. They want to use them to pivot to other high value systems. For that, you have to understand what the vulnerability does, and what the attack factor is.

This is one of my frustrations, that you can have something that has the highest score and it gives you a remote code execution. You can also have another thing that’s at the highest CVSS score, but the impact is denial of service. In other words, it makes a piece of software crash. The outcome is very different, as different as can be. Being able to execute code and just making something fall over are very different outcomes. Maybe both are useful to attackers in different cases, but the fact that we don’t distinguish between them with that score, or even outside that score, that we have to look at the description to glean those details, I think, is a problem with this system.

But there are also aspects of the CVSS that they can’t score for us. There’s the temporal score and the environmental score, which have to be filled out by whoever is using the CVSS. So the enterprise, or the business, or whoever’s scoring these vulnerabilities. It’s not really something that they’re ever going to have time to do. If we’re using the system right, companies are going through and putting an importance value, some kind of value, on every single asset they have, and custom scoring every single vulnerability they have. These are millions, in some cases. It’s just not going to happen.

Dave Bittner:

Now one of the things that you did in the report is, you went through your own client data at NopSec. So what did that dive reveal to you?

Sanja Nedic:

What we found with our client data was very similar to the overall trends, except that the inadequacy of the CVSS score was, I’d say, emphasized. Because it turns out that our clients are having more than half of their vulnerabilities being ranked as high severity based on the CVSS. We also saw around 93 unique vulnerabilities for our clients in the financial industry. If you multiply that by the number of assets that a financial organization may have, you get a very large number. We have also seen scans that had more than 300,000 unique vulnerabilities. In terms of the vulnerabilities we saw in our clients’ scans, we found around 10 to 11,000 different CVEs just over the last year alone. In terms of the vulnerable vendors, we were seeing the same of theirs as we see overall. The number one source of vulnerabilities was Microsoft, but this varied depending on the industry sector.

Dave Bittner:

You listed the top 20 CVEs, but one of the interesting things was the distribution of the types of vulnerabilities within those. Take me through that.

Adrian Sanabria:

One of the interesting things about the top 20 is that only about half of them are things that you can patch. So, traditionally when we think of vulnerabilities, like earlier, I was talking about exploits, I mean that’s a particular kind of vulnerability. It’s a software bug. But vulnerabilities can also be logical errors. They can be things like default credentials. Like if you’ve got a default username and password on something. People are going to take advantage of that, or if you configure something incorrectly. If you think it’s only available to your internal corporate network, but you mistakenly set up a firewall where it’s exposed to the outside. Maybe it’s a service that doesn’t even require credentials to get access to it. We see this often with different services. One in particular is Redis, which is a type of database that by default has no credentials. You can just connect to it and look at all the data.

I think it’s interesting that when we look at that top bit, everyone associates, “Oh I need to patch more quickly, and that’s going to solve a lot of this problem.” From what we’ve seen, that’s only going to solve half of your problem.

Dave Bittner:

One of the things you touch on here in the report is the use of machine learning when you’re calculating these things. Your risk scoring algorithms are making use of machine learning. What’s going on there? Why is that an important tool?

Sanja Nedic:

We approached the risk scoring problem as a supervised machine learning problem. We are trying to predict the probability that a particular vulnerability will be used in targeted attacks in the wild, or that there will be highly weaponized exploit kits and malware making use of it. We basically try to use historical data. So, we have a set of past vulnerabilities along with their features, and we use machine learning to find the best possible model that maps these vulnerabilities to their labels. Labels here being whether they have been used in malware or not.

It’s an automated way to find the best relationship between features or attributes of vulnerabilities that are likely to lead to a vulnerability being dangerous. We basically use the CVSS score in addition, with additional vulnerability information, from many threat feeds and social media to obtain that probability of a vulnerability being used in real-world attacks.

Adrian Sanabria:

I think a big part of Sanja’s job that a lot of people don’t think about is, when you talk about AI and machine learning, you think about something that you just set and forget. You set this up and this machine is just happily turning along, learning, getting smarter, solving normal problems. But our data is never pristine. It’s never 100 percent when you bring it in, like the threat intelligence we bring in, the exploit data. So Sanja’s constantly asking us questions, and we’re working together to try to determine, what’s the quality of this data? What bits and pieces in this data could throw off the bottle? Are we weighing things the correct way? Is there anything else we can put in there that would help?

An example of that, I actually did a webinar yesterday in preparation for this. This is actually the first time I’ve done the experiment, but I decided to tweet out the number of a CVE, like 2012-0002 I think it was, with the word malware in that same tweet. Sure enough, we looked at some threat intel out there, and the fact that I did that ended up in the threat intel, associated it more with malware than it was before. So, the fact that this is fallible, that this approach is something that people can mess with, they can falsify maybe, you have to take that into account. You have to actually take some of these and dive into it.

A good example of that is, when Sanja’s model predicts that a vulnerability is going to be used by malware in the future. We look at when that fails. It’s not 100 percent correct. We go back, and we look at those. We look at that list, and we think, “Okay, why did this not happen? Did the attackers just have enough? They just didn’t need to use these because they already had good exploits available to put into their malware, or did we miss something? Can we make the model better?”

Dave Bittner:

One of the things that the report noticed was that year-over-year, sometimes, the malware associated with certain vendors changes.

Sanja Nedic:

What we have noticed is that the big vendors, the big five like Microsoft, Google, Oracle, IBM, Apple, all tend to have a lot of vulnerabilities. This has stayed consistent over the years. On the other hand, we found that in terms of association with malware and exploit rates for vulnerabilities, there has been more change. That just because a vendor has a lot of vulnerabilities, does not necessarily mean that those are very dangerous. So for example, Microsoft you will see in the top five in terms of the malware, and the exploit kit rates as well, but the other four I’ve mentioned are not in the top five or top ten lists. Some vendors have a lot of vulnerabilities, others have a lot of very dangerous vulnerabilities.

Adrian Sanabria:

And interestingly, they’re Linux. A lot of them are Linux. Canonical is on the top. It’s associated with … They make Ubuntu, which a lot of people use. Red Hat’s there, who just got bought by IBM. Debian. So yeah, it’s not what you would think. This is one of the more interesting insights that I think came out of this year’s research.

Dave Bittner:

Looking at everything you gathered up for this year’s report, what are the conclusions? What are the take-homes this year?

Sanja Nedic:

I would say that a lot of the main takeaways is that the CVSS is not sufficient to quantify the risk a vulnerability poses, and that there are factors that are as important or more important. We incorporate a lot of additional information about vulnerabilities, and we use machine learning to prioritize the remediation of vulnerabilities that are likely to be exploited in the wild. Since, we have found that only a small fraction of vulnerabilities have functional exploits and even fewer are exploited in the wild.

So, our risk score comes from a model that’s trained to predict which vulnerabilities will be used in attacks. We find that by adding information from additional threat feeds and social media, information about vendors, information from natural language processing on the descriptions, we can prioritize better. We can predict which vulnerabilities will be used in targeted attacks better than the CVSS can, both within terms of false positives and false negatives.

Dave Bittner:

Yeah, interesting. Adrian, how about you?

Adrian Sanabria:

I mean, I tend to be a big picture kind of guy. I’m already thinking about next year’s report and where we want to go with this, and in our research too, because this is all based off of research that’s going on all year long. We don’t just do this just for the report. To me, the big thing is, how can we get ahead of this? Because in previous reports that we’ve done, we’ve shown that oftentimes, the bad guys are using these things before the patch is even available. Sometimes before people even know about it. The vulnerability isn’t even public. So there’s no amount of going faster or automating the remediation process that’s going to save you in those cases.

Can we take this prediction to the point where we can say … And I’d say there’s some obvious choices already. We touched on Microsoft and Adobe Flash, and things like that. We can already say, “Yeah, there’s probably going to be another bug. It’s going to be bad, and we should mitigate against that.” The mitigation is, Adobe is killing Flash entirely. That’s been such a big problem. But for others, can we predict, “Okay, this is such a big risk. We need to have some kind of mitigation in place that will stop attacks, or at least frustrate them, make them move on to something else before the vulnerability is even found.”

Dave Bittner:

Our thanks to Sanja Nedic and Adrian Sanabria from NopSec for joining us. The report is NopSec’s annual State of Vulnerability Risk Management Report.

Don’t forget to sign up for the Recorded Future Cyber Daily email, where every day you’ll receive the top results for trending technical indicators that are crossing the web, cyber news, targeted industries, threat actors, exploited vulnerabilities, malware, suspicious IP addresses, and much more. You can find that at recordedfuture.com/intel.

We hope you’ve enjoyed the show and that you’ll subscribe and help spread the word among your colleagues and online. The Recorded Future podcast team includes Coordinating Producer Amanda McKeon, Executive Producer Greg Barrette. The show is produced by Pratt Street Media, with Editor John Petrik, Executive Producer Peter Kilpe, and I’m Dave Bittner.

Thanks for listening.