Officials say agencies have demonstrated more dedication than ever in addressing a bug with astronomical reach, but organizations are at the mercy of product vendors to issue the patches they need to implement.
If there is a silver lining to all the hours cybersecurity personnel spent over the holiday break—and will continue to spend months into the future—working to secure their systems from log4j vulnerabilities, it could be in how the government approaches the remediation of such bugs going forward.
“I just want to footstomp the unprecedented nature of how we’ve been handling this because there are so many products and being able to track all of this in one place, really with the help of crowdsourcing from vendors, from the research community, has helped to create some order in the chaos,” Cybersecurity and Infrastructure Security Agency Director Jen Easterly said. “We’re learning a lot of lessons about how to deal with something that is as widespread as this particular vulnerability.”
Easterly and CISA Executive Assistant Director Eric Goldstein briefed reporters Monday on efforts to protect federal agencies and private-sector critical infrastructure from the exploitation of vulnerabilities in log4j, a library of open-source code used to log data such as computer performance and security issues.
CISA is particularly concerned with one of three vulnerabilities—log4Shell—discovered in the library, which is incorporated into thousands of software products and could affect hundreds of millions of devices across the world. Opportunistic adversaries are already installing code into systems to enslave victims’ computers in botnets for crypto mining and other purposes. An attacker deciding to fully leverage the log4Shell vulnerability could gain deep access into a target network by sending just 12 characters through a text or email, Easterly said. They could then use that access to pilfer information, launch ransomware or conduct other nefarious activities.
Easterly said this ease of execution, the ubiquity of the vulnerability, and the case-by-case nature of the remediation process makes the log4Shell the most serious vulnerability she’s seen in her career.
“Because the library is embedded into thousands of separate products, the vendor for each individual product must produce their own unique patch,” she said.
The crowdsourcing approach she referred to has materialized in a GitHub-hosted list of software products— currently 2,900 lines long—that attempts to track which vendors are affected and which of those affected vendors have issued patches that end users like federal agencies can then apply.
“We’ve seen extraordinary attention on this vulnerability across federal agencies,” Goldstein said. “I think, frankly, the most dedicated focus that we have ever seen. Agencies have remediated thousands of vulnerable internet connected assets across their networks.”
But there could be thousands upon thousands more in what he said “will be a long tail of remediation.” And it’s not just a matter of trudging through a sea of potentially affected devices. Goldstein acknowledged instances where vendors disagreed about whether their products were actually affected and didn’t seem to be taking their responsibility seriously.
“So certainly we are taking that sort of scenario extremely seriously. We are working with our federal partners as well as critical infrastructure to understand which vendors are doing the right thing to provide needed support and mitigations to their customers,” he said. “Both vendors were made aware of the seriousness of the issue and the seriousness with which we CISA and the broader US government are taking it.”
Going forward, in addition to stressing the value of engaging security researchers, Easterly and Goldstein said the log4j vulnerabilities have heightened the importance of standardizing use of a software bill of materials.
“Certainly what we’ve seen over the past month as a result of log4j is catalyzing and accelerating these efforts, to include software bill of materials,” she said, noting members of the open source community will attend a meeting the White House plans to hold on the issue and that CISA aimed to host its own meeting with the community Monday.
The debate over what vendors of software products should have to disclose in a SBOM that consumers could ostensibly use to identify whether they may be exposed to problematic code libraries such as log4j is far from settled. The vendors assert, for example, that just because a vulnerability is present in their code, doesn’t necessarily mean it’s exploitable. They have also expressed concerns over being forced to reveal their intellectual property, which some CISA officials have said are unfounded.
Goldstein said the SBOM and other advanced technologies will be important to prevent vulnerabilities like those in log4j occurring to begin with.
“We’re working with the Department of Homeland Security to develop tools to perform the packet analysis of open source software, including leveraging artificial intelligence and machine learning to identify vulnerabilities and flaws, scale and minimize the need for individual code reviewers recognizing that if we can use technology at scale we’re going to again, reduce the number of weaknesses that we are seeing in this sort of software,” he said.
He also expects to see increased government support for the development and maintenance of open source software in general, referring to it as a “foundational building block of modern technology.”
“Certainly this vulnerability will catalyze further attention, further focus further investment, which will manifest better security,” he said.
