How to Detect and Defeat the Log4j2 Vulnerability with Deepfence

How to Detect and Defeat the Log4j2 Vulnerability with Deepfence
January 10, 2022

Introduction to log4j2 mitigation

The log4j2 vulnerability like the OpenSSL Heartbleed and Apache Struts vulnerabilities that came before it are poignant reminders to digital businesses that it's not just enough to respond to a vulnerability by redeploying applications once a patch is available, you also have to be able to discover instances of the vulnerability being exploited in real time in your production platform and stop them. In this tutorial, we’ll show you how to use Deepfence ThreatMapper and ThreatStryker to help you do just that.

Deepfence ThreatMapper is an open source security observability platform that hunts for vulnerabilities – including log4j2 – in applications in production across containers, Kubernetes, clouds, serverless environments, VMs, and bare metal, and then ranks them based on their risk of exploit. ThreatMapper eliminates the noise and false positives generated by scanning tools by further calculating the risk of exploit for each of these vulnerabilities, so that you can target the issues that present the greatest risk to the security of your applications. 

Deepfence ThreatStryker extends ThreatMapper and measures the unique behavior of attackers within your application – capturing network traffic, identifying when attacks are in progress, and stopping them immediately by carrying out the security policies you define.

In this tutorial, we show you a demo environment with a container image that has the log4j2 vulnerability in a Java application, how it can be remotely exploited by an attacker, and how to use ThreatMapper and ThreatStryker to detect and protect against log4j2 attacks in real time.

How to find your most exploitable vulnerabilities like log4j2 with ThreatMapper

After installing ThreatMapper and starting your first vulnerability scan (see steps 1 through 4 in this tutorial), you can view your most critical vulnerabilities on the Most Exploitable Vulnerabilities report in the ThreatMapper console.

In the image below, we show you the top half of the Most Exploitable Vulnerabilities report for our scan of the demo environment. From all of the potential vulnerabilities found, ThreatMapper highlights the ones that pose the greatest risk and show the attack paths by which they may be exploited.

Below you can see the log4j container sitting behind HAproxy (called out in text for easy reference). This is the attack path that could be used to exploit the log4j2 vulnerability.

When you mouse over the critical node in the attack path above, you’ll see additional information about the vulnerability (such as node deepfenceio/log4j-vulnerable-app:latest has Attack Vector type: network with Top CVE: CVE-2021-44228, accessible on Port: 8080).

You can also see additional information at a glance as you scroll down the page to your ranked list:

When you click a particular vulnerability, you can dive deeper and see even more information in both table and JSON views.

In our demo, we clicked our #1 ranked vulnerability which is log4j to see the information provided. ThreatMapper pulls in more than 50 different threat feeds and gives you the most up to date one for a particular vulnerability where possible. ThreatMapper also includes links to external resources to get late breaking information about the vulnerability. 

In the table view, for example, you can see the offending package, where it resides (i.e the container it's running in), the CVSS score, the release in which it was fixed, and other data that helps you understand the reasons this is a critical vulnerability and how to fix it. 

How to identify log4j2 attacks in progress and stop them with ThreatStryker

Knowing the presence of the most exploitable vulnerabilities like log4j2 is very useful information. You can prioritize the remediation of these vulnerabilities with your dev team, and you may wish to apply temporary WAF rules to limit how attackers can access and exploit the vulnerabilities.

But, are attackers already one step ahead of you?

  • What are attackers doing to try and find vulnerabilities? 
  • Have they already found an initial exploit? 
  • Are they using that to spread laterally and gain additional control within your application? 
  • How can you stop them immediately?

That’s where ThreatStryker comes in. ThreatStryker builds on the static analysis performed by ThreatMapper, adding runtime inspection to determine what attackers have been doing to exploit potential vulnerabilities.

ThreatStryker captures two main types of runtime telemetry from your applications:

  • Indicators of Attack: Deep packet captures that may indicate reconnaissance, lateral spread, command-and-control, exfiltration, and other attack activities
  • Indicators of Compromise: On-host captures of process and file system anomalies that may indicate an attacker has successfully exploited a vulnerable component 

ThreatStryker’s packet capture runs in the background, sampling traffic from all nodes and containers. For the purposes of this demo, we’ll start a packet capture manually, selecting a specific process to monitor:

manual packet capture ThreatStryker

ThreatStryker’s packet capture is performed without a proxy, and without a kernel module. ThreatStryker uses eBPF to grab traffic, sample it, and then match that against runtime threat feeds to identify recon and other types of malicious traffic.

After the traffic analysis and inspection has gathered sufficient data, you can look at the alerts that have been generated as a result.

ThreatStryker models attack behavior using an extended version of the Cyber Kill Chain framework. The Intent Stage maps an event to a stage in the Cyber Kill Chain, and the attack class types and severity provide additional context. Then, clicking into one of the alerts in the list at the bottom reveals deeper information. 

For example, below is what we see after clicking an attempted administrator privilege gain alert. The bottom half includes a Sankey diagram that displays the malicious activity as it unfolds over time so that you can hunt in, apply policies, and understand exploits as they're happening. The top half includes a verbose listing of information gathered that triggered the alert, in both Table and JSON views, that you can scroll through vertically to see a wealth of information including the packet payload and the offending string. In the image below, we paused the vertical scroll at the infamous ${jndi:ldap string that’s a signature of the log4j2 attack.

The example above shows inbound (or ingress) traffic inspection that led to the exploitation alert of an attempted administrator privilege gain of the log4j2 vulnerability. But inspecting inbound traffic alone is not enough. ThreatStryker also inspects outbound (or egress) traffic and a wide range of on-host anomalies.

Let’s look at another alert, resulting from on-host inspection: Suspicious tracing event process strace /bin/ls -l:23408 trying to ptrace: 23409

Somehow, a process is trying to perform a system trace; an uncommon event in normal operation and a strong indicator that an attacker has found a way to compromise the container or application:

In isolation, each of these events is potentially concerning, and indicates possible stages in the evolution of an attack. When ThreatStryker assigns intent and classifications to an event, it correlates these indicators of attack and indicators of compromise over time, to tell the story of how an attack is evolving and how the risk of an exploit is growing.

You can track the ThreatStryker event logs over time using a range of monitoring tools, or you can configure security measures within ThreatStryker to step in and neutralize an attack in progress once it reaches a threshold of certainty.

Defining security policies – both quarantine policies and network policies – to stop attacks in progress starts in the Protection Policies dashboard in ThreatStryker, shown below.

In our demo environment, we do not yet have any protection policies set up, so let’s create a few. To do so, click the Network Policies tab and then click on the Add Policy button in the upper right corner. You’ll be presented with a menu of options where you can select severity, alert type, intent type, and more.

Then you can block both inbound and outbound traffic for an amount of time you choose – custom or indefinitely.

But just blocking an attack from proceeding only solves half of the problem. ThreatStryker can also identify events on a host that may indicate it has been compromised, including strange file system modifications, process activity, or other behavior that indicates in some way that the host or workload has been tainted by attack behavior. At this point we employ quarantine policies. Quarantine policies enable you to freeze, block, terminate, or delete the host to your preference in order to prevent the tainted effect from spreading any further.

To create a quarantine policy, head to the Quarantine Policies tab on the Protection Policies dashboard and click Add Policy to begin. Here’s an example policy we created in our demo environment.

Conclusion and What’s Next

In summary, ThreatMapper is an open source platform that learns the topology of your applications, identifies potentially vulnerable workloads, and scans those against over 50 different threat feeds, and gives you real time, accurate information about the vulnerabilities that each of those workloads may exhibit. ThreatMapper, then correlates that with the attack surface telling you which vulnerabilities like log4j2 are at the greatest risk of exploit so that you know what to fix first. 

ThreatStryker builds on the vulnerability map and information that ThreatMapper creates in order to identify attack behavior and block and neutralize attackers in their steps, preventing them from exploiting your applications.

Next steps: