Security researchers recently disclosed a zero-day remote code execution vulnerability, tracked as (Common Vulnerabilities and Exposures) CVE-2021-44228 in the popular Log4j2 Java logging library. Dragos Intelligence assesses with high confidence that this vulnerability will impact Operational Technology (OT) networks, based on the ubiquity of the library and the rapidly growing methodologies of exploitation.
Dragos Intelligence has observed both attempted and successful exploitation of the Log4j vulnerability in the wild and based on these observations already coordinated a takedown of one of the malicious domains used in early exploitation attempts.
This cross-cutting vulnerability, which is vendor-agnostic and affects both proprietary and open-source software, will leave a wide swathe of industries exposed to remote exploitation, including electric power, water, food and beverage, manufacturing, transportation, and more. Log4j is found in popular open-source repositories used in numerous industrial applications, such as Object Linking and Embedding for Process Control (OPC) Foundation’s Unified Architecture (UA) Java Legacy. Additionally, adversaries can leverage this vulnerability in proprietary Supervisory Control and Data Acquisition (SCADA) and Energy Management Systems (EMS) which make use of Java in their codebase.
Major OT vendors have begun to disclose the impact of this vulnerability on their software and equipment, and additional disclosures will continue as vendors work to identify the use of Log4j across their product lines. Unfortunately, the nature of the Log4j vulnerability makes it challenging to identify potentially impacted servers on a given network. While network-facing services written in Java are most obviously at risk, the vulnerability can technically impact any server (including back-end resources) where user-supplied data is processed and logged.
Vulnerable Log4j code allows adversaries to achieve remote code execution using specially constructed data request packets. The Java Naming and Directory Interface (JNDI) lookup feature of Log4j allows variables to be retrieved through JNDI, so when a new log entry is created and Log4j encounters a JNDI reference, it tries to connect to the (adversary controlled) supplied resource to fetch whatever is needed to resolve the variable. An adversary can send data containing a malicious JNDI reference to a server using any protocol which, when evaluated by Log4j, triggers the vulnerability.
Observed TTPs for CVE-2021-44228
While the Lightweight Directory Access Protocol (LDAP) attack vector received most of the initial attention, Dragos has observed attempted attacks using Domain Name System (DNS) and Remote Method Invocation (RMI) as well, and the list of potential protocols and methods for exploitation will continue to grow.
For an OT network incorporating robust segmentation, the risk from these protocols will be mitigated to an extent. However, OT elements such as SCADA/EMS configured for remote access may utilize LDAP for password management and therefore be vulnerable to exploitation, particularly by an adversary moving laterally within a network. OT networks with weak segmentation from business Information Technology (IT) networks will be most impacted by this vulnerability, even more so as additional protocols are discovered to enable exploitation.
The appearance of nested and obfuscated payloads to avoid Web Application Firewalls is a good example of early innovations in adversary Tactics, Techniques, and Procedures (TTPs). Applications using Log4j 1.x may be impacted if their configuration uses JNDI, however, we have not yet observed any adversaries actively exploiting this vulnerability. It is important to note that any project still using Log4x 1.x is running a deprecated and unsupported version that could contain other known vulnerabilities.
Dragos assesses with moderate confidence that as network defenders close off more simplistic exploit paths and advanced adversaries incorporate the vulnerability in their attacks, more sophisticated variations of Log4j exploits will emerge with a higher likelihood of directly impacting Operational Technology networks.
Accordingly, Dragos provides the following recommendations for immediate implementation.
Recommendations for Immediate Implementation
Given that Log4j has been a ubiquitous logging solution for Enterprise Java development for decades, Log4j has the potential to become a vulnerability that will persist within Industrial Control Systems (ICS) environments for years to come. Within ICS environments, Dragos anticipates OT operators will face one of three scenarios when working to mitigate Log4j vulnerabilities.
1 | A Java-based hardware or software solution that leverages Log4j and is under active development or support from the software vendor
As Log4j is frequently an embedded component of Java-based ICS hardware and software, OT operators must rely upon their vendors to provide patches to impacted software products. Also, it is important to note that the presence of Log4j in a Java-based software or hardware solution does not necessarily mean that it is exploitable.
2 | A Java-based hardware or software solution that leverages Log4j and is no longer supported by the software vendor
With the longer hardware and software refresh cycles we see in ICS environments, it is common to find ICS products that are no longer under active support or whose software vendors no longer exist. In these cases, remediation may not be possible, but steps can be taken to mitigate the risk.
These mitigation steps should be weighed against their potential to cause an unintentional operational impact. Defenders should test any mitigation steps that a vendor has not directly approved in a lab environment prior to running them in production. And finally, as no two Java software solutions are identical, we cannot guarantee that these mitigations will not have a negative impact on a Java application.
First, you will need to determine if the unsupported application utilizes the Log4j class.
- If the host system is Linux-based, you can run one or more of the following commands to detect the presence of Log4j. Note these commands will not be able to search within VMs, Docker containers or Kubernetes containers. You will need to interrogate virtual machines and containers separately.
dpkg -l | grep liblog4jdpkg -l | grep log4find / -name log4j-core-*.jarlocate log4j|grep -v log4js
- On Windows, you can run the following commands to find Log4j:
dir c: /S /b | findstr /C:"log4"
Within Java projects, it is possible to embed a Jar file such as Log4j inside of another Jar file. So, it is also possible the Log4j jar file could exist within a larger Jar file. A Jar file is just a zip file that contains various Java project files.
The Log4j project has proposed the following mitigations for use if updating Log4j is not an option.
For those who cannot upgrade to 2.15.0:
- In releases >=2.10, this behavior can be mitigated by setting either the system property log4j2.formatMsgNoLookups or the environment variable LOG4j_FORMAT_MSG_NO_LOOKUPS to true.
- For releases >=2.7 and <=2.14.1, all PatternLayout patterns can be modified to specify the message converter as %m{nolookups} instead of just %m.
- For releases >=2.0-beta9 and <=2.10.0, the mitigation is to remove the JndiLookup class from the classpath:
zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class
3 | A Java-based hardware or software solution that leverages Log4j and has been developed internally by the company
Java software solutions that have been created internally by an organization can be mitigated by upgrading Log4j 2.x to 2.15.0 (requires Java 8). The Log4j feature that enabled this vulnerability was Log4j’s allowance of Lookups to appear in log messages. As of Log4j 2.15.0 this feature is now disabled by default. While Log4j has provided an option to enable Lookups in this fashion, users are strongly discouraged from enabling it.
Details of the Log4j patch can be found in the following Log4j pull request: https://github.com/apache/logging-log4j2/pull/608
What’s Next?
As part of Dragos KP-2021-10, Dragos will be providing network-based detections for ICS-based active Log4j exploitation, including some of the obfuscated exploit attempts that we have observed. We expect KP-2021-10 to be available this week for Early Access (EA) deployment to the Dragos Platform. Prior to having KP-2021-10, Dragos Platform customers can leverage Log4j hunt queries that have been provided to the System Architect team to search for active Log4j exploitation within their ICS environments.
Given the ease of exploitation for this vulnerability, the scope of potentially impacted applications, and the time that will be required to apply comprehensive fixes across large organizations, Dragos recommends an “assume-breach mentality” and active hunting for post-exploitation activity. Dragos Intelligence continues to monitor ongoing exploitation attempts for any indications of intentional targeting of, or incidental impact on, Dragos customers and OT networks worldwide – which has already resulted in multiple notifications to companies regarding vulnerable systems to facilitate mitigation.
Ready to put your insights into action?
Take the next steps and contact our team today.