AWS Open Source Blog

Hotpatch for Apache Log4j

CVE-2021-44228 has made for a busy weekend trying to patch or mitigate the vulnerability in a pervasively used open source logging platform, Apache Log4j.

We recommend that those running affected applications upgrade Log4j to version 2.16 to address this vulnerability. However, this isn’t always quick, so folks from the Coretto team spent some time building a tool to hotpatch vulnerable log4j deployments. This tool is designed to hotpatch a running JVM using any Log4j 2.0+. The tool is idempotent, meaning that you can run this multiple times on the same JVM without changing the result past the initial application. This tool also looks for all the running JVMs and attempts to mitigate the vulnerability. You can also use this to patch shaded jars that include log4j as a dependency, and to patch multiple log4j instances on the classpath.

How does it work?

This tool injects a Java agent into a running JVM process. The agent attempts to patch the lookup() method of all loaded org.apache.logging.log4j.core.lookup.JndiLookup instances to unconditionally return the string “Patched JndiLookup::lookup()”. This is designed to address the CVE-2021-44228 remote code execution vulnerability in Log4j without restarting the Java process.

If you have the possibility of redeploying your Java processes, you can also use it as a static agent, meaning that you can include this patch in your runtime without directly logging in to your servers. Both of these modes of operation are described in the README.

Great – where can I get it?

This tool is available on GitHub today.

Caveats

So, as with all open source software, you’re using this at your own risk. Note that Hotpatch has been tested with JDK8 and JDK11 on Linux. On JDK17, only the static agent mode works. A full list of caveats can be found in the README.

Conclusion

This vulnerability is severe and due to the widespread adoption of Apache Log4j, its impact is large. We highly encourage you to review, patch, or mitigate this vulnerability. This tool may help you mitigate the risk when updating is not immediately possible.

Finally, thanks to the Corretto team for spending days, nights, and the weekend to write, harden, and ship this code.

Update as of December 14, 2021, 10:18PM GMT: This tool also addresses CVE-2021-45046 in addition to the original CVE-2021-44228. We continue to update the README and implement a host of new improvements for the tool. Thank you to the open source community for the ongoing feedback and encouragement.

David Nalley

David Nalley

David Nalley has been involved in open source for nearly two decades. He is the head of open source strategy and marketing at AWS and currently serves as the President of the Apache Software Foundation and serves on the Board of Directors for the Internet Security Research Group. You can follow him on Twitter (@ke4qqq).

Volker Simonis

Volker Simonis

Volker Simonis is a Principal Software Engineer in the Corretto team at AWS and represents Amazon in the Executive Committee of the JCP. He works on Java Virtual Machines since 2004 and is an OpenJDK Member, Reviewer and Committer right from the beginning. Before joining Amazon he worked for SAP, Sun Microsystems and the University of Tübingen from where he holds a master and PhD degree in Computer Science. He represented SAP in the Executive Committee of the JCP and was a member of the Java SE 9-13 JCP Expert Groups. Volker is a passionate and frequent speaker at conferences around the globe and can easily be contacted at @volker_simonis.