NoClassDefFoundError for org.apache.logging.log4j.spi.ExtendedLogger in Tomcat 11 with Java 21

[Dushyant-GitHub]

Problem Description

Our web application, which runs in Tomcat 11 and Java 21, uses SLF4J + Logback for logging. However, it allows dynamically importing extensions along with JARs at runtime using the server classloader. One of these extensions depends on log4j-api-2.17.1.jar, which is correctly present in the classpath.

After deploying the extension, calling a service that uses Log4j results in the following error:

java.lang.NoClassDefFoundError: org/apache/logging/log4j/spi/ExtendedLogger

The error mentioned above is being received by executing this line Logger logger = LogManager.getLogger(); in extension service.

However, other Log4j classes, such as LogManager and Logger, are loading without issues.

Environment Details

• Application Type : WAR-based Java web application

• Application Server : Apache Tomcat 11

• Java Version : 21

• Logging Framework in Main App : SLF4J with Logback

• Extension Support : Allows dynamic loading of JARs at runtime using the server classloader

• Problematic Dependency : log4j-api-2.17.1.jar (used by an extension)

Debugging Steps & Findings

1. Verified that log4j-api-2.17.1.jar is present in the classpath and contains org/apache/logging/log4j/spi/ExtendedLogger.class.
2. Checked for multiple versions of log4j-api dependency in our application. Result : Only one instance of log4j-api-2.17.1.jar is found.
3. Printed the class loaders for Log4j classes:

   System.out.println(org.apache.logging.log4j.LogManager.class);
   System.out.println(org.apache.logging.log4j.Logger.class);
   System.out.println(org.apache.logging.log4j.spi.ExtendedLogger.class);  
   

Result:
LogManager and Logger printed correctly.
ExtendedLogger.class throws NoClassDefFoundError.

Additional Notes

• Same application worked in Tomcat 9 without requiring a restart.

• Tomcat 11 requires a restart for ExtendedLogger to be found.

• After restarting Tomcat 11, everything works correctly.

Request for Help

We need guidance on :

• Why is ExtendedLogger missing, even though other Log4j classes load fine?

• Does Tomcat 11’s classloader affect Log4j differently compared to Tomcat 9?

• Are there additional configurations we should try in Tomcat 11?

Would appreciate any insights from the community!

[ppkarwasz]

However, it allows dynamically importing extensions along with JARs at runtime using the server classloader.

Can you clarify more what do you mean by "server classloader"? Are you talking about the parent of all web-app classloaders (see Tomcat classloading?

How do you dynamically add JARs to a running classloader? Classloader are usually designed to be immutable.

ExtendedLogger.class throws NoClassDefFoundError.

Do you have a root cause for the NoClassDefFoundError? This error is thrown for a variety of reasons, for example once the classloader fails to load a class, all successive attempt to load that class will throw a NoClassDefFoundError.

If o.a.l.l.spi.ExtendedLogger were to dynamically appear in a classloader, but the classloader already attempted to load the class, a NoClassDefFoundError will be thrown.

Some libraries, for example spring-jcl, try to load o.a.l.l.spi.ExtendedLogger to test if log4j-api is present on the classpath.

[Dushyant-GitHub]

Can you clarify more what do you mean by "server classloader"? Are you talking about the parent of all web-app classloaders (see Tomcat classloading?

When I mentioned "server classloader," I was referring to the classloader obtained via:
Thread.currentThread().getContextClassLoader()
This is the classloader that our application is using when dynamically loading extensions at runtime.

How do you dynamically add JARs to a running classloader? Classloader are usually designed to be immutable.

We use a custom URLClassLoader reflection-based approach to dynamically load JARs at runtime. Below is the code we are using:

 URLClassLoader webappClassLoader = (URLClassLoader) Thread.currentThread().getContextClassLoader();
 Method method = URLClassLoader.class.getDeclaredMethod("addURL", parameters);                                   
 method.setAccessible(true);                                                                                      
 method.invoke(webappClassLoader, new Object[] { jarUrl }); 

With this approach, we are successfully loading JARs at runtime without any immediate errors or exceptions. We have verified through debugging that the log4j-api JAR is also loaded properly into the classloader.

However, despite being loaded, we still encounter NoClassDefFoundError for org.apache.logging.log4j.spi.ExtendedLogger.

The same application worked without issues on Tomcat 9. However, after upgrading to Tomcat 11, we observed this NoClassDefFoundError.

Do you have a root cause for the NoClassDefFoundError?

The issue disappears after restarting the Tomcat 11 server. Once the server is restarted, everything works fine, and Log4j classes load without any errors.

If o.a.l.l.spi.ExtendedLogger were to dynamically appear in a classloader, but the classloader already attempted to load the class, a NoClassDefFoundError will be thrown.

Before loading the extension (which contains log4j-api), we explicitly checked whether the class org.apache.logging.log4j.spi.ExtendedLogger was already present in the classloader using:

try {
    Class.forName("org.apache.logging.log4j.spi.ExtendedLogger");
    System.out.println("ExtendedLogger is already present.");
} catch (ClassNotFoundException e) {
    System.out.println("ExtendedLogger is NOT present before extension import.");
}

This check confirmed that the class was NOT present before loading the extension.
Even after successfully adding log4j-api to the classloader dynamically, we still get NoClassDefFoundError when trying to use Log4j, which is unexpected.

We would appreciate any insights into why this behavior might occur in Tomcat 11 and whether Log4j's classloading mechanism has specific dependencies or behaviors that could be affected by dynamic JAR loading.

[ppkarwasz]

If o.a.l.l.spi.ExtendedLogger were to dynamically appear in a classloader, but the classloader already attempted to load the class, a NoClassDefFoundError will be thrown.

Sorry, this statement is incorrect: classloaders don't have a negative cache of classes that don't exist.

However, Tomcat introduced one in its October 2024 releases (see 9.0.97 release notes for example):

Cache not found results when searching for web application class loader resources. This addresses performance problems caused by components such as java.sql.DriverManager which, in some circumstances, will search for the same class repeatedly. In a large web application this can cause performance problems. The size of the cache can be controlled via the new notFoundClassResourceCacheSize on the StandardContext. (markt)

This is why your code fails on newer Tomcat releases: by calling Class.forName("org.apache.logging.log4j.spi.ExtendedLogger"), Tomcat will cache the error. As I mentioned above, testing for o.a.l.l.spi.ExtendedLogger is something libraries like spring-jcl do.

I didn't test it, but setting notFoundClassResourceCacheSize to 0, should solve your problem:

<Context notFoundClassResourceCacheSize="0">
  ...
</Context>

[ppkarwasz]

A side question:

I know that upgrading is painful, especially since we had approximately one new release per month last year, but 2.17.x is a very old release and we don't maintain it anymore. The upgrade risk from 2.17.1 to 2.24.3 should be minimal, but it is still there, since we allow for behavioral changes in minor releases (change in default values, stricter interpretation of configuration errors, for example).

What could we do, in your opinion, to motivate users to use maintained versions of Log4j (currently only the last minor release of 2.x)? What are your main reasons for an upgrade (new features, known vulnerability, maintenance status of the major/minor branch)? Would an LTM (long term maintenance) branch be something you would upgrade to?

Note: I am using "maintenance" instead of "support", since the level of "support" for all Log4j releases is the same: we answer questions if we remember how things worked in that release (even for Log4j 1). Of course more recent releases offer better "support". 😉

[Dushyant-GitHub]

Thank you for the clarification.

You were absolutely right—this issue was introduced in newer Tomcat versions due to the caching of "not found" class resources, as mentioned in the Tomcat 9.0.97 release notes. Since our application dynamically loads extensions and their dependencies at runtime, Tomcat's negative cache was causing a NoClassDefFoundError for org.apache.logging.log4j.spi.ExtendedLogger.

We were able to resolve the issue by explicitly setting:

<Context notFoundClassResourceCacheSize="0" />

After making this change, our application correctly loads the dynamically added JARs without requiring a server restart.

Thanks again for pointing us in the right direction!

[Dushyant-GitHub]

A side question:

I know that upgrading is painful, especially since we had approximately one new release per month last year, but 2.17.x is a very old release and we don't maintain it anymore. The upgrade risk from 2.17.1 to 2.24.3 should be minimal, but it is still there, since we allow for behavioral changes in minor releases (change in default values, stricter interpretation of configuration errors, for example).

What could we do, in your opinion, to motivate users to use maintained versions of Log4j (currently only the last minor release of 2.x)? What are your main reasons for an upgrade (new features, known vulnerability, maintenance status of the major/minor branch)? Would an LTM (long term maintenance) branch be something you would upgrade to?

Note: I am using "maintenance" instead of "support", since the level of "support" for all Log4j releases is the same: we answer questions if we remember how things worked in that release (even for Log4j 1). Of course more recent releases offer better "support". 😉

We understand the importance of staying on maintained versions. In our case, the log4j-api 2.17.1 dependency is used within customer extensions, and the decision to upgrade ultimately lies with them. We will convey your recommendation to our customers so they can evaluate the upgrade based on their requirements and risk assessments. We’ll pass along this feedback to our customers and let them decide on the best approach for their use case.

Thanks again for your support!! :)