Performance Degradation Observed When Upgrading org.graalvm.js from 22.0.0.2 to 23.1.7

[Tonys-L]

Describe GraalVM and your environment :

• GraalVM version or commit id if built from source: [23.1.7]
• CE or EE: [CE]
• JDK version: [JDK17]
• OS and OS Version: [Windows 10]
• Architecture: [x86]
• The output of java -Xinternalversion:

Describe the issue
We are using GraalVM's JS engine to execute dynamically provided JavaScript scripts. Our backend uses a thread pool with 10 threads to execute script tasks. Since org.graalvm.polyglot.Context is not thread-safe, we create a new Context for each script execution.

After upgrading from org.graalvm.js:22.0.0.2 to org.graalvm.js:23.1.7, we observed a 50% drop in performance. Through profiling and analysis, we traced the bottleneck to the Context initialization phase.

🔍 Performance Bottleneck

From the thread dump, we found that script execution threads are blocked at:

com.oracle.truffle.polyglot.InstrumentCache.load(InstrumentCache.java:144)

This is a synchronized block, meaning that all 10 threads are serialized during Context creation, causing contention and performance degradation.

"DeviceServiceExecutor-22" #318 prio=5 os_prio=0 cpu=27730.25ms elapsed=88.06s tid=0x0000007fa0017900 nid=0x6736 waiting for monitor entry  [0x0000007ed9a22000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at com.oracle.truffle.polyglot.InstrumentCache.load(InstrumentCache.java:144)
	- waiting to lock <0x00000000f5062938> (a java.lang.Class for com.oracle.truffle.polyglot.InstrumentCache)
	at com.oracle.truffle.polyglot.PolyglotEngineImpl.initializeInstruments(PolyglotEngineImpl.java:931)
	at com.oracle.truffle.polyglot.PolyglotEngineImpl.<init>(PolyglotEngineImpl.java:285)
	at com.oracle.truffle.polyglot.PolyglotImpl.buildEngine(PolyglotImpl.java:355)
	at org.graalvm.polyglot.Engine$Builder.build(Engine.java:756)
	at org.graalvm.polyglot.Context$Builder.build(Context.java:1925)

🧩 Root Cause Analysis

We traced the issue further into the internal loading mechanism:
EngineAccessor.locatorOrDefaultLoaders() ➜ TruffleLocator.loaders() ➜ Truffle.getRuntime().getCapability(TruffleLocator.class) ➜ DefaultTruffleRuntime.Loader.load()

In GraalVM 22.0.0.2, the DefaultTruffleRuntime.Loader.load() method calls ServiceLoader.load() once.

In GraalVM 23.1.7, it calls ServiceLoader.load() twice, which increases the overhead, especially under high concurrency.

🚨 Impact

All threads are serialized when creating Context instances.
Performance degradation is significant under concurrent execution.
This issue makes it difficult to scale script execution in high-throughput environments.

❓ Question

How can we resolve this performance bottleneck in a multi-threaded environment?
More specifically:
Is there a way to avoid synchronization in InstrumentCache.load()?
Can we reduce the number of ServiceLoader.load() calls in DefaultTruffleRuntime.Loader.load()?
Are there safe ways to reuse Context or pre-initialize engines to avoid contention?

[selhagani]

Hi @Tonys-L,

Thank you for reaching out to us. There's actually a new version of GraalJS which is GraalJS 24.2.2. Could you please try testing using this version?

If you're still noticing the performance degradation after the upgrade then could you please share a reproducer alongside the steps needed to encounter the issue please?

[chumer]

Hi Thank you for the report and I am sorry you are seeing a regression.

How can we resolve this performance bottleneck in a multi-threaded environment?

I would recommend to follow this guide:
https://www.graalvm.org/latest/reference-manual/embed-languages/#code-caching-across-multiple-contexts

It shows how you can share the engine across multiple contexts. Especially if you create many contexts, I would strongly suggest that.

Is there a way to avoid synchronization in InstrumentCache.load()?

Yes. I think we could optimize this, but it is not needed if you share the engine, as this code only runs once per engine creation. Would you mind sharing a fully runnable example that shows your performance problem? Especially when it comes to multi-threaded problems they are hard to reproduce just from description alone. Thank you!

Can we reduce the number of ServiceLoader.load() calls in DefaultTruffleRuntime.Loader.load()?

See previous answer.

Are there safe ways to reuse Context or pre-initialize engines to avoid contention?

Assuming that you are running a single tenant per process (security wise) then you can typically share a single engine across all contexts of a process. They are fully thread-safe and this is also the situation we are optimizing for.

Depending on your situation you can also reuse contexts. For example context pooling is quite commonly done:
Here is an example:
https://github.com/graalvm/graal-languages-demos/blob/main/graalwasm/graalwasm-micronaut-photon/src/main/java/com/example/PhotonPool.java#L22

GraalVM version or commit id if built from source: [23.1.7]
JDK version: [JDK17]

You are using a partially support version of Truffle and Graal.js which does not support runtime optimization. For Graal.js 23.1 I would recommend to upgrade to GraalVM JDK 21 for runtime optimization support, if possible.

More information here:
https://www.graalvm.org/latest/reference-manual/embed-languages/#runtime-optimization-support

Hope this helps!

[Tonys-L]

Hi @selhagani , @chumer . Thank you for your response.

This project ( test-graalvm-js-context-build) benchmarks GraalVM JS performance across two branches:

• master (version 24.2.2)
• graalvm.js_22.0.0.2 (version 22.0.0.2)

I conducted a throughput benchmark using JMH for the following scenarios:

• Directly creating Context instances
• Creating Context instances with a shared Engine
• Using ThreadLocal to cache Context instances (the approach currently used by our project team)

---

Test Environment

# JMH version: 1.36
# VM version: JDK 17.0.13, OpenJDK 64-Bit Server VM, 17.0.13+11
# VM invoker: D:\Java\eclipse_temurin_17.0.13\bin\java.exe
# VM options: -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -javaagent:D:\tools\idea\ideaIU-2023.1.1.win\lib\idea_rt.jar=35822:D:\tools\idea\ideaIU-2023.1.1.win\bin -Dfile.encoding=UTF-8
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 5 s each
# Measurement: 5 iterations, 6 s each
# Timeout: 10 min per iteration
# Threads: 10 threads, will synchronize iterations
# Benchmark mode: Throughput, ops/time

Test Result

24.2.2

Benchmark                       Mode  Cnt     Score    Error   Units
Main.forceCreateContext        thrpt    5     2.821 ±  0.156  ops/ms
Main.shareEngineContext        thrpt    5    10.066 ±  0.632  ops/ms
Main.threadLocalCachedContext  thrpt    5  2807.094 ± 80.844  ops/ms

22.0.0.2

Benchmark                       Mode  Cnt     Score     Error   Units
Main.forceCreateContext        thrpt    5     3.888 ±   0.176  ops/ms
Main.shareEngineContext        thrpt    5    10.696 ±   0.413  ops/ms
Main.threadLocalCachedContext  thrpt    5  2854.559 ± 103.283  ops/ms

Conclusion

• The time taken to create a Context in version 22.0.0.2 is shorter than in version 24.2.2.
• Using a shared engine improves performance by 3-4 times, with minimal performance difference between versions 22.0.0.2 and 24.2.2.
• Caching Context instances with ThreadLocal results in a dramatic performance improvement (qualitative leap).

[Tonys-L]

Sorry, I had forgotten to submit the graalvm.js_22.0.0.2 branch earlier, but it has now been submitted.

[chumer]

Caching Context instances with ThreadLocal results in a dramatic performance improvement (qualitative leap).

Please keep in mind that this shares global state of the JS context. So it is semantically not equivalent to the other solutions.

[Tonys-L]

Oh yes, in our actual projects, after the JS execution completes, we clear the defined JS variables by setting them to undefined.