Skip to content

Commit 0572197

Browse files
committed
Switch to JDK 25 and Oracle GraalVM 25 in benchmarks + clean up sbt commands for benchmarks
1 parent 0805a0c commit 0572197

File tree

5 files changed

+30
-30
lines changed

5 files changed

+30
-30
lines changed

.jvmopts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,4 @@
44
-XX:InitialCodeCacheSize=512m
55
-XX:ReservedCodeCacheSize=512m
66
-Dfile.encoding=UTF8
7+
--enable-native-access=ALL-UNNAMED

README.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -668,35 +668,35 @@ Learn how to write benchmarks in [JMH samples](https://hg.openjdk.java.net/code-
668668

669669
List of available options can be printed by:
670670
```sh
671-
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/jmh:run -h'
671+
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/Jmh/run -h'
672672
```
673673

674674
Results of benchmark can be stored in different formats: *.csv, *.json, etc. All supported formats can be listed by:
675675
```sh
676-
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/jmh:run -lrf'
676+
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/Jmh/run -lrf'
677677
```
678678

679679
JMH allows running benchmarks with different profilers, to get a list of supported use (can require entering of user
680680
password):
681681
```sh
682-
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/jmh:run -lprof'
682+
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/Jmh/run -lprof'
683683
```
684684

685685
Help for profiler options can be printed by following command (`<profiler_name>` should be replaced by the name of the
686686
supported profiler from the command above):
687687
```sh
688-
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/jmh:run -prof <profiler_name>:help'
688+
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/Jmh/run -prof <profiler_name>:help'
689689
```
690690

691691
For parametrized benchmarks the constant value(s) for parameter(s) can be set by `-p` option:
692692
```sh
693-
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/jmh:run -p size=1,10,100,1000 ArrayOf.*'
693+
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/Jmh/run -p size=1,10,100,1000 ArrayOf.*'
694694
```
695695

696696
To see throughput with the allocation rate of generated codecs run benchmarks with GC profiler using the following
697697
command:
698698
```sh
699-
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/jmh:run -prof gc .*Reading.*'
699+
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/Jmh/run -prof gc .*Reading.*'
700700
```
701701

702702
Results that are stored in JSON can be easy plotted in [JMH Visualizer](https://jmh.morethan.io/) by drugging & dropping
@@ -705,13 +705,13 @@ of your file to the drop zone or using the `source` parameter with an HTTP link
705705

706706
On Linux the perf profiler can be used to see CPU event statistics normalized per ops:
707707
```sh
708-
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/jmh:run -prof perfnorm TwitterAPIReading.jsoniterScala'
708+
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/Jmh/run -prof perfnorm TwitterAPIReading.jsoniterScala'
709709
```
710710

711711
Also, it can be run with a specified list of events. Here is an example of benchmarking using 16 threads to check of CPU
712712
stalls:
713713
```sh
714-
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/jmh:run -t 16 -prof "perfnorm:event=cycles,instructions,uops_executed.core,uops_executed.stall_cycles,cache-references,cache-misses,cycle_activity.stalls_total,cycle_activity.stalls_mem_any,cycle_activity.stalls_l3_miss,cycle_activity.stalls_l2_miss,cycle_activity.stalls_l1d_miss" .*'
714+
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/Jmh/run -t 16 -prof "perfnorm:event=cycles,instructions,uops_executed.core,uops_executed.stall_cycles,cache-references,cache-misses,cycle_activity.stalls_total,cycle_activity.stalls_mem_any,cycle_activity.stalls_l3_miss,cycle_activity.stalls_l2_miss,cycle_activity.stalls_l1d_miss" .*'
715715
```
716716

717717
List of available events for the perf profiler can be retrieved by the following command:
@@ -721,7 +721,7 @@ perf list
721721

722722
To get a result for some benchmarks with an in-flight recording file from JFR profiler use command like this:
723723
```sh
724-
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/jmh:run -prof "jfr:dir=target/jfr-reports" -wi 10 -i 60 TwitterAPIReading.jsoniterScala'
724+
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/Jmh/run -prof "jfr:dir=target/jfr-reports" -wi 10 -i 60 TwitterAPIReading.jsoniterScala'
725725
```
726726
You will get the profile in the `jsoniter-scala-benchmark/jvm/target/jfr-reports` directory.
727727

@@ -734,7 +734,7 @@ sudo sysctl kernel.kptr_restrict=0
734734

735735
Then use command like this:
736736
```sh
737-
sbt -java-home /usr/lib/jvm/jdk-21 jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/jmh:run -prof "async:dir=target/async-reports;interval=1000000;output=flamegraph;libPath=/opt/async-profiler/lib/libasyncProfiler.so" -jvmArgsAppend "-XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints" --p size=128 -wi 5 -i 10 jsoniterScala'
737+
sbt -java-home /usr/lib/jvm/jdk-21 jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/Jmh/run -prof "async:dir=target/async-reports;interval=1000000;output=flamegraph;libPath=/opt/async-profiler/lib/libasyncProfiler.so" -jvmArgsAppend "-XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints" --p size=128 -wi 5 -i 10 jsoniterScala'
738738
```
739739
Now you can open direct and reverse flame graphs in the `jsoniter-scala-benchmark/jvmtarget/async-reports` directory.
740740

@@ -771,7 +771,7 @@ Perf events:
771771
Following command can be used to profile and print assembly code of the hottest methods, but it requires [a setup of
772772
`hsdis` library to make PrintAssembly feature enabled](https://builds.shipilev.net/hsdis/):
773773
```sh
774-
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/jmh:run -prof perfasm -wi 10 -i 10 -p size=128 BigIntReading.jsoniterScala'
774+
sbt jsoniter-scala-benchmarkJVM/clean 'jsoniter-scala-benchmarkJVM/Jmh/run -prof perfasm -wi 10 -i 10 -p size=128 BigIntReading.jsoniterScala'
775775
```
776776

777777
More info about extras, options, and ability to generate flame graphs see in [Sbt-JMH docs](https://github.com/ktoso/sbt-jmh)

benchmark-t16.sh

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
#!/bin/bash
2-
sbt -batch -java-home /usr/lib/jvm/jdk-24 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff jdk-24-t16.json .*' 2>&1 | tee jdk-24-t16.txt
3-
sbt -batch -java-home /usr/lib/jvm/jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff jdk-21-t16.json .*' 2>&1 | tee jdk-21-t16.txt
4-
sbt -batch -java-home /usr/lib/jvm/jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff jdk-17-t16.json .*' 2>&1 | tee jdk-17-t16.txt
5-
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-24 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-jdk-24-t16.json .*' 2>&1 | tee graalvm-jdk-24-t16.txt
6-
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-jdk-21-t16.json .*' 2>&1 | tee graalvm-jdk-21-t16.txt
7-
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-jdk-17-t16.json .*' 2>&1 | tee graalvm-jdk-17-t16.txt
8-
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-24 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-community-jdk-24-t16.json .*' 2>&1 | tee graalvm-community-jdk-24-t16.txt
9-
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-community-jdk-21-t16.json .*' 2>&1 | tee graalvm-community-jdk-21-t16.txt
10-
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-community-jdk-17-t16.json .*' 2>&1 | tee graalvm-community-jdk-17-t16.txt
2+
sbt -batch -java-home /usr/lib/jvm/jdk-25 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff jdk-25-t16.json .*' 2>&1 | tee jdk-25-t16.txt
3+
sbt -batch -java-home /usr/lib/jvm/jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff jdk-21-t16.json .*' 2>&1 | tee jdk-21-t16.txt
4+
sbt -batch -java-home /usr/lib/jvm/jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff jdk-17-t16.json .*' 2>&1 | tee jdk-17-t16.txt
5+
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-25 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-jdk-25-t16.json .*' 2>&1 | tee graalvm-jdk-25-t16.txt
6+
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-jdk-21-t16.json .*' 2>&1 | tee graalvm-jdk-21-t16.txt
7+
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-jdk-17-t16.json .*' 2>&1 | tee graalvm-jdk-17-t16.txt
8+
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-24 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-community-jdk-24-t16.json .*' 2>&1 | tee graalvm-community-jdk-24-t16.txt
9+
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-community-jdk-21-t16.json .*' 2>&1 | tee graalvm-community-jdk-21-t16.txt
10+
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -t 16 -p size=128 -prof gc -rf json -rff graalvm-community-jdk-17-t16.json .*' 2>&1 | tee graalvm-community-jdk-17-t16.txt

benchmark.sh

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
#!/bin/bash
2-
sbt -batch -java-home /usr/lib/jvm/jdk-24 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff jdk-24.json .*' 2>&1 | tee jdk-24.txt
3-
sbt -batch -java-home /usr/lib/jvm/jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff jdk-21.json .*' 2>&1 | tee jdk-21.txt
4-
sbt -batch -java-home /usr/lib/jvm/jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff jdk-17.json .*' 2>&1 | tee jdk-17.txt
5-
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-24 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-jdk-24.json .*' 2>&1 | tee graalvm-jdk-24.txt
6-
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-jdk-21.json .*' 2>&1 | tee graalvm-jdk-21.txt
7-
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-jdk-17.json .*' 2>&1 | tee graalvm-jdk-17.txt
8-
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-24 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-community-jdk-24.json .*' 2>&1 | tee graalvm-community-jdk-24.txt
9-
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-community-jdk-21.json .*' 2>&1 | tee graalvm-community-jdk-21.txt
10-
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/jmh:run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-community-jdk-17.json .*' 2>&1 | tee graalvm-community-jdk-17.txt
2+
sbt -batch -java-home /usr/lib/jvm/jdk-25 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff jdk-25.json .*' 2>&1 | tee jdk-25.txt
3+
sbt -batch -java-home /usr/lib/jvm/jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff jdk-21.json .*' 2>&1 | tee jdk-21.txt
4+
sbt -batch -java-home /usr/lib/jvm/jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff jdk-17.json .*' 2>&1 | tee jdk-17.txt
5+
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-25 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-jdk-25.json .*' 2>&1 | tee graalvm-jdk-25.txt
6+
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-jdk-21.json .*' 2>&1 | tee graalvm-jdk-21.txt
7+
sbt -batch -java-home /usr/lib/jvm/graalvm-jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-jdk-17.json .*' 2>&1 | tee graalvm-jdk-17.txt
8+
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-24 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-community-jdk-24.json .*' 2>&1 | tee graalvm-community-jdk-24.txt
9+
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-21 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-community-jdk-21.json .*' 2>&1 | tee graalvm-community-jdk-21.txt
10+
sbt -batch -java-home /usr/lib/jvm/graalvm-community-jdk-17 ++3.6.2 clean 'jsoniter-scala-benchmarkJVM/Jmh/run -jvmArgsAppend "-Djmh.executor=FJP" -p size=128 -prof gc -rf json -rff graalvm-community-jdk-17.json .*' 2>&1 | tee graalvm-community-jdk-17.txt

jsoniter-scala-benchmark/shared/src/main/scala/com/github/plokhotnyuk/jsoniter_scala/benchmark/CommonParams.scala

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,6 @@ import org.openjdk.jmh.annotations._
3333
"-XX:+PerfDisableSharedMem", // See https://github.com/Simonis/mmap-pause#readme
3434
"-XX:-UseDynamicNumberOfCompilerThreads",
3535
"-XX:-UsePerfData",
36-
"-XX:-UseNotificationThread",
3736
"-XX:+UnlockExperimentalVMOptions",
3837
"-XX:+TrustFinalNonStaticFields"
3938
))

0 commit comments

Comments
 (0)