Skip to content

Commit a85c06c

Browse files
authored
Add new benchmark results
1 parent 7922986 commit a85c06c

File tree

1 file changed

+33
-4
lines changed

1 file changed

+33
-4
lines changed

Benchmarks/ClientBenchmarkApp/README.md

Lines changed: 33 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,27 +4,56 @@ The benchmarks are built with [BenchmarkDotNet](https://benchmarkdotnet.org) and
44

55
`dotnet run ClientBenchmarkApp.csproj -c Release`
66

7-
# Results - Mar 21, 2024
7+
# Results
8+
9+
The following results are from benchmarks run on my local MBP against a HiveMQ v4 broker running in a Docker container over localhost.
10+
11+
## Legend
12+
```
13+
Mean : Arithmetic mean of all measurements
14+
Error : Half of 99.9% confidence interval
15+
StdDev : Standard deviation of all measurements
16+
Median : Value separating the higher half of all measurements (50th percentile)
17+
1 us : 1 Microsecond (0.000001 sec)
18+
1 ms : 1,000 Microseconds
19+
```
20+
21+
## Mar 22, 2024
22+
23+
| Method | Mean | Error | StdDev | Median |
24+
|---------------------------------------------------------------- |--------------:|------------:|-------------:|---------------:|
25+
| 'Publish a QoS 0 messages to the broker.' | 57.27 us | 158.55 us | 467.50 us | 9.084 us |
26+
| 'Publish a QoS 1 messages to the broker.' | 2,291.28 us | 903.01 us | 2,662.56 us | 1,357.063 us |
27+
| 'Publish a QoS 2 messages to the broker.' | 2,058.05 us | 1,048.91 us | 3,092.73 us | 1,292.396 us |
28+
| 'Publish 100 256b length payload QoS 0 messages to the broker.' | 138.29 us | 183.38 us | 540.69 us | 79.604 us |
29+
| 'Publish 100 256b length payload QoS 1 messages to the broker.' | 45,813.98 us | 4,838.62 us | 14,266.78 us | 42,482.520 us |
30+
| 'Publish 100 256b length payload QoS 2 messages to the broker.' | 88,589.38 us | 3,877.02 us | 11,431.48 us | 85,640.167 us |
31+
| 'Publish 100 256k length payload QoS 0 messages to the broker.' | 124.92 us | 173.22 us | 510.74 us | 69.709 us |
32+
| 'Publish 100 256k length payload QoS 1 messages to the broker.' | 270,043.05 us | 8,850.72 us | 26,096.56 us | 266,506.583 us |
33+
| 'Publish 100 256k length payload QoS 2 messages to the broker.' | 300,923.38 us | 5,704.22 us | 16,819.03 us | 296,254.688 us |
34+
35+
36+
## Mar 21, 2024
837

938
With release [v0.11.0](https://github.com/hivemq/hivemq-mqtt-client-dotnet/releases/tag/v0.11.0) there was a big performance improvement. All messaging performance was improved but particularly publishing a QoS level 2 message went from ~206ms down to ~1.6ms.
1039

11-
## Previous Performance
40+
### Previous Performance
1241

1342
| Method | Mean | Error | StdDev | Median |
1443
|------------------------------------------ |-------------:|------------:|------------:|---------------:|
1544
| 'Publish a QoS 0 message' | 390.8 us | 1,842.5 us | 1,218.7 us | 5.646 us |
1645
| 'Publish a QoS 1 message' | 103,722.8 us | 4,330.0 us | 2,864.1 us | 103,536.375 us |
1746
| 'Publish a QoS 2 message' | 202,367.9 us | 26,562.9 us | 17,569.7 us | 206,959.834 us |
1847

19-
## First Pass Refactor Performance
48+
### First Pass Refactor Performance
2049

2150
| Method | Mean | Error | StdDev | Median |
2251
|------------------------------------------ |-----------:|-----------:|-----------:|-------------:|
2352
| 'Publish a QoS 0 message' | 401.9 us | 1,876.3 us | 1,241.0 us | 9.250 us |
2453
| 'Publish a QoS 1 message' | 2,140.0 us | 3,568.2 us | 2,360.1 us | 1,324.251 us |
2554
| 'Publish a QoS 2 message' | 4,217.2 us | 5,803.7 us | 3,838.8 us | 2,569.166 us |
2655

27-
## Final Refactor Performance Results (for now 👻)
56+
### Final Refactor Performance Results (for now 👻)
2857

2958
| Method | Mean | Error | StdDev | Median |
3059
|------------------------------------------ |------------:|----------:|------------:|-------------:|

0 commit comments

Comments
 (0)