[Questions] RabbitMQ 4.x has lower test results than 3.12.x on a 2 core Windows machine

[ElioHaddadCAH]

Community Support Policy

• I have read RabbitMQ's Community Support Policy
• I run RabbitMQ 4.x, the only series currently covered by community support
• I promise to provide all relevant information (versions, logs from all nodes, rabbitmq-diagnostics output, detailed reproduction steps)

RabbitMQ version used

4.0.6

Erlang version used

27.2.x

Operating system (distribution) used

Windows 2022 Server

How is RabbitMQ deployed?

Windows installer

Steps to deploy RabbitMQ cluster

We use the windows installer for both RMQ and Erlang, then after installation we configure the needed plugins, then restart the service.

Steps to reproduce the behavior in question

We run a load test that has a lot of messages to be published

What problem are you trying to solve?

We used to run our load tests RMQ 3.12.8 with Erlang 26.1.2, and when we started using RMQ 4.0.7 with Erlang 27.2.4, those load tests are facing performance severity issues. We found out that the ack was taking way more time, and noticed the number of ready messages was peaking high.
We have tried the following :

• Upgraded to amqp-client:5.22.0
• Tried correlated and simple publisher confirms
• Tried using lazy queues (although it was not advised to do so)
• Tried using the Khepri store instead of the Mnesia store

All of those solutions were dead ends, but the only thing that worked was running the tests on a server that had 4Vcores on its cpu instead of 2Vcores, keeping in mind that RMQ 3.12.8 runs the tests on the 2 Vcores servers perfectly fine

[lukebakken]

Hello, thanks for using RabbitMQ.

I'm wondering what you expect from Team RabbitMQ. Do you expect us to set up a Windows cluster, install various RabbitMQ versions, run some benchmarks (who knows what parameters, queue types, etc), and report back? That is several hours worth of free work, at a minimum.

Do you expect us to read your extremely limited report and immediately know what is going on?

If you can provide a PerfTest workload that demonstrates your issue, we might take the time to look into this.

Erlang 27.3 was recently released. I would start by using that version, as well as RabbitMQ 4.0.7.

I will re-open this discussion if you reply with actionable information.

[michaelklishin]

A two core Windows machine is a curious choice for a "peak performance" workload. It very likely uses HDDs, too.

[michaelklishin]

@ElioHaddadCAH we cannot suggest anything with a description like that.

Lazy queues are gone in 4.x, the key won't have any effect since CQv2s are always "lazy". Messages are not stored in Khepri.

We know absolutely nothing about the workload and you are neither a paying customer nor a contributor.
So it's entirely on you to describe in detail how the test was conducted, in what kind of environment,
simulate the workload using PerfTest and share all the relevant metrics.

We don't know if your Erlang build has the JIT enabled or not, how it was provisioned, and so on.

Oh, never mind.

the only thing that worked was running the tests on a server that had 4Vcores
on its cpu instead of 2Vcores

RabbitMQ was not designed for one or two core CPUs. A queue is a unit of concurrency, and each connection, queue replica, stream replica, AMQP 0-9-1 channel will all happily use the extra available cores. The number of cores is just as important as the I/O throughput and latency (with quorum queues and particularly streams).

For workloads with a large number of mostly idle connections/queues there is recommendation that can significantly reduce CPU load without any practical difference.

[michaelklishin]

Plus there is a set of tunable values specifically for quorum queues, and peak load features should seriously consider adopting streams and partitioned streams.