Piecewise linear workloads #11

gpkc · 2025-08-12T14:28:34Z

Added piecewise linear workloads. Example:

name: 'bell-throughput'

topics: 1
partitionsPerTopic: 60
messageSize: 1024
subscriptionsPerTopic: 3
consumerPerSubscription: 6
producersPerTopic: 12

consumerBacklogSizeGB: 0
testDurationMinutes: 30

useRandomizedPayloads: true
randomBytesRatio: 0.8
randomizedPayloadPoolSize: 1000

producerRate: 10000
producerRateTimeline:
  - { timeSeconds: 1, rate: 10000 }
  - { timeSeconds: 33, rate: 10526 }
  - { timeSeconds: 73, rate: 12602 }
  - { timeSeconds: 129, rate: 17917 }
  - { timeSeconds: 198, rate: 28215 }
  - { timeSeconds: 280, rate: 44610 }
  - { timeSeconds: 372, rate: 66724 }
  - { timeSeconds: 474, rate: 92070 }
  - { timeSeconds: 582, rate: 116159 }
  - { timeSeconds: 695, rate: 133611 }
  - { timeSeconds: 810, rate: 140000 }
  - { timeSeconds: 990, rate: 140000 }
  - { timeSeconds: 1105, rate: 133611 }
  - { timeSeconds: 1218, rate: 116159 }
  - { timeSeconds: 1326, rate: 92070 }
  - { timeSeconds: 1428, rate: 66724 }
  - { timeSeconds: 1520, rate: 44610 }
  - { timeSeconds: 1602, rate: 28215 }
  - { timeSeconds: 1671, rate: 17917 }
  - { timeSeconds: 1727, rate: 12602 }
  - { timeSeconds: 1767, rate: 10526 }
  - { timeSeconds: 1800, rate: 10000 }

Workload would look like:

I've added changes also to WorkloadSetTemplate and the WorkloadGenerator tool, even though we're not using those AFAIK, but we might in the future so the feature is there.

The algorithm here basically will interpolate linearly the values between each point defined above. Sampling rate is optionally defined via producerRateTimelineIntervalSeconds (defaults to 1s).

Copilot

Pull Request Overview

This PR implements piecewise linear workloads that allow dynamic producer rate changes over time through linear interpolation between defined rate points. The feature enables creating bell-curve or other complex rate patterns for more realistic load testing scenarios.

Adds support for time-based producer rate timelines with linear interpolation between points
Implements scheduling mechanism to adjust producer rates at configurable intervals
Extends workload configuration to support rate point definitions and timeline intervals

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
Workload.java	Adds RatePoint class and timeline fields to define piecewise linear rate schedules
WorkloadGenerator.java	Implements the core scheduling logic with linear interpolation and rate adjustment
WorkloadSetTemplate.java	Extends template configuration to support rate timeline definitions
tool/WorkloadGenerator.java	Adds timeline support to the workload generation tool
benchmark-framework/pom.xml	Downgrades jetty and async-http-client dependency versions

benchmark-framework/src/main/java/io/openmessaging/benchmark/WorkloadGenerator.java

benchmark-framework/src/main/java/io/openmessaging/benchmark/Workload.java

jeqo · 2025-08-18T07:29:14Z

benchmark-framework/src/main/java/io/openmessaging/benchmark/WorkloadGenerator.java

+                (workload.producerRateTimelineIntervalSeconds != null)
+                        ? Math.max(1, workload.producerRateTimelineIntervalSeconds)
+                        : 1;
+        final double initialRate = Math.max(1.0, workload.producerRate);


I wonder if it's easier to have either a separate configuration, e.g. producerInitialRate, so it's known that this applies to this dynamic changes (we could reuse this on the find max throughput that is hardcoded at 10K); or use the first entry and enforce it starts at zero.

Hmm good point
I'm changing it then:

we now have producerInitialRate, it is required if we are using producerRateTimeline and producerRate will then be ignored

we use producerInitialRate now also for the probe mode (find max throughput)

benchmark-framework/src/main/java/io/openmessaging/benchmark/WorkloadGenerator.java

benchmark-framework/pom.xml

jeqo · 2025-08-18T07:49:20Z

tool/src/main/java/io/openmessaging/benchmark/tool/workload/WorkloadGenerator.java

        workload.testDurationMinutes = template.testDurationMinutes;
        workload.warmupDurationMinutes = template.warmupDurationMinutes;
        workload.useRandomizedPayloads = template.useRandomizedPayloads;
+        if (template.producerRateTimeline != null && !template.producerRateTimeline.isEmpty()) {


Good finding! Wasn't aware of this tool. Will have a look

Let me know if you find a good use case!

jeqo · 2025-08-19T09:33:50Z

benchmark-framework/src/main/java/io/openmessaging/benchmark/WorkloadGenerator.java

-        if (workload.consumerBacklogSizeGB > 0 && workload.producerRate == 0) {
+        if (workload.consumerBacklogSizeGB > 0
+                && workload.producerRate == 0
+                && workload.producerRateTimeline.isEmpty()) {


Could you add a WorkloadGeneratorTest to test this validation?

Maybe these need to be separated checks: backlog cannot be defined if producerRate is zero. If producer rate is >0, then check that timeline and initial rate are not defined.

jeqo · 2025-08-19T09:35:06Z

benchmark-framework/src/main/java/io/openmessaging/benchmark/WorkloadGenerator.java

+            if (workload.producerInitialRate == null || workload.producerInitialRate <= 0) {
+                throw new IllegalArgumentException(
+                        "producerInitialRate must be > 0 when producerRateTimeline is configured");
+            }


Let's move this to the constructor as well, and test the validation

jeqo · 2025-08-19T09:36:08Z

benchmark-framework/src/main/java/io/openmessaging/benchmark/WorkloadGenerator.java

+            if (workload.producerInitialRate == null || workload.producerInitialRate <= 0) {
+                throw new IllegalArgumentException(
+                        "producerInitialRate must be > 0 when probing maximum sustainable rate "
+                                + "(producerRate == 0 and no timeline)");


Same here, move this to constructor.

jeqo · 2025-08-19T09:37:15Z

benchmark-framework/src/main/java/io/openmessaging/benchmark/WorkloadGenerator.java

+                        "producerInitialRate must be > 0 when probing maximum sustainable rate "
+                                + "(producerRate == 0 and no timeline)");
+            }
+            targetPublishRate = workload.producerInitialRate;


wonder if we should default initial rate at 10K to have same default behavior as max throughput.
Would this make it harder to reuse with piecewise workload?

gpkc requested review from Copilot and jeqo August 12, 2025 14:30

Copilot AI reviewed Aug 12, 2025

View reviewed changes

jeqo reviewed Aug 18, 2025

View reviewed changes

Add piecewise linear workloads

b663d22

gpkc force-pushed the gpkc-piecewise-linear-workloads branch from 96c3baf to b663d22 Compare August 18, 2025 15:02

gpkc requested a review from jeqo August 19, 2025 07:44

jeqo reviewed Aug 19, 2025

View reviewed changes

Piecewise linear workloads #11

Are you sure you want to change the base?

Piecewise linear workloads #11

Uh oh!

Conversation

gpkc commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gpkc commented Aug 12, 2025 •

edited

Loading