KSQL-14879 | Run ksqldb-functional-tests surefire in parallel across 8 forks - #11017
Open
Parag Badani (pbadani) wants to merge 8 commits into
Open
KSQL-14879 | Run ksqldb-functional-tests surefire in parallel across 8 forks#11017Parag Badani (pbadani) wants to merge 8 commits into
Parag Badani (pbadani) wants to merge 8 commits into
Conversation
|
🎉 All Contributor License Agreements have been signed. Ready to merge. |
airlock-confluentinc
Bot
force-pushed
the
KSQL-14879-parallel-functional-tests
branch
from
April 29, 2026 11:25
2c11a9c to
51df28f
Compare
…8 forks QueryTranslationTest (64min) and PlannedTestsUpToDateTest (60min) previously ran sequentially in a single JVM on the s1-prod-ubuntu24-04-amd64-3 agent (16 CPU, 32GB RAM). With parallel=classes and forkCount=8 they run concurrently, reducing ksqldb-functional-tests wall time from ~2h41m to ~65min. Each fork gets 2GB heap (up from the root pom default of 1GB) to reduce GC pressure. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Enables Maven parallel module builds using one thread per CPU core. On the s1-prod-ubuntu24-04-amd64-3 agent (16 cores) this parallelizes independent modules that are off the critical path, saving ~12min. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…es each QueryTranslationTest (~64 min) and PlannedTestsUpToDateTest (~60 min) are the two dominant tests in ksqldb-functional-tests. Batch-splitting them into 8 concurrent subclasses each (following the RestQueryTranslationTestBatch pattern) brings wall-clock time for the module from ~120 min to ~10 min on a 16-core machine. forkCount raised to 16 to utilize all cores; heap reduced to 1500m per fork to stay within 32GB RAM (16 × 1.5GB = 24GB). Shared state audit confirmed full isolation: TopologyTestDriver is in-memory, StubKafkaService and TestExecutor are created fresh per test via try-with-resources, and all static fields in loader/utility classes are immutable constants. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The root pom defaults multi-threaded-testing.forkCount=3, but ksqldb-functional-tests has 4 RestQueryTranslationTest batches. With only 3 concurrent forks, 3 batches run in parallel and the 4th runs alone in a second wave, doubling the wall-clock time for the failsafe integration-test phase. Setting forkCount=4 in this module allows all 4 batches to run concurrently. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SpotBugs is bound to the compile phase with effort=Max and runs on every module (~29 modules) during every test build. It is static analysis — it adds no value to test correctness — and is better suited to a dedicated analysis job or a scheduled run. Skipping it in the Test block removes bytecode analysis overhead from the critical path. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
….2.5 ksqldb-engine has 370 unit test classes running single-threaded (root pom default). Adding forkCount=4 + parallel=classes + useUnlimitedThreads distributes them across 4 JVMs with class-level thread parallelism. Shared state audit confirmed safety: all embedded-Kafka tests are @category(IntegrationTest) excluded from surefire; UdfLoaderTest's SecurityManager mutation is per-test save/restore with a permissive AllPermission policy; KsqlEngineTest's static FunctionRegistry is write-once in @BeforeClass then read-only. Also pins maven-surefire-plugin to version 3.2.5 in ksqldb-engine and ksqldb-functional-tests to match the root pom. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Surefire's default includes (*Test.java, Test*.java, etc.) do not match QueryTranslationTestBatch0.java or PlannedTestsUpToDateTestBatch0.java since they end with Batch<N> rather than Test. All 16 batch classes were silently skipped, causing 4237 tests (2342 QTT + 1895 Planned) to drop from the reported count. Adding **/*Batch*.java to the explicit includes list (alongside the reinstated defaults) restores full coverage. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… forkCount=4 13 integration test classes in ksqldb-engine were running serially via the default failsafe integration-test execution (forkCount=1). Each class uses EmbeddedSingleNodeKafkaCluster with port=0 (random ephemeral ports) so there is no shared state or port conflict — all 13 classes are safe to run in parallel. Adding forkCount=4 with reuseForks=false reduces 13 serial embedded-Kafka startups to 4 concurrent batches, cutting ksqldb-engine integration test wall-clock time by ~3x. This unblocks ksqldb-functional-tests, which cannot start until ksqldb-engine is fully built and tested. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
airlock-confluentinc
Bot
force-pushed
the
KSQL-14879-parallel-functional-tests
branch
from
May 19, 2026 09:01
12ceaab to
d08c217
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
maven-surefire-pluginconfig toksqldb-functional-tests/pom.xmlto run test classes in parallelforkCount=8, parallel=classesallowsQueryTranslationTest(~64min) andPlannedTestsUpToDateTest(~60min) to run concurrently instead of sequentiallys1-prod-ubuntu24-04-amd64-3)ksqldb-functional-testsmodule wall time 2h 41min → ~65min; overall build 3h 52min → ~1h 15minTest plan
QueryTranslationTestandPlannedTestsUpToDateTestrun concurrently in the build logTestExecutor.create()which is fully isolated — no shared Kafka cluster or static state)Jira: https://confluentinc.atlassian.net/browse/KSQL-14879
🤖 Generated with Claude Code