Skip to content

KSQL-15022 | REST endpoint hardening: connection close iteration + 406 fall-through + CommandTopic backup close - #11056

Open
Parag Badani (pbadani) wants to merge 3 commits into
7.6.xfrom
pbadani/KSQL-15022
Open

KSQL-15022 | REST endpoint hardening: connection close iteration + 406 fall-through + CommandTopic backup close#11056
Parag Badani (pbadani) wants to merge 3 commits into
7.6.xfrom
pbadani/KSQL-15022

Conversation

@pbadani

@pbadani Parag Badani (pbadani) commented May 23, 2026

Copy link
Copy Markdown
Member

Description

Second in a series of small, focused backports from PR #11037 onto 7.6.x (epic KSQL-14548, this work tracked in KSQL-15022). This PR contains three REST-layer fixes targeting per-event resource leaks.

1. ConnectionQueryManager — iterate a snapshot in the Vert.x close handler.
ConnectionQueries.handle() iterated the live Set<PushQueryHolder> while calling query.close() on each. close() invokes the registered close-handler (this::removeQuery), which mutates the same set mid-iteration — throwing ConcurrentModificationException on the second query and leaving every subsequent push query (and its KafkaStreams resources) never closed. Each dropped HTTP connection that held more than one push query leaked resources.

Fix: iterate over an ArrayList snapshot so close() can safely remove from the underlying set.

Backport of b4e6928aa3f from pbadani/fix-bugs.

2. QueryStreamHandler.handlePrintPublisher — missing return after 406.
When the client's Accept header for a print-topic request was not the delimited content type, the handler sent 406 Not Acceptable and called response().end() — but then fell through to subscribe a PrintSubscriber to the BlockingPrintPublisher. That started a KafkaConsumer poll loop writing into an already-ended response: every write was silently dropped, but the consumer kept polling broker connections until the end-handler chain eventually closed it.

Fix: add the missing return after the 406 path.

Backport of f87cc6623fa from pbadani/fix-bugs.

3. CommandTopic.close() — per-client try/catch so backup close always runs.
close() ran commandConsumer.close() followed by commandTopicBackup.close() with no exception handling. A transient Kafka error in the consumer's close (broker reset on shutdown, partition rebalance in flight, etc.) skipped the backup close, leaking the backup file handle and any watcher resources for the lifetime of the JVM.

Fix: wrap each close in its own try/catch so a failure in one does not prevent the other from running.

Backport of 66c5b290d46 from pbadani/fix-bugs.

Testing done

./mvnw -pl ksqldb-rest-app test -Dtest='ConnectionQueryManagerTest,QueryStreamHandlerTest,CommandTopicTest'

  • QueryStreamHandlerTest: 6/6 pass (includes new shouldReturnAfter406ForPrintTopicWithUnacceptableAcceptHeader)
  • ConnectionQueryManagerTest: 1/1 pass (new test covering multi-query connection close without CME)
  • CommandTopicTest: 17/17 pass (includes new test ensuring backup close runs when consumer close throws)

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")
  • Do these changes have compatibility implications for rollback?

The Vert.x connection-close handler ConnectionQueries.handle() iterated the
live queries set while calling PushQueryHolder.close() on each. close() invokes
the closeHandler (this::removeQuery), which mutates the same HashSet
mid-iteration — throwing ConcurrentModificationException on the second query
and leaving every subsequent push query (and its KafkaStreams resources)
never closed. Each dropped HTTP connection that held more than one push query
leaked resources.

Iterate over a snapshot so close() can safely remove from the set.
…isher

When the client's Accept header for a print-topic request was not the
delimited content type, handlePrintPublisher sent a 406 and called
response().end() — but then FELL THROUGH to subscribe the PrintSubscriber to
the BlockingPrintPublisher. That started a KafkaConsumer poll loop writing
into an already-ended response: every write was silently dropped, but the
consumer kept polling broker connections until the end handler chain
eventually closed it.

Add the missing return.
…ose throws

CommandTopic.close() ran commandConsumer.close() followed by
commandTopicBackup.close() with no exception handling. A transient Kafka
error in the consumer's close (broker reset on shutdown, partition rebalance
in flight, etc.) skipped the backup close, leaking the backup file handle
and any watcher resources for the lifetime of the JVM.

Wrap each close in its own try/catch so a failure in one does not prevent
the other from running.
@pbadani
Parag Badani (pbadani) requested a review from a team as a code owner May 23, 2026 06:44
@confluent-cla-assistant

Copy link
Copy Markdown

🎉 All Contributor License Agreements have been signed. Ready to merge.
Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant