KSQL-15022 | REST endpoint hardening: connection close iteration + 406 fall-through + CommandTopic backup close - #11056
Open
Parag Badani (pbadani) wants to merge 3 commits into
Open
KSQL-15022 | REST endpoint hardening: connection close iteration + 406 fall-through + CommandTopic backup close#11056Parag Badani (pbadani) wants to merge 3 commits into
Parag Badani (pbadani) wants to merge 3 commits into
Conversation
The Vert.x connection-close handler ConnectionQueries.handle() iterated the live queries set while calling PushQueryHolder.close() on each. close() invokes the closeHandler (this::removeQuery), which mutates the same HashSet mid-iteration — throwing ConcurrentModificationException on the second query and leaving every subsequent push query (and its KafkaStreams resources) never closed. Each dropped HTTP connection that held more than one push query leaked resources. Iterate over a snapshot so close() can safely remove from the set.
…isher When the client's Accept header for a print-topic request was not the delimited content type, handlePrintPublisher sent a 406 and called response().end() — but then FELL THROUGH to subscribe the PrintSubscriber to the BlockingPrintPublisher. That started a KafkaConsumer poll loop writing into an already-ended response: every write was silently dropped, but the consumer kept polling broker connections until the end handler chain eventually closed it. Add the missing return.
…ose throws CommandTopic.close() ran commandConsumer.close() followed by commandTopicBackup.close() with no exception handling. A transient Kafka error in the consumer's close (broker reset on shutdown, partition rebalance in flight, etc.) skipped the backup close, leaking the backup file handle and any watcher resources for the lifetime of the JVM. Wrap each close in its own try/catch so a failure in one does not prevent the other from running.
|
🎉 All Contributor License Agreements have been signed. Ready to merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Second in a series of small, focused backports from PR #11037 onto 7.6.x (epic KSQL-14548, this work tracked in KSQL-15022). This PR contains three REST-layer fixes targeting per-event resource leaks.
1.
ConnectionQueryManager— iterate a snapshot in the Vert.x close handler.ConnectionQueries.handle()iterated the liveSet<PushQueryHolder>while callingquery.close()on each.close()invokes the registered close-handler (this::removeQuery), which mutates the same set mid-iteration — throwingConcurrentModificationExceptionon the second query and leaving every subsequent push query (and its KafkaStreams resources) never closed. Each dropped HTTP connection that held more than one push query leaked resources.Fix: iterate over an
ArrayListsnapshot soclose()can safely remove from the underlying set.Backport of
b4e6928aa3ffrompbadani/fix-bugs.2.
QueryStreamHandler.handlePrintPublisher— missingreturnafter 406.When the client's
Acceptheader for a print-topic request was not the delimited content type, the handler sent406 Not Acceptableand calledresponse().end()— but then fell through to subscribe aPrintSubscriberto theBlockingPrintPublisher. That started aKafkaConsumerpoll loop writing into an already-ended response: every write was silently dropped, but the consumer kept polling broker connections until the end-handler chain eventually closed it.Fix: add the missing
returnafter the 406 path.Backport of
f87cc6623fafrompbadani/fix-bugs.3.
CommandTopic.close()— per-client try/catch so backup close always runs.close()rancommandConsumer.close()followed bycommandTopicBackup.close()with no exception handling. A transient Kafka error in the consumer's close (broker reset on shutdown, partition rebalance in flight, etc.) skipped the backup close, leaking the backup file handle and any watcher resources for the lifetime of the JVM.Fix: wrap each close in its own try/catch so a failure in one does not prevent the other from running.
Backport of
66c5b290d46frompbadani/fix-bugs.Testing done
./mvnw -pl ksqldb-rest-app test -Dtest='ConnectionQueryManagerTest,QueryStreamHandlerTest,CommandTopicTest'QueryStreamHandlerTest: 6/6 pass (includes newshouldReturnAfter406ForPrintTopicWithUnacceptableAcceptHeader)ConnectionQueryManagerTest: 1/1 pass (new test covering multi-query connection close without CME)CommandTopicTest: 17/17 pass (includes new test ensuring backup close runs when consumer close throws)Reviewer checklist