-
Notifications
You must be signed in to change notification settings - Fork 436
Open
Description
Description
After upgrading Kafka Connect to version 4.1.0 and the Elasticsearch sink connector to 15.0.0, the Kafka Connect worker fails to load during plugin scanning.
Environment
- Kafka Connect version: 4.1.0
- Connector:
io.confluent:kafka-connect-elasticsearch:15.0.0
- Platform: Kubernetes (Connect container with mounted plugin directory)
Kafka Connect configuration snippet
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
name: my-cluster
annotations:
strimzi.io/use-connector-resources: "true"
spec:
version: 4.1.0
replicas: 1
bootstrapServers: my-cluster-kafka-bootstrap:9092
config:
config.providers: env
config.providers.env.class: org.apache.kafka.common.config.provider.EnvVarConfigProvider
group.id: connect-cluster
offset.storage.topic: connect-cluster-offsets
config.storage.topic: connect-cluster-configs
status.storage.topic: connect-cluster-status
config.storage.replication.factor: -1
offset.storage.replication.factor: -1
status.storage.replication.factor: -1
build:
output:
type: docker
image: ttl.sh/my-cluster-connect-4.1.0:24h
plugins:
- name: kafka-connect-file
artifacts:
- type: maven
group: org.mongodb.kafka
artifact: mongo-kafka-connect
version: 2.0.1
- name: kafka-connect-elasticsearch-file
artifacts:
- type: maven
repository: https://packages.confluent.io/maven
group: io.confluent
artifact: kafka-connect-elasticsearch
version: 15.0.0
Observed behavior
When Kafka Connect starts, it fails to load the Elasticsearch connector plugin due to a missing method in
org.apache.kafka.common.utils.Utils
:
2025-10-08 13:41:18 INFO [main] PluginScanner:75 - Loading plugin from: /opt/kafka/plugins/kafka-connect-elasticsearch-file
2025-10-08 13:41:19 ERROR [main] PluginScanner:138 - Failed to discover Converter in /opt/kafka/plugins/kafka-connect-elasticsearch-file: Plugin class has a dependency which is missing or invalid
java.util.ServiceConfigurationError: org.apache.kafka.connect.storage.Converter: Provider org.apache.kafka.connect.json.JsonConverter could not be instantiated
at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) ~[?:?]
at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:813) ~[?:?]
at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729) ~[?:?]
at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403) ~[?:?]
at org.apache.kafka.connect.runtime.isolation.PluginScanner.handleLinkageError(PluginScanner.java:176) ~[connect-runtime-4.1.0.jar:?]
at org.apache.kafka.connect.runtime.isolation.PluginScanner.getServiceLoaderPluginDesc(PluginScanner.java:136) [connect-runtime-4.1.0.jar:?]
at org.apache.kafka.connect.runtime.isolation.ServiceLoaderScanner.scanPlugins(ServiceLoaderScanner.java:61) [connect-runtime-4.1.0.jar:?]
at org.apache.kafka.connect.runtime.isolation.PluginScanner.scanUrlsAndAddPlugins(PluginScanner.java:79) [connect-runtime-4.1.0.jar:?]
at org.apache.kafka.connect.runtime.isolation.PluginScanner.discoverPlugins(PluginScanner.java:67) [connect-runtime-4.1.0.jar:?]
at org.apache.kafka.connect.runtime.isolation.Plugins.initLoaders(Plugins.java:88) [connect-runtime-4.1.0.jar:?]
at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:80) [connect-runtime-4.1.0.jar:?]
at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:69) [connect-runtime-4.1.0.jar:?]
at org.apache.kafka.connect.cli.AbstractConnectCli.startConnect(AbstractConnectCli.java:121) [connect-runtime-4.1.0.jar:?]
at org.apache.kafka.connect.cli.AbstractConnectCli.run(AbstractConnectCli.java:95) [connect-runtime-4.1.0.jar:?]
at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:112) [connect-runtime-4.1.0.jar:?]
Caused by: java.lang.NoSuchMethodError: 'java.util.Set org.apache.kafka.common.utils.Utils.mkSet(java.lang.Object[])'
at org.apache.kafka.connect.json.JsonConverter.<init>(JsonConverter.java:251) ~[connect-json-7.9.0-ccs.jar:?]
at org.apache.kafka.connect.json.JsonConverter.<init>(JsonConverter.java:239) ~[connect-json-7.9.0-ccs.jar:?]
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[?:?]
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:789) ~[?:?]
... 13 more
I am looking for suggestions as to how to fix this. Any help would be appreciated. Thank you!
Metadata
Metadata
Assignees
Labels
No labels