Skip to content

Plugin fails to load on Kafka Connect 4.1.0 due to NoSuchMethodError in JsonConverter (Utils.mkSet) #897

@drprolegend

Description

@drprolegend

Description

After upgrading Kafka Connect to version 4.1.0 and the Elasticsearch sink connector to 15.0.0, the Kafka Connect worker fails to load during plugin scanning.


Environment

  • Kafka Connect version: 4.1.0
  • Connector: io.confluent:kafka-connect-elasticsearch:15.0.0
  • Platform: Kubernetes (Connect container with mounted plugin directory)

Kafka Connect configuration snippet

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: my-cluster
  annotations:
    strimzi.io/use-connector-resources: "true"
spec:
  version: 4.1.0
  replicas: 1
  bootstrapServers: my-cluster-kafka-bootstrap:9092
  config:
    config.providers: env
    config.providers.env.class: org.apache.kafka.common.config.provider.EnvVarConfigProvider
    group.id: connect-cluster
    offset.storage.topic: connect-cluster-offsets
    config.storage.topic: connect-cluster-configs
    status.storage.topic: connect-cluster-status
    config.storage.replication.factor: -1
    offset.storage.replication.factor: -1
    status.storage.replication.factor: -1
  build:
    output:
      type: docker
      image: ttl.sh/my-cluster-connect-4.1.0:24h
    plugins:
      - name: kafka-connect-file
        artifacts:
          - type: maven
            group: org.mongodb.kafka
            artifact: mongo-kafka-connect
            version: 2.0.1
      - name: kafka-connect-elasticsearch-file
        artifacts:
          - type: maven
            repository: https://packages.confluent.io/maven
            group: io.confluent
            artifact: kafka-connect-elasticsearch
            version: 15.0.0

Observed behavior

When Kafka Connect starts, it fails to load the Elasticsearch connector plugin due to a missing method in
org.apache.kafka.common.utils.Utils:

2025-10-08 13:41:18 INFO  [main] PluginScanner:75 - Loading plugin from: /opt/kafka/plugins/kafka-connect-elasticsearch-file
2025-10-08 13:41:19 ERROR [main] PluginScanner:138 - Failed to discover Converter in /opt/kafka/plugins/kafka-connect-elasticsearch-file: Plugin class has a dependency which is missing or invalid
java.util.ServiceConfigurationError: org.apache.kafka.connect.storage.Converter: Provider org.apache.kafka.connect.json.JsonConverter could not be instantiated
	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) ~[?:?]
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:813) ~[?:?]
	at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729) ~[?:?]
	at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403) ~[?:?]
	at org.apache.kafka.connect.runtime.isolation.PluginScanner.handleLinkageError(PluginScanner.java:176) ~[connect-runtime-4.1.0.jar:?]
	at org.apache.kafka.connect.runtime.isolation.PluginScanner.getServiceLoaderPluginDesc(PluginScanner.java:136) [connect-runtime-4.1.0.jar:?]
	at org.apache.kafka.connect.runtime.isolation.ServiceLoaderScanner.scanPlugins(ServiceLoaderScanner.java:61) [connect-runtime-4.1.0.jar:?]
	at org.apache.kafka.connect.runtime.isolation.PluginScanner.scanUrlsAndAddPlugins(PluginScanner.java:79) [connect-runtime-4.1.0.jar:?]
	at org.apache.kafka.connect.runtime.isolation.PluginScanner.discoverPlugins(PluginScanner.java:67) [connect-runtime-4.1.0.jar:?]
	at org.apache.kafka.connect.runtime.isolation.Plugins.initLoaders(Plugins.java:88) [connect-runtime-4.1.0.jar:?]
	at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:80) [connect-runtime-4.1.0.jar:?]
	at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:69) [connect-runtime-4.1.0.jar:?]
	at org.apache.kafka.connect.cli.AbstractConnectCli.startConnect(AbstractConnectCli.java:121) [connect-runtime-4.1.0.jar:?]
	at org.apache.kafka.connect.cli.AbstractConnectCli.run(AbstractConnectCli.java:95) [connect-runtime-4.1.0.jar:?]
	at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:112) [connect-runtime-4.1.0.jar:?]
Caused by: java.lang.NoSuchMethodError: 'java.util.Set org.apache.kafka.common.utils.Utils.mkSet(java.lang.Object[])'
	at org.apache.kafka.connect.json.JsonConverter.<init>(JsonConverter.java:251) ~[connect-json-7.9.0-ccs.jar:?]
	at org.apache.kafka.connect.json.JsonConverter.<init>(JsonConverter.java:239) ~[connect-json-7.9.0-ccs.jar:?]
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[?:?]
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:789) ~[?:?]
	... 13 more

I am looking for suggestions as to how to fix this. Any help would be appreciated. Thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions