Skip to content

OpenSearch plugin does not work AWS OpenSearch Serverless. [404] #149

@H-Sugimoto-999

Description

@H-Sugimoto-999

Steps to replicate

fluent.conf

<match *>
    @type opensearch
    logstash_format true
    flush_interval 1s
    with_transporter_log true

    <endpoint>
      url https://xxxxxxx.xx-xxxx-1.aoss.amazonaws.com
      region xx-xxxx-1
      access_key_id "xxxx"
      secret_access_key "xxxx"
      aws_service_name aoss
    </endpoint>
</match>

Expected Behavior or What you need to ask

I'm running a local VM which uses fluentd/fluent-plugin-opensearch.
I'm getting the following message:

2024-12-12 16:48:25 +0900 [info]: #0 fluent/log.rb:362:info: GET https://xxxxxxx.xx-xxxx-1.aoss.amazonaws.com:443/ [status:404, request:0.518s, query:N/A]
2024-12-12 16:48:25 +0900 [debug]: #0 fluent/log.rb:341:debug: <
2024-12-12 16:48:25 +0900 [fatal]: #0 fluent/log.rb:425:fatal: [404]
2024-12-12 16:48:27 +0900 [warn]: #0 fluent/log.rb:383:warn: Could not communicate to OpenSearch, resetting connection and trying again. [404]

AWS Support says,

After verifying the issue locally and checking with the product department, we found that Fluentd is likely not compatible with Amazon OpenSearch Serverless collections.

Although Fluentd may be accessing the root path of the OpenSearch Serverless collection endpoint, OpenSearch Serverless does not currently allow access to the root path.

Is this really ?
Plugin does not support Amazon OpenSearch Serverless ?
...

Using Fluentd and OpenSearch plugin versions

  • OS version Ubuntu 24.04
  • fluent-package 5.0.5 fluentd 1.16.6
  • fluent-plugin-opensearch' version 1.1.4

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions