Docker compose example from official docs broken?

[stemabo]

Describe the bug

I am (to the best of my knowledge) strictly following the official docker-compose example (https://docs.fluentd.org/container-deployment/docker-compose) but the Fluentd image runs into an error installing fluent-plugin-elasticsearch:

fluentd_1 | 2022-02-21 08:44:31 +0000 [info]: gem 'fluentd' version '1.12.0'
fluentd_1 | /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in 'require': cannot load such file -- elasticsearch/transport/transport/connections/selector (LoadError)

Is also experienced by others (@https://stackoverflow.com/users/18044852/henry-kao):
https://stackoverflow.com/questions/71120621/efk-system-is-build-on-docker-but-fluentd-cant-start-up/71126320

To Reproduce

follow https://docs.fluentd.org/container-deployment/docker-compose

Expected behavior

Fluentd container to start up with no errors...

Your Environment

- Fluentd version: fluent/fluentd:v1.12.0-debian-1.0 (as per the docs, newer version seems to make no difference) 
- Operating system: Ubuntu 20.04.4 LTS
- Kernel version: 5.4.0-89-generic

Your Configuration

as in [https://docs.fluentd.org/container-deployment/docker-compose](url)

Your Error Log

/usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in 'require': cannot load such file -- elasticsearch/transport/transport/connections/selector (LoadError)

Additional context

I would assume this is me getting something wrong, but as others confirm this behavior it might actually be an issue...

[BrianDouglas]

Can confirm as I'm having the exact same issue. Been having a bunch of trouble trying to get the EFK stack working. Seems like a lot of articles and tutorials out there are out of date. A real shame that even the official documentation doesn't work.

[Xernosch]

Hi!

I came across the same Problem and I got it to work. Not sure if this solution fits for your situation.

While following the official docs ( https://docs.fluentd.org/container-deployment/docker-compose ) I changed the config to use the newest versions up to date.

1. In the fluentd Dockerfile change the version from "5.0.3" to the newest version

RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.2.2"]

2. Since this only seems to work with elastic >= 8, you have to update elastic and kibana in the docker-compose.yml as well. I also needed to disable TLS for elastic as well.

version: "3"
services:
[...]
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.1.2
    container_name: elasticsearch
    environment:
      - "discovery.type=single-node"
      - xpack.security.enabled=false
    expose:
      - "9200"
    ports:
      - "9200:9200"

  kibana:
    image: docker.elastic.co/kibana/kibana:8.1.2
[...]

This works for me. Of course, disabling security (xpack.security.enabled=false) is just for proof of concept. Otherwise elastic will complain about the http connections where it expects https. Right now I am not aware how to configure fluentd to do this.

Hope this helps!
Cheers

[SEJSCloud]

Thank you @Xernosch
I have suffered for 1 week beacuse of version compatibility of fluentd ...
It works perfectely!!

[iiienapg]

1. In the fluentd Dockerfile change the version from "5.0.3" to the newest version

RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.2.2"]

2. Since this only seems to work with elastic >= 8, you have to update elastic and kibana in the docker-compose.yml as well. I also needed to disable TLS for elastic as well.

If you don't use ES8 for any reasons install elasticsearch gem version < 8 just before fluent-plugin-elasticsearch:

RUN ["gem", "install", "elasticsearch", "--no-document", "--version", "< 8"]
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.2.2"]

and don't switch to ES8 in docker-compose.yml

[valentinacala]

Hi, I tried to apply the suggested approach. Now I receive a new error:

2023-07-19 15:24:59 +0000 [error]: #0 unexpected error error_class=Errno::EACCES error="Permission denied @ dir_s_mkdir - /var/log/td-agent"
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:247:in `mkdir'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:247:in `fu_mkdir'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:228:in `block (2 levels) in mkdir_p'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:226:in `reverse_each'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:226:in `block in mkdir_p'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:211:in `each'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/fileutils.rb:211:in `mkdir_p'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/plugin/in_tail.rb:243:in `start'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:203:in `block in start'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:191:in `each'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:191:in `block in lifecycle'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:178:in `each'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:178:in `lifecycle'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:202:in `start'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/engine.rb:248:in `start'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/engine.rb:147:in `run'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/supervisor.rb:720:in `block in run_worker'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/supervisor.rb:971:in `main_process'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/supervisor.rb:711:in `run_worker'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/lib/fluent/command/fluentd.rb:376:in `<top (required)>'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.14.6/bin/fluentd:15:in `<top (required)>'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/bin/fluentd:23:in `load'
  2023-07-19 15:24:59 +0000 [error]: #0 /usr/local/bundle/bin/fluentd:23:in `<main>'
2023-07-19 15:24:59 +0000 [error]: #0 unexpected error error_class=Errno::EACCES error="Permission denied @ dir_s_mkdir - /var/log/td-agent"
  2023-07-19 15:24:59 +0000 [error]: #0 suppressed same stacktrace
2023-07-19 15:24:59 +0000 [error]: Worker 0 finished unexpectedly with status 1

elasticsearch/kibana version: 7.10.2
fluentd version: v1.12.0-debian-1.0

Any suggestion on why having this error and a possible solution?

Thank you in advance!

[paulsjohnson91]

For those having problems, it all seems to come down to version incompatibility. This worked for me:

version: "3"
services:
  web:
    container_name: web
    image: httpd
    ports:
      - "80:80"
    links:
      - fluentd
    logging:
      driver: "fluentd"
      options:
        fluentd-address: localhost:24224
        tag: httpd.access

  fluentd:
    container_name: fluentd
    build: ./fluentd
    volumes:
      - ./fluentd/conf:/fluentd/etc
    links:
      - "elasticsearch"
    ports:
      - "24224:24224"
      - "24224:24224/udp"

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.1.2
    container_name: elasticsearch
    environment:
      - "discovery.type=single-node"
      - "xpack.security.enabled=false"
    expose:
      - "9200"
    ports:
      - "9200:9200"

  kibana:
    container_name: kibana
    image: docker.elastic.co/kibana/kibana:8.1.2
    links:
      - "elasticsearch"
    ports:
      - "5601:5601"

With the Dockerfile being

# fluentd/Dockerfile

FROM fluent/fluentd:v1.12.0-debian-1.0
USER root
RUN ["gem", "install", "elasticsearch", "--no-document", "--version", "8.1.2"]
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.2.2"]
USER fluent

You might need to restart the fluentd pod if it fails before the elasticsearch pod has finished initialising

[reason211]

mark

[willzhang]

elasticsearch 8.x dockerfile

FROM fluent/fluentd:v1.16.2-debian-1.1
USER root
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.3.0"]
USER fluent

docker-compose.yaml

version: "3"
services:
  web:
    image: httpd
    ports:
      - "80:80"
    links:
      - fluentd
    logging:
      driver: "fluentd"
      options:
        fluentd-address: localhost:24224
        tag: httpd.access

  fluentd:
    image: registry.cn-shenzhen.aliyuncs.com/cnregistry/fluentd-es:v1.16.2-debian-1.1
    volumes:
      - ./fluentd/conf:/fluentd/etc
    links:
      - "elasticsearch"
    ports:
      - "24224:24224"
      - "24224:24224/udp"

  elasticsearch:
    container_name: elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:8.10.2
    ports:
      - 9200:9200
    environment:
      - http.host=0.0.0.0
      - transport.host=127.0.0.1
      - xpack.security.enabled=false

  kibana:
    image: docker.elastic.co/kibana/kibana:8.10.2
    links:
      - "elasticsearch"
    ports:
      - "5601:5601"

[bestwebua]

@paulsjohnson91 Thanks for your notes! It works! ❤️ ❤️ ❤️