Skip to content

Failing to provide a + in the date pattern of the index property silently fails. #1082

Open
@nicenemo

Description

@nicenemo

Logstash information:

  1. Using bundled JDK: /usr/share/logstash/jdk logstash 8.4.1 But also fails on older versions
  2. Debian packages from Elastic. But also fails on Mac.
  3. Either as service or as command line
  4. plugin came packaged with logstash

We run Logstash both as systemd service as well as commandline tool.
We run on Debian Bullseye and MacOS.
Debian is ran on AWS ARM & Intel based systems. It is run on Windows WSL2 based Debian instances too.

JVM (e.g. java -version):

Bundled

Steps to reproduce:

Use a config like this:

input {
  # We read from backup files. Note that the start of the path MUST be absolute
  file {
    path => "/opt/data/backup/ecorp-fluffy-data-2022.33.json"
    sincedb_path => "/tmp/sincedb_restore-ecorp.db"
    start_position => "beginning"
    file_chunk_size => 268435456
    mode => "read"
    codec => "json"
    file_completed_action => "log"
    file_completed_log_path => "/opt/data/logstash/logs/restore-ecorp-fluffy-data.log"
  }
}
filter {
  if "_jsonparsefailure" in [tags] {
    drop { }
  }
# sort
  json {
    source => "message"
  }
}

output {
  # We write to the "new" cluster
  elasticsearch {
    manage_template => false
    sniffing => false
    ilm_enabled => false
     # does not work and fails silent because of missing +.
     index => "ecorp-fluffy-%{YYYY.MM}.01"
     # Does work. the added fixed 01 is to work around another bug/feature
     # index => "ecorp-fluffy-%{+YYYY.MM}.01"
    doc_as_upsert => true
    action => "update"
    document_id => "%{identifier}"
    http_compression => true
    ssl => true
    ssl_certificate_verification => false
    hosts => ["https://localhost:9200"]
    api_key => " not going to tell you"
  }
  # We print dots to see it in action
  stdout {
    codec => "dots"
  }
}

It will fail silently.

Provide logs (if relevant):

That's the problem.... Logstash is silent ;)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions