[NEW SCENARIO] Add CSV ingestion example

[agardnerIT]

Add a new scenario which demonstrates how to watch a CSV file and send it into Dynatrace via the collector.

API Token

Create an Dynatrace API Token with logs.ingest permissions.

config.yaml

# Collector will monitor the included files
# and poll for new content every poll_interval
receivers:
  filelog:
    
    include:
      - "C:\\path\\to\\a file.csv"
    #start_at: beginning
    poll_interval: 60s
    include_file_name: true
    include_file_path: true
    operators:
      - type: csv_parser
        # Add your first row names here
        # for example:
        # header: Post,Asked By,Answered,Date Posted
        delimiter: ","

# Enrich the logs with the host metadata so you know where the log is coming from
# Batch up the records rather than sending line by line
# Send in batches of 50 OR send whatever you have every 1s
# Whichever comes first
processors:
  resourcedetection:
    detectors: ["system"]
    system:
      hostname_sources: ["os"]
  batch:
    timeout: 1s
    send_batch_size: 50

  # Drop telemetry if...
  # Condition are an OR
  filter:
    # Field is empty
    error_mode: ignore
    logs:
      log_record:
        - IsMatch(log.attributes["Post"], "")
# Where should the data be sent?
exporters:
  otlphttp:
    endpoint: "${env:DT_ENDPOINT}"
    headers:
      Authorization: "Api-Token ${env:DT_API_TOKEN}"

# Now wire it all together
service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [filter, resourcedetection, batch]
      exporters: [otlphttp]

Start Collector

set DT_ENDPOINT=https://abc12345.live.dynatrace.com/api/v2/otlp
set DT_API_TOKEN=dt0c01.sample.secret
"C:\path\to\collector.exe" --config="C:\path\to\config.yaml"