Skip to content

Commit 8f58c7a

Browse files
authored
Merge pull request #207 from logstash-plugins/custom-headers
Adding support for custom headers
2 parents 56d923f + 6ef0ffc commit 8f58c7a

File tree

3 files changed

+47
-16
lines changed

3 files changed

+47
-16
lines changed

docs/index.asciidoc

Lines changed: 25 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ include::{include_path}/plugin_header.asciidoc[]
2323

2424
Read from an Elasticsearch cluster, based on search query results.
2525
This is useful for replaying test logs, reindexing, etc.
26-
You can periodically schedule ingestion using a cron syntax
26+
You can periodically schedule ingestion using a cron syntax
2727
(see `schedule` setting) or run the query one time to load
2828
data into Logstash.
2929

@@ -112,6 +112,7 @@ Please check out <<plugins-{type}s-{plugin}-obsolete-options>> for details.
112112
| <<plugins-{type}s-{plugin}-cloud_auth>> |<<password,password>>|No
113113
| <<plugins-{type}s-{plugin}-cloud_id>> |<<string,string>>|No
114114
| <<plugins-{type}s-{plugin}-connect_timeout_seconds>> | <<number,number>>|No
115+
| <<plugins-{type}s-{plugin}-custom_headers>> |<<hash,hash>>|No
115116
| <<plugins-{type}s-{plugin}-docinfo>> |<<boolean,boolean>>|No
116117
| <<plugins-{type}s-{plugin}-docinfo_fields>> |<<array,array>>|No
117118
| <<plugins-{type}s-{plugin}-docinfo_target>> |<<string,string>>|No
@@ -205,8 +206,18 @@ For more info, check out the
205206
The maximum amount of time, in seconds, to wait while establishing a connection to Elasticsearch.
206207
Connect timeouts tend to occur when Elasticsearch or an intermediate proxy is overloaded with requests and has exhausted its connection pool.
207208

209+
[id="plugins-{type}s-{plugin}-custom_headers"]
210+
===== `custom_headers`
211+
212+
* Value type is <<hash,hash>>
213+
* Default value is empty
214+
215+
Pass a set of key value pairs as the headers sent in each request to an elasticsearch node.
216+
The headers will be used for any kind of request.
217+
These custom headers will override any headers previously set by the plugin such as the User Agent or Authorization headers.
218+
208219
[id="plugins-{type}s-{plugin}-docinfo"]
209-
===== `docinfo`
220+
===== `docinfo`
210221

211222
* Value type is <<boolean,boolean>>
212223
* Default value is `false`
@@ -257,7 +268,7 @@ Example
257268

258269

259270
[id="plugins-{type}s-{plugin}-docinfo_fields"]
260-
===== `docinfo_fields`
271+
===== `docinfo_fields`
261272

262273
* Value type is <<array,array>>
263274
* Default value is `["_index", "_type", "_id"]`
@@ -268,7 +279,7 @@ option lists the metadata fields to save in the current event. See
268279
more information.
269280

270281
[id="plugins-{type}s-{plugin}-docinfo_target"]
271-
===== `docinfo_target`
282+
===== `docinfo_target`
272283

273284
* Value type is <<string,string>>
274285
* Default value depends on whether <<plugins-{type}s-{plugin}-ecs_compatibility>> is enabled:
@@ -292,7 +303,7 @@ this option names the field under which to store the metadata fields as subfield
292303
Controls this plugin's compatibility with the {ecs-ref}[Elastic Common Schema (ECS)].
293304

294305
[id="plugins-{type}s-{plugin}-hosts"]
295-
===== `hosts`
306+
===== `hosts`
296307

297308
* Value type is <<array,array>>
298309
* There is no default value for this setting.
@@ -302,18 +313,18 @@ can be either IP, HOST, IP:port, or HOST:port. The port defaults to
302313
9200.
303314

304315
[id="plugins-{type}s-{plugin}-index"]
305-
===== `index`
316+
===== `index`
306317

307318
* Value type is <<string,string>>
308319
* Default value is `"logstash-*"`
309320

310-
The index or alias to search.
321+
The index or alias to search.
311322
Check out {ref}/api-conventions.html#api-multi-index[Multi Indices
312323
documentation] in the Elasticsearch documentation for info on
313324
referencing multiple indices.
314325

315326
[id="plugins-{type}s-{plugin}-password"]
316-
===== `password`
327+
===== `password`
317328

318329
* Value type is <<password,password>>
319330
* There is no default value for this setting.
@@ -333,7 +344,7 @@ An empty string is treated as if proxy was not set, this is useful when using
333344
environment variables e.g. `proxy => '${LS_PROXY:}'`.
334345

335346
[id="plugins-{type}s-{plugin}-query"]
336-
===== `query`
347+
===== `query`
337348

338349
* Value type is <<string,string>>
339350
* Default value is `'{ "sort": [ "_doc" ] }'`
@@ -381,7 +392,7 @@ The default is 0 (no retry). This value should be equal to or greater than zero.
381392
NOTE: Partial failures - such as errors in a subset of all slices - can result in the entire query being retried, which can lead to duplication of data. Avoiding this would require Logstash to store the entire result set of a query in memory which is often not possible.
382393

383394
[id="plugins-{type}s-{plugin}-schedule"]
384-
===== `schedule`
395+
===== `schedule`
385396

386397
* Value type is <<string,string>>
387398
* There is no default value for this setting.
@@ -393,7 +404,7 @@ There is no schedule by default. If no schedule is given, then the statement is
393404
exactly once.
394405

395406
[id="plugins-{type}s-{plugin}-scroll"]
396-
===== `scroll`
407+
===== `scroll`
397408

398409
* Value type is <<string,string>>
399410
* Default value is `"1m"`
@@ -416,7 +427,7 @@ The query requires at least one `sort` field, as described in the <<plugins-{typ
416427
`scroll` uses {ref}/paginate-search-results.html#scroll-search-results[scroll] API to search, which is no longer recommended.
417428

418429
[id="plugins-{type}s-{plugin}-size"]
419-
===== `size`
430+
===== `size`
420431

421432
* Value type is <<number,number>>
422433
* Default value is `1000`
@@ -606,7 +617,7 @@ It is also possible to target an entry in the event's metadata, which will be av
606617

607618

608619
[id="plugins-{type}s-{plugin}-user"]
609-
===== `user`
620+
===== `user`
610621

611622
* Value type is <<string,string>>
612623
* There is no default value for this setting.
@@ -620,7 +631,7 @@ empty string authentication will be disabled.
620631
==== Elasticsearch Input Obsolete Configuration Options
621632

622633
WARNING: As of version `5.0.0` of this plugin, some configuration options have been replaced.
623-
The plugin will fail to start if it contains any of these obsolete options.
634+
The plugin will fail to start if it contains any of these obsolete options.
624635

625636

626637
[cols="<,<",options="header",]

lib/logstash/inputs/elasticsearch.rb

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,9 @@
2424
# called `http.content_type.required`. If this option is set to `true`, and you
2525
# are using Logstash 2.4 through 5.2, you need to update the Elasticsearch input
2626
# plugin to version 4.0.2 or higher.
27-
#
27+
#
2828
# ================================================================================
29-
#
29+
#
3030
# Read from an Elasticsearch cluster, based on search query results.
3131
# This is useful for replaying test logs, reindexing, etc.
3232
# It also supports periodically scheduling lookup enrichments
@@ -166,6 +166,9 @@ class LogStash::Inputs::Elasticsearch < LogStash::Inputs::Base
166166
# http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_document_metadata.html
167167
config :docinfo_fields, :validate => :array, :default => ['_index', '_type', '_id']
168168

169+
# Custom headers for Elasticsearch requests
170+
config :custom_headers, :validate => :hash, :default => {}
171+
169172
# Basic Auth - username
170173
config :user, :validate => :string
171174

@@ -298,6 +301,7 @@ def register
298301
transport_options[:headers].merge!(setup_basic_auth(user, password))
299302
transport_options[:headers].merge!(setup_api_key(api_key))
300303
transport_options[:headers].merge!({'user-agent' => prepare_user_agent()})
304+
transport_options[:headers].merge!(@custom_headers) unless @custom_headers.empty?
301305
transport_options[:request_timeout] = @request_timeout_seconds unless @request_timeout_seconds.nil?
302306
transport_options[:connect_timeout] = @connect_timeout_seconds unless @connect_timeout_seconds.nil?
303307
transport_options[:socket_timeout] = @socket_timeout_seconds unless @socket_timeout_seconds.nil?

spec/inputs/elasticsearch_spec.rb

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,22 @@
116116
expect( extract_transport(client).options[:transport_options][:headers] ).to match hash_including("x-elastic-product-origin"=>"logstash-input-elasticsearch")
117117
end
118118
end
119+
120+
context "with custom headers" do
121+
let(:config) do
122+
{
123+
"schedule" => "* * * * * UTC",
124+
"custom_headers" => { "Custom-Header-1" => "Custom Value 1", "Custom-Header-2" => "Custom Value 2" }
125+
}
126+
end
127+
128+
129+
it "sets custom headers" do
130+
plugin.register
131+
client = plugin.send(:client)
132+
expect( extract_transport(client).options[:transport_options][:headers] ).to match hash_including(config["custom_headers"])
133+
end
134+
end
119135
end
120136

121137
context "retry" do

0 commit comments

Comments
 (0)