You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Authentication to a secure Elasticsearch cluster is possible using _one_ of the following options:
75
+
76
+
* <<plugins-{type}s-{plugin}-user>> AND <<plugins-{type}s-{plugin}-password>>
77
+
* <<plugins-{type}s-{plugin}-cloud_auth>>
78
+
* <<plugins-{type}s-{plugin}-api_key>>
79
+
80
+
[id="plugins-{type}s-{plugin}-autz"]
81
+
==== Authorization
82
+
83
+
Authorization to a secure Elasticsearch cluster requires `read` permission at index level and `monitoring` permissions at cluster level.
84
+
The `monitoring` permission at cluster level is necessary to perform periodic connectivity checks.
85
+
86
+
[id="plugins-{type}s-{plugin}-ecs"]
87
+
==== Compatibility with the Elastic Common Schema (ECS)
88
+
89
+
When ECS compatibility is disabled, `docinfo_target` uses the `"@metadata"` field as a default, with ECS enabled the plugin
90
+
uses a naming convention `"[@metadata][input][elasticsearch]"` as a default target for placing document information.
91
+
92
+
The plugin logs a warning when ECS is enabled and `target` isn't set.
93
+
94
+
TIP: Set the `target` option to avoid potential schema conflicts.
95
+
96
+
[id="plugins-{type}s-{plugin}-failure-handling"]
97
+
==== Failure handling
98
+
99
+
When this input plugin cannot create a structured `Event` from a hit result, it will instead create an `Event` that is tagged with `_elasticsearch_input_failure` whose `[event][original]` is a JSON-encoded string representation of the entire hit.
100
+
101
+
Common causes are:
102
+
103
+
- When the hit result contains top-level fields that are {logstash-ref}/processing.html#reserved-fields[reserved in Logstash] but do not have the expected shape. Use the <<plugins-{type}s-{plugin}-target>> directive to avoid conflicts with the top-level namespace.
104
+
- When <<plugins-{type}s-{plugin}-docinfo>> is enabled and the docinfo fields cannot be merged into the hit result. Combine <<plugins-{type}s-{plugin}-target>> and <<plugins-{type}s-{plugin}-docinfo_target>> to avoid conflict.
51
105
52
106
[id="plugins-{type}s-{plugin}-cursor"]
53
107
==== Tracking a field's value across runs
54
108
109
+
NOTE: experimental:[] `tracking_field` and related settings are experimental and subject to change in the future
110
+
55
111
It is sometimes desirable to track the value of a particular field between two jobs:
56
112
* avoid re-processing the entire result set of a long query after an unplanned restart
57
113
* only grab new data from an index instead of processing the entire set on each job
@@ -163,61 +219,6 @@ With this setup, as new documents are indexed an `test-*` index, the next schedu
163
219
. use PIT+search_after to paginate through all the data;
164
220
. update the value of the field at the end of the pagination.
165
221
166
-
[id="plugins-{type}s-{plugin}-scheduling"]
167
-
==== Scheduling
168
-
169
-
Input from this plugin can be scheduled to run periodically according to a specific
170
-
schedule. This scheduling syntax is powered by https://github.com/jmettraux/rufus-scheduler[rufus-scheduler].
171
-
The syntax is cron-like with some extensions specific to Rufus (e.g. timezone support ).
Authentication to a secure Elasticsearch cluster is possible using _one_ of the following options:
190
-
191
-
* <<plugins-{type}s-{plugin}-user>> AND <<plugins-{type}s-{plugin}-password>>
192
-
* <<plugins-{type}s-{plugin}-cloud_auth>>
193
-
* <<plugins-{type}s-{plugin}-api_key>>
194
-
195
-
[id="plugins-{type}s-{plugin}-autz"]
196
-
==== Authorization
197
-
198
-
Authorization to a secure Elasticsearch cluster requires `read` permission at index level and `monitoring` permissions at cluster level.
199
-
The `monitoring` permission at cluster level is necessary to perform periodic connectivity checks.
200
-
201
-
[id="plugins-{type}s-{plugin}-ecs"]
202
-
==== Compatibility with the Elastic Common Schema (ECS)
203
-
204
-
When ECS compatibility is disabled, `docinfo_target` uses the `"@metadata"` field as a default, with ECS enabled the plugin
205
-
uses a naming convention `"[@metadata][input][elasticsearch]"` as a default target for placing document information.
206
-
207
-
The plugin logs a warning when ECS is enabled and `target` isn't set.
208
-
209
-
TIP: Set the `target` option to avoid potential schema conflicts.
210
-
211
-
[id="plugins-{type}s-{plugin}-failure-handling"]
212
-
==== Failure handling
213
-
214
-
When this input plugin cannot create a structured `Event` from a hit result, it will instead create an `Event` that is tagged with `_elasticsearch_input_failure` whose `[event][original]` is a JSON-encoded string representation of the entire hit.
215
-
216
-
Common causes are:
217
-
218
-
- When the hit result contains top-level fields that are {logstash-ref}/processing.html#reserved-fields[reserved in Logstash] but do not have the expected shape. Use the <<plugins-{type}s-{plugin}-target>> directive to avoid conflicts with the top-level namespace.
219
-
- When <<plugins-{type}s-{plugin}-docinfo>> is enabled and the docinfo fields cannot be merged into the hit result. Combine <<plugins-{type}s-{plugin}-target>> and <<plugins-{type}s-{plugin}-docinfo_target>> to avoid conflict.
0 commit comments