In our cluster, we pre-create the index with custom mappings, analyzers, and other configurations.
Currently, the connector always attempts to create the index, which is unnecessary in our setup:
|
private void ensureIndexExists(String index) { |
|
if (!indexCache.contains(index)) { |
|
log.info("Creating index {}.", index); |
|
client.createIndexOrDataStream(index); |
|
indexCache.add(index); |
Although it does not create it again, it does issue an unnecessary call(and generates confusing logs) to our cluster
|
GetIndexRequest request = new GetIndexRequest(index); |
|
return callWithRetries( |
|
"check if index " + index + " exists", |
|
() -> client.indices().exists(request, RequestOptions.DEFAULT) |
|
); |
The connector should be configurable to assume the index already exists and skip index creation attempts.
In our cluster, we pre-create the index with custom mappings, analyzers, and other configurations.
Currently, the connector always attempts to create the index, which is unnecessary in our setup:
kafka-connect-elasticsearch/src/main/java/io/confluent/connect/elasticsearch/ElasticsearchSinkTask.java
Lines 240 to 244 in d394475
Although it does not create it again, it does issue an unnecessary call(and generates confusing logs) to our cluster
kafka-connect-elasticsearch/src/main/java/io/confluent/connect/elasticsearch/ElasticsearchClient.java
Lines 408 to 412 in d394475
The connector should be configurable to assume the index already exists and skip index creation attempts.