Data Stream naming is far too restrictive

[morrone]

The approach to Data Stream naming in the connector is too restrictive. Right now the only option we have is to pattern the data stream name like so:

<data.stream.type>-<data.stream.dataset>-<topic>

This pattern doesn't even really allow us to match the common data steam naming scheme which, in kafka terms, would look a bit more like:

<type>-<topic>-<namespace>

And further, the data.stream.type can only be "metrics" or "logs".

None of these restrictions exist in elasticsearch itself; they are only imposed by the connector.

We should really be able to specify the entire data stream name, using some basic substitution variables (for topic name, etc.) in the string.

[rz1027]

+1

[rz1027]

Was this problem ever tackled ? @sduff @mcb

[rz1027]

The method is using
data.stream.namespace property to change the output datastream name