proposal for adding a sidecar to kafka pods #185

dinesh-murugiah · 2025-11-11T16:39:56Z

Proposal: Native Sidecar Container Support for Strimzi Components

Summary

Provide a brief summary of the feature you are proposing to add to Strimzi.

This proposal adds first‑class, declarative sidecar container support to Strimzi-managed workloads (starting with Kafka brokers), configured via CRDs under spec.kafka.template.pod.sidecarContainers. Users can attach monitoring, logging, security, and networking sidecars without webhooks or forks. The design introduces a stable CRD schema (SidecarContainer), a validation and conversion pipeline, and generic interfaces so other Strimzi components (KafkaConnect, KafkaBridge, MirrorMaker2, etc.) can adopt it.

Current situation

Describe the current capability Strimzi has in this area.

Strimzi currently lacks native sidecar support. Users rely on:

Mutating webhooks/admission controllers
Forking and patching Strimzi code
Ephemeral containers for ad‑hoc troubleshooting

Limitations of these approaches include fragile upgrades, higher operational complexity, lack of declarative control, and weak integration with Strimzi lifecycle and security primitives.

Motivation

Explain the motivation why this should be added, and what value it brings.

Business value

Operational flexibility: Enable Flexibility for adding additional sidecars as per the custom need of the users
Simplified ops: manage sidecars natively via Strimzi CRDs and GitOps
Cloud‑native patterns: support log forwarders, APM agents, service‑mesh proxies

Representative use cases

Monitoring/Observability: prometheus exporters, APM agents (Datadog, New Relic)
Logging/Auditing: Fluent Bit/Filebeat; audit collectors
Security/Compliance: Vault agent; cert rotation helpers
Networking: traffic analyzers/proxies
Data tooling: lightweight data checks or protocol helpers

Proposal

Provide an introduction to the proposal. Use sub sections to call out considerations, possible delivery mechanisms etc.

This proposal introduces a custom SidecarContainer CRD model (stable, Fabric8‑independent) and a component‑agnostic sidecar pipeline:

Schema & API surface (SidecarContainer) kept minimal and serializable
Validation in CR processing; conversion to Fabric8 Container at runtime
Component interface (SidecarInterface) + SidecarUtils for reuse
PodTemplate placement for per‑pool (KRaft) configuration
Automatic NetworkPolicy enrichment for declared sidecar ports
Volume access model supporting Kafka data (read‑only), ConfigMaps/Secrets, and emptyDir

Rationale for Custom `SidecarContainer` Abstraction

We use a custom SidecarContainer instead of Fabric8 io.fabric8.kubernetes.api.model.Container in CRDs.

Technical justification

Problem – CRD generation incompatibility: Fabric8's Container exposes IntOrString fields (e.g., ports, probe handlers). Strimzi's CRD generator cannot reliably serialize IntOrString to OpenAPI v3, breaking validation and schema generation.
Solution – Custom abstraction: SidecarContainer mirrors essential fields with simple types (String, List, standard objects). Probes use a Strimzi type (SidecarProbe) instead of Fabric8 Probe.

Supported configuration (concise)

Core: name, image, imagePullPolicy, command, args
Env & storage: env (ContainerEnvVar), volumeMounts
Networking: ports (ContainerPort)
Resources: resources (ResourceRequirements) mandatory
Security: securityContext (SecurityContext)
Health: livenessProbe, readinessProbe (custom SidecarProbe)

Benefits

CRD compatibility: Ensures generated CRDs are valid and parseable by Kubernetes
Simplified API surface: Exposes only the configuration options relevant to Strimzi sidecars
Maintainability: Decouples Strimzi's API from potential breaking changes in Fabric8's container model
Type safety: Avoids runtime serialization errors caused by ambiguous field types

API design (small snippet)

@Description("Sidecar container alongside main component")
public class SidecarContainer { 
  private String name;
  private String image;
  private List<ContainerPort> ports;
  private List<EnvVar> env;
  private ResourceRequirements resources; // mandatory
  private List<VolumeMount> volumeMounts;
  private SecurityContext securityContext;
}

CRD location in PodTemplate (Kafka example)

spec:
  kafka:
    template:
      pod:
        sidecarContainers:
          - name: fluent-bit
            image: fluent/fluent-bit:2.2.0
            resources:
              limits: { cpu: "200m", memory: "256Mi" }

Validation & conversion pipeline

Hierarchy: (1) CRD schema → (2) Strimzi API validation → (3) component rules → (4) pod creation.
Key rules: unique names; no conflict with Strimzi-managed names; resource limits required; ports must not collide with component‑reserved ports; volume mounts must exist and avoid conflicts.

Interface & utils (signatures only)

public interface SidecarInterface {
  void validateSidecarContainers(...);
  List<Container> createSidecarContainers(PodTemplate pod);
}

// Conversion
List<Container> convertSidecarContainers(PodTemplate pod);

Network policies & volumes

NetworkPolicy: Extract sidecar ports from all pools; add ingress rules for those ports by default. Users can further restrict via standard policies.
Volumes: Support read‑only Kafka data mounts for log access, user‑defined volumes in pod.template, Secrets/ConfigMaps, and emptyDir.

SidecarProbe (overview)

A Strimzi probe type used by sidecars to avoid IntOrString in CRDs and keep schemas simple.

Compact class sketch

@JsonInclude(NON_NULL)
@JsonPropertyOrder({ "execCommand", "httpGetPath", "httpGetPort", "httpGetScheme",
  "tcpSocketPort", "initialDelaySeconds", "timeoutSeconds", "periodSeconds",
  "successThreshold", "failureThreshold" })
public class SidecarProbe implements UnknownPropertyPreserving { 
  private List<String> execCommand;
  private String httpGetPath, httpGetPort, httpGetScheme, tcpSocketPort;
  private int initialDelaySeconds = 15, timeoutSeconds = 5, periodSeconds = 10;
  private int successThreshold = 1, failureThreshold = 3;
  private Map<String,Object> additionalProperties;
}

Highlights

Supports exec, httpGet, tcpSocket styles via simple String fields (no IntOrString).
Sensible defaults: initialDelaySeconds=15, timeoutSeconds=5, periodSeconds=10.
Min validations (e.g., timeout/period/success/failure thresholds ≥ 1).
Preserves unknown properties for forward compatibility.

Minimal examples

Single sidecar (log forwarder)

spec:
  kafka:
    template:
      pod:
        sidecarContainers:
        - name: fluent-bit
          image: fluent/fluent-bit:2.2.0
          resources:
            limits: { cpu: "200m", memory: "256Mi" }

Per‑pool configuration (KRaft)

kind: KafkaNodePool
spec:
  roles: [broker]
  template:
    pod:
      sidecarContainers:
      - name: log-forwarder
        image: fluent/fluent-bit:2.2.0
        resources:
          limits: { cpu: "200m", memory: "256Mi" }

Affected/not affected projects

Call out the projects in the Strimzi organisation that are/are not affected by this proposal.

Affected (Phase 1)

strimzi-kafka-operator (api: new SidecarContainer; operator: SidecarInterface, SidecarUtils, KafkaCluster integration; crd‑generator updates)

Future (Phase 2+)

KafkaConnect, KafkaBridge, MirrorMaker2, EntityOperator (implement interface; add reserved ports/names)

Not affected

strimzi-kafka-oauth, test-container, drain-cleaner, access-operator (no changes)

Compatibility

Call out any future or backwards compatibility considerations this proposal has accounted for.

Backward compatible: Field is optional; no change for existing CRs.
Forward compatible: Stable custom model avoids Fabric8 IntOrString issues in schema generation.
Rolling behavior: Add/modify/remove sidecars via standard rolling updates.
Versioning: Introduce in Strimzi ≥ 0.50.0; other components can adopt incrementally.

Rejected alternatives

Call out options that were considered while creating this proposal, but then later rejected, along with reasons why.

Expose Fabric8 Container directly — rejected due to OpenAPI/CRD stability and IntOrString schema issues.
Separate KafkaSidecar CRD — rejected for not having a generic placeholder for future expandability.
Operator‑level auto‑injection — rejected to keep explicit user control and avoid risky implicit coupling.
InitContainer/DaemonSet patterns — do not satisfy long‑running, pod‑local sidecar use cases.
Helm‑only customization — not portable; breaks on operator‑managed rollouts.

References

References (informative)

Strimzi Pod Templates (docs)
Kubernetes multi‑container pods (docs)
Implementation PR: strimzi/strimzi‑kafka‑operator#12121

…dd for others later Signed-off-by: dinesh-murugiah <[email protected]>

scholzj

Thanks for the proposal. I left some comments ... but some generic notes:

My main concern is that this is oversimplified. It would work for simple usecases where sidecars are not the best choice in the first place and where if you want to use sidecars you can easily do so through Mutating Admission Controllers.
It does not solve the real issues sidecars would need to solve which are Kafka-aware proxies. But because this creates the API, it will make it harder to add support for it later. I believe this cannot be tackled as the easy parts first but it needs to start from the hard parts to make sure the API and UX are able to handle the hardest use-cases before we adopt it.
The CRD side with the whole Sidecar API is a major concern. The CRDs are already today at the edge of their size and we have to strip the descriptions to make them easy to use. Again something where mutating admission controlers do a better job.

Aside from the technicl comments, better formatting would help the readability. Thigs such as:

Sentence per line
Empty lines before and after headers
etc.

scholzj · 2025-11-11T19:52:23Z