ArgoCD: periodic sync since helm 3.13.0: annotations are null

[volker-raschek]

Hello everyone,
ArgoCD has installed version 3.13.0 of the metrics-server helm chart. This version introduces annotations, which are null. Attributes with null values are removed or not saved by kubernetes. ArgoCD recognises this as a drift, as it is a deviation between the defined state (helm) and the cluster (live). ArgoCD therefore triggers a rolling update to restore annotations: null. However, Kubernetes will remove the annotations: null again. The deployment is therefore permanently in an OutOfSync status and triggers every three minutes a rolling update to avoid the drift. See screenshot.

As a temporary workaround, the ArgoCD application specification has been adjusted to ignore the annotation: null in the diff.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: metrics-server
spec:
  project: default
  ignoreDifferences:
  - group: apiregistration.k8s.io
    kind: APIService
    name: v1beta1.metrics.k8s.io
    jsonPointers:
    - /metadata/annotations

However, the goal should be that annotation: null is not rendered at all in the helm chart. To avoid a regression in future, give the helm plugin unittest a try and implement unittests to avoid templating empty annotations.

Volker

[richabanker]

/triage accepted
cc @dgrisonnet