Skip to content

[BUG] Invalid value for resources results in misleading error message. #591

Open
@AndrzejBaldys

Description

@AndrzejBaldys

System info:

  • OS: [e.g. Linux? MaxOS? Windows?]
  • Linux
  • KubeLinter v0.6.2-0-g191de10fd5

Describe the bug
We are using Terraform provider to deploy our charts. This allows us to provide some values at deployment time (usually infrastructure resources IDs etc.). Recently we've added alerts for pods resource consumption. To be DRY we've moved real values for resources to Terraform files from which they can be used for chart deployments and as alerts thresholds. To be able to template chart we still needed to have some values in resources so we've used "DEPLOY_TIME_VARIABLE" like we do for other deployment time values. After this change kube-linter is failing with error message: .../my-service/templates/service.yaml: (object: <no namespace>/my-service-api-svc /v1, Kind=Service) no pods found matching service labels (map[app:myservice]) (check: dangling-service, remediation: Confirm that your service's selector correctly matches the labels on one of your deployments.) which seems to be not valid as the only change that was done was to values of the resources.

From:

# ./values.yaml

#...
app:
  replicaCount: 3
  resources:
    requests:
      cpu: 15m
      memory: 160Mi
    limits:
      cpu: 60m
      memory: 640Mi

#...

to:

# ./values.yaml

#...
app:
  replicaCount: 3
  resources:
    requests:
      cpu: "DEPLOY_TIME_VARIABLE"
      memory: "DEPLOY_TIME_VARIABLE"
    limits:
      cpu: "DEPLOY_TIME_VARIABLE"
      memory: "DEPLOY_TIME_VARIABLE"
#...

Sample YAML input

# ./Chart.yaml

apiVersion: v0.1
name: app-configuration-proxy
description: App Configuration Proxy Service
type: application
version: __RELEASE_TAG__
appVersion: __RELEASE_TAG__
# ./values.yaml

image:
  registry: __ACR_NAME__
  repository: app-configuration-proxy
  tag: __RELEASE_TAG__

app:
  replicaCount: 3
  resources:
    requests:
      cpu: "DEPLOY_TIME_VARIABLE"
      memory: "DEPLOY_TIME_VARIABLE"
    limits:
      cpu: "DEPLOY_TIME_VARIABLE"
      memory: "DEPLOY_TIME_VARIABLE"

identity:
  type: 0
  resourceId: "DEPLOY_TIME_VARIABLE"
  clientId: "DEPLOY_TIME_VARIABLE"
  serviceAccountName: "DEPLOY_TIME_VARIABLE"

azureAd:
  tenantId: "DEPLOY_TIME_VARIABLE"
  domain: "DEPLOY_TIME_VARIABLE"
  clientId: "DEPLOY_TIME_VARIABLE"
  swaggerclientId: "DEPLOY_TIME_VARIABLE"
  groups:
    deploymentSlot: "DEPLOY_TIME_VARIABLE"
  allowToBeAuthorizedByAcl: "true"

keyvaultName: "DEPLOY_TIME_VARIABLE"

ingress:
  pathPrefix: "/app-configuration-proxy-service"
  defaultHost:
    dnsname: "DEPLOY_TIME_VARIABLE"
    defaultSecretName: "DEPLOY_TIME_VARIABLE"
  sitespecificHosts:
    enabled: false
    dnsZoneName: "DEPLOY_TIME_VARIABLE"
    sitecodes: ["DEPLOY_TIME_VARIABLE"]
    siteSpecificSecretName: "DEPLOY_TIME_VARIABLE"
  wafPolicyId: "DEPLOY_TIME_VARIABLE"

appInfo:
  applicationName: "DEPLOY_TIME_VARIABLE"
  deploymentStampName: "DEPLOY_TIME_VARIABLE"
  slotName: "DEPLOY_TIME_VARIABLE"
  serviceName: "DEPLOY_TIME_VARIABLE"
  environment: "DEPLOY_TIME_VARIABLE"
  component: "DEPLOY_TIME_VARIABLE"

azureAppConfigEndpoint: "DEPLOY_TIME_VARIABLE"

exposeSwagger: "EXPOSE_SWAGGER"
swaggerDescription: __SWAGGER_DESCRIPTION__
swaggerContactEmail: __SWAGGER_CONTACT_EMAIL__

applicationInsights:
  logLevel:
    default: __ApplicationInsights_LogLevel_Default__

logging:
  logLevel:
    default: __Logging_LogLevel_Default__

secretstore:
  deploymentStamp:
    keyvaultName: kv-__APP_NAME__-app-__ENVIRONMENT__
    secrets:
      - kvName: "auto-acpx-sp-client-secret"
        envName: "AzureAd__Secret"

defaultDomain:
  dtlsKeyvaultCertName: my-cert-name

sitespecificDomain:
  enabled: false
  siteTlsKeyvaultCertName: my-cert-name
# ./templates/_helpers.tpl

{{- define "app-configuration-proxy-service.name" -}}
app-configuration-proxy
{{- end }}

{{- define "app-configuration-proxy-service.fullname" -}}
{{ include "app-configuration-proxy-service.name" . }}
{{- end }}

{{- define "app-configuration-proxy-service.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{- define "app-configuration-proxy-service.labels" -}}
helm.sh/chart: {{ include "app-configuration-proxy-service.chart" . }}
{{ include "app-configuration-proxy-service.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{- define "app-configuration-proxy-service.selectorLabels" -}}
app.kubernetes.io/name: {{ include "app-configuration-proxy-service.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{- define "app-configuration-proxy-service.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "app-configuration-proxy-service.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
# ./templates/deployment-stamp-kv-secrets.yaml

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: "{{ include "app-configuration-proxy-service.name" . }}-stamp-secretproviderclass"
spec:
  provider: azure
  secretObjects:
  - data:
      {{- range .Values.secretstore.deploymentStamp.secrets }}
      - objectName: {{ .kvName }}
        key: {{ .kvName }}
      {{- end }}
    secretName: "{{ include "app-configuration-proxy-service.name" . }}-stamp-secrets"
    type: Opaque

  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "false"
    clientID: "{{ .Values.identity.clientId }}"
    keyvaultName: "{{ .Values.secretstore.deploymentStamp.keyvaultName }}"
    cloudName: ""
    objects:  |
      array:
        {{- range .Values.secretstore.deploymentStamp.secrets }}
        - |
          objectName: {{ .kvName }}
          objectType: secret
          objectVersion: ""
        {{- end }}
    tenantId: "{{ .Values.azureAd.tenantId }}"
# ./templates/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "app-configuration-proxy-service.fullname" . }}-deployment
  labels:
    app: appconfigurationproxyservice
    {{- include "app-configuration-proxy-service.labels" . | nindent 4 }}
spec:
  replicas: {{ .Values.app.replicaCount }}
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: appconfigurationproxyservice
  template:
    metadata:
      labels:
        app: appconfigurationproxyservice
        azure.workload.identity/use: "true"
        {{- include "app-configuration-proxy-service.labels" . | nindent 8 }}
    spec:
      serviceAccountName: {{ .Values.identity.serviceAccountName }}
      securityContext:
        runAsUser: 1000
        runAsGroup: 2000
        fsGroup: 2000
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - appconfigurationproxyservice
              topologyKey: kubernetes.io/hostname
      containers:
      - name: app-configuration-proxy
        image: {{ .Values.image.registry }}.azurecr.io/{{ .Values.image.repository }}:{{ .Values.image.tag }}
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
        resources:
          requests:
            cpu: {{ .Values.app.resources.requests.cpu }}
            memory: {{ .Values.app.resources.requests.memory }}
          limits:
            cpu: {{ .Values.app.resources.limits.cpu }}
            memory: {{ .Values.app.resources.limits.memory }}
        ports:
        - containerPort: 8080
        envFrom:
        - secretRef:
            name: {{ include "app-configuration-proxy-service.fullname" . }}-azuread
        env:
          - name: PathPrefix
            value: {{ .Values.ingress.pathPrefix }}
          - name: ExposeSwagger
            value: {{ .Values.exposeSwagger | quote }}
          - name: SwaggerDescription
            value: {{ .Values.swaggerDescription | quote }}
          - name: SwaggerContactEmail
            value: {{ .Values.swaggerContactEmail | quote }}
          - name: APPINSIGHTS_INSTRUMENTATIONKEY
            value: {{ .Values.app_insights_instrumentation_key }}
          - name: AppInfo__ApplicationName
            value: {{ .Values.appInfo.applicationName }}
          - name: AppInfo__DeploymentStampName
            value: {{ .Values.appInfo.deploymentStampName }}
          - name: AppInfo__SlotName
            value: {{ .Values.appInfo.slotName }}
          - name: AppInfo__ServiceName
            value: {{ .Values.appInfo.serviceName }}
          - name: AppInfo__Environment
            value: {{ .Values.appInfo.environment }}
          - name: AppInfo__Component
            value: {{ .Values.appInfo.component }}
          - name: AzureAppConfigEndpoint
            value: {{ .Values.azureAppConfigEndpoint }}
          {{- range .Values.secretstore.deploymentStamp.secrets}}
          - name: {{ .envName}}
            valueFrom:
              secretKeyRef:
                name: "{{ include "app-configuration-proxy-service.name" . }}-stamp-secrets"
                key: {{ .kvName}}
          {{- end }}
        volumeMounts:
          - name: secrets-store-stamp
            mountPath: /mnt/secrets-stamp
            readOnly: true
          - name: tmp
            mountPath: /tmp
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 8080
          periodSeconds: 3
          timeoutSeconds: 1
        livenessProbe:
          httpGet:
            path: /health/live
            port: 8080
          periodSeconds: 3
          timeoutSeconds: 1
      volumes:
      - name: secrets-store-stamp
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: "{{ include "app-configuration-proxy-service.name" . }}-stamp-secretproviderclass"
      - name: tmp
        emptyDir: {}
# ./templates/ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ include "app-configuration-proxy-service.fullname" . }}-api-ingress
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-path-prefix: "/"
    appgw.ingress.kubernetes.io/ssl-redirect: "true"
    appgw.ingress.kubernetes.io/waf-policy-for-path: "{{ .Values.ingress.wafPolicyId }}"
spec:
  rules:
  - host: "{{ .Values.ingress.defaultHost.dnsname }}"
    http:
      paths:
      - path: "{{ .Values.ingress.pathPrefix }}/*"
        pathType: ImplementationSpecific
        backend:
          service:
            name: {{ include "app-configuration-proxy-service.fullname" . }}-api-svc
            port:
              number: 80
  {{- if .Values.ingress.sitespecificHosts.enabled }}
  {{- range .Values.ingress.sitespecificHosts.sitecodes }}
  - host: "{{ . }}.{{ $.Values.ingress.sitespecificHosts.dnsZoneName }}"
    http:
      paths:
        - path: "{{ $.Values.ingress.pathPrefix }}/*"
          pathType: ImplementationSpecific
          backend:
            service:
              name: {{ include "app-configuration-proxy-service.fullname" . }}-api-svc
              port:
                number: 80
  {{- end }}
  {{- end }}
  tls:
  - hosts:
    - "{{ .Values.ingress.defaultHost.dnsname }}"
    secretName: {{ .Values.ingress.defaultHost.defaultSecretName }}
  {{- if .Values.ingress.sitespecificHosts.enabled }}
  - hosts:
    {{- range .Values.ingress.sitespecificHosts.sitecodes }}
    - "{{ . }}.{{ $.Values.ingress.sitespecificHosts.dnsZoneName }}"
    {{- end }}
    secretName: {{ .Values.ingress.sitespecificHosts.siteSpecificSecretName }}
  {{- end }}
# ./templates/secrets.yaml

apiVersion: v1
kind: Secret
metadata:
  name: {{ include "app-configuration-proxy-service.fullname" . }}-azuread
type: Opaque
data:
  AzureAd__Instance: {{ printf "https://login.microsoftonline.com" | b64enc | quote }}
  AzureAd__TenantId: {{ .Values.azureAd.tenantId | b64enc | quote }}
  AzureAd__Domain: {{ .Values.azureAd.domain | b64enc | quote }}
  AzureAd__ClientId: {{ .Values.azureAd.clientId | b64enc | quote }}
  AzureAd__Groups__DeploymentSlot: {{ .Values.azureAd.groups.deploymentSlot | b64enc | quote }}
  AzureAd__AllowWebApiToBeAuthorizedByACL: {{ .Values.azureAd.allowToBeAuthorizedByAcl | b64enc | quote }}
  SwaggerUI__ClientId: {{ .Values.azureAd.swaggerclientId | b64enc | quote }}
# ./templates/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: {{ include "app-configuration-proxy-service.fullname" . }}-api-svc
spec:
  selector:
    app: appconfigurationproxyservice
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
# ./templates/serviceaccounts.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: {{ .Values.identity.clientId }}
  labels:
    azure.workload.identity/use: "true"
  name: {{ .Values.identity.serviceAccountName }}
automountServiceAccountToken: true

Expected behavior

  • Error messages which would clearly indicate root-cause of the issue.
  • It should be possible to configure linter to accept any value for resources including placeholders and only warn user if it cannot check if the value seems to be reasonable but it should not count it as error.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions