Kube-Prometheus-Stack And ArgoCD 2.3 - How to remove a workaround

Kube-Prometheus-Stack And ArgoCD 2.3 - How to remove a workaround

"prometheuses.monitoring.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

!!Update!!

See my latest article on using `Server-Side apply``

image.png

This is an update of my previous article:

Because with the release of ArgoCD 2.3, we now have a better way to solve this issue.

muppets-swedish-chef.gif

This involves two easy steps in our recipe, with splitting the deployment in two sepereate ArgoCD Applications:

Application I: skipCrds

With the pull request #8012, the team around ArgoCD introduced following cool feature: "feat: add skipCrds flag for helm charts". This means we can skip the installation of CRDs.

What are CRS, short refresher from the official kubernetes documentation:

Custom Resources

Custom resources are extensions of the Kubernetes API. This page discusses when to add a custom resource to your Kubernetes cluster and when to use a standalone service. It describes the two methods for adding custom resources and how to choose between them.

Helm installs custom resource definitions in the crds folder by default if they are not existing and with skipCrds we can now control this behaviour.

To get kube-prometheus-stack working, we will set this flag to false. The first application defintions looks like this:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: kube-prometheus-stack
  annotations:
    argocd.argoproj.io/sync-wave: "3"
spec:
  destination:
    name: in-cluster
    namespace: monitoring
  project: default
  source:
    repoURL: 'https://prometheus-community.github.io/helm-charts'
    targetRevision: 34.1.1
    helm:
      skipCrds: true
      values: |-
        #Snip the values
    chart: kube-prometheus-stack
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Application II: Sync options Replace

The second part, is to use Replace then by default, ArgoCD executes kubectl apply operation to apply the configuration stored in Git. This is now not suitable as the resource spec might is too big and won't fit into kubectl.kubernetes.io/last-applied-configuration.

If the Replace=true sync option is set, ArgoCD will use kubectl replace or kubectl create command to apply changes.

The second application defintion looks like this:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: kube-prometheus-stack-crds
  annotations:
    argocd.argoproj.io/sync-wave: "2"
spec:
  destination:
    name: in-cluster
    namespace: monitoring
  project: default
  source:
    repoURL: https://github.com/prometheus-community/helm-charts.git
    path: charts/kube-prometheus-stack/crds/
    targetRevision: kube-prometheus-stack-34.1.1
    directory:
      recurse: true
  syncPolicy:
    syncOptions:
      - CreateNamespace=true
      - Replace=true
    automated:
      prune: true
      selfHeal: true

See the documentation for more details on sync options.

image.png

The End

No external code this time

image.png