Kube-Prometheus-Stack And ArgoCD 2.3 - How to remove a workaround
"prometheuses.monitoring.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes
!!Update!!
See my latest article on using `Server-Side apply``
This is an update of my previous article:
Because with the release of ArgoCD 2.3, we now have a better way to solve this issue.
This involves two easy steps in our recipe, with splitting the deployment in two sepereate ArgoCD Applications
:
Application I: skipCrds
With the pull request #8012, the team around ArgoCD introduced following cool feature: "feat: add skipCrds flag for helm charts". This means we can skip the installation of CRDs.
What are CRS, short refresher from the official kubernetes documentation:
Custom Resources
Custom resources are extensions of the Kubernetes API. This page discusses when to add a custom resource to your Kubernetes cluster and when to use a standalone service. It describes the two methods for adding custom resources and how to choose between them.
Helm installs custom resource definitions in the crds
folder by default if they are not existing and with skipCrds
we can now control this behaviour.
To get kube-prometheus-stack
working, we will set this flag to false
. The first application defintions looks like this:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: kube-prometheus-stack
annotations:
argocd.argoproj.io/sync-wave: "3"
spec:
destination:
name: in-cluster
namespace: monitoring
project: default
source:
repoURL: 'https://prometheus-community.github.io/helm-charts'
targetRevision: 34.1.1
helm:
skipCrds: true
values: |-
#Snip the values
chart: kube-prometheus-stack
syncPolicy:
automated:
prune: true
selfHeal: true
Application II: Sync options Replace
The second part, is to use Replace
then by default, ArgoCD executes kubectl apply operation to apply the configuration stored in Git. This is now not suitable as the resource spec might is too big and won't fit into kubectl.kubernetes.io/last-applied-configuration
.
If the Replace=true
sync option is set, ArgoCD will use kubectl replace
or kubectl create
command to apply changes.
The second application defintion looks like this:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: kube-prometheus-stack-crds
annotations:
argocd.argoproj.io/sync-wave: "2"
spec:
destination:
name: in-cluster
namespace: monitoring
project: default
source:
repoURL: https://github.com/prometheus-community/helm-charts.git
path: charts/kube-prometheus-stack/crds/
targetRevision: kube-prometheus-stack-34.1.1
directory:
recurse: true
syncPolicy:
syncOptions:
- CreateNamespace=true
- Replace=true
automated:
prune: true
selfHeal: true
See the documentation for more details on sync options.
The End
No external code this time