![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![Version: 0.38.0](https://img.shields.io/badge/Version-0.38.0-informational?style=flat-square) [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/victoriametrics)](https://artifacthub.io/packages/helm/victoriametrics/victoria-metrics-operator) Victoria Metrics Operator ## Prerequisites * Install the follow packages: ``git``, ``kubectl``, ``helm``, ``helm-docs``. See this [tutorial](https://docs.victoriametrics.com/helm/requirements/). * PV support on underlying infrastructure. ## ArgoCD issues When running operator using ArgoCD without Cert Manager (`.Values.admissionWebhooks.certManager.enabled: false`) it will rerender webhook certificates on each sync since Helm `lookup` function is not respected by ArgoCD. To prevent this please update you operator Application `spec.syncPolicy` and `spec.ignoreDifferences` with a following: ```yaml apiVersion: argoproj.io/v1alpha1 kind: Application ... spec: ... syncPolicy: syncOptions: # https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#respect-ignore-difference-configs # argocd must also ignore difference during apply stage # otherwise it ll silently override changes and cause a problem - RespectIgnoreDifferences=true ignoreDifferences: - group: "" kind: Secret name: -validation namespace: kube-system jsonPointers: - /data - group: admissionregistration.k8s.io kind: ValidatingWebhookConfiguration name: -admission jqPathExpressions: - '.webhooks[]?.clientConfig.caBundle' ``` where `` is output of `{{ include "vm-operator.fullname" }}` for your setup ## Upgrade guide During release an issue with helm CRD was discovered. So for upgrade from version less then 0.1.3 you have to two options: 1) use helm management for CRD, enabled by default. 2) use own management system, need to add variable: --set createCRD=false. If you choose helm management, following steps must be done before upgrade: 1) define namespace and helm release name variables ``` export NAMESPACE=default export RELEASE_NAME=operator ``` execute kubectl commands: ``` kubectl get crd | grep victoriametrics.com | awk '{print $1 }' | xargs -i kubectl label crd {} app.kubernetes.io/managed-by=Helm --overwrite kubectl get crd | grep victoriametrics.com | awk '{print $1 }' | xargs -i kubectl annotate crd {} meta.helm.sh/release-namespace="$NAMESPACE" meta.helm.sh/release-name="$RELEASE_NAME" --overwrite ``` run helm upgrade command. ## Chart Details This chart will do the following: * Rollout victoria metrics operator ## How to install Access a Kubernetes cluster. ### Setup chart repository (can be omitted for OCI repositories) Add a chart helm repository with follow commands: ```console helm repo add vm https://victoriametrics.github.io/helm-charts/ helm repo update ``` List versions of `vm/victoria-metrics-operator` chart available to installation: ```console helm search repo vm/victoria-metrics-operator -l ``` ### Install `victoria-metrics-operator` chart Export default values of `victoria-metrics-operator` chart to file `values.yaml`: - For HTTPS repository ```console helm show values vm/victoria-metrics-operator > values.yaml ``` - For OCI repository ```console helm show values oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator > values.yaml ``` Change the values according to the need of the environment in ``values.yaml`` file. Test the installation with command: - For HTTPS repository ```console helm install vmo vm/victoria-metrics-operator -f values.yaml -n NAMESPACE --debug --dry-run ``` - For OCI repository ```console helm install vmo oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator -f values.yaml -n NAMESPACE --debug --dry-run ``` Install chart with command: - For HTTPS repository ```console helm install vmo vm/victoria-metrics-operator -f values.yaml -n NAMESPACE ``` - For OCI repository ```console helm install vmo oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator -f values.yaml -n NAMESPACE ``` Get the pods lists by running this commands: ```console kubectl get pods -A | grep 'vmo' ``` Get the application by running this command: ```console helm list -f vmo -n NAMESPACE ``` See the history of versions of `vmo` application with command. ```console helm history vmo -n NAMESPACE ``` ## Validation webhook Its possible to use validation of created resources with operator. For now, you need cert-manager to easily certificate management https://cert-manager.io/docs/ ```yaml admissionWebhooks: enabled: true # what to do in case, when operator not available to validate request. certManager: # enables cert creation and injection by cert-manager enabled: true ``` ## How to uninstall Remove application with command. ```console helm uninstall vmo -n NAMESPACE ``` ## Documentation of Helm Chart Install ``helm-docs`` following the instructions on this [tutorial](https://docs.victoriametrics.com/helm/requirements/). Generate docs with ``helm-docs`` command. ```bash cd charts/victoria-metrics-operator helm-docs ``` The markdown generation is entirely go template driven. The tool parses metadata from charts and generates a number of sub-templates that can be referenced in a template file (by default ``README.md.gotmpl``). If no template file is provided, the tool has a default internal template that will generate a reasonably formatted README. ## Parameters The following tables lists the configurable parameters of the chart and their default values. Change the values according to the need of the environment in ``victoria-metrics-operator/values.yaml`` file.
Key Type Default Description
admissionWebhooks object
certManager:
    enabled: false
    issuer: {}
enabled: true
enabledCRDValidation:
    vlogs: true
    vmagent: true
    vmalert: true
    vmalertmanager: true
    vmalertmanagerconfig: true
    vmauth: true
    vmcluster: true
    vmrule: true
    vmsingle: true
    vmuser: true
keepTLSSecret: true
policy: Fail
tls:
    caCert: null
    cert: null
    key: null

Configures resource validation

admissionWebhooks.certManager object
enabled: false
issuer: {}

Enables custom ca bundle, if you are not using cert-manager. In case of custom ca, you have to create secret - {chart-name}-validation with keys: tls.key, tls.crt, ca.crt

admissionWebhooks.certManager.enabled bool
false

Enables cert creation and injection by cert-manager.

admissionWebhooks.certManager.issuer object
{}

If needed, provide own issuer. Operator will create self-signed if empty.

admissionWebhooks.enabled bool
true

Enables validation webhook.

admissionWebhooks.policy string
Fail

What to do in case, when operator not available to validate request.

affinity object
{}

Pod affinity

annotations object
{}

Annotations to be added to the all resources

crds.cleanup.enabled bool
false

Tells helm to clean up all the vm resources under this release’s namespace when uninstalling

crds.cleanup.image object
pullPolicy: IfNotPresent
repository: bitnami/kubectl
tag: ""

Image configuration for CRD cleanup Job

crds.cleanup.resources object
limits:
    cpu: 500m
    memory: 256Mi
requests:
    cpu: 100m
    memory: 56Mi

Cleanup hook resources

crds.enabled bool
true

manages CRD creation. Disables CRD creation only in combination with crds.plain: false due to helm dependency conditions limitation

crds.plain bool
false

check if plain or templated CRDs should be created. with this option set to false, all CRDs will be rendered from templates. with this option set to true, all CRDs are immutable and require manual upgrade.

env list
[]

Extra settings for the operator deployment. Full list here

envFrom list
[]

Specify alternative source for env variables

extraArgs object
{}

Operator container additional commandline arguments

extraContainers list
[]

Extra containers to run in a pod with operator

extraHostPathMounts list
[]

Additional hostPath mounts

extraLabels object
{}

Labels to be added to the all resources

extraObjects list
[]

Add extra specs dynamically to this chart

extraVolumeMounts list
[]

Extra Volume Mounts for the container

extraVolumes list
[]

Extra Volumes for the pod

fullnameOverride string
""

Overrides the full name of server component resources

global.cluster.dnsDomain string
cluster.local.

K8s cluster domain suffix, uses for building storage pods’ FQDN. Details are here

global.compatibility object
openshift:
    adaptSecurityContext: auto

Openshift security context compatibility configuration

global.image.registry string
""

Image registry, that can be shared across multiple helm charts

global.imagePullSecrets list
[]

Image pull secrets, that can be shared across multiple helm charts

image object
pullPolicy: IfNotPresent
registry: ""
repository: victoriametrics/operator
tag: ""
variant: ""

operator image configuration

image.pullPolicy string
IfNotPresent

Image pull policy

image.registry string
""

Image registry

image.repository string
victoriametrics/operator

Image repository

image.tag string
""

Image tag override Chart.AppVersion

imagePullSecrets list
[]

Secret to pull images

lifecycle object
{}

Operator lifecycle. See this article for details.

logLevel string
info

VM operator log level. Possible values: info and error.

nameOverride string
""

Override chart name

nodeSelector object
{}

Pod’s node selector. Details are here

operator.disable_prometheus_converter bool
false

By default, operator converts prometheus-operator objects.

operator.enable_converter_ownership bool
false

Enables ownership reference for converted prometheus-operator objects, it will remove corresponding victoria-metrics objects in case of deletion prometheus one.

operator.prometheus_converter_add_argocd_ignore_annotations bool
false

Compare-options and sync-options for prometheus objects converted by operator for properly use with ArgoCD

operator.useCustomConfigReloader bool
false

Enables custom config-reloader, bundled with operator. It should reduce vmagent and vmauth config sync-time and make it predictable.

podDisruptionBudget object
enabled: false
labels: {}

See kubectl explain poddisruptionbudget.spec for more or check these docs

podLabels object
{}

extra Labels for Pods only

podSecurityContext object
enabled: true

Pod’s security context. Details are here

probe.liveness object
failureThreshold: 3
initialDelaySeconds: 5
periodSeconds: 15
tcpSocket:
    port: probe
timeoutSeconds: 5

Liveness probe

probe.readiness object
failureThreshold: 3
httpGet:
    port: probe
initialDelaySeconds: 5
periodSeconds: 15
timeoutSeconds: 5

Readiness probe

probe.startup object
{}

Startup probe

rbac.aggregatedClusterRoles object
enabled: true
labels:
    admin:
        rbac.authorization.k8s.io/aggregate-to-admin: "true"
    view:
        rbac.authorization.k8s.io/aggregate-to-view: "true"

Create aggregated clusterRoles for CRD readonly and admin permissions

rbac.aggregatedClusterRoles.labels object
admin:
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
view:
    rbac.authorization.k8s.io/aggregate-to-view: "true"

Labels attached to according clusterRole

rbac.create bool
true

Specifies whether the RBAC resources should be created

replicaCount int
1

Number of operator replicas

resources object
{}

Resource object

securityContext object
enabled: true

Security context to be added to server pods

service.annotations object
{}

Service annotations

service.clusterIP string
""

Service ClusterIP

service.externalIPs string
""

Service external IPs. Check here for details

service.externalTrafficPolicy string
""

Service external traffic policy. Check here for details

service.healthCheckNodePort string
""

Health check node port for a service. Check here for details

service.ipFamilies list
[]

List of service IP families. Check here for details.

service.ipFamilyPolicy string
""

Service IP family policy. Check here for details.

service.labels object
{}

Service labels

service.loadBalancerIP string
""

Service load balacner IP

service.loadBalancerSourceRanges list
[]

Load balancer source range

service.servicePort int
8080

Service port

service.type string
ClusterIP

Service type

service.webhookPort int
9443

Service webhook port

serviceAccount.create bool
true

Specifies whether a service account should be created

serviceAccount.name string
""

The name of the service account to use. If not set and create is true, a name is generated using the fullname template

serviceMonitor object
annotations: {}
basicAuth: {}
enabled: false
extraLabels: {}
interval: ""
relabelings: []
scheme: ""
scrapeTimeout: ""
tlsConfig: {}

Configures monitoring with serviceScrape. VMServiceScrape must be pre-installed

terminationGracePeriodSeconds int
30

Graceful pod termination timeout. See this article for details.

tolerations list
[]

Array of tolerations object. Spec is here

topologySpreadConstraints list
[]

Pod Topology Spread Constraints. Spec is here

watchNamespaces list
[]

By default, the operator will watch all the namespaces If you want to override this behavior, specify the namespace. Operator supports multiple namespaces for watching.