VictoriaMetrics/docs/operator/resources/README.md at e5b4812d770319e9309f14e8f2d76e491169aa39

mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00

Automatic update operator docs from VictoriaMetrics/operator@fe90f57 (#7026 )

Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>

2024-09-17 19:30:09 +02:00

14 KiB

Raw Blame History

This documentation section describes the design and interaction between the custom resource definitions (CRD) that the Victoria Metrics Operator introduces. Operator introduces the following custom resources:

Here is the scheme of relations between the custom resources:

Specification

You can find the specification for the custom resources on API Docs.

Extra arguments

If you can't find necessary field in the specification of custom resource, you can use extraArgs field for passing additional arguments to the application.

Field extraArgs is supported for the following custom resources:

Supported flags for each application can be found the in the corresponding documentation:

Usage example:

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMSingle
metadata:
  name: vmsingle-example-extraargs
spec:
  retentionPeriod: "1"
  extraArgs:
    dedup.minScrapeInterval: 60s
  # ...

Extra environment variables

Flag can be replaced with environment variable, it's useful for retrieving value from secret. You can use extraEnvs field for passing additional arguments to the application.

Usage example:

kind: VMSingle
metadata:
  name: vmsingle-example-extraenvs
spec:
  retentionPeriod: "1"
  extraEnvs:
    - name: DEDUP_MINSCRAPEINTERVAL
      valueFrom:
        secretKeyRef:
          name: vm-secret
          key: dedup

This feature really useful for using with -envflag.enable command-line argument.

Examples

Page for every custom resource contains examples section:

In addition, you can find examples of the custom resources for VictoriaMetrics operator in the examples directory of operator repository.

Managing versions of VM

Every custom resource with deployable application has a fields for specifying version (docker image) of component:

Managing resources

Every custom resource with deployable application has a fields and operator parameters for specifying resources for the component:

High availability

VictoriaMetrics operator support high availability for each component of the monitoring stack:

In addition, these CRD support common features, that can be used to increase high availability - resources above have the following fields:

affinity - to schedule pods on different nodes (affinity and anti-affinity in kubernetes docs),
tolerations - to schedule pods on nodes with taints (taints and tolerations in kubernetes docs),
nodeSelector - to schedule pods on nodes with specific labels (node selector in kubernetes docs),
topologySpreadConstraints - to schedule pods on different nodes in the same topology (topology spread constraints in kubernetes docs).

See details about these fields in the Specification.

Enterprise features

Operator supports following Enterprise features for VictoriaMetrics components:

More information about enterprise features you can read on VictoriaMetrics Enterprise page.

Configuration synchronization

Basic concepts

VictoriaMetrics applications, like many other applications with configuration file deployed at Kubernetes, uses ConfigMaps and Secrets for configuration files. Usually, it's out of application scope to watch for configuration on-disk changes. Applications reload their configuration by a signal from a user or some other tool, that knows how to watch for updates. At Kubernetes, the most popular design for this case is a sidecar container, that watches for configuration file changes and sends an HTTP request to the application.

Configmap or Secret that mounted at Pod holds a copy of its content. Kubernetes component kubelet is responsible for content synchronization between an object at Kubernetes API and a file served on disk. It's not efficient to sync its content immediately, and kubelet eventually synchronizes it. There is a configuration option, that controls this period.

That's why, applications managed by operator don't receive changes immediately. It usually takes 1-2 min, before content will be updated.

It may trigger errors when an application was deleted, but VMAgent still tries to scrape it.

Possible mitigations

The naive solution for this case decrease the synchronization period. But it configures globally and may be hard for operator users.

That's why operator uses a few hacks.

For ConfigMap updates, operator changes annotation with a time of Configmap content update. It triggers ConfigMap's content synchronization by kubelet immediately. It's the case for VMAlert, it uses ConfigMap as a configuration source.

For Secret it doesn't work. And operator offers its implementation for side-car container. It can be configured with env variable for operator:

- name: VM_USECUSTOMCONFIGRELOADER
  value: "true"

If it's defined, operator uses own config-reloader instead of prometheus-config-reload.

It watches corresponding Secret for changes with Kubernetes API watch call and writes content into emptyDir. This emptyDir shared with the application. In case of content changes, config-reloader sends HTTP requests to the application. It greatly reduces the time for configuration synchronization.

14 KiB Raw Blame History