VictoriaMetrics/docs/helm/victoria-metrics-agent/README.md

Victoria Metrics Agent - collects metrics from various sources and stores them to VictoriaMetrics

Prerequisites

• Install the follow packages: git, kubectl, helm, helm-docs. See this tutorial.

How to install

Access a Kubernetes cluster.

Setup chart repository (can be omitted for OCI repositories)

Add a chart helm repository with follow commands:

helm repo add vm https://victoriametrics.github.io/helm-charts/

helm repo update

List versions of vm/victoria-metrics-agent chart available to installation:

helm search repo vm/victoria-metrics-agent -l

Install victoria-metrics-agent chart

Export default values of victoria-metrics-agent chart to file values.yaml:

• For HTTPS repository

  helm show values vm/victoria-metrics-agent > values.yaml
  

• For OCI repository

  helm show values oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-agent > values.yaml
  

Change the values according to the need of the environment in values.yaml file.

Test the installation with command:

• For HTTPS repository

  helm install vma vm/victoria-metrics-agent -f values.yaml -n NAMESPACE --debug --dry-run
  

• For OCI repository

  helm install vma oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-agent -f values.yaml -n NAMESPACE --debug --dry-run
  

Install chart with command:

• For HTTPS repository

  helm install vma vm/victoria-metrics-agent -f values.yaml -n NAMESPACE
  

• For OCI repository

  helm install vma oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-agent -f values.yaml -n NAMESPACE
  

Get the pods lists by running this commands:

kubectl get pods -A | grep 'vma'

Get the application by running this command:

helm list -f vma -n NAMESPACE

See the history of versions of vma application with command.

helm history vma -n NAMESPACE

Upgrade guide

Upgrade to 0.13.0

• replace remoteWriteUrls to remoteWrite:

Given below config

remoteWriteUrls:
- http://address1/api/v1/write
- http://address2/api/v1/write

should be changed to

remoteWrite:
- url: http://address1/api/v1/write
- url: http://address2/api/v1/write

How to uninstall

Remove application with command.

helm uninstall vma -n NAMESPACE

Documentation of Helm Chart

Install helm-docs following the instructions on this tutorial.

Generate docs with helm-docs command.

cd charts/victoria-metrics-agent

helm-docs

The markdown generation is entirely go template driven. The tool parses metadata from charts and generates a number of sub-templates that can be referenced in a template file (by default README.md.gotmpl). If no template file is provided, the tool has a default internal template that will generate a reasonably formatted README.

Parameters

The following tables lists the configurable parameters of the chart and their default values.

Change the values according to the need of the environment in victoria-metrics-agent/values.yaml file.

Key	Type	Default	Description
affinity	object	{}	Pod affinity
annotations	object	{}	Annotations to be added to the deployment
config	object	global: scrape_interval: 10s scrape_configs: - job_name: vmagent static_configs: - targets: - localhost:8429 - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token job_name: kubernetes-apiservers kubernetes_sd_configs: - role: endpoints relabel_configs: - action: keep regex: default;kubernetes;https source_labels: - __meta_kubernetes_namespace - __meta_kubernetes_service_name - __meta_kubernetes_endpoint_port_name scheme: https tls_config: ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecure_skip_verify: true - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token job_name: kubernetes-nodes kubernetes_sd_configs: - role: node relabel_configs: - action: labelmap regex: __meta_kubernetes_node_label_(.+) - replacement: kubernetes.default.svc:443 target_label: __address__ - regex: (.+) replacement: /api/v1/nodes/$1/proxy/metrics source_labels: - __meta_kubernetes_node_name target_label: __metrics_path__ scheme: https tls_config: ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecure_skip_verify: true - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token honor_timestamps: false job_name: kubernetes-nodes-cadvisor kubernetes_sd_configs: - role: node relabel_configs: - action: labelmap regex: __meta_kubernetes_node_label_(.+) - replacement: kubernetes.default.svc:443 target_label: __address__ - regex: (.+) replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor source_labels: - __meta_kubernetes_node_name target_label: __metrics_path__ scheme: https tls_config: ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecure_skip_verify: true - job_name: kubernetes-service-endpoints kubernetes_sd_configs: - role: endpointslices relabel_configs: - action: drop regex: true source_labels: - __meta_kubernetes_pod_container_init - action: keep_if_equal source_labels: - __meta_kubernetes_service_annotation_prometheus_io_port - __meta_kubernetes_pod_container_port_number - action: keep regex: true source_labels: - __meta_kubernetes_service_annotation_prometheus_io_scrape - action: replace regex: (https?) source_labels: - __meta_kubernetes_service_annotation_prometheus_io_scheme target_label: __scheme__ - action: replace regex: (.+) source_labels: - __meta_kubernetes_service_annotation_prometheus_io_path target_label: __metrics_path__ - action: replace regex: ([^:]+)(?::\d+)?;(\d+) replacement: $1:$2 source_labels: - __address__ - __meta_kubernetes_service_annotation_prometheus_io_port target_label: __address__ - action: labelmap regex: __meta_kubernetes_service_label_(.+) - source_labels: - __meta_kubernetes_pod_name target_label: pod - source_labels: - __meta_kubernetes_pod_container_name target_label: container - source_labels: - __meta_kubernetes_namespace target_label: namespace - source_labels: - __meta_kubernetes_service_name target_label: service - replacement: ${1} source_labels: - __meta_kubernetes_service_name target_label: job - action: replace source_labels: - __meta_kubernetes_pod_node_name target_label: node - job_name: kubernetes-service-endpoints-slow kubernetes_sd_configs: - role: endpointslices relabel_configs: - action: drop regex: true source_labels: - __meta_kubernetes_pod_container_init - action: keep_if_equal source_labels: - __meta_kubernetes_service_annotation_prometheus_io_port - __meta_kubernetes_pod_container_port_number - action: keep regex: true source_labels: - __meta_kubernetes_service_annotation_prometheus_io_scrape_slow - action: replace regex: (https?) source_labels: - __meta_kubernetes_service_annotation_prometheus_io_scheme target_label: __scheme__ - action: replace regex: (.+) source_labels: - __meta_kubernetes_service_annotation_prometheus_io_path target_label: __metrics_path__ - action: replace regex: ([^:]+)(?::\d+)?;(\d+) replacement: $1:$2 source_labels: - __address__ - __meta_kubernetes_service_annotation_prometheus_io_port target_label: __address__ - action: labelmap regex: __meta_kubernetes_service_label_(.+) - source_labels: - __meta_kubernetes_pod_name target_label: pod - source_labels: - __meta_kubernetes_pod_container_name target_label: container - source_labels: - __meta_kubernetes_namespace target_label: namespace - source_labels: - __meta_kubernetes_service_name target_label: service - replacement: ${1} source_labels: - __meta_kubernetes_service_name target_label: job - action: replace source_labels: - __meta_kubernetes_pod_node_name target_label: node scrape_interval: 5m scrape_timeout: 30s - job_name: kubernetes-services kubernetes_sd_configs: - role: service metrics_path: /probe params: module: - http_2xx relabel_configs: - action: keep regex: true source_labels: - __meta_kubernetes_service_annotation_prometheus_io_probe - source_labels: - __address__ target_label: __param_target - replacement: blackbox target_label: __address__ - source_labels: - __param_target target_label: instance - action: labelmap regex: __meta_kubernetes_service_label_(.+) - source_labels: - __meta_kubernetes_namespace target_label: namespace - source_labels: - __meta_kubernetes_service_name target_label: service - job_name: kubernetes-pods kubernetes_sd_configs: - role: pod relabel_configs: - action: drop regex: true source_labels: - __meta_kubernetes_pod_container_init - action: keep_if_equal source_labels: - __meta_kubernetes_pod_annotation_prometheus_io_port - __meta_kubernetes_pod_container_port_number - action: keep regex: true source_labels: - __meta_kubernetes_pod_annotation_prometheus_io_scrape - action: replace regex: (.+) source_labels: - __meta_kubernetes_pod_annotation_prometheus_io_path target_label: __metrics_path__ - action: replace regex: ([^:]+)(?::\d+)?;(\d+) replacement: $1:$2 source_labels: - __address__ - __meta_kubernetes_pod_annotation_prometheus_io_port target_label: __address__ - action: labelmap regex: __meta_kubernetes_pod_label_(.+) - source_labels: - __meta_kubernetes_pod_name target_label: pod - source_labels: - __meta_kubernetes_pod_container_name target_label: container - source_labels: - __meta_kubernetes_namespace target_label: namespace - action: replace source_labels: - __meta_kubernetes_pod_node_name target_label: node	VMAgent scrape configuration
configMap	string	""	VMAgent scraping configuration use existing configmap if specified otherwise .config values will be used
containerWorkingDir	string	/	Container working directory
deployment	object	strategy: {}	K8s Deployment specific variables
deployment.strategy	object	{}	Deployment stragegy. Check here for details
emptyDir	object	{}	Empty dir configuration for a case, when persistence is disabled
env	list	[]	Additional environment variables (ex.: secret tokens, flags). Check here for more details.
envFrom	list	[]	Specify alternative source for env variables
extraArgs	object	envflag.enable: "true" envflag.prefix: VM_ loggerFormat: json	VMAgent extra command line arguments
extraContainers	list	[]	Extra containers to run in a pod with vmagent
extraHostPathMounts	list	[]	Additional hostPath mounts
extraLabels	object	{}	Extra labels for Deployment and Statefulset
extraObjects	list	[]	Add extra specs dynamically to this chart
extraScrapeConfigs	list	[]	Extra scrape configs that will be appended to config
extraVolumeMounts	list	[]	Extra Volume Mounts for the container
extraVolumes	list	[]	Extra Volumes for the pod
fullnameOverride	string	""	Override resources fullname
global.cluster.dnsDomain	string	cluster.local.	K8s cluster domain suffix, uses for building storage pods’ FQDN. Details are here
global.compatibility	object	openshift: adaptSecurityContext: auto	Openshift security context compatibility configuration
global.image.registry	string	""	Image registry, that can be shared across multiple helm charts
global.imagePullSecrets	list	[]	Image pull secrets, that can be shared across multiple helm charts
horizontalPodAutoscaling	object	enabled: false maxReplicas: 10 metrics: [] minReplicas: 1	Horizontal Pod Autoscaling. Note that it is not intended to be used for vmagents which perform scraping. In order to scale scraping vmagents check here
horizontalPodAutoscaling.enabled	bool	false	Use HPA for vmagent
horizontalPodAutoscaling.maxReplicas	int	10	Maximum replicas for HPA to use to to scale vmagent
horizontalPodAutoscaling.metrics	list	[]	Metric for HPA to use to scale vmagent
horizontalPodAutoscaling.minReplicas	int	1	Minimum replicas for HPA to use to scale vmagent
image.pullPolicy	string	IfNotPresent	Image pull policy
image.registry	string	""	Image registry
image.repository	string	victoriametrics/vmagent	Image repository
image.tag	string	""	Image tag, set to Chart.AppVersion by default
image.variant	string	""	Variant of the image to use. e.g. enterprise, scratch
imagePullSecrets	list	[]	Image pull secrets
ingress.annotations	object	{}	Ingress annotations
ingress.enabled	bool	false	Enable deployment of ingress for agent
ingress.extraLabels	object	{}	Ingress extra labels
ingress.hosts	list	- name: vmagent.local path: - / port: http	Array of host objects
ingress.ingressClassName	string	""	Ingress controller class name
ingress.pathType	string	Prefix	Ingress path type
ingress.tls	list	[]	Array of TLS objects
initContainers	list	[]	Init containers for vmagent
license	object	key: "" secret: key: "" name: ""	Enterprise license key configuration for VictoriaMetrics enterprise. Required only for VictoriaMetrics enterprise. Check docs here, for more information, visit site. Request a trial license here Supported starting from VictoriaMetrics v1.94.0
license.key	string	""	License key
license.secret	object	key: "" name: ""	Use existing secret with license key
license.secret.key	string	""	Key in secret with license key
license.secret.name	string	""	Existing secret name
nameOverride	string	""	Override chart name
nodeSelector	object	{}	Pod’s node selector. Details are here
persistence.accessModes	list	- ReadWriteOnce	Array of access modes. Must match those of existing PV or dynamic provisioner. Details are here
persistence.annotations	object	{}	Persistant volume annotations
persistence.enabled	bool	false	Create/use Persistent Volume Claim for server component. Empty dir if false
persistence.existingClaim	string	""	Existing Claim name. If defined, PVC must be created manually before volume will be bound
persistence.extraLabels	object	{}	Persistant volume additional labels
persistence.matchLabels	object	{}	Bind Persistent Volume by labels. Must match all labels of targeted PV.
persistence.size	string	10Gi	Size of the volume. Should be calculated based on the logs you send and retention policy you set.
persistence.storageClassName	string	""	StorageClass to use for persistent volume. Requires server.persistentVolume.enabled: true. If defined, PVC created automatically
podAnnotations	object	{}	Annotations to be added to pod
podDisruptionBudget	object	enabled: false labels: {}	See kubectl explain poddisruptionbudget.spec for more or check official documentation
podLabels	object	{}	Extra labels for Pods only
podSecurityContext	object	enabled: true	Security context to be added to pod
priorityClassName	string	""	Priority class to be assigned to the pod(s)
probe.liveness	object	initialDelaySeconds: 5 periodSeconds: 15 tcpSocket: {} timeoutSeconds: 5	Liveness probe
probe.readiness	object	httpGet: {} initialDelaySeconds: 5 periodSeconds: 15	Readiness probe
probe.startup	object	{}	Startup probe
rbac.annotations	object	{}	Role/RoleBinding annotations
rbac.create	bool	true	Enables Role/RoleBinding creation
rbac.extraLabels	object	{}	Role/RoleBinding labels
rbac.namespaced	bool	false	If true and rbac.enabled, will deploy a Role/RoleBinding instead of a ClusterRole/ClusterRoleBinding
remoteWrite	list	[]	Generates remoteWrite.* flags and config maps with value content for values, that are of type list of map. Each item should contain url param to pass validation.
replicaCount	int	1	Replica count
resources	object	{}	Resource object. Details are here
securityContext	object	enabled: true	Security context to be added to pod’s containers
service.annotations	object	{}	Service annotations
service.clusterIP	string	""	Service ClusterIP
service.enabled	bool	false	Enable agent service
service.externalIPs	list	[]	Service external IPs. Check here for details
service.externalTrafficPolicy	string	""	Service external traffic policy. Check here for details
service.extraLabels	object	{}	Service labels
service.healthCheckNodePort	string	""	Health check node port for a service. Check here for details
service.ipFamilies	list	[]	List of service IP families. Check here for details.
service.ipFamilyPolicy	string	""	Service IP family policy. Check here for details.
service.loadBalancerIP	string	""	Service load balacner IP
service.loadBalancerSourceRanges	list	[]	Load balancer source range
service.servicePort	int	8429	Service port
service.targetPort	string	http	Target port
service.type	string	ClusterIP	Service type
serviceAccount.annotations	object	{}	Annotations to add to the service account
serviceAccount.create	bool	true	Specifies whether a service account should be created
serviceAccount.name	string	null	The name of the service account to use. If not set and create is true, a name is generated using the fullname template
serviceMonitor.annotations	object	{}	Service Monitor annotations
serviceMonitor.basicAuth	object	{}	Basic auth params for Service Monitor
serviceMonitor.enabled	bool	false	Enable deployment of Service Monitor for server component. This is Prometheus operator object
serviceMonitor.extraLabels	object	{}	Service Monitor labels
serviceMonitor.metricRelabelings	list	[]	Service Monitor metricRelabelings
serviceMonitor.relabelings	list	[]	Service Monitor relabelings
serviceMonitor.targetPort	string	http	Service Monitor targetPort
statefulSet	object	clusterMode: false enabled: false replicationFactor: 1 updateStrategy: {}	K8s StatefulSet specific variables
statefulSet.clusterMode	bool	false	create cluster of vmagents. Check here available since v1.77.2
statefulSet.replicationFactor	int	1	replication factor for vmagent in cluster mode
statefulSet.updateStrategy	object	{}	StatefulSet update strategy. Check here for details.
tolerations	list	[]	Node tolerations for server scheduling to nodes with taints. Details are here
topologySpreadConstraints	list	[]	Pod topologySpreadConstraints