2022-01-21 10:05:58 +00:00
|
|
|
---
|
2023-10-02 12:50:08 +00:00
|
|
|
sort: 1
|
|
|
|
weight: 1
|
|
|
|
title: QuickStart
|
2023-10-05 10:43:28 +00:00
|
|
|
menu:
|
|
|
|
docs:
|
|
|
|
parent: "operator"
|
|
|
|
weight: 1
|
2023-11-22 11:59:07 +00:00
|
|
|
aliases:
|
|
|
|
- /operator/quick-start.html
|
2022-01-21 10:05:58 +00:00
|
|
|
---
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# VictoriaMetrics Operator QuickStart
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
VictoriaMetrics Operator serves to make running VictoriaMetrics applications on top of Kubernetes as easy as possible
|
|
|
|
while preserving Kubernetes-native configuration options.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
The shortest way to deploy full-stack monitoring cluster with VictoriaMetrics Operator is
|
|
|
|
to use Helm-chart [victoria-metrics-k8s-stack](https://victoriametrics.github.io/helm-charts/charts/victoria-metrics-k8s-stack/).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Also you can follow the other steps in documentation to use VictoriaMetrics Operator:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
- [Setup](./setup.md)
|
|
|
|
- [Security](./security.md)
|
|
|
|
- [Configuration](./configuration.md)
|
|
|
|
- [Migration from Prometheus](./migration.md)
|
|
|
|
- [Monitoring](./monitoring.md)
|
|
|
|
- [Authorization and exposing components](./auth.md)
|
|
|
|
- [High Availability](./high-availability.md)
|
|
|
|
- [Enterprise](./enterprise.md)
|
|
|
|
- [Custom resources](./resources/README.md)
|
|
|
|
- [FAQ (Frequency Asked Questions)](./FAQ.md)
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
But if you want to deploy VictoriaMetrics Operator quickly from scratch (without using templating for custom resources),
|
|
|
|
you can follow this guide:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
- [Setup operator](#setup-operator)
|
|
|
|
- [Deploy components](#deploy-components)
|
|
|
|
- [VMCluster](#vmcluster-vmselect-vminsert-vmstorage)
|
|
|
|
- [Scraping](#scraping)
|
|
|
|
- [VMAgent](#vmagent)
|
|
|
|
- [VMServiceScrape](#vmservicescrape)
|
|
|
|
- [Access](#access)
|
|
|
|
- [VMAuth](#vmauth)
|
|
|
|
- [VMUser](#vmuser)
|
|
|
|
- [Alerting](#alerting)
|
|
|
|
- [VMAlertmanager](#vmalertmanager)
|
|
|
|
- [VMAlert](#vmalert)
|
|
|
|
- [VMRule](#vmrule)
|
|
|
|
- [VMUser](#vmuser-update)
|
|
|
|
- [Anythings else?](#anythings-else)
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Let's start!
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
## Setup operator
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
You can find out how to and instructions for installing the VictoriaMetrics operator into your kubernetes cluster
|
|
|
|
on the [Setup page](./setup.md).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Here we will elaborate on just one of the ways - for instance, we will install operator via Helm-chart
|
|
|
|
[victoria-metrics-operator](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-operator/README.md):
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Add repo with helm-chart:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
helm repo add vm https://victoriametrics.github.io/helm-charts/
|
|
|
|
helm repo update
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Render `values.yaml` with default operator configuration:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
helm show values vm/victoria-metrics-operator > values.yaml
|
|
|
|
```
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Now you can configure operator - open rendered `values.yaml` file in your text editor. For example:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
code values.yaml
|
|
|
|
```
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
<img src="quick-start_values.png" width="1200">
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Now you can change configuration in `values.yaml`. For more details about configuration options and methods,
|
|
|
|
see [configuration -> victoria-metrics-operator](./configuration.md#victoria-metrics-operator).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
If you migrated from prometheus-operator, you can read about prometheus-operator objects conversion on
|
|
|
|
the [migration from prometheus-operator](./migration.md).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Since we're looking at installing from scratch, let's disable prometheus-operator objects conversion,
|
|
|
|
and also let's set some resources for operator in `values.yaml`:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```yaml
|
|
|
|
# ...
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
operator:
|
|
|
|
# -- By default, operator converts prometheus-operator objects.
|
|
|
|
disable_prometheus_converter: true
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# -- Resources for operator
|
|
|
|
resources:
|
|
|
|
limits:
|
|
|
|
cpu: 500m
|
|
|
|
memory: 500Mi
|
|
|
|
requests:
|
|
|
|
cpu: 100m
|
|
|
|
memory: 150Mi
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# ...
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
You will need a kubernetes namespace to deploy the operator and VM components. Let's create it:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl create namespace vm
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
After finishing with `values.yaml` and creating namespace, you can test the installation with command:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
helm install vmoperator vm/victoria-metrics-operator -f values.yaml -n vm --debug --dry-run
|
|
|
|
```
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Where `vm` is the namespace where you want to install operator.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
If everything is ok, you can install operator with command:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
helm install vmoperator vm/victoria-metrics-operator -f values.yaml -n vm
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# NAME: vmoperator
|
|
|
|
# LAST DEPLOYED: Thu Sep 14 15:13:04 2023
|
|
|
|
# NAMESPACE: vm
|
|
|
|
# STATUS: deployed
|
|
|
|
# REVISION: 1
|
|
|
|
# TEST SUITE: None
|
|
|
|
# NOTES:
|
|
|
|
# victoria-metrics-operator has been installed. Check its status by running:
|
|
|
|
# kubectl --namespace vm get pods -l "app.kubernetes.io/instance=vmoperator"
|
|
|
|
#
|
|
|
|
# Get more information on https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-operator.
|
|
|
|
# See "Getting started guide for VM Operator" on https://docs.victoriametrics.com/guides/getting-started-with-vm-operator.html .
|
|
|
|
```
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
And check that operator is running:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl get pods -n vm -l "app.kubernetes.io/instance=vmoperator"
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# NAME READY STATUS RESTARTS AGE
|
|
|
|
# vmoperator-victoria-metrics-operator-7b88bd6df9-q9qwz 1/1 Running 0 98s
|
|
|
|
```
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
## Deploy components
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Now you can create instances of VictoriaMetrics applications.
|
|
|
|
Let's create fullstack monitoring cluster with
|
|
|
|
[`vmagent`](./resources/vmagent.md),
|
|
|
|
[`vmauth`](./resources/vmauth.md),
|
|
|
|
[`vmalert`](./resources/vmalert.md),
|
|
|
|
[`vmalertmanager`](./resources/vmalertmanager.md),
|
|
|
|
[`vmcluster`](./resources/vmcluster.md)
|
|
|
|
(a component for deploying a cluster version of
|
|
|
|
[VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#architecture-overview)
|
|
|
|
consisting of `vmstorage`, `vmselect` and `vminsert`):
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
<img src="quick-start_cluster-scheme.png" width="1200">
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
More details about resources of VictoriaMetrics operator you can find on the [resources page](./resources/README.md).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
### VMCluster (vmselect, vminsert, vmstorage)
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Let's start by deploying the [`vmcluster`](./resources/vmcluster.md) resource.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Create file `vmcluster.yaml`
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
code vmcluster.yaml
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
with the following content:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
|
|
|
```yaml
|
2023-10-02 12:50:08 +00:00
|
|
|
# vmcluster.yaml
|
2022-01-21 10:05:58 +00:00
|
|
|
apiVersion: operator.victoriametrics.com/v1beta1
|
|
|
|
kind: VMCluster
|
|
|
|
metadata:
|
2023-10-02 12:50:08 +00:00
|
|
|
name: demo
|
2022-01-21 10:05:58 +00:00
|
|
|
spec:
|
2023-10-02 12:50:08 +00:00
|
|
|
retentionPeriod: "1"
|
2022-01-21 10:05:58 +00:00
|
|
|
replicationFactor: 2
|
|
|
|
vmstorage:
|
|
|
|
replicaCount: 2
|
|
|
|
storageDataPath: "/vm-data"
|
|
|
|
storage:
|
|
|
|
volumeClaimTemplate:
|
|
|
|
spec:
|
|
|
|
resources:
|
|
|
|
requests:
|
2023-10-02 12:50:08 +00:00
|
|
|
storage: "10Gi"
|
2022-01-21 10:05:58 +00:00
|
|
|
resources:
|
|
|
|
limits:
|
2023-10-02 12:50:08 +00:00
|
|
|
cpu: "1"
|
|
|
|
memory: "1Gi"
|
2022-01-21 10:05:58 +00:00
|
|
|
vmselect:
|
|
|
|
replicaCount: 2
|
|
|
|
cacheMountPath: "/select-cache"
|
|
|
|
storage:
|
|
|
|
volumeClaimTemplate:
|
|
|
|
spec:
|
|
|
|
resources:
|
|
|
|
requests:
|
2023-10-02 12:50:08 +00:00
|
|
|
storage: "1Gi"
|
2022-01-21 10:05:58 +00:00
|
|
|
resources:
|
|
|
|
limits:
|
2023-10-02 12:50:08 +00:00
|
|
|
cpu: "1"
|
|
|
|
memory: "1Gi"
|
|
|
|
requests:
|
|
|
|
cpu: "0.5"
|
|
|
|
memory: "500Mi"
|
2022-01-21 10:05:58 +00:00
|
|
|
vminsert:
|
|
|
|
replicaCount: 2
|
2023-10-02 12:50:08 +00:00
|
|
|
resources:
|
|
|
|
limits:
|
|
|
|
cpu: "1"
|
|
|
|
memory: "1Gi"
|
|
|
|
requests:
|
|
|
|
cpu: "0.5"
|
|
|
|
memory: "500Mi"
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
After that you can deploy `vmcluster` resource to the kubernetes cluster:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl apply -f vmcluster.yaml -n vm
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# vmcluster.operator.victoriametrics.com/demo created
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Check that `vmcluster` is running:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl get pods -n vm -l "app.kubernetes.io/instance=demo"
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# NAME READY STATUS RESTARTS AGE
|
|
|
|
# vminsert-demo-8688d88ff7-fnbnw 1/1 Running 0 3m39s
|
|
|
|
# vminsert-demo-8688d88ff7-5wbj7 1/1 Running 0 3m39s
|
|
|
|
# vmselect-demo-0 1/1 Running 0 3m39s
|
|
|
|
# vmselect-demo-1 1/1 Running 0 3m39s
|
|
|
|
# vmstorage-demo-1 1/1 Running 0 22s
|
|
|
|
# vmstorage-demo-0 1/1 Running 0 6s
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Now you can see that 6 components of your demo vmcluster is running.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
In addition, you can see that the operator created services for each of the component type:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl get svc -n vm -l "app.kubernetes.io/instance=demo"
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
|
|
|
# vmstorage-demo ClusterIP None <none> 8482/TCP,8400/TCP,8401/TCP 8m3s
|
|
|
|
# vmselect-demo ClusterIP None <none> 8481/TCP 8m3s
|
|
|
|
# vminsert-demo ClusterIP 192.168.194.183 <none> 8480/TCP 8m3s
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
We'll need them in the next steps.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
More information about `vmcluster` resource you can find on
|
|
|
|
the [vmcluster page](./resources/vmcluster.md).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
### Scraping
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
#### VMAgent
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Now let's deploy [`vmagent`](./resources/vmagent.md) resource.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Create file `vmagent.yaml`
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
code vmagent.yaml
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
with the following content:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
|
|
|
```yaml
|
|
|
|
apiVersion: operator.victoriametrics.com/v1beta1
|
2023-10-02 12:50:08 +00:00
|
|
|
kind: VMAgent
|
2022-01-21 10:05:58 +00:00
|
|
|
metadata:
|
2023-10-02 12:50:08 +00:00
|
|
|
name: demo
|
2022-01-21 10:05:58 +00:00
|
|
|
spec:
|
2023-10-02 12:50:08 +00:00
|
|
|
selectAllByDefault: true
|
|
|
|
remoteWrite:
|
|
|
|
- url: "http://vminsert-demo.vm.svc:8480/insert/0/prometheus/api/v1/write"
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
After that you can deploy `vmagent` resource to the kubernetes cluster:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl apply -f vmagent.yaml -n vm
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# vmagent.operator.victoriametrics.com/demo created
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Check that `vmagent` is running:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl get pods -n vm -l "app.kubernetes.io/instance=demo" -l "app.kubernetes.io/name=vmagent"
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# NAME READY STATUS RESTARTS AGE
|
|
|
|
# vmagent-demo-6785f7d7b9-zpbv6 2/2 Running 0 72s
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
More information about `vmagent` resource you can find on
|
|
|
|
the [vmagent page](./resources/vmagent.md).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
#### VMServiceScrape
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Now we have the timeseries database (vmcluster) and the tool to collect metrics (vmagent) and send it to the database.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
But we need to tell vmagent what metrics to collect. For this we will use [`vmservicescrape`](./resources/vmservicescrape.md) resource
|
|
|
|
or [other `*scrape` resources](./resources/README.md).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
By default, operator creates `vmservicescrape` resource for each component that it manages. More details about this you can find on
|
|
|
|
the [monitoring page](./configuration.md#monitoring-of-cluster-components).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
For instance, we can create `vmservicescrape` for VictoriaMetrics operator manually. Let's create file `vmservicescrape.yaml`:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
code vmservicescrape.yaml
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
with the following content:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
|
|
|
```yaml
|
|
|
|
apiVersion: operator.victoriametrics.com/v1beta1
|
|
|
|
kind: VMServiceScrape
|
|
|
|
metadata:
|
2023-10-02 12:50:08 +00:00
|
|
|
name: vmoperator-demo
|
2022-01-21 10:05:58 +00:00
|
|
|
spec:
|
|
|
|
selector:
|
|
|
|
matchLabels:
|
2023-10-02 12:50:08 +00:00
|
|
|
app.kubernetes.io/instance: vmoperator
|
|
|
|
app.kubernetes.io/name: victoria-metrics-operator
|
|
|
|
namespaceSelector:
|
|
|
|
matchNames:
|
|
|
|
- vm
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
After that you can deploy `vmservicescrape` resource to the kubernetes cluster:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl apply -f vmservicescrape.yaml -n vm
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# vmservicescrape.operator.victoriametrics.com/vmoperator-demo created
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
### Access
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
We need to look at the results of what we got. Up until now, we've just been looking only at the status of the pods.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
#### VMAuth
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Let's expose our components with [`vmauth`](./resources/vmauth.md).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Create file `vmauth.yaml`
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
code vmauth.yaml
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
with the following content:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
|
|
|
```yaml
|
|
|
|
apiVersion: operator.victoriametrics.com/v1beta1
|
2023-10-02 12:50:08 +00:00
|
|
|
kind: VMAuth
|
2022-01-21 10:05:58 +00:00
|
|
|
metadata:
|
2023-10-02 12:50:08 +00:00
|
|
|
name: demo
|
2022-01-21 10:05:58 +00:00
|
|
|
spec:
|
2023-10-02 12:50:08 +00:00
|
|
|
userNamespaceSelector: {}
|
|
|
|
userSelector: {}
|
|
|
|
ingress:
|
|
|
|
class_name: nginx # <-- change this to your ingress-controller
|
|
|
|
host: vm-demo.k8s.orb.local # <-- change this to your domain
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
**Note** that content of `ingress` field depends on your ingress-controller and domain.
|
|
|
|
Your cluster will have them differently.
|
|
|
|
Also, for simplicity, we don't use tls, but in real environments not having tls is unsafe.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
#### VMUser
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
To get authorized access to our data it is necessary to create a user using
|
|
|
|
the [vmuser](./resources/vmuser.md) resource.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Create file `vmuser.yaml`
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
code vmuser.yaml
|
|
|
|
```
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
with the following content:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
|
|
|
```yaml
|
|
|
|
apiVersion: operator.victoriametrics.com/v1beta1
|
2023-10-02 12:50:08 +00:00
|
|
|
kind: VMUser
|
2022-01-21 10:05:58 +00:00
|
|
|
metadata:
|
2023-10-02 12:50:08 +00:00
|
|
|
name: demo
|
2022-01-21 10:05:58 +00:00
|
|
|
spec:
|
2023-10-02 12:50:08 +00:00
|
|
|
name: demo
|
|
|
|
username: demo
|
|
|
|
generatePassword: true
|
|
|
|
targetRefs:
|
|
|
|
# vmui + vmselect
|
|
|
|
- crd:
|
|
|
|
kind: VMCluster/vmselect
|
|
|
|
name: demo
|
|
|
|
namespace: vm
|
|
|
|
target_path_suffix: "/select/0"
|
|
|
|
paths:
|
|
|
|
- "/vmui"
|
|
|
|
- "/vmui/.*"
|
|
|
|
- "/prometheus/api/v1/query"
|
|
|
|
- "/prometheus/api/v1/query_range"
|
|
|
|
- "/prometheus/api/v1/series"
|
|
|
|
- "/prometheus/api/v1/label/"
|
|
|
|
- "/prometheus/api/v1/label/[^/]+/values"
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
After that you can deploy `vmauth` and `vmuser` resources to the kubernetes cluster:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl apply -f vmauth.yaml -n vm
|
|
|
|
kubectl apply -f vmuser.yaml -n vm
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# vmauth.operator.victoriametrics.com/demo created
|
|
|
|
# vmuser.operator.victoriametrics.com/demo created
|
|
|
|
```
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Operator automatically creates a secret with username/password token for `VMUser` resource with `generatePassword=true`:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl get secret -n vm -l "app.kubernetes.io/instance=demo" -l "app.kubernetes.io/name=vmuser"
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# NAME TYPE DATA AGE
|
|
|
|
# vmuser-demo Opaque 3 29m
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
You can get password for your user with command:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl get secret -n vm vmuser-demo -o jsonpath="{.data.password}" | base64 --decode
|
2023-05-10 07:50:41 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# Yt3N2r3cPl
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Now you can get access to your data with url `http://vm-demo.k8s.orb.local/vmui`, username `demo`
|
|
|
|
and your given password (`Yt3N2r3cPl` in our case):
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
<img src="quick-start_select-1.png" width="1200">
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
<img src="quick-start_select-2.png" width="1200">
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
### Alerting
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
The remaining components will be needed for alerting.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
#### VMAlertmanager
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Let's start with [`vmalertmanager`](./resources/vmalertmanager.md).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Create file `vmuser.yaml`
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
code vmuser.yaml
|
|
|
|
```
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
with the following content:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
|
|
|
```yaml
|
|
|
|
apiVersion: operator.victoriametrics.com/v1beta1
|
2023-10-02 12:50:08 +00:00
|
|
|
kind: VMAlertmanager
|
2022-01-21 10:05:58 +00:00
|
|
|
metadata:
|
2023-10-02 12:50:08 +00:00
|
|
|
name: demo
|
2022-01-21 10:05:58 +00:00
|
|
|
spec:
|
2023-10-02 12:50:08 +00:00
|
|
|
configRawYaml: |
|
|
|
|
global:
|
|
|
|
resolve_timeout: 5m
|
|
|
|
route:
|
|
|
|
group_wait: 30s
|
|
|
|
group_interval: 5m
|
|
|
|
repeat_interval: 12h
|
|
|
|
receiver: 'webhook'
|
|
|
|
receivers:
|
|
|
|
- name: 'webhook'
|
|
|
|
webhook_configs:
|
|
|
|
- url: 'http://your-webhook-url'
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
where webhook-url is the address of the webhook to receive notifications
|
|
|
|
(configuration of AlertManager notifications will remain out of scope).
|
|
|
|
You can find more details about `alertmanager` configuration in
|
|
|
|
the [Alertmanager documentation](https://prometheus.io/docs/alerting/latest/configuration/).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
After that you can deploy `vmalertmanager` resource to the kubernetes cluster:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl apply -f vmalertmanager.yaml -n vm
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# vmalertmanager.operator.victoriametrics.com/demo created
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Check that `vmalertmanager` is running:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl get pods -n vm -l "app.kubernetes.io/instance=demo" -l "app.kubernetes.io/name=vmalertmanager"
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# NAME READY STATUS RESTARTS AGE
|
|
|
|
# vmalertmanager-demo-0 2/2 Running 0 107s
|
|
|
|
```
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
#### VMAlert
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
And now you can create [`vmalert`](./resources/vmalert.md) resource.
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Create file `vmalert.yaml`
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
code vmalert.yaml
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
with the following content:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
|
|
|
```yaml
|
|
|
|
apiVersion: operator.victoriametrics.com/v1beta1
|
|
|
|
kind: VMAlert
|
|
|
|
metadata:
|
2023-10-02 12:50:08 +00:00
|
|
|
name: demo
|
2022-01-21 10:05:58 +00:00
|
|
|
spec:
|
|
|
|
datasource:
|
2023-10-02 12:50:08 +00:00
|
|
|
url: "http://vmselect-demo.vm.svc:8481/select/0/prometheus"
|
|
|
|
remoteWrite:
|
|
|
|
url: "http://vminsert-demo.vm.svc:8480/insert/0/prometheus"
|
|
|
|
remoteRead:
|
|
|
|
url: "http://vmselect-demo.vm.svc:8481/select/0/prometheus"
|
2022-01-21 10:05:58 +00:00
|
|
|
notifier:
|
2023-10-02 12:50:08 +00:00
|
|
|
url: "http://vmalertmanager-demo.vm.svc:9093"
|
|
|
|
evaluationInterval: "30s"
|
|
|
|
selectAllByDefault: true
|
|
|
|
# for accessing to vmalert via vmauth with path prefix
|
|
|
|
extraArgs:
|
|
|
|
http.pathPrefix: /vmalert
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
After that you can deploy `vmalert` resource to the kubernetes cluster:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl apply -f vmalert.yaml -n vm
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# vmalert.operator.victoriametrics.com/demo created
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Check that `vmalert` is running:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl get pods -n vm -l "app.kubernetes.io/instance=demo" -l "app.kubernetes.io/name=vmalert"
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# NAME READY STATUS RESTARTS AGE
|
|
|
|
# vmalert-demo-bf75c67cb-hh4qd 2/2 Running 0 5s
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
#### VMRule
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Now you can create [vmrule](./resources/vmrule.md) resource
|
|
|
|
for [vmalert](./resources/vmalert.md).
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Create file `vmrule.yaml`
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
code vmrule.yaml
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
with the following content:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
|
|
|
```yaml
|
|
|
|
apiVersion: operator.victoriametrics.com/v1beta1
|
|
|
|
kind: VMRule
|
|
|
|
metadata:
|
2023-10-02 12:50:08 +00:00
|
|
|
name: demo
|
2022-01-21 10:05:58 +00:00
|
|
|
spec:
|
|
|
|
groups:
|
|
|
|
- name: vmalert
|
|
|
|
rules:
|
|
|
|
- alert: vmalert config reload error
|
|
|
|
expr: delta(vmalert_config_last_reload_errors_total[5m]) > 0
|
|
|
|
for: 10s
|
|
|
|
labels:
|
|
|
|
severity: major
|
2023-10-02 12:50:08 +00:00
|
|
|
job: "{{ $labels.job }}"
|
|
|
|
annotations:
|
|
|
|
value: "{{ $value }}"
|
|
|
|
description: 'error reloading vmalert config, reload count for 5 min {{ $value }}'
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
After that you can deploy `vmrule` resource to the kubernetes cluster:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl apply -f vmrule.yaml -n vm
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
# vmrule.operator.victoriametrics.com/demo created
|
|
|
|
```
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
#### VMUser update
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
Let's update our user with access to `vmalert` and `vmalertmanager`:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
code vmuser.yaml
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
apiVersion: operator.victoriametrics.com/v1beta1
|
2023-10-02 12:50:08 +00:00
|
|
|
kind: VMUser
|
2022-01-21 10:05:58 +00:00
|
|
|
metadata:
|
2023-10-02 12:50:08 +00:00
|
|
|
name: demo
|
2022-01-21 10:05:58 +00:00
|
|
|
spec:
|
2023-10-02 12:50:08 +00:00
|
|
|
name: demo
|
|
|
|
username: demo
|
|
|
|
generatePassword: true
|
|
|
|
targetRefs:
|
|
|
|
# vmui + vmselect
|
|
|
|
- crd:
|
|
|
|
kind: VMCluster/vmselect
|
|
|
|
name: demo
|
|
|
|
namespace: vm
|
|
|
|
target_path_suffix: "/select/0"
|
|
|
|
paths:
|
|
|
|
- "/vmui"
|
|
|
|
- "/vmui/.*"
|
|
|
|
- "/prometheus/api/v1/query"
|
|
|
|
- "/prometheus/api/v1/query_range"
|
|
|
|
- "/prometheus/api/v1/series"
|
|
|
|
- "/prometheus/api/v1/label/"
|
|
|
|
- "/prometheus/api/v1/label/[^/]+/values"
|
|
|
|
# vmalert
|
|
|
|
- crd:
|
|
|
|
kind: VMAlert
|
|
|
|
name: demo
|
|
|
|
namespace: vm
|
|
|
|
paths:
|
|
|
|
- "/vmalert"
|
|
|
|
- "/vmalert/.*"
|
|
|
|
- "/api/v1/groups"
|
|
|
|
- "/api/v1/alert"
|
|
|
|
- "/api/v1/alerts"
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
After that you can deploy `vmuser` resource to the kubernetes cluster:
|
2022-01-21 10:05:58 +00:00
|
|
|
|
2023-10-02 12:50:08 +00:00
|
|
|
```shell
|
|
|
|
kubectl apply -f vmuser.yaml -n vm
|
|
|
|
|
|
|
|
# vmuser.operator.victoriametrics.com/demo created
|
2022-01-21 10:05:58 +00:00
|
|
|
```
|
2023-10-02 12:50:08 +00:00
|
|
|
|
|
|
|
And now you can get access to your data with url `http://vm-demo.k8s.orb.local/vmalert`
|
|
|
|
(for your environment it most likely will be different) with username `demo`:
|
|
|
|
|
|
|
|
<img src="quick-start_alert-1.png" width="1200">
|
|
|
|
|
|
|
|
<img src="quick-start_alert-2.png" width="1200">
|
|
|
|
|
|
|
|
## Anything else
|
|
|
|
|
|
|
|
That's it. We obtained a monitoring cluster corresponding to the target topology:
|
|
|
|
|
|
|
|
<img src="quick-start_cluster-scheme.png" width="1200">
|
|
|
|
|
|
|
|
You have a full-stack monitoring cluster with VictoriaMetrics Operator.
|
|
|
|
|
|
|
|
You can find information about these and other resources of operator on the [Custom resources page](./resources/README.md).
|
|
|
|
|
|
|
|
In addition, check out other sections of the documentation for VictoriaMetrics Operator:
|
|
|
|
|
|
|
|
- [Setup](./setup.md)
|
|
|
|
- [Security](./security.md)
|
|
|
|
- [Configuration](./configuration.md)
|
|
|
|
- [Migration from Prometheus](./migration.md)
|
|
|
|
- [Monitoring](./monitoring.md)
|
|
|
|
- [Authorization and exposing components](./auth.md)
|
|
|
|
- [High Availability](./high-availability.md)
|
|
|
|
- [Enterprise](./enterprise.md)
|
|
|
|
|
|
|
|
If you have any questions, check out our [FAQ](./FAQ.md)
|
|
|
|
and feel free to can ask them:
|
|
|
|
- [VictoriaMetrics Slack](https://victoriametrics.slack.com/)
|
|
|
|
- [VictoriaMetrics Telegram](https://t.me/VictoriaMetrics_en)
|
|
|
|
|
|
|
|
If you have any suggestions or find a bug, please create an issue
|
|
|
|
on [GitHub](https://github.com/VictoriaMetrics/operator/issues/new).
|