mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-11-21 14:44:00 +00:00
added k8s guide for managed VM
This commit is contained in:
parent
df1e545c0e
commit
79527441ec
3 changed files with 121 additions and 0 deletions
121
docs/managed_victoriametrics/how-to-monitor-k8s.md
Normal file
121
docs/managed_victoriametrics/how-to-monitor-k8s.md
Normal file
|
@ -0,0 +1,121 @@
|
||||||
|
# Kubernetes Monitoring with Managed VictoriaMetrics
|
||||||
|
|
||||||
|
Monitoring kubernetes cluster is necessary to build SLO/SLI, to analyze performance and cost-efficiency of your workloads.
|
||||||
|
|
||||||
|
To enable enable kubernetes cluster monitoring, we will be collecting metrics about cluster performance and utilization from kubernetes components like `kube-api-server`, `kube-controller-manager`, `kube-scheduler`, `kube-state-metrics`, `etcd`, `core-dns`, `kubelet` and `kube-proxy`. We will also install some recording rules, alert rules and dashboards to provide visibility of cluster performance, as well as alerting for cluster metrics.
|
||||||
|
For node resource utilization we will be collecting metrics from `node-exporter`. We will also install dashboard and alerts for node related metrics
|
||||||
|
|
||||||
|
For workloads monitoring in kubernetes cluster we will have [VictoriaMetrics Operator](https://docs.victoriametrics.com/operator/VictoriaMetrics-Operator.html). It enables us to define scrape jobs using kubernetes CRDs [VMServiceScrape](https://docs.victoriametrics.com/operator/design.html#vmservicescrape), [VMPodScrape](https://docs.victoriametrics.com/operator/design.html#vmpodscrape). To add alerts or recording rules for workloads we can use [VMRule](https://docs.victoriametrics.com/operator/design.html#vmrule) CRD
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
In this guide we will be using [victoria-metrics-k8s-stack](https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-k8s-stack) helm chart
|
||||||
|
|
||||||
|
This chart will install `VMOperator`, `VMAgent`, `NodeExporter`, `kube-state-metrics`, `grafana` and some service scrape configurations to start monitoring kuberentes cluster components
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
- Active Managed VictoriaMetrics instance. You can learn how to signup for Managed VictoriaMetrics [here](https://docs.victoriametrics.com/managed-victoriametrics/quickstart.html#how-to-register).
|
||||||
|
- Access to your kubernetes cluster
|
||||||
|
- Helm binary. You can find installation [here]https://helm.sh/docs/intro/install/()
|
||||||
|
|
||||||
|
### Installation steps
|
||||||
|
Install the Helm chart in a custom namespace
|
||||||
|
|
||||||
|
|
||||||
|
1. Create a unique Kubernetes namespace, for example monitoring
|
||||||
|
```
|
||||||
|
kubectl create namespace monitoring
|
||||||
|
```
|
||||||
|
2. Create kubernetes-secrets with token to access your dbaas deployment
|
||||||
|
```
|
||||||
|
kubectl --namespace monitoring create secret generic dbaas-write-access-token --from-literal=bearerToken=your-token
|
||||||
|
|
||||||
|
kubectl --namespace monitoring create secret generic dbaas-read-access-token --from-literal=bearerToken=your-token
|
||||||
|
```
|
||||||
|
|
||||||
|
> You can find your access token on the "Access" tab of your deployment
|
||||||
|
> ![access tijen](./k8s-monitoring-access-token.png)
|
||||||
|
|
||||||
|
|
||||||
|
3. Set up a Helm repository using the following commands:
|
||||||
|
```
|
||||||
|
helm repo add grafana https://grafana.github.io/helm-charts
|
||||||
|
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
|
||||||
|
helm repo add vm https://victoriametrics.github.io/helm-charts
|
||||||
|
helm repo update
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Create a YAML file of Helm values called dbaas.yaml with following content
|
||||||
|
```yaml
|
||||||
|
externalVM:
|
||||||
|
read:
|
||||||
|
url:
|
||||||
|
bearerTokenSecret:
|
||||||
|
name: dbaas-write-access-token
|
||||||
|
key: bearerToken
|
||||||
|
write:
|
||||||
|
url:
|
||||||
|
bearerTokenSecret:
|
||||||
|
name: dbaas-read-access-token
|
||||||
|
key: bearerToken
|
||||||
|
|
||||||
|
vmsingle:
|
||||||
|
enabled: false
|
||||||
|
|
||||||
|
vmcluster:
|
||||||
|
enabled: false
|
||||||
|
|
||||||
|
vmalert:
|
||||||
|
enabled: true
|
||||||
|
spec:
|
||||||
|
evaluationInterval: 15s
|
||||||
|
|
||||||
|
vmagent:
|
||||||
|
enabled: true
|
||||||
|
|
||||||
|
spec:
|
||||||
|
scrapeInterval: 30s
|
||||||
|
externalLabels:
|
||||||
|
cluster: cluster-name
|
||||||
|
|
||||||
|
# dependencies
|
||||||
|
# Grafana dependency chart configuration. For possible values refer to https://github.com/grafana/helm-charts/tree/main/charts/grafana#configuration
|
||||||
|
grafana:
|
||||||
|
enabled: true
|
||||||
|
```
|
||||||
|
|
||||||
|
5. Install VictoriaMetrics-k8s-stack helm chart
|
||||||
|
|
||||||
|
```
|
||||||
|
helm --namespace monitoring install vm vm/victoria-metrics-k8s-stack -f dbaas.yaml -n monitoring
|
||||||
|
```
|
||||||
|
|
||||||
|
### Connect to grafana and create your datasource
|
||||||
|
|
||||||
|
> If you are using external grafana, you can skip steps 1-3 and you will need to import dashboards that can be found here manually
|
||||||
|
|
||||||
|
|
||||||
|
1. Get grafana password
|
||||||
|
```
|
||||||
|
kubectl --namespace monitoring get secret vm-grafana -o jsonpath="{.data.admin-password}" | base64 -d
|
||||||
|
```
|
||||||
|
2. Connect to grafana
|
||||||
|
```
|
||||||
|
kubectl --namespace monitoring port-forward service/vm-grafana 3000:80
|
||||||
|
```
|
||||||
|
3. Open grafana in your browser http://localhost:3000/datasources
|
||||||
|
|
||||||
|
Use admin as username and password from previous step
|
||||||
|
|
||||||
|
4. Click on add datasource
|
||||||
|
|
||||||
|
Choose VictoriaMetrics or Prometheus as datasource type. Make sure you made this datasource as default for dashboards to work.
|
||||||
|
|
||||||
|
![datasource](./k8s-monitoring-datasource.png)
|
||||||
|
|
||||||
|
> You can find token and URL in your deployment, on access tab
|
||||||
|
|
||||||
|
### Test it
|
||||||
|
|
||||||
|
You should be able to see data that was sent to your dbaas using VMAgent dashboard http://localhost:3000/d/G7Z9GzMGz/victoriametrics-vmagent/
|
||||||
|
You also will be able to see bunch of kuberentes dashboards in your grafana
|
BIN
docs/managed_victoriametrics/k8s-monitoring-access-token.png
Normal file
BIN
docs/managed_victoriametrics/k8s-monitoring-access-token.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 949 KiB |
BIN
docs/managed_victoriametrics/k8s-monitoring-datasource.png
Normal file
BIN
docs/managed_victoriametrics/k8s-monitoring-datasource.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 269 KiB |
Loading…
Reference in a new issue