Update PerTenantStatistic.md

This commit is contained in:
dereksfoster99 2021-04-14 11:44:24 -04:00 committed by Aliaksandr Valialkin
parent 113a612c86
commit 1b6d2467ff

View file

@ -6,27 +6,27 @@ sort: 10
<img alt="cluster-per-tenant-stat" src="per-tenant-stats.jpg"> <img alt="cluster-per-tenant-stat" src="per-tenant-stats.jpg">
Enterprise version of Victoria Metrics Cluster exposes usage statistic for each tenant. The enterprise version of VictoriaMetrics cluster exposes the usage statistics for each tenant.
The next statistic is exposed: When the next statistic is exposed:
- `vminsert` - `vminsert`
* `vm_tenant_inserted_rows_total` - the ingestion rate by tenant * `vm_tenant_inserted_rows_total` - the ingestion rate by tenant
- `vmselect` - `vmselect`
* `vm_tenant_select_requests_duration_ms_total` - query latency by tenant. It can be usefull for identifing tenant with the most heaviest queries * `vm_tenant_select_requests_duration_ms_total` - query latency by tenant. It can be useful to identify the tenant with the heaviest queries
* `vm_tenant_select_requests_total` - total requests. You can calculate request rate (qps) by using this metric * `vm_tenant_select_requests_total` - total requests. You can calculate request rate (qps) with this metric
- `vmstorage` - `vmstorage`
* `vm_tenant_active_timeseries` - the number of active timeseries * `vm_tenant_active_timeseries` - the number of active timeseries
* `vm_tenant_used_tenant_bytes` - the disk spaces is consumed by metrics for particular tenant * `vm_tenant_used_tenant_bytes` - the disk space consumed by the metrics for a particular tenant
* `vm_tenant_timeseries_created_total` - the total number for timeseries by tenant * `vm_tenant_timeseries_created_total` - the total number for timeseries by tenant
The information should be scrapped by the agent (`vmagent`, `victoriametrics`, prometheus, etc) and stored in TSDB. This can be the same cluster but a different tenant, but we encourage to use one more instance of TSDB (more lightweight, eg. VM single) for monitoring of monitoring. The information should be scraped by the agent (`vmagent`, `victoriametrics`, prometheus, etc) and stored in the TSDB. This can be the same cluster but a different tenant however, we encourage the use of one more instance of TSDB (more lightweight, eg. VM single) for the monitoring of monitoring.
the config example for statistic scrapping the config example for statistic scraping
```yaml ```yaml
scrape_configs: scrape_configs:
@ -38,18 +38,18 @@ scrape_configs:
### Visualization ### Visualization
Visualisation of statistic can be done in grafana using this dashboard [link](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster/dashboards/clusterbytenant.json) Visualisation of statistics can be done in Grafana using this dashboard [link](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster/dashboards/clusterbytenant.json)
### Integration with vmgateway ### Integration with vmgateway
Per Tenant Statistic is the source data for `vmgateway` rate limiter. More information can be found [here](https://victoriametrics.github.io/vmgateway.html) Per Tenant Statistics are the source data for the `vmgateway` rate limiter. More information can be found [here](https://victoriametrics.github.io/vmgateway.html)
### Integration with vmalert ### Integration with vmalert
You can generate alerts based on each tenant resource usage and notify the system/people about reaching the limits. You can generate alerts based on each tenants' resource usage and notify the system/users that they are reaching the limits.
here is an example of alert by high churn rate by the tenant Here is an example of an alert for high churn rate by the tenant
```yaml ```yaml
@ -65,8 +65,8 @@ here is an example of alert by high churn rate by the tenant
severity: warning severity: warning
annotations: annotations:
summary: "Churn rate is more than 10% for the last 15m" summary: "Churn rate is more than 10% for the last 15m"
description: "VM constantly creates new time series at tenant: {{ $labels.accountID }}:{{ $labels.projectID }}.\n description: "VM constantly creates new time series in the tenant: {{ $labels.accountID }}:{{ $labels.projectID }}.\n
This effect is known as Churn Rate.\n This effect is known as Churn Rate.\n
High Churn Rate tightly connected with database performance and may High Churn Rate is tightly connected with database performance and may
result in unexpected OOM's or slow queries." result in unexpected OOM's or slow queries."
``` ```