github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-11 14:53:49 +00:00
Code Issues Projects Releases Packages Wiki Activity

Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files

Browse source
This commit is contained in:
Aliaksandr Valialkin 2022-06-19 23:05:31 +03:00
commit cacd3d6f6d
No known key found for this signature in database
GPG key ID: A72BEC6CD3D0DED1
76 changed files with 2571 additions and 770 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

91
README.md
View file

@ -164,7 +164,7 @@ Then apply new config via the following command:
<div class="with-copy" markdown="1">
```bash
```console
kill -HUP `pidof prometheus`
```
@ -328,7 +328,7 @@ VictoriaMetrics doesn't check `DD_API_KEY` param, so it can be set to arbitrary
Example on how to send data to VictoriaMetrics via DataDog "submit metrics" API from command line:
```bash
```console
echo '
{
"series": [
@ -354,7 +354,7 @@ The imported data can be read via [export API](https://docs.victoriametrics.com/
<div class="with-copy" markdown="1">
```bash
```console
curl http://localhost:8428/api/v1/export -d 'match[]=system.load.1'
```
@ -369,6 +369,16 @@ This command should return the following output if everything is OK:
Extra labels may be added to all the written time series by passing `extra_label=name=value` query args.
For example, `/datadog/api/v1/series?extra_label=foo=bar` would add `{foo="bar"}` label to all the ingested metrics.
DataDog agent sends the [configured tags](https://docs.datadoghq.com/getting_started/tagging/) to
undocumented endpoint - `/datadog/intake`. This endpoint isn't supported by VictoriaMetrics yet.
This prevents from adding the configured tags to DataDog agent data sent into VictoriaMetrics.
The workaround is to run a sidecar [vmagent](https://docs.victoriametrics.com/vmagent.html) alongside every DataDog agent,
which must run with `DD_DD_URL=http://localhost:8429/datadog` environment variable.
The sidecar `vmagent` must be configured with the needed tags via `-remoteWrite.label` command-line flag and must forward
incoming data with the added tags to a centralized VictoriaMetrics specified via `-remoteWrite.url` command-line flag.
See [these docs](https://docs.victoriametrics.com/vmagent.html#adding-labels-to-metrics) for details on how to add labels to metrics at `vmagent`.
## How to send data from InfluxDB-compatible agents such as [Telegraf](https://www.influxdata.com/time-series-platform/telegraf/)
Use `http://<victoriametric-addr>:8428` url instead of InfluxDB url in agents' configs.
@ -408,7 +418,7 @@ to local VictoriaMetrics using `curl`:
<div class="with-copy" markdown="1">
```bash
```console
curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/write'
```
@ -419,7 +429,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match={__name__=~"measurement_.*"}'
```
@ -447,7 +457,7 @@ Comma-separated list of expected databases can be passed to VictoriaMetrics via
Enable Graphite receiver in VictoriaMetrics by setting `-graphiteListenAddr` command line flag. For instance,
the following command will enable Graphite receiver in VictoriaMetrics on TCP and UDP port `2003`:
```bash
```console
/path/to/victoria-metrics-prod -graphiteListenAddr=:2003
```
@ -456,7 +466,7 @@ to the VictoriaMetrics host in `StatsD` configs.
Example for writing data with Graphite plaintext protocol to local VictoriaMetrics using `nc`:
```bash
```console
echo "foo.bar.baz;tag1=value1;tag2=value2 123 `date +%s`" | nc -N localhost 2003
```
@ -466,7 +476,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match=foo.bar.baz'
```
@ -478,6 +488,8 @@ The `/api/v1/export` endpoint should return the following response:
{"metric":{"__name__":"foo.bar.baz","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1560277406000]}
```
[Graphite relabeling](https://docs.victoriametrics.com/vmagent.html#graphite-relabeling) can be used if the imported Graphite data is going to be queried via [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html).
## Querying Graphite data
Data sent to VictoriaMetrics via `Graphite plaintext protocol` may be read via the following APIs:
@ -492,6 +504,9 @@ VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series w
The `__graphite__` pseudo-label supports e.g. alternate regexp filters such as `(value1|...|valueN)`. They are transparently converted to `{value1,...,valueN}` syntax [used in Graphite](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). This allows using [multi-value template variables in Grafana](https://grafana.com/docs/grafana/latest/variables/formatting-multi-value-variables/) inside `__graphite__` pseudo-label. For example, Grafana expands `{__graphite__=~"foo.($bar).baz"}` into `{__graphite__=~"foo.(x|y).baz"}` if `$bar` template variable contains `x` and `y` values. In this case the query is automatically converted into `{__graphite__=~"foo.{x,y}.baz"}` before execution.
VictoriaMetrics also supports Graphite query language - see [these docs](#graphite-render-api-usage).
## How to send data from OpenTSDB-compatible agents
VictoriaMetrics supports [telnet put protocol](http://opentsdb.net/docs/build/html/api_telnet/put.html)
@ -503,7 +518,7 @@ The same protocol is used for [ingesting data in KairosDB](https://kairosdb.gith
Enable OpenTSDB receiver in VictoriaMetrics by setting `-opentsdbListenAddr` command line flag. For instance,
the following command enables OpenTSDB receiver in VictoriaMetrics on TCP and UDP port `4242`:
```bash
```console
/path/to/victoria-metrics-prod -opentsdbListenAddr=:4242
```
@ -513,7 +528,7 @@ Example for writing data with OpenTSDB protocol to local VictoriaMetrics using `
<div class="with-copy" markdown="1">
```bash
```console
echo "put foo.bar.baz `date +%s` 123 tag1=value1 tag2=value2" | nc -N localhost 4242
```
@ -524,7 +539,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match=foo.bar.baz'
```
@ -541,7 +556,7 @@ The `/api/v1/export` endpoint should return the following response:
Enable HTTP server for OpenTSDB `/api/put` requests by setting `-opentsdbHTTPListenAddr` command line flag. For instance,
the following command enables OpenTSDB HTTP server on port `4242`:
```bash
```console
/path/to/victoria-metrics-prod -opentsdbHTTPListenAddr=:4242
```
@ -551,7 +566,7 @@ Example for writing a single data point:
<div class="with-copy" markdown="1">
```bash
```console
curl -H 'Content-Type: application/json' -d '{"metric":"x.y.z","value":45.34,"tags":{"t1":"v1","t2":"v2"}}' http://localhost:4242/api/put
```
@ -561,7 +576,7 @@ Example for writing multiple data points in a single request:
<div class="with-copy" markdown="1">
```bash
```console
curl -H 'Content-Type: application/json' -d '[{"metric":"foo","value":45.34},{"metric":"bar","value":43}]' http://localhost:4242/api/put
```
@ -571,7 +586,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match[]=x.y.z' -d 'match[]=foo' -d 'match[]=bar'
```
@ -741,7 +756,7 @@ The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is pos
by setting it via `<ROOT_IMAGE>` environment variable.
For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-victoria-metrics
```
@ -855,7 +870,7 @@ Each JSON line contains samples for a single time series. An example output:
Optional `start` and `end` args may be added to the request in order to limit the time frame for the exported data. These args may contain either
unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/api/v1/export -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/api/v1/export -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -869,7 +884,7 @@ of time series data. This enables gzip compression for the exported data. Exampl
<div class="with-copy" markdown="1">
```bash
```console
curl -H 'Accept-Encoding: gzip' http://localhost:8428/api/v1/export -d 'match[]={__name__!=""}' > data.jsonl.gz
```
@ -903,7 +918,7 @@ for metrics to export.
Optional `start` and `end` args may be added to the request in order to limit the time frame for the exported data. These args may contain either
unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/api/v1/export/csv -d 'format=<format>' -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/api/v1/export/csv -d 'format=<format>' -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -920,7 +935,7 @@ for metrics to export. Use `{__name__=~".*"}` selector for fetching all the time
On large databases you may experience problems with limit on the number of time series, which can be exported. In this case you need to adjust `-search.maxExportSeries` command-line flag:
```bash
```console
# count unique timeseries in database
wget -O- -q 'http://your_victoriametrics_instance:8428/api/v1/series/count' | jq '.data[0]'
@ -930,7 +945,7 @@ wget -O- -q 'http://your_victoriametrics_instance:8428/api/v1/series/count' | jq
Optional `start` and `end` args may be added to the request in order to limit the time frame for the exported data. These args may contain either
unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/api/v1/export/native -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/api/v1/export/native -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -962,7 +977,7 @@ Time series data can be imported into VictoriaMetrics via any supported data ing
Example for importing data obtained via [/api/v1/export](#how-to-export-data-in-json-line-format):
```bash
```console
# Export the data from <source-victoriametrics>:
curl http://source-victoriametrics:8428/api/v1/export -d 'match={__name__!=""}' > exported_data.jsonl
@ -972,7 +987,7 @@ curl -X POST http://destination-victoriametrics:8428/api/v1/import -T exported_d
Pass `Content-Encoding: gzip` HTTP request header to `/api/v1/import` for importing gzipped data:
```bash
```console
# Export gzipped data from <source-victoriametrics>:
curl -H 'Accept-Encoding: gzip' http://source-victoriametrics:8428/api/v1/export -d 'match={__name__!=""}' > exported_data.jsonl.gz
@ -993,7 +1008,7 @@ The specification of VictoriaMetrics' native format may yet change and is not fo
If you have a native format file obtained via [/api/v1/export/native](#how-to-export-data-in-native-format) however this is the most efficient protocol for importing data in.
```bash
```console
# Export the data from <source-victoriametrics>:
curl http://source-victoriametrics:8428/api/v1/export/native -d 'match={__name__!=""}' > exported_data.bin
@ -1034,14 +1049,14 @@ Each request to `/api/v1/import/csv` may contain arbitrary number of CSV lines.
Example for importing CSV data via `/api/v1/import/csv`:
```bash
```console
curl -d "GOOG,1.23,4.56,NYSE" 'http://localhost:8428/api/v1/import/csv?format=2:metric:ask,3:metric:bid,1:label:ticker,4:label:market'
curl -d "MSFT,3.21,1.67,NASDAQ" 'http://localhost:8428/api/v1/import/csv?format=2:metric:ask,3:metric:bid,1:label:ticker,4:label:market'
```
After that the data may be read via [/api/v1/export](#how-to-export-data-in-json-line-format) endpoint:
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match[]={ticker!=""}'
```
@ -1067,7 +1082,7 @@ via `/api/v1/import/prometheus` path. For example, the following line imports a
<div class="with-copy" markdown="1">
```bash
```console
curl -d 'foo{bar="baz"} 123' -X POST 'http://localhost:8428/api/v1/import/prometheus'
```
@ -1077,7 +1092,7 @@ The following command may be used for verifying the imported data:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match={__name__=~"foo"}'
```
@ -1093,7 +1108,7 @@ Pass `Content-Encoding: gzip` HTTP request header to `/api/v1/import/prometheus`
<div class="with-copy" markdown="1">
```bash
```console
# Import gzipped data to <destination-victoriametrics>:
curl -X POST -H 'Content-Encoding: gzip' http://destination-victoriametrics:8428/api/v1/import/prometheus -T prometheus_data.gz
```
@ -1132,7 +1147,9 @@ Example contents for `-relabelConfig` file:
regex: true
```
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details about relabeling in VictoriaMetrics.
VictoriaMetrics components provide additional relabeling features such as Graphite-style relabeling.
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details.
## Federation
@ -1142,7 +1159,7 @@ at `http://<victoriametrics-addr>:8428/federate?match[]=<timeseries_selector_for
Optional `start` and `end` args may be added to the request in order to scrape the last point for each selected time series on the `[start ... end]` interval.
`start` and `end` may contain either unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/federate -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/federate -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -1196,7 +1213,7 @@ See also [cardinality limiter](#cardinality-limiter) and [capacity planning docs
* Install multiple VictoriaMetrics instances in distinct datacenters (availability zones).
* Pass addresses of these instances to [vmagent](https://docs.victoriametrics.com/vmagent.html) via `-remoteWrite.url` command-line flag:
```bash
```console
/path/to/vmagent -remoteWrite.url=http://<victoriametrics-addr-1>:8428/api/v1/write -remoteWrite.url=http://<victoriametrics-addr-2>:8428/api/v1/write
```
@ -1215,7 +1232,7 @@ remote_write:
* Apply the updated config:
```bash
```console
kill -HUP `pidof prometheus`
```
@ -1396,7 +1413,7 @@ For example, substitute `-graphiteListenAddr=:2003` with `-graphiteListenAddr=<i
If you plan to store more than 1TB of data on `ext4` partition or plan extending it to more than 16TB,
then the following options are recommended to pass to `mkfs.ext4`:
```bash
```console
mkfs.ext4 ... -O 64bit,huge_file,extent -T huge
```
@ -1457,7 +1474,7 @@ In this case VictoriaMetrics puts query trace into `trace` field in the output J
For example, the following command:
```bash
```console
curl http://localhost:8428/api/v1/query_range -d 'query=2*rand()' -d 'start=-1h' -d 'step=1m' -d 'trace=1' | jq '.trace'
```
@ -1718,7 +1735,7 @@ VictoriaMetrics provides handlers for collecting the following [Go profiles](htt
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8428/debug/pprof/heap > mem.pprof
```
@ -1728,7 +1745,7 @@ curl http://0.0.0.0:8428/debug/pprof/heap > mem.pprof
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8428/debug/pprof/profile > cpu.pprof
```

59
app/vmagent/README.md
View file

@ -73,7 +73,7 @@ Pass `-help` to `vmagent` in order to see [the full list of supported command-li
* Sending `SUGHUP` signal to `vmagent` process:
```bash
```console
kill -SIGHUP `pidof vmagent`
```
@ -252,12 +252,13 @@ Labels can be added to metrics by the following mechanisms:
VictoriaMetrics components (including `vmagent`) support Prometheus-compatible relabeling.
They provide the following additional actions on top of actions from the [Prometheus relabeling](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config):
* `replace_all`: replaces all of the occurences of `regex` in the values of `source_labels` with the `replacement` and stores the results in the `target_label`.
* `labelmap_all`: replaces all of the occurences of `regex` in all the label names with the `replacement`.
* `keep_if_equal`: keeps the entry if all the label values from `source_labels` are equal.
* `drop_if_equal`: drops the entry if all the label values from `source_labels` are equal.
* `keep_metrics`: keeps all the metrics with names matching the given `regex`.
* `drop_metrics`: drops all the metrics with names matching the given `regex`.
* `replace_all`: replaces all of the occurences of `regex` in the values of `source_labels` with the `replacement` and stores the results in the `target_label`
* `labelmap_all`: replaces all of the occurences of `regex` in all the label names with the `replacement`
* `keep_if_equal`: keeps the entry if all the label values from `source_labels` are equal
* `drop_if_equal`: drops the entry if all the label values from `source_labels` are equal
* `keep_metrics`: keeps all the metrics with names matching the given `regex`
* `drop_metrics`: drops all the metrics with names matching the given `regex`
* `graphite`: applies Graphite-style relabeling to metric name. See [these docs](#graphite-relabeling)
The `regex` value can be split into multiple lines for improved readability and maintainability. These lines are automatically joined with `|` char when parsed. For example, the following configs are equivalent:
@ -305,6 +306,38 @@ You can read more about relabeling in the following articles:
* [Extracting labels from legacy metric names](https://www.robustperception.io/extracting-labels-from-legacy-metric-names)
* [relabel_configs vs metric_relabel_configs](https://www.robustperception.io/relabel_configs-vs-metric_relabel_configs)
## Graphite relabeling
VictoriaMetrics components support `action: graphite` relabeling rules, which allow extracting various parts from Graphite-style metrics
into the configured labels with the syntax similar to [Glob matching in statsd_exporter](https://github.com/prometheus/statsd_exporter#glob-matching).
Note that the `name` field must be substituted with explicit `__name__` option under `labels` section.
If `__name__` option is missing under `labels` section, then the original Graphite-style metric name is left unchanged.
For example, the following relabeling rule generates `requests_total{job="app42",instance="host124:8080"}` metric
from "app42.host123.requests.total" Graphite-style metric:
```yaml
- action: graphite
match: "*.*.*.total"
labels:
__name__: "${3}_total"
job: "$1"
instance: "${2}:8080"
```
Important notes about `action: graphite` relabeling rules:
- The relabeling rule is applied only to metrics, which match the given `match` expression. Other metrics remain unchanged.
- The `*` matches the maximum possible number of chars until the next dot or until the next part of the `match` expression whichever comes first.
It may match zero chars if the next char is `.`.
For example, `match: "app*foo.bar"` matches `app42foo.bar` and `42` becomes available to use at `labels` section via `$1` capture group.
- The `$0` capture group matches the original metric name.
- The relabeling rules are executed in order defined in the original config.
The `action: graphite` relabeling rules are easier to write and maintain than `action: replace` for labels extraction from Graphite-style metric names.
Additionally, the `action: graphite` relabeling rules usually work much faster than the equivalent `action: replace` rules.
## Prometheus staleness markers
`vmagent` sends [Prometheus staleness markers](https://www.robustperception.io/staleness-and-promql) to `-remoteWrite.url` in the following cases:
@ -560,7 +593,7 @@ Every Kafka message may contain multiple lines in `influx`, `prometheus`, `graph
The following command starts `vmagent`, which reads metrics in InfluxDB line protocol format from Kafka broker at `localhost:9092` from the topic `metrics-by-telegraf` and sends them to remote storage at `http://localhost:8428/api/v1/write`:
```bash
```console
./bin/vmagent -remoteWrite.url=http://localhost:8428/api/v1/write \
-kafka.consumer.topic.brokers=localhost:9092 \
-kafka.consumer.topic.format=influx \
@ -622,13 +655,13 @@ Two types of auth are supported:
* sasl with username and password:
```bash
```console
./bin/vmagent -remoteWrite.url=kafka://localhost:9092/?topic=prom-rw&security.protocol=SASL_SSL&sasl.mechanisms=PLAIN -remoteWrite.basicAuth.username=user -remoteWrite.basicAuth.password=password
```
* tls certificates:
```bash
```console
./bin/vmagent -remoteWrite.url=kafka://localhost:9092/?topic=prom-rw&security.protocol=SSL -remoteWrite.tlsCAFile=/opt/ca.pem -remoteWrite.tlsCertFile=/opt/cert.pem -remoteWrite.tlsKeyFile=/opt/key.pem
```
@ -657,7 +690,7 @@ The `<PKG_TAG>` may be manually set via `PKG_TAG=foobar make package-vmagent`.
The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
by setting it via `<ROOT_IMAGE>` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-vmagent
```
@ -685,7 +718,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8429/debug/pprof/heap > mem.pprof
```
@ -695,7 +728,7 @@ curl http://0.0.0.0:8429/debug/pprof/heap > mem.pprof
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8429/debug/pprof/profile > cpu.pprof
```

24
app/vmalert/README.md
View file

@ -36,7 +36,7 @@ implementation and aims to be compatible with its syntax.
To build `vmalert` from sources:
```bash
```console
git clone https://github.com/VictoriaMetrics/VictoriaMetrics
cd VictoriaMetrics
make vmalert
@ -52,12 +52,13 @@ To start using `vmalert` you will need the following things:
aggregating alerts, and sending notifications. Please note, notifier address also supports Consul and DNS Service Discovery via
[config file](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmalert/notifier/config.go).
* remote write address [optional] - [remote write](https://prometheus.io/docs/prometheus/latest/storage/#remote-storage-integrations)
compatible storage to persist rules and alerts state info;
compatible storage to persist rules and alerts state info. To persist results to multiple destinations use vmagent
configured with multiple remote writes as a proxy;
* remote read address [optional] - MetricsQL compatible datasource to restore alerts state from.
Then configure `vmalert` accordingly:
```bash
```console
./bin/vmalert -rule=alert.rules \ # Path to the file with rules configuration. Supports wildcard
-datasource.url=http://localhost:8428 \ # PromQL compatible datasource
-notifier.url=http://localhost:9093 \ # AlertManager URL (required if alerting rules are used)
@ -424,6 +425,21 @@ Flags `-remoteRead.url` and `-notifier.url` are omitted since we assume only rec
See also [downsampling docs](https://docs.victoriametrics.com/#downsampling).
#### Multiple remote writes
For persisting recording or alerting rule results `vmalert` requires `-remoteWrite.url` to be set.
But this flag supports only one destination. To persist rule results to multiple destinations
we recommend using [vmagent](https://docs.victoriametrics.com/vmagent.html) as fan-out proxy:
<img alt="vmalert multiple remote write destinations" src="vmalert_multiple_rw.png">
In this topology, `vmalert` is configured to persist rule results to `vmagent`. And `vmagent`
is configured to fan-out received data to two or more destinations.
Using `vmagent` as a proxy provides additional benefits such as
[data persisting when storage is unreachable](https://docs.victoriametrics.com/vmagent.html#replication-and-high-availability),
or time series modification via [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling).
### Web
`vmalert` runs a web-server (`-httpListenAddr`) for serving metrics and alerts endpoints:
@ -1022,7 +1038,7 @@ It is recommended using
You can build `vmalert` docker image from source and push it to your own docker repository.
Run the following commands from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics):
```bash
```console
make package-vmalert
docker tag victoria-metrics/vmalert:version my-repo:my-version-name
docker push my-repo:my-version-name

6
app/vmalert/group.go
View file

@ -438,11 +438,17 @@ func (e *executor) exec(ctx context.Context, rule Rule, ts time.Time, resolveDur
return nil
}
wg := sync.WaitGroup{}
for _, nt := range e.notifiers() {
wg.Add(1)
go func(nt notifier.Notifier) {
if err := nt.Send(ctx, alerts); err != nil {
errGr.Add(fmt.Errorf("rule %q: failed to send alerts to addr %q: %w", rule, nt.Addr(), err))
}
wg.Done()
}(nt)
}
wg.Wait()
return errGr.Err()
}

39
app/vmalert/group_test.go
View file

@ -413,3 +413,42 @@ func TestPurgeStaleSeries(t *testing.T) {
[]Rule{&AlertingRule{RuleID: 1}, &AlertingRule{RuleID: 2}},
)
}
func TestFaultyNotifier(t *testing.T) {
fq := &fakeQuerier{}
fq.add(metricWithValueAndLabels(t, 1, "__name__", "foo", "job", "bar"))
r := newTestAlertingRule("instant", 0)
r.q = fq
fn := &fakeNotifier{}
e := &executor{
notifiers: func() []notifier.Notifier {
return []notifier.Notifier{
&faultyNotifier{},
fn,
}
},
}
delay := 5 * time.Second
ctx, cancel := context.WithTimeout(context.Background(), delay)
defer cancel()
go func() {
_ = e.exec(ctx, r, time.Now(), 0, 10)
}()
tn := time.Now()
deadline := tn.Add(delay / 2)
for {
if fn.getCounter() > 0 {
return
}
if tn.After(deadline) {
break
}
tn = time.Now()
time.Sleep(time.Millisecond * 100)
}
t.Fatalf("alive notifier didn't receive notification by %v", deadline)
}

12
app/vmalert/helpers_test.go
View file

@ -87,6 +87,18 @@ func (fn *fakeNotifier) getAlerts() []notifier.Alert {
return fn.alerts
}
type faultyNotifier struct {
fakeNotifier
}
func (fn *faultyNotifier) Send(ctx context.Context, _ []notifier.Alert) error {
d, ok := ctx.Deadline()
if ok {
time.Sleep(time.Until(d))
}
return fmt.Errorf("send failed")
}
func metricWithValueAndLabels(t *testing.T, value float64, labels ...string) datasource.Metric {
return metricWithValuesAndLabels(t, []float64{value}, labels...)
}

2
app/vmalert/notifier/init.go
View file

@ -145,7 +145,7 @@ func notifiersFromFlags(gen AlertURLGenerator) ([]Notifier, error) {
}
addr = strings.TrimSuffix(addr, "/")
am, err := NewAlertManager(addr+alertManagerPath, gen, authCfg, nil, time.Minute)
am, err := NewAlertManager(addr+alertManagerPath, gen, authCfg, nil, time.Second*10)
if err != nil {
return nil, err
}

17
app/vmalert/utils/err_group.go
View file

@ -3,24 +3,34 @@ package utils
import (
"fmt"
"strings"
"sync"
)
// ErrGroup accumulates multiple errors
// and produces single error message.
type ErrGroup struct {
mu sync.Mutex
errs []error
}
// Add adds a new error to group.
// Isn't thread-safe.
// Is thread-safe.
func (eg *ErrGroup) Add(err error) {
eg.mu.Lock()
eg.errs = append(eg.errs, err)
eg.mu.Unlock()
}
// Err checks if group contains at least
// one error.
func (eg *ErrGroup) Err() error {
if eg == nil || len(eg.errs) == 0 {
if eg == nil {
return nil
}
eg.mu.Lock()
defer eg.mu.Unlock()
if len(eg.errs) == 0 {
return nil
}
return eg
@ -28,6 +38,9 @@ func (eg *ErrGroup) Err() error {
// Error satisfies Error interface
func (eg *ErrGroup) Error() string {
eg.mu.Lock()
defer eg.mu.Unlock()
if len(eg.errs) == 0 {
return ""
}

27
app/vmalert/utils/err_group_test.go
View file

@ -2,6 +2,7 @@ package utils
import (
"errors"
"fmt"
"testing"
)
@ -36,3 +37,29 @@ func TestErrGroup(t *testing.T) {
}
}
}
// TestErrGroupConcurrent supposed to test concurrent
// use of error group.
// Should be executed with -race flag
func TestErrGroupConcurrent(t *testing.T) {
eg := new(ErrGroup)
const writersN = 4
payload := make(chan error, writersN)
for i := 0; i < writersN; i++ {
go func() {
for err := range payload {
eg.Add(err)
}
}()
}
const iterations = 500
for i := 0; i < iterations; i++ {
payload <- fmt.Errorf("error %d", i)
if i%10 == 0 {
_ = eg.Err()
}
}
close(payload)
}

711
app/vmalert/vmalert_multiple_rw.excalidraw Normal file
View file

@ -0,0 +1,711 @@
{
"type": "excalidraw",
"version": 2,
"source": "https://excalidraw.com",
"elements": [
{
"type": "rectangle",
"version": 797,
"versionNonce": 1977657992,
"isDeleted": false,
"id": "VgBUzo0blGR-Ijd2mQEEf",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 289.6802978515625,
"y": 399.3895568847656,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 123.7601318359375,
"height": 72.13211059570312,
"seed": 1194011660,
"groupIds": [
"iBaXgbpyifSwPplm_GO5b"
],
"strokeSharpness": "sharp",
"boundElements": [
{
"type": "arrow",
"id": "miEbzHxOPXe4PEYvXiJp5"
},
{
"type": "arrow",
"id": "rcmiQfIWtfbTTlwxqr1sl"
},
{
"type": "arrow",
"id": "P-dpWlSTtnsux-zr5oqgF"
},
{
"type": "arrow",
"id": "oAToSPttH7aWoD_AqXGFX"
},
{
"type": "arrow",
"id": "wRO0q9xKPHc8e8XPPsQWh"
},
{
"type": "arrow",
"id": "sxEhnxlbT7ldlSsmHDUHp"
},
{
"type": "arrow",
"id": "pD9DcILMxa6GaR1U5YyMO"
},
{
"type": "arrow",
"id": "HPEwr85wL4IedW0AgdArp"
},
{
"type": "arrow",
"id": "EyecK0YM9Cc8T6ju-nTOc"
},
{
"id": "xpdAlCCGgIMAgSaqQ4K65",
"type": "arrow"
}
],
"updated": 1655372487772,
"link": null,
"locked": false
},
{
"type": "text",
"version": 671,
"versionNonce": 1438327288,
"isDeleted": false,
"id": "e9TDm09y-GhPm84XWt0Jv",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 311.22686767578125,
"y": 420.4738006591797,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 83,
"height": 24,
"seed": 327273100,
"groupIds": [
"iBaXgbpyifSwPplm_GO5b"
],
"strokeSharpness": "sharp",
"boundElements": [],
"updated": 1655372487772,
"link": null,
"locked": false,
"fontSize": 20,
"fontFamily": 3,
"text": "vmagent",
"baseline": 20,
"textAlign": "center",
"verticalAlign": "middle",
"containerId": null,
"originalText": "vmagent"
},
{
"type": "rectangle",
"version": 1247,
"versionNonce": 1809504904,
"isDeleted": false,
"id": "Sa4OBd1ZjD6itohm7Ll8z",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 542.2673645019531,
"y": 308.46409606933594,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 219.1235961914062,
"height": 44.74725341796875,
"seed": 126267060,
"groupIds": [
"ek-pq3umtz1yN-J_-preq"
],
"strokeSharpness": "sharp",
"boundElements": [
{
"type": "arrow",
"id": "wRO0q9xKPHc8e8XPPsQWh"
},
{
"type": "arrow",
"id": "he-SpFjCxEQEWpWny2kKP"
},
{
"type": "arrow",
"id": "-pjrKo16rOsasM8viZPJ-"
},
{
"id": "HPEwr85wL4IedW0AgdArp",
"type": "arrow"
}
],
"updated": 1655372610014,
"link": null,
"locked": false
},
{
"type": "text",
"version": 1149,
"versionNonce": 1939391880,
"isDeleted": false,
"id": "we766A079lfGYu2_aC4Pl",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 629.1559448242188,
"y": 318.8975372314453,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 48,
"height": 24,
"seed": 478660236,
"groupIds": [
"ek-pq3umtz1yN-J_-preq"
],
"strokeSharpness": "sharp",
"boundElements": [],
"updated": 1655372621140,
"link": null,
"locked": false,
"fontSize": 20,
"fontFamily": 3,
"text": "vm-1",
"baseline": 20,
"textAlign": "center",
"verticalAlign": "top",
"containerId": null,
"originalText": "vm-1"
},
{
"type": "arrow",
"version": 337,
"versionNonce": 1739475336,
"isDeleted": false,
"id": "HPEwr85wL4IedW0AgdArp",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 423.70701599121094,
"y": 431.0309448437124,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 107.82342529296875,
"height": 100.61778190120276,
"seed": 389863732,
"groupIds": [],
"strokeSharpness": "round",
"boundElements": [],
"updated": 1655372610015,
"link": null,
"locked": false,
"startBinding": {
"elementId": "VgBUzo0blGR-Ijd2mQEEf",
"focus": 0.6700023593531782,
"gap": 10.266586303710938
},
"endBinding": {
"elementId": "Sa4OBd1ZjD6itohm7Ll8z",
"focus": 0.9042666945544442,
"gap": 10.736923217773438
},
"lastCommittedPoint": null,
"startArrowhead": null,
"endArrowhead": "arrow",
"points": [
[
0,
0
],
[
107.82342529296875,
-100.61778190120276
]
]
},
{
"type": "arrow",
"version": 429,
"versionNonce": 252631288,
"isDeleted": false,
"id": "EyecK0YM9Cc8T6ju-nTOc",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 424.7585906982422,
"y": 432.4328003132737,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 119.94342041015625,
"height": 83.58206327156176,
"seed": 981082124,
"groupIds": [],
"strokeSharpness": "round",
"boundElements": [],
"updated": 1655372623571,
"link": null,
"locked": false,
"startBinding": {
"elementId": "VgBUzo0blGR-Ijd2mQEEf",
"focus": -0.6826568395144794,
"gap": 11.318161010742188
},
"endBinding": {
"elementId": "lXpACjXQqK7SZF_vrACjJ",
"focus": -0.8650156795513397,
"gap": 3.6341629028320312
},
"lastCommittedPoint": null,
"startArrowhead": null,
"endArrowhead": "arrow",
"points": [
[
0,
0
],
[
119.94342041015625,
83.58206327156176
]
]
},
{
"type": "rectangle",
"version": 979,
"versionNonce": 896077192,
"isDeleted": false,
"id": "X08ptHmEm7tCgoFbQntAR",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": -4.634010314941406,
"y": 402.69072341918945,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 123.7601318359375,
"height": 72.13211059570312,
"seed": 1000953848,
"groupIds": [
"IAd7y_6yDxq13U11FuJvH"
],
"strokeSharpness": "sharp",
"boundElements": [
{
"type": "arrow",
"id": "miEbzHxOPXe4PEYvXiJp5"
},
{
"type": "arrow",
"id": "rcmiQfIWtfbTTlwxqr1sl"
},
{
"type": "arrow",
"id": "P-dpWlSTtnsux-zr5oqgF"
},
{
"type": "arrow",
"id": "oAToSPttH7aWoD_AqXGFX"
},
{
"type": "arrow",
"id": "wRO0q9xKPHc8e8XPPsQWh"
},
{
"type": "arrow",
"id": "sxEhnxlbT7ldlSsmHDUHp"
},
{
"type": "arrow",
"id": "pD9DcILMxa6GaR1U5YyMO"
},
{
"type": "arrow",
"id": "HPEwr85wL4IedW0AgdArp"
},
{
"type": "arrow",
"id": "EyecK0YM9Cc8T6ju-nTOc"
},
{
"id": "xpdAlCCGgIMAgSaqQ4K65",
"type": "arrow"
}
],
"updated": 1655372487773,
"link": null,
"locked": false
},
{
"type": "text",
"version": 844,
"versionNonce": 2073980664,
"isDeleted": false,
"id": "4lz3UmUePrjYOJGyMEsNo",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 16.912559509277344,
"y": 423.7749671936035,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 82,
"height": 24,
"seed": 808600456,
"groupIds": [
"IAd7y_6yDxq13U11FuJvH"
],
"strokeSharpness": "sharp",
"boundElements": [],
"updated": 1655372487773,
"link": null,
"locked": false,
"fontSize": 20,
"fontFamily": 3,
"text": "vmalert",
"baseline": 19,
"textAlign": "center",
"verticalAlign": "middle",
"containerId": null,
"originalText": "vmalert"
},
{
"id": "xpdAlCCGgIMAgSaqQ4K65",
"type": "arrow",
"x": 127.58199310302739,
"y": 437.3415815729096,
"width": 154.43469238281244,
"height": 0.2578931190849971,
"angle": 0,
"strokeColor": "black",
"backgroundColor": "transparent",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"groupIds": [],
"strokeSharpness": "round",
"seed": 1769759112,
"version": 140,
"versionNonce": 1727929480,
"isDeleted": false,
"boundElements": null,
"updated": 1655372487773,
"link": null,
"locked": false,
"points": [
[
0,
0
],
[
154.43469238281244,
0.2578931190849971
]
],
"lastCommittedPoint": null,
"startBinding": {
"elementId": "X08ptHmEm7tCgoFbQntAR",
"focus": -0.042373209435744755,
"gap": 8.45587158203125
},
"endBinding": {
"elementId": "VgBUzo0blGR-Ijd2mQEEf",
"focus": -0.062483627408895646,
"gap": 7.663612365722656
},
"startArrowhead": null,
"endArrowhead": "arrow"
},
{
"type": "text",
"version": 896,
"versionNonce": 619040760,
"isDeleted": false,
"id": "d_hJkkcPArQGdFiPDbjtp",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 129.2102279663086,
"y": 404.1378517150879,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 141,
"height": 19,
"seed": 2108447992,
"groupIds": [],
"strokeSharpness": "sharp",
"boundElements": [
{
"type": "arrow",
"id": "wRO0q9xKPHc8e8XPPsQWh"
}
],
"updated": 1655372487773,
"link": null,
"locked": false,
"fontSize": 16,
"fontFamily": 3,
"text": "persist results",
"baseline": 15,
"textAlign": "left",
"verticalAlign": "top",
"containerId": null,
"originalText": "persist results"
},
{
"id": "P35cFQroIm2nrmm3Jlqgp",
"type": "text",
"x": -7.461128234863281,
"y": 483.3255958557129,
"width": 301,
"height": 20,
"angle": 0,
"strokeColor": "black",
"backgroundColor": "transparent",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"groupIds": [],
"strokeSharpness": "sharp",
"seed": 1314060792,
"version": 179,
"versionNonce": 139280376,
"isDeleted": false,
"boundElements": null,
"updated": 1655372636346,
"link": null,
"locked": false,
"text": " -remoteWrite.url=http://vmagent",
"fontSize": 16,
"fontFamily": 3,
"textAlign": "left",
"verticalAlign": "top",
"baseline": 16,
"containerId": null,
"originalText": " -remoteWrite.url=http://vmagent"
},
{
"type": "rectangle",
"version": 1339,
"versionNonce": 812947448,
"isDeleted": false,
"id": "lXpACjXQqK7SZF_vrACjJ",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 548.3361740112305,
"y": 487.1258888244629,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 219.1235961914062,
"height": 44.74725341796875,
"seed": 333549960,
"groupIds": [
"vuLTnxw8A0DXtmDYT1F4r"
],
"strokeSharpness": "sharp",
"boundElements": [
{
"type": "arrow",
"id": "wRO0q9xKPHc8e8XPPsQWh"
},
{
"type": "arrow",
"id": "he-SpFjCxEQEWpWny2kKP"
},
{
"type": "arrow",
"id": "-pjrKo16rOsasM8viZPJ-"
},
{
"id": "EyecK0YM9Cc8T6ju-nTOc",
"type": "arrow"
}
],
"updated": 1655372623571,
"link": null,
"locked": false
},
{
"type": "text",
"version": 1244,
"versionNonce": 666803448,
"isDeleted": false,
"id": "v9qzZSsHdJ_ETRlP4Msn5",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 635.2247543334961,
"y": 497.55932998657227,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 48,
"height": 24,
"seed": 1105210104,
"groupIds": [
"vuLTnxw8A0DXtmDYT1F4r"
],
"strokeSharpness": "sharp",
"boundElements": [],
"updated": 1655372625794,
"link": null,
"locked": false,
"fontSize": 20,
"fontFamily": 3,
"text": "vm-2",
"baseline": 20,
"textAlign": "center",
"verticalAlign": "top",
"containerId": null,
"originalText": "vm-2"
},
{
"id": "yb3B2pFN0OZOd4yLmSU2m",
"type": "text",
"x": 449.79036712646484,
"y": 406.616886138916,
"width": 442,
"height": 20,
"angle": 0,
"strokeColor": "black",
"backgroundColor": "transparent",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"groupIds": [],
"strokeSharpness": "sharp",
"seed": 1374332808,
"version": 196,
"versionNonce": 526480264,
"isDeleted": false,
"boundElements": null,
"updated": 1655372596999,
"link": null,
"locked": false,
"text": "-remoteWrite.url=https://vm-1:8428/api/v1/write",
"fontSize": 16,
"fontFamily": 3,
"textAlign": "left",
"verticalAlign": "top",
"baseline": 16,
"containerId": null,
"originalText": "-remoteWrite.url=https://vm-1:8428/api/v1/write"
},
{
"type": "text",
"version": 242,
"versionNonce": 1304477832,
"isDeleted": false,
"id": "8CXNdrDePIAAwgJB2b8YT",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 450.0511703491211,
"y": 432.6653480529785,
"strokeColor": "black",
"backgroundColor": "transparent",
"width": 442,
"height": 20,
"seed": 1349606392,
"groupIds": [],
"strokeSharpness": "sharp",
"boundElements": [],
"updated": 1655372600292,
"link": null,
"locked": false,
"fontSize": 16,
"fontFamily": 3,
"text": "-remoteWrite.url=https://vm-2:8428/api/v1/write",
"baseline": 16,
"textAlign": "left",
"verticalAlign": "top",
"containerId": null,
"originalText": "-remoteWrite.url=https://vm-2:8428/api/v1/write"
},
{
"type": "text",
"version": 1195,
"versionNonce": 1912405496,
"isDeleted": false,
"id": "Ev-VujoFglVNIh5GIhsba",
"fillStyle": "hachure",
"strokeWidth": 1,
"strokeStyle": "solid",
"roughness": 0,
"opacity": 100,
"angle": 0,
"x": 357.2894821166992,
"y": 370.6587562561035,
"strokeColor": "#000000",
"backgroundColor": "transparent",
"width": 114,
"height": 20,
"seed": 1289300104,
"groupIds": [],
"strokeSharpness": "sharp",
"boundElements": [
{
"type": "arrow",
"id": "wRO0q9xKPHc8e8XPPsQWh"
}
],
"updated": 1655372703770,
"link": null,
"locked": false,
"fontSize": 16,
"fontFamily": 3,
"text": "fan-out data",
"baseline": 16,
"textAlign": "left",
"verticalAlign": "top",
"containerId": null,
"originalText": "fan-out data"
}
],
"appState": {
"gridSize": null,
"viewBackgroundColor": "#ffffff"
},
"files": {}
}

BIN
app/vmalert/vmalert_multiple_rw.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 80 KiB

12
app/vmauth/README.md
View file

@ -10,7 +10,7 @@ The `-auth.config` can point to either local file or to http url.
Just download `vmutils-*` archive from [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases), unpack it
and pass the following flag to `vmauth` binary in order to start authorizing and routing requests:
```bash
```console
/path/to/vmauth -auth.config=/path/to/auth/config.yml
```
@ -129,7 +129,7 @@ It is expected that all the backend services protected by `vmauth` are located i
Do not transfer Basic Auth headers in plaintext over untrusted networks. Enable https. This can be done by passing the following `-tls*` command-line flags to `vmauth`:
```bash
```console
-tls
Whether to enable TLS (aka HTTPS) for incoming requests. -tlsCertFile and -tlsKeyFile must be set if -tls is set
-tlsCertFile string
@ -181,7 +181,7 @@ The `<PKG_TAG>` may be manually set via `PKG_TAG=foobar make package-vmauth`.
The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
by setting it via `<ROOT_IMAGE>` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-vmauth
```
@ -193,7 +193,7 @@ ROOT_IMAGE=scratch make package-vmauth
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8427/debug/pprof/heap > mem.pprof
```
@ -203,7 +203,7 @@ curl http://0.0.0.0:8427/debug/pprof/heap > mem.pprof
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8427/debug/pprof/profile > cpu.pprof
```
@ -217,7 +217,7 @@ The collected profiles may be analyzed with [go tool pprof](https://github.com/g
Pass `-help` command-line arg to `vmauth` in order to see all the configuration options:
```bash
```console
./vmauth -help
vmauth authenticates and authorizes incoming requests and proxies them to VictoriaMetrics.

18
app/vmbackup/README.md
View file

@ -28,7 +28,7 @@ creation of hourly, daily, weekly and monthly backups.
Regular backup can be performed with the following command:
```bash
```console
vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs://<bucket>/<path/to/new/backup>
```
@ -43,7 +43,7 @@ vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=h
If the destination GCS bucket already contains the previous backup at `-origin` path, then new backup can be sped up
with the following command:
```bash
```console
./vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs://<bucket>/<path/to/new/backup> -origin=gs://<bucket>/<path/to/existing/backup>
```
@ -54,7 +54,7 @@ It saves time and network bandwidth costs by performing server-side copy for the
Incremental backups are performed if `-dst` points to an already existing backup. In this case only new data is uploaded to remote storage.
It saves time and network bandwidth costs when working with big backups:
```bash
```console
./vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs://<bucket>/<path/to/existing/backup>
```
@ -64,7 +64,7 @@ Smart backups mean storing full daily backups into `YYYYMMDD` folders and creati
* Run the following command every hour:
```bash
```console
./vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs://<bucket>/latest
```
@ -73,7 +73,7 @@ The command will upload only changed data to `gs://<bucket>/latest`.
* Run the following command once a day:
```bash
```console
vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs://<bucket>/<YYYYMMDD> -origin=gs://<bucket>/latest
```
@ -129,7 +129,7 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-
for s3 (aws, minio or other s3 compatible storages):
```bash
```console
[default]
aws_access_key_id=theaccesskey
aws_secret_access_key=thesecretaccesskeyvalue
@ -155,7 +155,7 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-
* Usage with s3 custom url endpoint. It is possible to use `vmbackup` with s3 compatible storages like minio, cloudian, etc.
You have to add a custom url endpoint via flag:
```bash
```console
# for minio
-customS3Endpoint=http://localhost:9000
@ -165,7 +165,7 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-
* Run `vmbackup -help` in order to see all the available options:
```bash
```console
-concurrency int
The number of concurrent workers. Higher concurrency may reduce backup duration (default 10)
-configFilePath string
@ -280,6 +280,6 @@ The `<PKG_TAG>` may be manually set via `PKG_TAG=foobar make package-vmbackup`.
The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
by setting it via `<ROOT_IMAGE>` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-vmbackup
```

6
app/vmctl/README.md
View file

@ -15,7 +15,7 @@ Features:
To see the full list of supported modes
run the following command:
```bash
```console
$ ./vmctl --help
NAME:
vmctl - VictoriaMetrics command-line tool
@ -527,7 +527,7 @@ and specify `accountID` param.
In this mode, `vmctl` allows verifying correctness and integrity of data exported via [native format](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-export-data-in-native-format) from VictoriaMetrics.
You can verify exported data at disk before uploading it by `vmctl verify-block` command:
```bash
```console
# export blocks from VictoriaMetrics
curl localhost:8428/api/v1/export/native -g -d 'match[]={__name__!=""}' -o exported_data_block
# verify block content
@ -650,7 +650,7 @@ The `<PKG_TAG>` may be manually set via `PKG_TAG=foobar make package-vmctl`.
The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
by setting it via `<ROOT_IMAGE>` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-vmctl
```

10
app/vmgateway/README.md
View file

@ -54,7 +54,7 @@ Where:
Start the single version of VictoriaMetrics
```bash
```console
# single
# start node
./bin/victoria-metrics --selfScrapeInterval=10s
@ -62,19 +62,19 @@ Start the single version of VictoriaMetrics
Start vmgateway
```bash
```console
./bin/vmgateway -eula -enable.auth -read.url http://localhost:8428 --write.url http://localhost:8428
```
Retrieve data from the database
```bash
```console
curl 'http://localhost:8431/api/v1/series/count' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ2bV9hY2Nlc3MiOnsidGVuYW50X2lkIjp7fSwicm9sZSI6MX0sImV4cCI6MTkzOTM0NjIxMH0.5WUxEfdcV9hKo4CtQdtuZYOGpGXWwaqM9VuVivMMrVg'
```
A request with an incorrect token or without any token will be rejected:
```bash
```console
curl 'http://localhost:8431/api/v1/series/count'
curl 'http://localhost:8431/api/v1/series/count' -H 'Authorization: Bearer incorrect-token'
@ -124,7 +124,7 @@ limits:
cluster version of VictoriaMetrics is required for rate limiting.
```bash
```console
# start datasource for cluster metrics
cat << EOF > cluster.yaml

10
app/vmrestore/README.md
View file

@ -10,7 +10,7 @@ when restarting `vmrestore` with the same args.
VictoriaMetrics must be stopped during the restore process.
```bash
```console
vmrestore -src=gs://<bucket>/<path/to/backup> -storageDataPath=<local/path/to/restore>
```
@ -36,7 +36,7 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q
for s3 (aws, minio or other s3 compatible storages):
```bash
```console
[default]
aws_access_key_id=theaccesskey
aws_secret_access_key=thesecretaccesskeyvalue
@ -62,7 +62,7 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q
* Usage with s3 custom url endpoint. It is possible to use `vmrestore` with s3 api compatible storages, like minio, cloudian and other.
You have to add custom url endpoint with a flag:
```bash
```console
# for minio:
-customS3Endpoint=http://localhost:9000
@ -72,7 +72,7 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q
* Run `vmrestore -help` in order to see all the available options:
```bash
```console
-concurrency int
The number of concurrent workers. Higher concurrency may reduce restore duration (default 10)
-configFilePath string
@ -180,6 +180,6 @@ The `<PKG_TAG>` may be manually set via `PKG_TAG=foobar make package-vmrestore`.
The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
by setting it via `<ROOT_IMAGE>` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-vmrestore
```

8
app/vmselect/prometheus/prometheus.go
View file

@ -1099,6 +1099,14 @@ func getCommonParams(r *http.Request, startTime time.Time, requireNonEmptyMatch
if err != nil {
return nil, err
}
// Limit the `end` arg to the current time +2 days in the same way
// as it is limited during data ingestion.
// See https://github.com/VictoriaMetrics/VictoriaMetrics/blob/ea06d2fd3ccbbb6aa4480ab3b04f7b671408be2a/lib/storage/table.go#L378
// This should fix possible timestamp overflow - see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2669
maxTS := startTime.UnixNano()/1e6 + 2*24*3600*1000
if end > maxTS {
end = maxTS
}
if end < start {
end = start
}

4
app/vmselect/vmui/asset-manifest.json
View file

@ -1,12 +1,12 @@
{
"files": {
"main.css": "./static/css/main.7e6d0c89.css",
"main.js": "./static/js/main.f7185a13.js",
"main.js": "./static/js/main.fdf5a65f.js",
"static/js/27.939f971b.chunk.js": "./static/js/27.939f971b.chunk.js",
"index.html": "./index.html"
},
"entrypoints": [
"static/css/main.7e6d0c89.css",
"static/js/main.f7185a13.js"
"static/js/main.fdf5a65f.js"
]
}

2
app/vmselect/vmui/index.html
View file

@ -1 +1 @@
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="VM-UI is a metric explorer for Victoria Metrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700&display=swap"/><script src="./dashboards/index.js" type="module"></script><script defer="defer" src="./static/js/main.f7185a13.js"></script><link href="./static/css/main.7e6d0c89.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="VM-UI is a metric explorer for Victoria Metrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700&display=swap"/><script src="./dashboards/index.js" type="module"></script><script defer="defer" src="./static/js/main.fdf5a65f.js"></script><link href="./static/css/main.7e6d0c89.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>

2
app/vmselect/vmui/static/js/main.f7185a13.js
View file

File diff suppressed because one or more lines are too long

2
app/vmselect/vmui/static/js/main.fdf5a65f.js Normal file
View file

File diff suppressed because one or more lines are too long

0
app/vmselect/vmui/static/js/main.f7185a13.js.LICENSE.txt → app/vmselect/vmui/static/js/main.fdf5a65f.js.LICENSE.txt
View file

2
app/vmui/packages/vmui/.eslintrc.js
View file

@ -42,7 +42,7 @@ module.exports = {
"max-lines": [
"error",
{
"max": 150,
"max": 1000,
"skipBlankLines": true,
"skipComments": true,
}

6
app/vmui/packages/vmui/src/api/tsdb.ts
View file

@ -3,10 +3,12 @@ export interface CardinalityRequestsParams {
extraLabel: string | null,
match: string | null,
date: string | null,
focusLabel: string | null,
}
export const getCardinalityInfo = (server: string, requestsParam: CardinalityRequestsParams) => {
const match = requestsParam.match ? `&match[]=${requestsParam.match}` : "";
return `${server}/api/v1/status/tsdb?topN=${requestsParam.topN}&date=${requestsParam.date}${match}`;
const match = requestsParam.match ? "&match[]=" + encodeURIComponent(requestsParam.match) : "";
const focusLabel = requestsParam.focusLabel ? "&focusLabel=" + encodeURIComponent(requestsParam.focusLabel) : "";
return `${server}/api/v1/status/tsdb?topN=${requestsParam.topN}&date=${requestsParam.date}${match}${focusLabel}`;
};

38
app/vmui/packages/vmui/src/components/CardinalityPanel/CardinalityConfigurator/CardinalityConfigurator.tsx
View file

@ -17,6 +17,7 @@ export interface CardinalityConfiguratorProps {
onSetQuery: (query: string, index: number) => void;
onRunQuery: () => void;
onTopNChange: (e: ChangeEvent<HTMLTextAreaElement|HTMLInputElement>) => void;
onFocusLabelChange: (e: ChangeEvent<HTMLTextAreaElement|HTMLInputElement>) => void;
query: string;
topN: number;
error?: ErrorTypes | string;
@ -24,6 +25,7 @@ export interface CardinalityConfiguratorProps {
totalLabelValuePairs: number;
date: string | null;
match: string | null;
focusLabel: string | null;
}
const CardinalityConfigurator: FC<CardinalityConfiguratorProps> = ({
@ -34,10 +36,12 @@ const CardinalityConfigurator: FC<CardinalityConfiguratorProps> = ({
onRunQuery,
onSetQuery,
onTopNChange,
onFocusLabelChange,
totalSeries,
totalLabelValuePairs,
date,
match
match,
focusLabel
}) => {
const dispatch = useAppDispatch();
const {queryControls: {autocomplete}} = useAppState();
@ -50,40 +54,48 @@ const CardinalityConfigurator: FC<CardinalityConfiguratorProps> = ({
return <Box boxShadow="rgba(99, 99, 99, 0.2) 0px 2px 8px 0px;" p={4} pb={2} mb={2}>
<Box>
<Box display="grid" gridTemplateColumns="1fr auto auto" gap="4px" width="50%" mb={4}>
<Box display="grid" gridTemplateColumns="1fr auto auto auto auto" gap="4px" width="100%" mb={4}>
<QueryEditor
query={query} index={0} autocomplete={autocomplete} queryOptions={queryOptions}
error={error} setHistoryIndex={onSetHistory} runQuery={onRunQuery} setQuery={onSetQuery}
label={"Time series selector"}
/>
<Box display="flex" alignItems="center">
<Box ml={2}>
<Box mr={2}>
<TextField
label="Number of entries per table"
type="number"
size="small"
size="medium"
variant="outlined"
value={topN}
error={topN < 1}
helperText={topN < 1 ? "Number must be bigger than zero" : " "}
onChange={onTopNChange}/>
</Box>
<Tooltip title="Execute Query">
<IconButton onClick={onRunQuery} sx={{height: "49px", width: "49px"}}>
<PlayCircleOutlineIcon/>
</IconButton>
</Tooltip>
<Box mr={2}>
<TextField
label="Focus label"
type="text"
size="medium"
variant="outlined"
value={focusLabel}
onChange={onFocusLabelChange} />
</Box>
<Box>
<FormControlLabel label="Enable autocomplete"
control={<BasicSwitch checked={autocomplete} onChange={onChangeAutocomplete}/>}
/>
</Box>
</Box>
<Tooltip title="Execute Query">
<IconButton onClick={onRunQuery} sx={{height: "49px", width: "49px"}}>
<PlayCircleOutlineIcon/>
</IconButton>
</Tooltip>
</Box>
</Box>
<Box>
Analyzed <b>{totalSeries}</b> series with <b>{totalLabelValuePairs}</b> label=value pairs
at <b>{date}</b> {match && <span>for series selector <b>{match}</b></span>}. Show top {topN} entries per table.
Analyzed <b>{totalSeries}</b> series with <b>{totalLabelValuePairs}</b> &quot;label=value&quot; pairs
at <b>{date}</b> {match && <span>for series selector <b>{match}</b></span>}.
Show top {topN} entries per table.
</Box>
</Box>;
};

101
app/vmui/packages/vmui/src/components/CardinalityPanel/CardinalityPanel.tsx
View file

@ -2,24 +2,30 @@ import React, {ChangeEvent, FC, useState} from "react";
import {SyntheticEvent} from "react";
import {Alert} from "@mui/material";
import {useFetchQuery} from "../../hooks/useCardinalityFetch";
import {
METRIC_NAMES_HEADERS,
LABEL_NAMES_HEADERS,
LABEL_VALUE_PAIRS_HEADERS,
LABELS_WITH_UNIQUE_VALUES_HEADERS,
spinnerContainerStyles
} from "./consts";
import {defaultProperties, queryUpdater} from "./helpers";
import {queryUpdater} from "./helpers";
import {Data} from "../Table/types";
import CardinalityConfigurator from "./CardinalityConfigurator/CardinalityConfigurator";
import Spinner from "../common/Spinner";
import {useCardinalityDispatch, useCardinalityState} from "../../state/cardinality/CardinalityStateContext";
import MetricsContent from "./MetricsContent/MetricsContent";
import {DefaultActiveTab, Tabs, TSDBStatus, Containers} from "./types";
const spinnerContainerStyles = (height: string) => {
return {
width: "100%",
maxWidth: "100%",
position: "absolute",
height: height ?? "50%",
background: "rgba(255, 255, 255, 0.7)",
pointerEvents: "none",
zIndex: 1000,
};
};
const CardinalityPanel: FC = () => {
const cardinalityDispatch = useCardinalityDispatch();
const {topN, match, date} = useCardinalityState();
const {topN, match, date, focusLabel} = useCardinalityState();
const configError = "";
const [query, setQuery] = useState(match || "");
const [queryHistoryIndex, setQueryHistoryIndex] = useState(0);
@ -47,10 +53,13 @@ const CardinalityPanel: FC = () => {
cardinalityDispatch({type: "SET_TOP_N", payload: +e.target.value});
};
const {isLoading, tsdbStatus, error} = useFetchQuery();
const defaultProps = defaultProperties(tsdbStatus);
const [stateTabs, setTab] = useState(defaultProps.defaultState);
const onFocusLabelChange = (e: ChangeEvent<HTMLTextAreaElement|HTMLInputElement>) => {
cardinalityDispatch({type: "SET_FOCUS_LABEL", payload: e.target.value});
};
const {isLoading, appConfigurator, error} = useFetchQuery();
const [stateTabs, setTab] = useState(appConfigurator.defaultState.defaultActiveTab);
const {tsdbStatusData, defaultState, tablesHeaders} = appConfigurator;
const handleTabChange = (e: SyntheticEvent, newValue: number) => {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
@ -59,11 +68,16 @@ const CardinalityPanel: FC = () => {
const handleFilterClick = (key: string) => (e: SyntheticEvent) => {
const name = e.currentTarget.id;
const query = queryUpdater[key](name);
const query = queryUpdater[key](focusLabel, name);
setQuery(query);
setQueryHistory(prev => [...prev, query]);
setQueryHistoryIndex(prev => prev + 1);
cardinalityDispatch({type: "SET_MATCH", payload: query});
let newFocusLabel = "";
if (key === "labelValueCountByLabelName" || key == "seriesCountByLabelName") {
newFocusLabel = name;
}
cardinalityDispatch({type: "SET_FOCUS_LABEL", payload: newFocusLabel});
cardinalityDispatch({type: "RUN_QUERY"});
};
@ -79,56 +93,25 @@ const CardinalityPanel: FC = () => {
/>}
<CardinalityConfigurator error={configError} query={query} onRunQuery={onRunQuery} onSetQuery={onSetQuery}
onSetHistory={onSetHistory} onTopNChange={onTopNChange} topN={topN} date={date} match={match}
totalSeries={tsdbStatus.totalSeries} totalLabelValuePairs={tsdbStatus.totalLabelValuePairs}/>
totalSeries={tsdbStatusData.totalSeries} totalLabelValuePairs={tsdbStatusData.totalLabelValuePairs}
focusLabel={focusLabel} onFocusLabelChange={onFocusLabelChange}
/>
{error && <Alert color="error" severity="error" sx={{whiteSpace: "pre-wrap", mt: 2}}>{error}</Alert>}
{appConfigurator.keys(focusLabel).map((keyName) => (
<MetricsContent
sectionTitle={"Metric names with the highest number of series"}
activeTab={stateTabs.seriesCountByMetricName}
rows={tsdbStatus.seriesCountByMetricName as unknown as Data[]}
key={keyName}
sectionTitle={appConfigurator.sectionsTitles(focusLabel)[keyName]}
activeTab={stateTabs[keyName as keyof DefaultActiveTab]}
rows={tsdbStatusData[keyName as keyof TSDBStatus] as unknown as Data[]}
onChange={handleTabChange}
onActionClick={handleFilterClick("seriesCountByMetricName")}
tabs={defaultProps.tabs.seriesCountByMetricName}
chartContainer={defaultProps.containerRefs.seriesCountByMetricName}
totalSeries={tsdbStatus.totalSeries}
tabId={"seriesCountByMetricName"}
tableHeaderCells={METRIC_NAMES_HEADERS}
/>
<MetricsContent
sectionTitle={"Labels with the highest number of series"}
activeTab={stateTabs.seriesCountByLabelName}
rows={tsdbStatus.seriesCountByLabelName as unknown as Data[]}
onChange={handleTabChange}
onActionClick={handleFilterClick("seriesCountByLabelName")}
tabs={defaultProps.tabs.seriesCountByLabelName}
chartContainer={defaultProps.containerRefs.seriesCountByLabelName}
totalSeries={tsdbStatus.totalSeries}
tabId={"seriesCountByLabelName"}
tableHeaderCells={LABEL_NAMES_HEADERS}
/>
<MetricsContent
sectionTitle={"Label=value pairs with the highest number of series"}
activeTab={stateTabs.seriesCountByLabelValuePair}
rows={tsdbStatus.seriesCountByLabelValuePair as unknown as Data[]}
onChange={handleTabChange}
onActionClick={handleFilterClick("seriesCountByLabelValuePair")}
tabs={defaultProps.tabs.seriesCountByLabelValuePair}
chartContainer={defaultProps.containerRefs.seriesCountByLabelValuePair}
totalSeries={tsdbStatus.totalSeries}
tabId={"seriesCountByLabelValuePair"}
tableHeaderCells={LABEL_VALUE_PAIRS_HEADERS}
/>
<MetricsContent
sectionTitle={"Labels with the highest number of unique values"}
activeTab={stateTabs.labelValueCountByLabelName}
rows={tsdbStatus.labelValueCountByLabelName as unknown as Data[]}
onChange={handleTabChange}
onActionClick={handleFilterClick("labelValueCountByLabelName")}
tabs={defaultProps.tabs.labelValueCountByLabelName}
chartContainer={defaultProps.containerRefs.labelValueCountByLabelName}
totalSeries={-1}
tabId={"labelValueCountByLabelName"}
tableHeaderCells={LABELS_WITH_UNIQUE_VALUES_HEADERS}
onActionClick={handleFilterClick(keyName)}
tabs={defaultState.tabs[keyName as keyof Tabs]}
chartContainer={defaultState.containerRefs[keyName as keyof Containers<HTMLDivElement>]}
totalSeries={appConfigurator.totalSeries(keyName)}
tabId={keyName}
tableHeaderCells={tablesHeaders[keyName]}
/>
))}
</>
);
};

2
app/vmui/packages/vmui/src/components/CardinalityPanel/MetricsContent/MetricsContent.tsx
View file

@ -34,7 +34,7 @@ const MetricsContent: FC<MetricsProperties> = ({
tabId,
onActionClick,
sectionTitle,
tableHeaderCells
tableHeaderCells,
}) => {
const tableCells = (row: Data) => (
<TableCells

233
app/vmui/packages/vmui/src/components/CardinalityPanel/appConfigurator.ts Normal file
View file

@ -0,0 +1,233 @@
import {Containers, DefaultActiveTab, Tabs, TSDBStatus} from "./types";
import {useRef} from "preact/compat";
import {HeadCell} from "../Table/types";
interface AppState {
tabs: Tabs;
containerRefs: Containers<HTMLDivElement>;
defaultActiveTab: DefaultActiveTab,
}
export default class AppConfigurator {
private tsdbStatus: TSDBStatus;
private tabsNames: string[];
constructor() {
this.tsdbStatus = this.defaultTSDBStatus;
this.tabsNames = ["table", "graph"];
}
set tsdbStatusData(tsdbStatus: TSDBStatus) {
this.tsdbStatus = tsdbStatus;
}
get tsdbStatusData(): TSDBStatus {
return this.tsdbStatus;
}
get defaultTSDBStatus(): TSDBStatus {
return {
totalSeries: 0,
totalLabelValuePairs: 0,
seriesCountByMetricName: [],
seriesCountByLabelName: [],
seriesCountByFocusLabelValue: [],
seriesCountByLabelValuePair: [],
labelValueCountByLabelName: [],
};
}
keys(focusLabel: string | null): string[] {
let keys: string[] = [];
if (focusLabel) {
keys = keys.concat("seriesCountByFocusLabelValue");
}
keys = keys.concat(
"seriesCountByMetricName",
"seriesCountByLabelName",
"seriesCountByLabelValuePair",
"labelValueCountByLabelName",
);
return keys;
}
get defaultState(): AppState {
return this.keys("job").reduce((acc, cur) => {
return {
...acc,
tabs: {
...acc.tabs,
[cur]: this.tabsNames,
},
containerRefs: {
...acc.containerRefs,
[cur]: useRef<HTMLDivElement>(null),
},
defaultActiveTab: {
...acc.defaultActiveTab,
[cur]: 0,
},
};
}, {
tabs: {} as Tabs,
containerRefs: {} as Containers<HTMLDivElement>,
defaultActiveTab: {} as DefaultActiveTab,
} as AppState);
}
sectionsTitles(str: string | null): Record<string, string> {
return {
seriesCountByMetricName: "Metric names with the highest number of series",
seriesCountByLabelName: "Labels with the highest number of series",
seriesCountByFocusLabelValue: `Values for "${str}" label with the highest number of series`,
seriesCountByLabelValuePair: "Label=value pairs with the highest number of series",
labelValueCountByLabelName: "Labels with the highest number of unique values",
};
}
get tablesHeaders(): Record<string, HeadCell[]> {
return {
seriesCountByMetricName: METRIC_NAMES_HEADERS,
seriesCountByLabelName: LABEL_NAMES_HEADERS,
seriesCountByFocusLabelValue: FOCUS_LABEL_VALUES_HEADERS,
seriesCountByLabelValuePair: LABEL_VALUE_PAIRS_HEADERS,
labelValueCountByLabelName: LABEL_NAMES_WITH_UNIQUE_VALUES_HEADERS,
};
}
totalSeries(keyName: string): number {
if (keyName === "labelValueCountByLabelName") {
return -1;
}
return this.tsdbStatus.totalSeries;
}
}
const METRIC_NAMES_HEADERS = [
{
disablePadding: false,
id: "name",
label: "Metric name",
numeric: false,
},
{
disablePadding: false,
id: "value",
label: "Number of series",
numeric: false,
},
{
disablePadding: false,
id: "percentage",
label: "Percent of series",
numeric: false,
},
{
disablePadding: false,
id: "action",
label: "Action",
numeric: false,
}
] as HeadCell[];
const LABEL_NAMES_HEADERS = [
{
disablePadding: false,
id: "name",
label: "Label name",
numeric: false,
},
{
disablePadding: false,
id: "value",
label: "Number of series",
numeric: false,
},
{
disablePadding: false,
id: "percentage",
label: "Percent of series",
numeric: false,
},
{
disablePadding: false,
id: "action",
label: "Action",
numeric: false,
}
] as HeadCell[];
const FOCUS_LABEL_VALUES_HEADERS = [
{
disablePadding: false,
id: "name",
label: "Label value",
numeric: false,
},
{
disablePadding: false,
id: "value",
label: "Number of series",
numeric: false,
},
{
disablePadding: false,
id: "percentage",
label: "Percent of series",
numeric: false,
},
{
disablePadding: false,
id: "action",
label: "Action",
numeric: false,
}
] as HeadCell[];
export const LABEL_VALUE_PAIRS_HEADERS = [
{
disablePadding: false,
id: "name",
label: "Label=value pair",
numeric: false,
},
{
disablePadding: false,
id: "value",
label: "Number of series",
numeric: false,
},
{
disablePadding: false,
id: "percentage",
label: "Percent of series",
numeric: false,
},
{
disablePadding: false,
id: "action",
label: "Action",
numeric: false,
}
] as HeadCell[];
export const LABEL_NAMES_WITH_UNIQUE_VALUES_HEADERS = [
{
disablePadding: false,
id: "name",
label: "Label name",
numeric: false,
},
{
disablePadding: false,
id: "value",
label: "Number of unique values",
numeric: false,
},
{
disablePadding: false,
id: "action",
label: "Action",
numeric: false,
}
] as HeadCell[];

115
app/vmui/packages/vmui/src/components/CardinalityPanel/consts.ts
View file

@ -1,115 +0,0 @@
import {HeadCell} from "../Table/types";
export const METRIC_NAMES_HEADERS = [
{
disablePadding: false,
id: "name",
label: "Metric name",
numeric: false,
},
{
disablePadding: false,
id: "value",
label: "Number of series",
numeric: false,
},
{
disablePadding: false,
id: "percentage",
label: "Percent of series",
numeric: false,
},
{
disablePadding: false,
id: "action",
label: "Action",
numeric: false,
}
] as HeadCell[];
export const LABEL_NAMES_HEADERS = [
{
disablePadding: false,
id: "name",
label: "Label name",
numeric: false,
},
{
disablePadding: false,
id: "value",
label: "Number of series",
numeric: false,
},
{
disablePadding: false,
id: "percentage",
label: "Percent of series",
numeric: false,
},
{
disablePadding: false,
id: "action",
label: "Action",
numeric: false,
}
] as HeadCell[];
export const LABEL_VALUE_PAIRS_HEADERS = [
{
disablePadding: false,
id: "name",
label: "Label=value pair",
numeric: false,
},
{
disablePadding: false,
id: "value",
label: "Number of series",
numeric: false,
},
{
disablePadding: false,
id: "percentage",
label: "Percent of series",
numeric: false,
},
{
disablePadding: false,
id: "action",
label: "Action",
numeric: false,
}
]as HeadCell[];
export const LABELS_WITH_UNIQUE_VALUES_HEADERS = [
{
disablePadding: false,
id: "name",
label: "Label name",
numeric: false,
},
{
disablePadding: false,
id: "value",
label: "Number of unique values",
numeric: false,
},
{
disablePadding: false,
id: "action",
label: "Action",
numeric: false,
}
] as HeadCell[];
export const spinnerContainerStyles = (height: string) => {
return {
width: "100%",
maxWidth: "100%",
position: "absolute",
height: height ?? "50%",
background: "rgba(255, 255, 255, 0.7)",
pointerEvents: "none",
zIndex: 1000,
};
};

44
app/vmui/packages/vmui/src/components/CardinalityPanel/helpers.ts
View file

@ -1,45 +1,25 @@
import {Containers, DefaultState, QueryUpdater, Tabs, TSDBStatus} from "./types";
import {useRef} from "preact/compat";
import {QueryUpdater} from "./types";
export const queryUpdater: QueryUpdater = {
seriesCountByMetricName: (query: string): string => {
seriesCountByMetricName: (focusLabel: string | null, query: string): string => {
return getSeriesSelector("__name__", query);
},
seriesCountByLabelName: (query: string): string => `{${query}!=""}`,
seriesCountByLabelValuePair: (query: string): string => {
seriesCountByLabelName: (focusLabel: string | null, query: string): string => `{${query}!=""}`,
seriesCountByFocusLabelValue: (focusLabel: string | null, query: string): string => {
return getSeriesSelector(focusLabel, query);
},
seriesCountByLabelValuePair: (focusLabel: string | null, query: string): string => {
const a = query.split("=");
const label = a[0];
const value = a.slice(1).join("=");
return getSeriesSelector(label, value);
},
labelValueCountByLabelName: (query: string): string => `{${query}!=""}`,
labelValueCountByLabelName: (focusLabel: string | null, query: string): string => `{${query}!=""}`,
};
const getSeriesSelector = (label: string, value: string): string => {
const getSeriesSelector = (label: string | null, value: string): string => {
if (!label) {
return "";
}
return "{" + label + "=" + JSON.stringify(value) + "}";
};
export const defaultProperties = (tsdbStatus: TSDBStatus) => {
return Object.keys(tsdbStatus).reduce((acc, key) => {
if (key === "totalSeries" || key === "totalLabelValuePairs") return acc;
return {
...acc,
tabs:{
...acc.tabs,
[key]: ["table", "graph"],
},
containerRefs: {
...acc.containerRefs,
[key]: useRef<HTMLDivElement>(null),
},
defaultState: {
...acc.defaultState,
[key]: 0,
},
};
}, {
tabs:{} as Tabs,
containerRefs: {} as Containers<HTMLDivElement>,
defaultState: {} as DefaultState,
});
};

8
app/vmui/packages/vmui/src/components/CardinalityPanel/types.ts
View file

@ -5,6 +5,7 @@ export interface TSDBStatus {
totalLabelValuePairs: number;
seriesCountByMetricName: TopHeapEntry[];
seriesCountByLabelName: TopHeapEntry[];
seriesCountByFocusLabelValue: TopHeapEntry[];
seriesCountByLabelValuePair: TopHeapEntry[];
labelValueCountByLabelName: TopHeapEntry[];
}
@ -15,12 +16,13 @@ export interface TopHeapEntry {
}
export type QueryUpdater = {
[key: string]: (query: string) => string,
[key: string]: (focusLabel: string | null, query: string) => string,
}
export interface Tabs {
seriesCountByMetricName: string[];
seriesCountByLabelName: string[];
seriesCountByFocusLabelValue: string[];
seriesCountByLabelValuePair: string[];
labelValueCountByLabelName: string[];
}
@ -28,13 +30,15 @@ export interface Tabs {
export interface Containers<T> {
seriesCountByMetricName: MutableRef<T>;
seriesCountByLabelName: MutableRef<T>;
seriesCountByFocusLabelValue: MutableRef<T>;
seriesCountByLabelValuePair: MutableRef<T>;
labelValueCountByLabelName: MutableRef<T>;
}
export interface DefaultState {
export interface DefaultActiveTab {
seriesCountByMetricName: number;
seriesCountByLabelName: number;
seriesCountByFocusLabelValue: number;
seriesCountByLabelValuePair: number;
labelValueCountByLabelName: number;
}

27
app/vmui/packages/vmui/src/hooks/useCardinalityFetch.ts
View file

@ -5,34 +5,28 @@ import {CardinalityRequestsParams, getCardinalityInfo} from "../api/tsdb";
import {getAppModeEnable, getAppModeParams} from "../utils/app-mode";
import {TSDBStatus} from "../components/CardinalityPanel/types";
import {useCardinalityState} from "../state/cardinality/CardinalityStateContext";
import AppConfigurator from "../components/CardinalityPanel/appConfigurator";
const appModeEnable = getAppModeEnable();
const {serverURL: appServerUrl} = getAppModeParams();
const defaultTSDBStatus = {
totalSeries: 0,
totalLabelValuePairs: 0,
seriesCountByMetricName: [],
seriesCountByLabelName: [],
seriesCountByLabelValuePair: [],
labelValueCountByLabelName: [],
};
export const useFetchQuery = (): {
fetchUrl?: string[],
isLoading: boolean,
error?: ErrorTypes | string
tsdbStatus: TSDBStatus,
appConfigurator: AppConfigurator,
} => {
const {topN, extraLabel, match, date, runQuery} = useCardinalityState();
const appConfigurator = new AppConfigurator();
const {topN, extraLabel, match, date, runQuery, focusLabel} = useCardinalityState();
const {serverUrl} = useAppState();
const [isLoading, setIsLoading] = useState(false);
const [error, setError] = useState<ErrorTypes | string>();
const [tsdbStatus, setTSDBStatus] = useState<TSDBStatus>(defaultTSDBStatus);
const [tsdbStatus, setTSDBStatus] = useState<TSDBStatus>(appConfigurator.defaultTSDBStatus);
useEffect(() => {
if (error) {
setTSDBStatus(defaultTSDBStatus);
setTSDBStatus(appConfigurator.defaultTSDBStatus);
setIsLoading(false);
}
}, [error]);
@ -42,7 +36,7 @@ export const useFetchQuery = (): {
if (!server) return;
setError("");
setIsLoading(true);
setTSDBStatus(defaultTSDBStatus);
setTSDBStatus(appConfigurator.defaultTSDBStatus);
const url = getCardinalityInfo(server, requestParams);
try {
@ -54,7 +48,7 @@ export const useFetchQuery = (): {
setIsLoading(false);
} else {
setError(resp.error);
setTSDBStatus(defaultTSDBStatus);
setTSDBStatus(appConfigurator.defaultTSDBStatus);
setIsLoading(false);
}
} catch (e) {
@ -65,8 +59,9 @@ export const useFetchQuery = (): {
useEffect(() => {
fetchCardinalityInfo({topN, extraLabel, match, date});
fetchCardinalityInfo({topN, extraLabel, match, date, focusLabel});
}, [serverUrl, runQuery, date]);
return {isLoading, tsdbStatus, error};
appConfigurator.tsdbStatusData = tsdbStatus;
return {isLoading, appConfigurator: appConfigurator, error};
};

9
app/vmui/packages/vmui/src/state/cardinality/reducer.ts
View file

@ -7,6 +7,7 @@ export interface CardinalityState {
date: string | null
match: string | null
extraLabel: string | null
focusLabel: string | null
}
export type Action =
@ -14,12 +15,15 @@ export type Action =
| { type: "SET_DATE", payload: string | null }
| { type: "SET_MATCH", payload: string | null }
| { type: "SET_EXTRA_LABEL", payload: string | null }
| { type: "SET_FOCUS_LABEL", payload: string | null }
| { type: "RUN_QUERY" }
export const initialState: CardinalityState = {
runQuery: 0,
topN: getQueryStringValue("topN", 10) as number,
date: getQueryStringValue("date", dayjs(new Date()).format("YYYY-MM-DD")) as string,
focusLabel: getQueryStringValue("focusLabel", "") as string,
match: (getQueryStringValue("match", []) as string[]).join("&"),
extraLabel: getQueryStringValue("extra_label", "") as string,
};
@ -46,6 +50,11 @@ export function reducer(state: CardinalityState, action: Action): CardinalitySta
...state,
extraLabel: action.payload
};
case "SET_FOCUS_LABEL":
return {
...state,
focusLabel: action.payload,
};
case "RUN_QUERY":
return {
...state,

3
app/vmui/packages/vmui/src/utils/query-string.ts
View file

@ -17,7 +17,8 @@ const stateToUrlParams = {
"topN": "topN",
"date": "date",
"match": "match[]",
"extraLabel": "extra_label"
"extraLabel": "extra_label",
"focusLabel": "focusLabel"
}
};

4
deployment/marketplace/digitialocean/one-click-droplet/README.md
View file

@ -42,7 +42,7 @@ To check it, open the following in your browser `http://your_droplet_public_ipv4
Run the following command to query and retrieve a result from VictoriaMetrics Single with `curl`:
```bash
```console
curl -sg http://your_droplet_public_ipv4:8428/api/v1/query_range?query=vm_app_uptime_seconds | jq
```
@ -50,6 +50,6 @@ curl -sg http://your_droplet_public_ipv4:8428/api/v1/query_range?query=vm_app_up
Once the Droplet is created, you can use DigitalOcean's web console to start a session or SSH directly to the server as root:
```bash
```console
ssh root@your_droplet_public_ipv4
```

4
deployment/marketplace/digitialocean/one-click-droplet/RELEASE_GUIDE.md
View file

@ -6,13 +6,13 @@
2. API Token can be generated on [https://cloud.digitalocean.com/account/api/tokens](https://cloud.digitalocean.com/account/api/tokens) or use already generated from OnePassword.
3. Set variable `DIGITALOCEAN_API_TOKEN` for environment:
```bash
```console
export DIGITALOCEAN_API_TOKEN="your_token_here"
```
or set it by with make:
```bash
```console
make release-victoria-metrics-digitalocean-oneclick-droplet DIGITALOCEAN_API_TOKEN="your_token_here"
```

9
docs/CHANGELOG.md
View file

@ -18,7 +18,7 @@ The following tip changes can be tested by building VictoriaMetrics components f
**Update notes:** this release introduces backwards-incompatible changes to communication protocol between `vmselect` and `vmstorage` nodes in cluster version of VictoriaMetrics because of added [query tracing](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing), so `vmselect` and `vmstorage` nodes may log communication errors during the upgrade. These errors should stop after all the `vmselect` and `vmstorage` nodes are updated to new release. It is safe to downgrade to previous releases.
* FEATURE: support query tracing, which allows determining bottlenecks during query processing. See [these docs](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing) and [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add `cardinality` tab, which can help identifying the source of [high cardinality](https://docs.victoriametrics.com/FAQ.html#what-is-high-cardinality) and [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate) issues. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233) and [these docs](https://docs.victoriametrics.com/#cardinality-explorer).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add `cardinality` tab, which can help identifying the source of [high cardinality](https://docs.victoriametrics.com/FAQ.html#what-is-high-cardinality) and [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate) issues. See [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233) and [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2730) feature requests and [these docs](https://docs.victoriametrics.com/#cardinality-explorer).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): small UX enhancements according to [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2638).
* FEATURE: allow overriding default limits for in-memory cache `indexdb/tagFilters` via flag `-storage.cacheSizeIndexDBTagFilters`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2663).
* FEATURE: add support of `lowercase` and `uppercase` relabeling actions in the same way as [Prometheus 2.36.0 does](https://github.com/prometheus/prometheus/releases/tag/v2.36.0). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2664).
@ -28,12 +28,14 @@ The following tip changes can be tested by building VictoriaMetrics components f
* FEATURE: optimize performance for [/api/v1/labels](https://prometheus.io/docs/prometheus/latest/querying/api/#getting-label-names) and [/api/v1/label/.../values](https://prometheus.io/docs/prometheus/latest/querying/api/#querying-label-values) endpoints when `match[]`, `extra_label` or `extra_filters[]` query args are passed to these endpoints. This should help with [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1533).
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): support `limit` param per-group for limiting number of produced samples per each rule. Thanks to @Howie59 for [implementation](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2676).
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): remove dependency on Internet access at [web API pages](https://docs.victoriametrics.com/vmalert.html#web). Previously the functionality and the layout of these pages was broken without Internet access. See [shis issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594).
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): send alerts to the configured notifiers in parallel. Previously alerts were sent to notifiers sequentially. This could delay sending pending alerts when notifier blocks on the currently sent alert.
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): implement the `http://vmagent:8429/service-discovery` page in the same way as Prometheus does. This page shows the original labels for all the discovered targets alongside the resulting labels after the relabeling. This simplifies service discovery debugging.
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): remove dependency on Internet access at `http://vmagent:8429/targets` page. Previously the page layout was broken without Internet access. See [shis issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add support for `kubeconfig_file` option at [kubernetes_sd_configs](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config). It may be useful for Kubernetes monitoring by `vmagent` outside Kubernetes cluster. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1464).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): expose `/api/v1/status/config` endpoint in the same way as Prometheus does. See [these docs](https://prometheus.io/docs/prometheus/latest/querying/api/#config).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add `-promscrape.suppressScrapeErrorsDelay` command-line flag, which can be used for delaying and aggregating the logging of per-target scrape errors. This may reduce the amounts of logs when `vmagent` scrapes many unreliable targets. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2575). Thanks to @jelmd for [the initial implementation](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2576).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add `-promscrape.cluster.name` command-line flag, which allows proper data de-duplication when the same target is scraped from multiple [vmagent clusters](https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2679).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add `action: graphite` relabeling rules optimized for extracting labels from Graphite-style metric names. See [these docs](https://docs.victoriametrics.com/vmagent.html#graphite-relabeling) and [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2737).
* FEATURE: [VictoriaMetrics enterprise](https://victoriametrics.com/products/enterprise/): expose `vm_downsampling_partitions_scheduled` and `vm_downsampling_partitions_scheduled_size_bytes` metrics, which can be used for tracking the progress of initial [downsampling](https://docs.victoriametrics.com/#downsampling) for historical data. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2612).
* BUGFIX: support for data ingestion in [DataDog format](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-send-data-from-datadog-agent) from legacy clients / agents. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2670). Thanks to @elProxy for the fix.
@ -44,6 +46,9 @@ The following tip changes can be tested by building VictoriaMetrics components f
* BUGFIX: [vmui](https://docs.victoriametrics.com/#vmui): properly apply the selected time range when auto-refresh is enabled. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2693).
* BUGFIX: [vmui](https://docs.victoriametrics.com/#vmui): properly update the url with vmui state when new query is entered. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2692).
* BUGFIX: [Graphite render API](https://docs.victoriametrics.com/#graphite-render-api-usage): properly calculate sample timestamps when `moving*()` functions such as [movingAverage()](https://graphite.readthedocs.io/en/stable/functions.html#graphite.render.functions.movingAverage) are applied over [summarize()](https://graphite.readthedocs.io/en/stable/functions.html#graphite.render.functions.summarize).
* BUGFIX: limit the `end` query arg value to `+2 days` in the future at `/api/v1/*` endpoints, because VictoriaMetrics doesn't allow storing samples with timestamps bigger than +2 days in the future. This should help resolving [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2669).
* BUGFIX: properly register time series in per-day inverted index during the first hour after `indexdb` rotation. Previously this could lead to missing time series during querying if these time series stopped receiving new samples during the first hour after `indexdb` rotation. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2698).
* BUGFIX: do not register new series when `-storage.maxHourlySeries` or `-storage.maxDailySeries` limits were reached. Previously samples for new series weren't added to the database when the [cardinality limit](https://docs.victoriametrics.com/#cardinality-limiter) was reached, but series were still registered in the inverted index (aka `indexdb`). This could lead to unbound `indexdb` growth during [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate).
## [v1.77.2](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.77.2)
@ -1001,7 +1006,7 @@ Released at 26-11-2020
* FEATURE: added [Snap package for single-node VictoriaMetrics](https://snapcraft.io/victoriametrics). This simplifies installation under Ubuntu to a single command:
```bash
```console
snap install victoriametrics
```

6
docs/Cluster-VictoriaMetrics.md
View file

@ -115,7 +115,7 @@ By default images are built on top of [alpine](https://hub.docker.com/_/scratch)
It is possible to build an image on top of any other base image by setting it via `<ROOT_IMAGE>` environment variable.
For example, the following command builds images on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package
```
@ -448,7 +448,7 @@ Example command for collecting cpu profile from `vmstorage` (replace `0.0.0.0` w
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8482/debug/pprof/profile > cpu.pprof
```
@ -458,7 +458,7 @@ Example command for collecting memory profile from `vminsert` (replace `0.0.0.0`
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8480/debug/pprof/heap > mem.pprof
```

4
docs/Quick-Start.md
View file

@ -40,7 +40,7 @@ under the current directory:
<div class="with-copy" markdown="1">
```bash
```console
docker pull victoriametrics/victoria-metrics:latest
docker run -it --rm -v `pwd`/victoria-metrics-data:/victoria-metrics-data -p 8428:8428 victoriametrics/victoria-metrics:latest
```
@ -63,7 +63,7 @@ file.
<div class="with-copy" markdown="1">
```bash
```console
git clone https://github.com/VictoriaMetrics/VictoriaMetrics --branch cluster &&
cd VictoriaMetrics/deployment/docker &&
docker-compose up

95
docs/README.md
View file

@ -164,7 +164,7 @@ Then apply new config via the following command:
<div class="with-copy" markdown="1">
```bash
```console
kill -HUP `pidof prometheus`
```
@ -268,7 +268,8 @@ See the [example VMUI at VictoriaMetrics playground](https://play.victoriametric
VictoriaMetrics provides an ability to explore time series cardinality at `cardinality` tab in [vmui](#vmui) in the following ways:
- To identify metric names with the highest number of series.
- To idnetify labels with the highest number of series.
- To identify labels with the highest number of series.
- To identify values with the highest number of series for the selected label (aka `focusLabel`).
- To identify label=name pairs with the highest number of series.
- To identify labels with the highest number of unique values.
@ -327,7 +328,7 @@ VictoriaMetrics doesn't check `DD_API_KEY` param, so it can be set to arbitrary
Example on how to send data to VictoriaMetrics via DataDog "submit metrics" API from command line:
```bash
```console
echo '
{
"series": [
@ -353,7 +354,7 @@ The imported data can be read via [export API](https://docs.victoriametrics.com/
<div class="with-copy" markdown="1">
```bash
```console
curl http://localhost:8428/api/v1/export -d 'match[]=system.load.1'
```
@ -368,6 +369,16 @@ This command should return the following output if everything is OK:
Extra labels may be added to all the written time series by passing `extra_label=name=value` query args.
For example, `/datadog/api/v1/series?extra_label=foo=bar` would add `{foo="bar"}` label to all the ingested metrics.
DataDog agent sends the [configured tags](https://docs.datadoghq.com/getting_started/tagging/) to
undocumented endpoint - `/datadog/intake`. This endpoint isn't supported by VictoriaMetrics yet.
This prevents from adding the configured tags to DataDog agent data sent into VictoriaMetrics.
The workaround is to run a sidecar [vmagent](https://docs.victoriametrics.com/vmagent.html) alongside every DataDog agent,
which must run with `DD_DD_URL=http://localhost:8429/datadog` environment variable.
The sidecar `vmagent` must be configured with the needed tags via `-remoteWrite.label` command-line flag and must forward
incoming data with the added tags to a centralized VictoriaMetrics specified via `-remoteWrite.url` command-line flag.
See [these docs](https://docs.victoriametrics.com/vmagent.html#adding-labels-to-metrics) for details on how to add labels to metrics at `vmagent`.
## How to send data from InfluxDB-compatible agents such as [Telegraf](https://www.influxdata.com/time-series-platform/telegraf/)
Use `http://<victoriametric-addr>:8428` url instead of InfluxDB url in agents' configs.
@ -407,7 +418,7 @@ to local VictoriaMetrics using `curl`:
<div class="with-copy" markdown="1">
```bash
```console
curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/write'
```
@ -418,7 +429,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match={__name__=~"measurement_.*"}'
```
@ -446,7 +457,7 @@ Comma-separated list of expected databases can be passed to VictoriaMetrics via
Enable Graphite receiver in VictoriaMetrics by setting `-graphiteListenAddr` command line flag. For instance,
the following command will enable Graphite receiver in VictoriaMetrics on TCP and UDP port `2003`:
```bash
```console
/path/to/victoria-metrics-prod -graphiteListenAddr=:2003
```
@ -455,7 +466,7 @@ to the VictoriaMetrics host in `StatsD` configs.
Example for writing data with Graphite plaintext protocol to local VictoriaMetrics using `nc`:
```bash
```console
echo "foo.bar.baz;tag1=value1;tag2=value2 123 `date +%s`" | nc -N localhost 2003
```
@ -465,7 +476,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match=foo.bar.baz'
```
@ -477,6 +488,8 @@ The `/api/v1/export` endpoint should return the following response:
{"metric":{"__name__":"foo.bar.baz","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1560277406000]}
```
[Graphite relabeling](https://docs.victoriametrics.com/vmagent.html#graphite-relabeling) can be used if the imported Graphite data is going to be queried via [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html).
## Querying Graphite data
Data sent to VictoriaMetrics via `Graphite plaintext protocol` may be read via the following APIs:
@ -491,6 +504,9 @@ VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series w
The `__graphite__` pseudo-label supports e.g. alternate regexp filters such as `(value1|...|valueN)`. They are transparently converted to `{value1,...,valueN}` syntax [used in Graphite](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). This allows using [multi-value template variables in Grafana](https://grafana.com/docs/grafana/latest/variables/formatting-multi-value-variables/) inside `__graphite__` pseudo-label. For example, Grafana expands `{__graphite__=~"foo.($bar).baz"}` into `{__graphite__=~"foo.(x|y).baz"}` if `$bar` template variable contains `x` and `y` values. In this case the query is automatically converted into `{__graphite__=~"foo.{x,y}.baz"}` before execution.
VictoriaMetrics also supports Graphite query language - see [these docs](#graphite-render-api-usage).
## How to send data from OpenTSDB-compatible agents
VictoriaMetrics supports [telnet put protocol](http://opentsdb.net/docs/build/html/api_telnet/put.html)
@ -502,7 +518,7 @@ The same protocol is used for [ingesting data in KairosDB](https://kairosdb.gith
Enable OpenTSDB receiver in VictoriaMetrics by setting `-opentsdbListenAddr` command line flag. For instance,
the following command enables OpenTSDB receiver in VictoriaMetrics on TCP and UDP port `4242`:
```bash
```console
/path/to/victoria-metrics-prod -opentsdbListenAddr=:4242
```
@ -512,7 +528,7 @@ Example for writing data with OpenTSDB protocol to local VictoriaMetrics using `
<div class="with-copy" markdown="1">
```bash
```console
echo "put foo.bar.baz `date +%s` 123 tag1=value1 tag2=value2" | nc -N localhost 4242
```
@ -523,7 +539,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match=foo.bar.baz'
```
@ -540,7 +556,7 @@ The `/api/v1/export` endpoint should return the following response:
Enable HTTP server for OpenTSDB `/api/put` requests by setting `-opentsdbHTTPListenAddr` command line flag. For instance,
the following command enables OpenTSDB HTTP server on port `4242`:
```bash
```console
/path/to/victoria-metrics-prod -opentsdbHTTPListenAddr=:4242
```
@ -550,7 +566,7 @@ Example for writing a single data point:
<div class="with-copy" markdown="1">
```bash
```console
curl -H 'Content-Type: application/json' -d '{"metric":"x.y.z","value":45.34,"tags":{"t1":"v1","t2":"v2"}}' http://localhost:4242/api/put
```
@ -560,7 +576,7 @@ Example for writing multiple data points in a single request:
<div class="with-copy" markdown="1">
```bash
```console
curl -H 'Content-Type: application/json' -d '[{"metric":"foo","value":45.34},{"metric":"bar","value":43}]' http://localhost:4242/api/put
```
@ -570,7 +586,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match[]=x.y.z' -d 'match[]=foo' -d 'match[]=bar'
```
@ -740,7 +756,7 @@ The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is pos
by setting it via `<ROOT_IMAGE>` environment variable.
For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-victoria-metrics
```
@ -854,7 +870,7 @@ Each JSON line contains samples for a single time series. An example output:
Optional `start` and `end` args may be added to the request in order to limit the time frame for the exported data. These args may contain either
unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/api/v1/export -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/api/v1/export -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -868,7 +884,7 @@ of time series data. This enables gzip compression for the exported data. Exampl
<div class="with-copy" markdown="1">
```bash
```console
curl -H 'Accept-Encoding: gzip' http://localhost:8428/api/v1/export -d 'match[]={__name__!=""}' > data.jsonl.gz
```
@ -902,7 +918,7 @@ for metrics to export.
Optional `start` and `end` args may be added to the request in order to limit the time frame for the exported data. These args may contain either
unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/api/v1/export/csv -d 'format=<format>' -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/api/v1/export/csv -d 'format=<format>' -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -919,7 +935,7 @@ for metrics to export. Use `{__name__=~".*"}` selector for fetching all the time
On large databases you may experience problems with limit on the number of time series, which can be exported. In this case you need to adjust `-search.maxExportSeries` command-line flag:
```bash
```console
# count unique timeseries in database
wget -O- -q 'http://your_victoriametrics_instance:8428/api/v1/series/count' | jq '.data[0]'
@ -929,7 +945,7 @@ wget -O- -q 'http://your_victoriametrics_instance:8428/api/v1/series/count' | jq
Optional `start` and `end` args may be added to the request in order to limit the time frame for the exported data. These args may contain either
unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/api/v1/export/native -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/api/v1/export/native -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -961,7 +977,7 @@ Time series data can be imported into VictoriaMetrics via any supported data ing
Example for importing data obtained via [/api/v1/export](#how-to-export-data-in-json-line-format):
```bash
```console
# Export the data from <source-victoriametrics>:
curl http://source-victoriametrics:8428/api/v1/export -d 'match={__name__!=""}' > exported_data.jsonl
@ -971,7 +987,7 @@ curl -X POST http://destination-victoriametrics:8428/api/v1/import -T exported_d
Pass `Content-Encoding: gzip` HTTP request header to `/api/v1/import` for importing gzipped data:
```bash
```console
# Export gzipped data from <source-victoriametrics>:
curl -H 'Accept-Encoding: gzip' http://source-victoriametrics:8428/api/v1/export -d 'match={__name__!=""}' > exported_data.jsonl.gz
@ -992,7 +1008,7 @@ The specification of VictoriaMetrics' native format may yet change and is not fo
If you have a native format file obtained via [/api/v1/export/native](#how-to-export-data-in-native-format) however this is the most efficient protocol for importing data in.
```bash
```console
# Export the data from <source-victoriametrics>:
curl http://source-victoriametrics:8428/api/v1/export/native -d 'match={__name__!=""}' > exported_data.bin
@ -1033,14 +1049,14 @@ Each request to `/api/v1/import/csv` may contain arbitrary number of CSV lines.
Example for importing CSV data via `/api/v1/import/csv`:
```bash
```console
curl -d "GOOG,1.23,4.56,NYSE" 'http://localhost:8428/api/v1/import/csv?format=2:metric:ask,3:metric:bid,1:label:ticker,4:label:market'
curl -d "MSFT,3.21,1.67,NASDAQ" 'http://localhost:8428/api/v1/import/csv?format=2:metric:ask,3:metric:bid,1:label:ticker,4:label:market'
```
After that the data may be read via [/api/v1/export](#how-to-export-data-in-json-line-format) endpoint:
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match[]={ticker!=""}'
```
@ -1066,7 +1082,7 @@ via `/api/v1/import/prometheus` path. For example, the following line imports a
<div class="with-copy" markdown="1">
```bash
```console
curl -d 'foo{bar="baz"} 123' -X POST 'http://localhost:8428/api/v1/import/prometheus'
```
@ -1076,7 +1092,7 @@ The following command may be used for verifying the imported data:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match={__name__=~"foo"}'
```
@ -1092,7 +1108,7 @@ Pass `Content-Encoding: gzip` HTTP request header to `/api/v1/import/prometheus`
<div class="with-copy" markdown="1">
```bash
```console
# Import gzipped data to <destination-victoriametrics>:
curl -X POST -H 'Content-Encoding: gzip' http://destination-victoriametrics:8428/api/v1/import/prometheus -T prometheus_data.gz
```
@ -1131,7 +1147,9 @@ Example contents for `-relabelConfig` file:
regex: true
```
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details about relabeling in VictoriaMetrics.
VictoriaMetrics components provide additional relabeling features such as Graphite-style relabeling.
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details.
## Federation
@ -1141,7 +1159,7 @@ at `http://<victoriametrics-addr>:8428/federate?match[]=<timeseries_selector_for
Optional `start` and `end` args may be added to the request in order to scrape the last point for each selected time series on the `[start ... end]` interval.
`start` and `end` may contain either unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/federate -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/federate -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -1195,7 +1213,7 @@ See also [cardinality limiter](#cardinality-limiter) and [capacity planning docs
* Install multiple VictoriaMetrics instances in distinct datacenters (availability zones).
* Pass addresses of these instances to [vmagent](https://docs.victoriametrics.com/vmagent.html) via `-remoteWrite.url` command-line flag:
```bash
```console
/path/to/vmagent -remoteWrite.url=http://<victoriametrics-addr-1>:8428/api/v1/write -remoteWrite.url=http://<victoriametrics-addr-2>:8428/api/v1/write
```
@ -1214,7 +1232,7 @@ remote_write:
* Apply the updated config:
```bash
```console
kill -HUP `pidof prometheus`
```
@ -1395,7 +1413,7 @@ For example, substitute `-graphiteListenAddr=:2003` with `-graphiteListenAddr=<i
If you plan to store more than 1TB of data on `ext4` partition or plan extending it to more than 16TB,
then the following options are recommended to pass to `mkfs.ext4`:
```bash
```console
mkfs.ext4 ... -O 64bit,huge_file,extent -T huge
```
@ -1441,6 +1459,7 @@ VictoriaMetrics returns TSDB stats at `/api/v1/status/tsdb` page in the way simi
* `topN=N` where `N` is the number of top entries to return in the response. By default top 10 entries are returned.
* `date=YYYY-MM-DD` where `YYYY-MM-DD` is the date for collecting the stats. By default the stats is collected for the current day. Pass `date=1970-01-01` in order to collect global stats across all the days.
* `focusLabel=LABEL_NAME` returns label values with the highest number of time series for the given `LABEL_NAME` in the `seriesCountByFocusLabelValue` list.
* `match[]=SELECTOR` where `SELECTOR` is an arbitrary [time series selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors) for series to take into account during stats calculation. By default all the series are taken into account.
* `extra_label=LABEL=VALUE`. See [these docs](#prometheus-querying-api-enhancements) for more details.
@ -1455,7 +1474,7 @@ In this case VictoriaMetrics puts query trace into `trace` field in the output J
For example, the following command:
```bash
```console
curl http://localhost:8428/api/v1/query_range -d 'query=2*rand()' -d 'start=-1h' -d 'step=1m' -d 'trace=1' | jq '.trace'
```
@ -1716,7 +1735,7 @@ VictoriaMetrics provides handlers for collecting the following [Go profiles](htt
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8428/debug/pprof/heap > mem.pprof
```
@ -1726,7 +1745,7 @@ curl http://0.0.0.0:8428/debug/pprof/heap > mem.pprof
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8428/debug/pprof/profile > cpu.pprof
```

2
docs/Release-Guide.md
View file

@ -17,6 +17,8 @@ sort: 18
4. Push release tags to <https://github.com/VictoriaMetrics/VictoriaMetrics> : `git push origin v1.xx.y` and `git push origin v1.xx.y-cluster`. Do not push `-enterprise` tags to public repository.
5. Go to <https://github.com/VictoriaMetrics/VictoriaMetrics/releases> , create new release from the pushed tag on step 4 and upload `*.tar.gz` archive with the corresponding `_checksums.txt` from step 3.
6. Copy the [CHANGELOG](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/CHANGELOG.md) for this release to [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) page.
7. Bump version of the VictoriaMetrics cluster setup in for [sandbox environment](https://github.com/VictoriaMetrics/ops/blob/main/sandbox/manifests/benchmark-vm/vmcluster.yaml)
by [opening and merging PR](https://github.com/VictoriaMetrics/ops/pull/58).
## Building snap package

91
docs/Single-server-VictoriaMetrics.md
View file

@ -168,7 +168,7 @@ Then apply new config via the following command:
<div class="with-copy" markdown="1">
```bash
```console
kill -HUP `pidof prometheus`
```
@ -332,7 +332,7 @@ VictoriaMetrics doesn't check `DD_API_KEY` param, so it can be set to arbitrary
Example on how to send data to VictoriaMetrics via DataDog "submit metrics" API from command line:
```bash
```console
echo '
{
"series": [
@ -358,7 +358,7 @@ The imported data can be read via [export API](https://docs.victoriametrics.com/
<div class="with-copy" markdown="1">
```bash
```console
curl http://localhost:8428/api/v1/export -d 'match[]=system.load.1'
```
@ -373,6 +373,16 @@ This command should return the following output if everything is OK:
Extra labels may be added to all the written time series by passing `extra_label=name=value` query args.
For example, `/datadog/api/v1/series?extra_label=foo=bar` would add `{foo="bar"}` label to all the ingested metrics.
DataDog agent sends the [configured tags](https://docs.datadoghq.com/getting_started/tagging/) to
undocumented endpoint - `/datadog/intake`. This endpoint isn't supported by VictoriaMetrics yet.
This prevents from adding the configured tags to DataDog agent data sent into VictoriaMetrics.
The workaround is to run a sidecar [vmagent](https://docs.victoriametrics.com/vmagent.html) alongside every DataDog agent,
which must run with `DD_DD_URL=http://localhost:8429/datadog` environment variable.
The sidecar `vmagent` must be configured with the needed tags via `-remoteWrite.label` command-line flag and must forward
incoming data with the added tags to a centralized VictoriaMetrics specified via `-remoteWrite.url` command-line flag.
See [these docs](https://docs.victoriametrics.com/vmagent.html#adding-labels-to-metrics) for details on how to add labels to metrics at `vmagent`.
## How to send data from InfluxDB-compatible agents such as [Telegraf](https://www.influxdata.com/time-series-platform/telegraf/)
Use `http://<victoriametric-addr>:8428` url instead of InfluxDB url in agents' configs.
@ -412,7 +422,7 @@ to local VictoriaMetrics using `curl`:
<div class="with-copy" markdown="1">
```bash
```console
curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/write'
```
@ -423,7 +433,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match={__name__=~"measurement_.*"}'
```
@ -451,7 +461,7 @@ Comma-separated list of expected databases can be passed to VictoriaMetrics via
Enable Graphite receiver in VictoriaMetrics by setting `-graphiteListenAddr` command line flag. For instance,
the following command will enable Graphite receiver in VictoriaMetrics on TCP and UDP port `2003`:
```bash
```console
/path/to/victoria-metrics-prod -graphiteListenAddr=:2003
```
@ -460,7 +470,7 @@ to the VictoriaMetrics host in `StatsD` configs.
Example for writing data with Graphite plaintext protocol to local VictoriaMetrics using `nc`:
```bash
```console
echo "foo.bar.baz;tag1=value1;tag2=value2 123 `date +%s`" | nc -N localhost 2003
```
@ -470,7 +480,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match=foo.bar.baz'
```
@ -482,6 +492,8 @@ The `/api/v1/export` endpoint should return the following response:
{"metric":{"__name__":"foo.bar.baz","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1560277406000]}
```
[Graphite relabeling](https://docs.victoriametrics.com/vmagent.html#graphite-relabeling) can be used if the imported Graphite data is going to be queried via [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html).
## Querying Graphite data
Data sent to VictoriaMetrics via `Graphite plaintext protocol` may be read via the following APIs:
@ -496,6 +508,9 @@ VictoriaMetrics supports `__graphite__` pseudo-label for selecting time series w
The `__graphite__` pseudo-label supports e.g. alternate regexp filters such as `(value1|...|valueN)`. They are transparently converted to `{value1,...,valueN}` syntax [used in Graphite](https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards). This allows using [multi-value template variables in Grafana](https://grafana.com/docs/grafana/latest/variables/formatting-multi-value-variables/) inside `__graphite__` pseudo-label. For example, Grafana expands `{__graphite__=~"foo.($bar).baz"}` into `{__graphite__=~"foo.(x|y).baz"}` if `$bar` template variable contains `x` and `y` values. In this case the query is automatically converted into `{__graphite__=~"foo.{x,y}.baz"}` before execution.
VictoriaMetrics also supports Graphite query language - see [these docs](#graphite-render-api-usage).
## How to send data from OpenTSDB-compatible agents
VictoriaMetrics supports [telnet put protocol](http://opentsdb.net/docs/build/html/api_telnet/put.html)
@ -507,7 +522,7 @@ The same protocol is used for [ingesting data in KairosDB](https://kairosdb.gith
Enable OpenTSDB receiver in VictoriaMetrics by setting `-opentsdbListenAddr` command line flag. For instance,
the following command enables OpenTSDB receiver in VictoriaMetrics on TCP and UDP port `4242`:
```bash
```console
/path/to/victoria-metrics-prod -opentsdbListenAddr=:4242
```
@ -517,7 +532,7 @@ Example for writing data with OpenTSDB protocol to local VictoriaMetrics using `
<div class="with-copy" markdown="1">
```bash
```console
echo "put foo.bar.baz `date +%s` 123 tag1=value1 tag2=value2" | nc -N localhost 4242
```
@ -528,7 +543,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match=foo.bar.baz'
```
@ -545,7 +560,7 @@ The `/api/v1/export` endpoint should return the following response:
Enable HTTP server for OpenTSDB `/api/put` requests by setting `-opentsdbHTTPListenAddr` command line flag. For instance,
the following command enables OpenTSDB HTTP server on port `4242`:
```bash
```console
/path/to/victoria-metrics-prod -opentsdbHTTPListenAddr=:4242
```
@ -555,7 +570,7 @@ Example for writing a single data point:
<div class="with-copy" markdown="1">
```bash
```console
curl -H 'Content-Type: application/json' -d '{"metric":"x.y.z","value":45.34,"tags":{"t1":"v1","t2":"v2"}}' http://localhost:4242/api/put
```
@ -565,7 +580,7 @@ Example for writing multiple data points in a single request:
<div class="with-copy" markdown="1">
```bash
```console
curl -H 'Content-Type: application/json' -d '[{"metric":"foo","value":45.34},{"metric":"bar","value":43}]' http://localhost:4242/api/put
```
@ -575,7 +590,7 @@ After that the data may be read via [/api/v1/export](#how-to-export-data-in-json
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match[]=x.y.z' -d 'match[]=foo' -d 'match[]=bar'
```
@ -745,7 +760,7 @@ The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is pos
by setting it via `<ROOT_IMAGE>` environment variable.
For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-victoria-metrics
```
@ -859,7 +874,7 @@ Each JSON line contains samples for a single time series. An example output:
Optional `start` and `end` args may be added to the request in order to limit the time frame for the exported data. These args may contain either
unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/api/v1/export -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/api/v1/export -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -873,7 +888,7 @@ of time series data. This enables gzip compression for the exported data. Exampl
<div class="with-copy" markdown="1">
```bash
```console
curl -H 'Accept-Encoding: gzip' http://localhost:8428/api/v1/export -d 'match[]={__name__!=""}' > data.jsonl.gz
```
@ -907,7 +922,7 @@ for metrics to export.
Optional `start` and `end` args may be added to the request in order to limit the time frame for the exported data. These args may contain either
unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/api/v1/export/csv -d 'format=<format>' -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/api/v1/export/csv -d 'format=<format>' -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -924,7 +939,7 @@ for metrics to export. Use `{__name__=~".*"}` selector for fetching all the time
On large databases you may experience problems with limit on the number of time series, which can be exported. In this case you need to adjust `-search.maxExportSeries` command-line flag:
```bash
```console
# count unique timeseries in database
wget -O- -q 'http://your_victoriametrics_instance:8428/api/v1/series/count' | jq '.data[0]'
@ -934,7 +949,7 @@ wget -O- -q 'http://your_victoriametrics_instance:8428/api/v1/series/count' | jq
Optional `start` and `end` args may be added to the request in order to limit the time frame for the exported data. These args may contain either
unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/api/v1/export/native -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/api/v1/export/native -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -966,7 +981,7 @@ Time series data can be imported into VictoriaMetrics via any supported data ing
Example for importing data obtained via [/api/v1/export](#how-to-export-data-in-json-line-format):
```bash
```console
# Export the data from <source-victoriametrics>:
curl http://source-victoriametrics:8428/api/v1/export -d 'match={__name__!=""}' > exported_data.jsonl
@ -976,7 +991,7 @@ curl -X POST http://destination-victoriametrics:8428/api/v1/import -T exported_d
Pass `Content-Encoding: gzip` HTTP request header to `/api/v1/import` for importing gzipped data:
```bash
```console
# Export gzipped data from <source-victoriametrics>:
curl -H 'Accept-Encoding: gzip' http://source-victoriametrics:8428/api/v1/export -d 'match={__name__!=""}' > exported_data.jsonl.gz
@ -997,7 +1012,7 @@ The specification of VictoriaMetrics' native format may yet change and is not fo
If you have a native format file obtained via [/api/v1/export/native](#how-to-export-data-in-native-format) however this is the most efficient protocol for importing data in.
```bash
```console
# Export the data from <source-victoriametrics>:
curl http://source-victoriametrics:8428/api/v1/export/native -d 'match={__name__!=""}' > exported_data.bin
@ -1038,14 +1053,14 @@ Each request to `/api/v1/import/csv` may contain arbitrary number of CSV lines.
Example for importing CSV data via `/api/v1/import/csv`:
```bash
```console
curl -d "GOOG,1.23,4.56,NYSE" 'http://localhost:8428/api/v1/import/csv?format=2:metric:ask,3:metric:bid,1:label:ticker,4:label:market'
curl -d "MSFT,3.21,1.67,NASDAQ" 'http://localhost:8428/api/v1/import/csv?format=2:metric:ask,3:metric:bid,1:label:ticker,4:label:market'
```
After that the data may be read via [/api/v1/export](#how-to-export-data-in-json-line-format) endpoint:
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match[]={ticker!=""}'
```
@ -1071,7 +1086,7 @@ via `/api/v1/import/prometheus` path. For example, the following line imports a
<div class="with-copy" markdown="1">
```bash
```console
curl -d 'foo{bar="baz"} 123' -X POST 'http://localhost:8428/api/v1/import/prometheus'
```
@ -1081,7 +1096,7 @@ The following command may be used for verifying the imported data:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/api/v1/export' -d 'match={__name__=~"foo"}'
```
@ -1097,7 +1112,7 @@ Pass `Content-Encoding: gzip` HTTP request header to `/api/v1/import/prometheus`
<div class="with-copy" markdown="1">
```bash
```console
# Import gzipped data to <destination-victoriametrics>:
curl -X POST -H 'Content-Encoding: gzip' http://destination-victoriametrics:8428/api/v1/import/prometheus -T prometheus_data.gz
```
@ -1136,7 +1151,9 @@ Example contents for `-relabelConfig` file:
regex: true
```
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details about relabeling in VictoriaMetrics.
VictoriaMetrics components provide additional relabeling features such as Graphite-style relabeling.
See [these docs](https://docs.victoriametrics.com/vmagent.html#relabeling) for more details.
## Federation
@ -1146,7 +1163,7 @@ at `http://<victoriametrics-addr>:8428/federate?match[]=<timeseries_selector_for
Optional `start` and `end` args may be added to the request in order to scrape the last point for each selected time series on the `[start ... end]` interval.
`start` and `end` may contain either unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) values.
For example:
```bash
```console
curl http://<victoriametrics-addr>:8428/federate -d 'match[]=<timeseries_selector_for_export>' -d 'start=1654543486' -d 'end=1654543486'
curl http://<victoriametrics-addr>:8428/federate -d 'match[]=<timeseries_selector_for_export>' -d 'start=2022-06-06T19:25:48+00:00' -d 'end=2022-06-06T19:29:07+00:00'
```
@ -1200,7 +1217,7 @@ See also [cardinality limiter](#cardinality-limiter) and [capacity planning docs
* Install multiple VictoriaMetrics instances in distinct datacenters (availability zones).
* Pass addresses of these instances to [vmagent](https://docs.victoriametrics.com/vmagent.html) via `-remoteWrite.url` command-line flag:
```bash
```console
/path/to/vmagent -remoteWrite.url=http://<victoriametrics-addr-1>:8428/api/v1/write -remoteWrite.url=http://<victoriametrics-addr-2>:8428/api/v1/write
```
@ -1219,7 +1236,7 @@ remote_write:
* Apply the updated config:
```bash
```console
kill -HUP `pidof prometheus`
```
@ -1400,7 +1417,7 @@ For example, substitute `-graphiteListenAddr=:2003` with `-graphiteListenAddr=<i
If you plan to store more than 1TB of data on `ext4` partition or plan extending it to more than 16TB,
then the following options are recommended to pass to `mkfs.ext4`:
```bash
```console
mkfs.ext4 ... -O 64bit,huge_file,extent -T huge
```
@ -1461,7 +1478,7 @@ In this case VictoriaMetrics puts query trace into `trace` field in the output J
For example, the following command:
```bash
```console
curl http://localhost:8428/api/v1/query_range -d 'query=2*rand()' -d 'start=-1h' -d 'step=1m' -d 'trace=1' | jq '.trace'
```
@ -1722,7 +1739,7 @@ VictoriaMetrics provides handlers for collecting the following [Go profiles](htt
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8428/debug/pprof/heap > mem.pprof
```
@ -1732,7 +1749,7 @@ curl http://0.0.0.0:8428/debug/pprof/heap > mem.pprof
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8428/debug/pprof/profile > cpu.pprof
```

40
docs/guides/getting-started-with-vm-operator.md
View file

@ -22,7 +22,7 @@ See how to work with a [VictoriaMetrics Helm repository in previous guide](https
<div class="with-copy" markdown="1">
```bash
```console
helm install operator vm/victoria-metrics-operator
```
@ -30,7 +30,7 @@ helm install operator vm/victoria-metrics-operator
The expected output is:
```bash
```console
NAME: vmoperator
LAST DEPLOYED: Thu Sep 30 17:30:30 2021
NAMESPACE: default
@ -49,13 +49,13 @@ Run the following command to check that VM Operator is up and running:
<div class="with-copy" markdown="1">
```bash
```console
kubectl --namespace default get pods -l "app.kubernetes.io/instance=vmoperator"
```
</div>
The expected output:
```bash
```console
NAME READY STATUS RESTARTS AGE
vmoperator-victoria-metrics-operator-67cff44cd6-s47n6 1/1 Running 0 77s
```
@ -68,7 +68,7 @@ Run the following command to install [VictoriaMetrics Cluster](https://docs.vict
<div class="with-copy" markdown="1" id="example-cluster-config">
```bash
```console
cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMCluster
@ -89,7 +89,7 @@ EOF
The expected output:
```bash
```console
vmcluster.operator.victoriametrics.com/example-vmcluster-persistent created
```
@ -100,13 +100,13 @@ vmcluster.operator.victoriametrics.com/example-vmcluster-persistent created
Please note that it may take some time for the pods to start. To check that the pods are started, run the following command:
<div class="with-copy" markdown="1" id="example-cluster-config">
```bash
```console
kubectl get pods | grep vmcluster
```
</div>
The expected output:
```bash
```console
NAME READY STATUS RESTARTS AGE
vminsert-example-vmcluster-persistent-845849cb84-9vb6f 1/1 Running 0 5m15s
vminsert-example-vmcluster-persistent-845849cb84-r7mmk 1/1 Running 0 5m15s
@ -119,13 +119,13 @@ vmstorage-example-vmcluster-persistent-1 1/1 Running 0
There is an extra command to get information about the cluster state:
<div class="with-copy" markdown="1" id="services">
```bash
```console
kubectl get vmclusters
```
</div>
The expected output:
```bash
```console
NAME INSERT COUNT STORAGE COUNT SELECT COUNT AGE STATUS
example-vmcluster-persistent 2 2 2 5m53s operational
```
@ -136,14 +136,14 @@ To get the name of `vminsert` services, please run the following command:
<div class="with-copy" markdown="1" id="services">
```bash
```console
kubectl get svc | grep vminsert
```
</div>
The expected output:
```bash
```console
vminsert-example-vmcluster-persistent ClusterIP 10.107.47.136 <none> 8480/TCP 5m58s
```
@ -153,7 +153,7 @@ Here is an example of the full configuration that we need to apply:
<div class="with-copy" markdown="1">
```bash
```console
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
@ -177,7 +177,7 @@ EOF
The expected output:
```bash
```console
vmagent.operator.victoriametrics.com/example-vmagent created
```
@ -188,14 +188,14 @@ Verify that `VMAgent` is up and running by executing the following command:
<div class="with-copy" markdown="1">
```bash
```console
kubectl get pods | grep vmagent
```
</div>
The expected output is:
```bash
```console
vmagent-example-vmagent-7996844b5f-b5rzs 2/2 Running 0 9s
```
@ -207,13 +207,13 @@ Run the following command to make `VMAgent`'s port accessible from the local mac
</div>
```bash
```console
kubectl port-forward svc/vmagent-example-vmagent 8429:8429
```
The expected output is:
```bash
```console
Forwarding from 127.0.0.1:8429 -> 8429
Forwarding from [::1]:8429 -> 8429
```
@ -235,14 +235,14 @@ To get the new service name, please run the following command:
<div class="with-copy" markdown="1" id="services">
```bash
```console
kubectl get svc | grep vmselect
```
</div>
The expected output:
```bash
```console
vmselect-example-vmcluster-persistent ClusterIP None <none> 8481/TCP 7m
```

30
docs/guides/k8s-ha-monitoring-via-vm-cluster.md
View file

@ -65,7 +65,7 @@ EOF
The expected result of the command execution is the following:
```bash
```console
NAME: vmcluster
LAST DEPLOYED: Thu Jul 29 13:33:51 2021
NAMESPACE: default
@ -121,14 +121,14 @@ Verify that the VictoriaMetrics cluster pods are up and running by executing the
<div class="with-copy" markdown="1">
```bash
```console
kubectl get pods | grep vmcluster
```
</div>
The expected output is:
```bash
```console
vmcluster-victoria-metrics-cluster-vminsert-78b84d8cd9-4mh9d 1/1 Running 0 2m28s
vmcluster-victoria-metrics-cluster-vminsert-78b84d8cd9-4ppl7 1/1 Running 0 2m28s
vmcluster-victoria-metrics-cluster-vminsert-78b84d8cd9-782qk 1/1 Running 0 2m28s
@ -241,7 +241,7 @@ Verify that `vmagent`'s pod is up and running by executing the following command
<div class="with-copy" markdown="1">
```bash
```console
kubectl get pods | grep vmagent
```
</div>
@ -249,7 +249,7 @@ kubectl get pods | grep vmagent
The expected output is:
```bash
```console
vmagent-victoria-metrics-agent-57ddbdc55d-h4ljb 1/1 Running 0 13s
```
@ -258,14 +258,14 @@ vmagent-victoria-metrics-agent-57ddbdc55d-h4ljb 1/1 Running
Run the following command to check that VictoriaMetrics services are up and running:
<div class="with-copy" markdown="1">
```bash
```console
kubectl get pods | grep victoria-metrics
```
</div>
The expected output is:
```bash
```console
vmagent-victoria-metrics-agent-57ddbdc55d-h4ljb 1/1 Running 0 75s
vmcluster-victoria-metrics-cluster-vminsert-78b84d8cd9-s8v7x 1/1 Running 0 89s
vmcluster-victoria-metrics-cluster-vminsert-78b84d8cd9-xlm9d 1/1 Running 0 89s
@ -283,14 +283,14 @@ To verify that metrics are present in the VictoriaMetrics send a curl request to
Run the following command to see the list of services:
<div class="with-copy" markdown="1">
```bash
```console
k get svc | grep vmselect
```
</div>
The expected output:
```bash
```console
vmcluster-victoria-metrics-cluster-vmselect ClusterIP 10.88.2.69 <none> 8481/TCP 1m
```
@ -298,20 +298,20 @@ Run the following command to make `vmselect`'s port accessable from the local ma
<div class="with-copy" markdown="1">
```bash
```console
kubectl port-forward svc/vmcluster-victoria-metrics-cluster-vmselect 8481:8481
```
</div>
Execute the following command to get metrics via `curl`:
```bash
```console
curl -sg 'http://127.0.0.1:8481/select/0/prometheus/api/v1/query_range?query=count(up{kubernetes_pod_name=~".*vmselect.*"})&start=-10m&step=1m' | jq
```
The expected output is:
```bash
```console
{
"status": "success",
"isPartial": false,
@ -389,7 +389,7 @@ To test if High Availability works, we need to shutdown one of the `vmstorages`.
<div class="with-copy" markdown="1">
```bash
```console
kubectl scale sts vmcluster-victoria-metrics-cluster-vmstorage --replicas=2
```
</div>
@ -398,13 +398,13 @@ Verify that now we have two running `vmstorages` in the cluster by executing the
<div class="with-copy" markdown="1">
```bash
```console
kubectl get pods | grep vmstorage
```
</div>
The expected output is:
```bash
```console
vmcluster-victoria-metrics-cluster-vmstorage-0 1/1 Running 0 44m
vmcluster-victoria-metrics-cluster-vmstorage-1 1/1 Running 0 43m
```

22
docs/guides/k8s-monitoring-via-vm-cluster.md
View file

@ -28,7 +28,7 @@ You need to add the VictoriaMetrics Helm repository to install VictoriaMetrics c
<div class="with-copy" markdown="1">
```bash
```console
helm repo add vm https://victoriametrics.github.io/helm-charts/
```
@ -38,7 +38,7 @@ Update Helm repositories:
<div class="with-copy" markdown="1">
```bash
```console
helm repo update
```
@ -48,7 +48,7 @@ To verify that everything is set up correctly you may run this command:
<div class="with-copy" markdown="1">
```bash
```console
helm search repo vm/
```
@ -56,7 +56,7 @@ helm search repo vm/
The expected output is:
```bash
```console
NAME CHART VERSION APP VERSION DESCRIPTION
vm/victoria-metrics-agent 0.7.20 v1.62.0 Victoria Metrics Agent - collects metrics from ...
vm/victoria-metrics-alert 0.3.34 v1.62.0 Victoria Metrics Alert - executes a list of giv...
@ -100,7 +100,7 @@ EOF
As a result of this command you will see the following output:
```bash
```console
NAME: vmcluster
LAST DEPLOYED: Thu Jul 1 09:41:57 2021
NAMESPACE: default
@ -159,14 +159,14 @@ Verify that [VictoriaMetrics cluster](https://docs.victoriametrics.com/Cluster-V
<div class="with-copy" markdown="1">
```bash
```console
kubectl get pods
```
</div>
The expected output is:
```bash
```console
NAME READY STATUS RESTARTS AGE
vmcluster-victoria-metrics-cluster-vminsert-689cbc8f55-95szg 1/1 Running 0 16m
vmcluster-victoria-metrics-cluster-vminsert-689cbc8f55-f852l 1/1 Running 0 16m
@ -422,14 +422,14 @@ Verify that `vmagent`'s pod is up and running by executing the following command
<div class="with-copy" markdown="1">
```bash
```console
kubectl get pods | grep vmagent
```
</div>
The expected output is:
```bash
```console
vmagent-victoria-metrics-agent-69974b95b4-mhjph 1/1 Running 0 11m
```
@ -440,7 +440,7 @@ Add the Grafana Helm repository.
<div class="with-copy" markdown="1">
```bash
```console
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
```
@ -512,7 +512,7 @@ The second and the third will forward Grafana to `127.0.0.1:3000`:
<div class="with-copy" markdown="1">
```bash
```console
kubectl get secret --namespace default my-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=grafana,app.kubernetes.io/instance=my-grafana" -o jsonpath="{.items[0].metadata.name}")

22
docs/guides/k8s-monitoring-via-vm-single.md
View file

@ -28,7 +28,7 @@ You need to add the VictoriaMetrics Helm repository to install VictoriaMetrics c
<div class="with-copy" markdown="1">
```bash
```console
helm repo add vm https://victoriametrics.github.io/helm-charts/
```
@ -38,7 +38,7 @@ Update Helm repositories:
<div class="with-copy" markdown="1">
```bash
```console
helm repo update
```
@ -48,7 +48,7 @@ To verify that everything is set up correctly you may run this command:
<div class="with-copy" markdown="1">
```bash
```console
helm search repo vm/
```
@ -56,7 +56,7 @@ helm search repo vm/
The expected output is:
```bash
```console
NAME CHART VERSION APP VERSION DESCRIPTION
vm/victoria-metrics-agent 0.7.20 v1.62.0 Victoria Metrics Agent - collects metrics from ...
vm/victoria-metrics-alert 0.3.34 v1.62.0 Victoria Metrics Alert - executes a list of giv...
@ -74,7 +74,7 @@ Run this command in your terminal:
<div class="with-copy" markdown="1">.html
```bash
```console
helm install vmsingle vm/victoria-metrics-single -f https://docs.victoriametrics.com/guides/guide-vmsingle-values.yaml
```
@ -175,7 +175,7 @@ server:
As a result of the command you will see the following output:
```bash
```console
NAME: victoria-metrics
LAST DEPLOYED: Fri Jun 25 12:06:13 2021
NAMESPACE: default
@ -219,7 +219,7 @@ Verify that VictoriaMetrics pod is up and running by executing the following com
<div class="with-copy" markdown="1">
```bash
```console
kubectl get pods
```
@ -227,7 +227,7 @@ kubectl get pods
The expected output is:
```bash
```console
NAME READY STATUS RESTARTS AGE
vmsingle-victoria-metrics-single-server-0 1/1 Running 0 68s
```
@ -239,7 +239,7 @@ Add the Grafana Helm repository.
<div class="with-copy" markdown="1">
```bash
```console
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
```
@ -305,7 +305,7 @@ To see the password for Grafana `admin` user use the following command:
<div class="with-copy" markdown="1">
```bash
```console
kubectl get secret --namespace default my-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
```
@ -315,7 +315,7 @@ Expose Grafana service on `127.0.0.1:3000`:
<div class="with-copy" markdown="1">
```bash
```console
export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=grafana,app.kubernetes.io/instance=my-grafana" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace default port-forward $POD_NAME 3000

4
docs/guides/migrate-from-influx.md
View file

@ -72,7 +72,7 @@ supports [InfluxDB line protocol](https://docs.victoriametrics.com/#how-to-send-
for data ingestion. For example, to write a measurement to VictoriaMetrics we need to send an HTTP POST request with
payload in a line protocol format:
```bash
```console
curl -d 'census,location=klamath,scientist=anderson bees=23 1566079200000' -X POST 'http://<victoriametric-addr>:8428/write'
```
@ -83,7 +83,7 @@ Please note, an arbitrary number of lines delimited by `\n` (aka newline char) c
To get the written data back let's export all series matching the `location="klamath"` filter:
```bash
```console
curl -G 'http://<victoriametric-addr>:8428/api/v1/export' -d 'match={location="klamath"}'
```

2
docs/guides/multi-regional-setup-dedicated-regions.md
View file

@ -28,7 +28,7 @@ Using this schema, you can achieve:
* You need to pass two `-remoteWrite.url` command-line options to `vmagent`:
```bash
```console
/path/to/vmagent-prod \
-remoteWrite.url=<ground-control-1-remote-write> \
-remoteWrite.url=<ground-control-2-remote-write>

6
docs/keyConcepts.md
View file

@ -295,7 +295,7 @@ for [InfluxDB line protocol](https://docs.victoriametrics.com/Single-server-Vict
Creating custom clients or instrumenting the application for metrics writing is as easy as sending a POST request:
```bash
```console
curl -d '{"metric":{"__name__":"foo","job":"node_exporter"},"values":[0,1,2],"timestamps":[1549891472010,1549891487724,1549891503438]}' -X POST 'http://localhost:8428/api/v1/import'
```
@ -441,7 +441,7 @@ plot this data sample on the system of coordinates, it will have the following f
To get the value of `foo_bar` metric at some specific moment of time, for example `2022-05-10 10:03:00`, in
VictoriaMetrics we need to issue an **instant query**:
```bash
```console
curl "http://<victoria-metrics-addr>/api/v1/query?query=foo_bar&time=2022-05-10T10:03:00.000Z"
```
@ -504,7 +504,7 @@ step - step in seconds for evaluating query expression on the time range. If omi
To get the values of `foo_bar` on time range from `2022-05-10 09:59:00` to `2022-05-10 10:17:00`, in VictoriaMetrics we
need to issue a range query:
```bash
```console
curl "http://<victoria-metrics-addr>/api/v1/query_range?query=foo_bar&step=1m&start=2022-05-10T09:59:00.000Z&end=2022-05-10T10:17:00.000Z"
```

6
docs/operator/VictoriaMetrics-Operator.md
View file

@ -48,7 +48,7 @@ and disable CRD controller with flag: `--controller.disableCRDOwnership=true`
## Troubleshooting
- cannot apply crd at kubernetes 1.18 + version and kubectl reports error:
```bash
```console
Error from server (Invalid): error when creating "release/crds/crd.yaml": CustomResourceDefinition.apiextensions.k8s.io "vmalertmanagers.operator.victoriametrics.com" is invalid: [spec.validation.openAPIV3Schema.properties[spec].properties[initContainers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property]
Error from server (Invalid): error when creating "release/crds/crd.yaml": CustomResourceDefinition.apiextensions.k8s.io "vmalerts.operator.victoriametrics.com" is invalid: [
```
@ -62,12 +62,12 @@ Error from server (Invalid): error when creating "release/crds/crd.yaml": Custom
- minikube or kind
start:
```bash
```console
make run
```
for test execution run:
```bash
```console
#unit tests
make test

4
docs/operator/high-availability.MD
View file

@ -280,7 +280,7 @@ EOF
Then wait for the cluster becomes ready
```bash
```console
kubectl get vmclusters -w
NAME INSERT COUNT STORAGE COUNT SELECT COUNT AGE STATUS
example-vmcluster-persistent 2 2 2 2s expanding
@ -289,7 +289,7 @@ example-vmcluster-persistent 2 2 2 30s
Get links for connection by executing the command:
```bash
```console
kubectl get svc -l app.kubernetes.io/instance=example-vmcluster-persistent
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
vminsert-example-vmcluster-persistent ClusterIP 10.96.34.94 <none> 8480/TCP 69s

54
docs/operator/quick-start.MD
View file

@ -13,7 +13,7 @@ Obtain release from releases page:
We suggest use the latest release.
```bash
```console
# Get latest release version from https://github.com/VictoriaMetrics/operator/releases/latest
export VM_VERSION=`basename $(curl -fs -o/dev/null -w %{redirect_url} https://github.com/VictoriaMetrics/operator/releases/latest)`
wget https://github.com/VictoriaMetrics/operator/releases/download/$VM_VERSION/bundle_crd.zip
@ -24,7 +24,7 @@ unzip bundle_crd.zip
> sed -i "s/namespace: monitoring-system/namespace: YOUR_NAMESPACE/g" release/operator/*
First of all, you have to create [custom resource definitions](https://github.com/VictoriaMetrics/operator)
```bash
```console
kubectl apply -f release/crds
```
@ -32,13 +32,13 @@ Then you need RBAC for operator, relevant configuration for the release can be f
Change configuration for operator at `release/operator/manager.yaml`, possible settings: [operator-settings](/vars.MD)
and apply it:
```bash
```console
kubectl apply -f release/operator/
```
Check the status of operator
```bash
```console
kubectl get pods -n monitoring-system
#NAME READY STATUS RESTARTS AGE
@ -74,19 +74,19 @@ You can change [operator-settings](/vars.MD), or use your custom namespace see [
Build template
```bash
```console
kustomize build . -o monitoring.yaml
```
Apply manifests
```bash
```console
kubectl apply -f monitoring.yaml
```
Check the status of operator
```bash
```console
kubectl get pods -n monitoring-system
#NAME READY STATUS RESTARTS AGE
@ -269,7 +269,7 @@ EOF
It requires access to Kubernetes API and you can create RBAC for it first, it can be found at `release/examples/VMAgent_rbac.yaml`
Or you can use default rbac account, that will be created for `VMAgent` by operator automatically.
```bash
```console
kubectl apply -f release/examples/vmagent_rbac.yaml
```
@ -538,7 +538,7 @@ EOF
```
Check status for pods:
```bash
```console
kubectl get pods
NAME READY STATUS RESTARTS AGE
example-app-594f97677c-g72v8 1/1 Running 0 23s
@ -552,7 +552,7 @@ vmsingle-example-vmsingle-persisted-794b59ccc6-fnkpt 1/1 Running 0
```
Checking logs for `VMAgent`:
```bash
```console
kubectl logs vmagent-example-vmagent-5777fdf7bf-tctcv vmagent
2020-08-02T18:18:17.226Z info VictoriaMetrics/app/vmagent/remotewrite/remotewrite.go:98 Successfully reloaded relabel configs
2020-08-02T18:18:17.229Z info VictoriaMetrics/lib/promscrape/scraper.go:137 found changes in "/etc/vmagent/config_out/vmagent.env.yaml"; applying these changes
@ -606,7 +606,7 @@ EOF
Let's check `VMAgent` logs (you have to wait some time for config sync, usually its around 1 min):
```bash
```console
kubectl logs vmagent-example-vmagent-5777fdf7bf-tctcv vmagent --tail 100
2020-08-03T08:24:13.312Z info VictoriaMetrics/lib/promscrape/scraper.go:106 SIGHUP received; reloading Prometheus configs from "/etc/vmagent/config_out/vmagent.env.yaml"
2020-08-03T08:24:13.312Z info VictoriaMetrics/app/vmagent/remotewrite/remotewrite.go:98 Successfully reloaded relabel configs
@ -668,7 +668,7 @@ EOF
```
Ensure, that pods started:
```bash
```console
kubectl get pods
NAME READY STATUS RESTARTS AGE
example-app-594f97677c-g72v8 1/1 Running 0 3m40s
@ -700,7 +700,7 @@ EOF
```
Lets check `VMAgent` logs:
```bash
```console
kubectl logs vmagent-example-vmagent-5777fdf7bf-tctcv vmagent --tail 100
2020-08-03T08:51:13.582Z info VictoriaMetrics/app/vmagent/remotewrite/remotewrite.go:98 Successfully reloaded relabel configs
2020-08-03T08:51:13.585Z info VictoriaMetrics/lib/promscrape/scraper.go:137 found changes in "/etc/vmagent/config_out/vmagent.env.yaml"; applying these changes
@ -736,7 +736,7 @@ EOF
```
Ensure, that it started and ready:
```bash
```console
kubectl get pods -l app.kubernetes.io/name=vmalert
NAME READY STATUS RESTARTS AGE
vmalert-example-vmalert-6f8748c6f9-hcfrr 2/2 Running 0 2m26s
@ -779,7 +779,7 @@ EOF
{% endraw %}
Ensure, that new alert was started:
```bash
```console
kubectl logs vmalert-example-vmalert-6f8748c6f9-hcfrr vmalert
2020-08-03T09:07:49.772Z info VictoriaMetrics/app/vmalert/web.go:45 api config reload was called, sending sighup
2020-08-03T09:07:49.772Z info VictoriaMetrics/app/vmalert/main.go:115 SIGHUP received. Going to reload rules ["/etc/vmalert/config/vm-example-vmalert-rulefiles-0/*.yaml"] ...
@ -817,14 +817,14 @@ EOF
{% endraw %}
`VMAlert` will report incorrect rule config and fire alert:
```bash
```console
2020-08-03T09:11:40.672Z info VictoriaMetrics/app/vmalert/main.go:115 SIGHUP received. Going to reload rules ["/etc/vmalert/config/vm-example-vmalert-rulefiles-0/*.yaml"] ...
2020-08-03T09:11:40.672Z info VictoriaMetrics/app/vmalert/manager.go:83 reading rules configuration file from "/etc/vmalert/config/vm-example-vmalert-rulefiles-0/*.yaml"
2020-08-03T09:11:40.673Z error VictoriaMetrics/app/vmalert/main.go:119 error while reloading rules: cannot parse configuration file: invalid group "incorrect rule" in file "/etc/vmalert/config/vm-example-vmalert-rulefiles-0/default-example-vmrule-incorrect-rule.yaml": invalid rule "incorrect rule"."vmalert bad config": invalid expression: unparsed data left: "expression"
```
Clean up incorrect rule:
```bash
```console
kubectl delete vmrule example-vmrule-incorrect-rule
```
@ -976,7 +976,7 @@ EOF
Ensure, that pods are ready:
```bash
```console
kubectl get pods
NAME READY STATUS RESTARTS AGE
prometheus-blackbox-exporter-5b5f44bd9c-2szdj 1/1 Running 0 3m3s
@ -986,7 +986,7 @@ vmsingle-example-vmsingle-persisted-8584486b68-mqg6b 1/1 Running 0
Now define some `VMProbe`, lets start with basic static target and probe `VMAgent` with its service address, for accessing
blackbox exporter, you have to specify its url at `VMProbe` config. Lets get both services names:
```bash
```console
kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4h21m
@ -1095,7 +1095,7 @@ spec:
EOF
```
2 targets must be added to `VMAgent` scrape config:
```bash
```console
static_configs: added targets: 2, removed targets: 0; total targets: 2
```
@ -1200,7 +1200,7 @@ EOF
```
Check its status
```bash
```console
kubectl get pods
NAME READY STATUS RESTARTS AGE
@ -1233,7 +1233,7 @@ EOF
Configuration changes for `VMAuth` takes some time, coz of mounted secret, its eventually updated by kubelet. Check vmauth log for changes:
```bash
```console
kubectl logs vmauth-example-ffcc78fcc-xddk7 vmauth -f --tail 10
2021-05-31T10:46:40.171Z info VictoriaMetrics/app/vmauth/auth_config.go:168 Loaded information about 1 users from "/opt/vmauth/config.yaml"
2021-05-31T10:46:40.171Z info VictoriaMetrics/app/vmauth/main.go:37 started vmauth in 0.000 seconds
@ -1249,7 +1249,7 @@ kubectl logs vmauth-example-ffcc78fcc-xddk7 vmauth -f --tail 10
Now lets try to access protected endpoints, i will use port-forward for that:
```bash
```console
kubectl port-forward vmauth-example-ffcc78fcc-xddk7 8427
# at separate terminal execute:
@ -1263,7 +1263,7 @@ curl localhost:8427/api/v1/groups -u 'simple-user:simple-password'
Check create secret for application access:
```bash
```console
kubectl get secrets vmuser-example
NAME TYPE DATA AGE
vmuser-example Opaque 2 6m33s
@ -1275,7 +1275,7 @@ By default, the operator converts all existing prometheus-operator API objects i
You can control this behaviour by setting env variable for operator:
```bash
```console
#disable convertion for each object
VM_ENABLEDPROMETHEUSCONVERTER_PODMONITOR=false
VM_ENABLEDPROMETHEUSCONVERTER_SERVICESCRAPE=false
@ -1326,7 +1326,7 @@ spec:
By default the operator doesn't make converted objects disappear after original ones are deleted. To change this behaviour
configure adding `OwnerReferences` to converted objects:
```bash
```console
VM_ENABLEDPROMETHEUSCONVERTEROWNERREFERENCES=true
```
Converted objects will be linked to the original ones and will be deleted by kubernetes after the original ones are deleted.
@ -1404,7 +1404,7 @@ to the rule config:
Example for Kubernetes Nginx ingress [doc](https://kubernetes.github.io/ingress-nginx/examples/auth/basic/)
```bash
```console
#generate creds
htpasswd -c auth foo

82
docs/url-examples.md
View file

@ -11,7 +11,7 @@ sort: 21
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl 'http://<victoriametrics-addr>:8428/api/v1/admin/tsdb/delete_series?match[]=vm_http_request_errors_total'
```
@ -20,7 +20,7 @@ curl 'http://<victoriametrics-addr>:8428/api/v1/admin/tsdb/delete_series?match[]
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl 'http://<vmselect>:8481/delete/0/prometheus/api/v1/admin/tsdb/delete_series?match[]=vm_http_request_errors_total'
```
@ -37,7 +37,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl 'http://<victoriametrics-addr>:8428/api/v1/export/csv?format=__name__,__value__,__timestamp__:unix_s&match=vm_http_request_errors_total' > filename.txt
```
@ -46,7 +46,7 @@ curl 'http://<victoriametrics-addr>:8428/api/v1/export/csv?format=__name__,__val
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmselect>:8481/select/0/prometheus/api/v1/export/csv?format=__name__,__value__,__timestamp__:unix_s&match=vm_http_request_errors_total' > filename.txt
```
@ -64,7 +64,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<victoriametrics-addr>:8428/api/v1/export/native?match[]=vm_http_request_errors_total' > filename.txt
```
@ -73,7 +73,7 @@ curl -G 'http://<victoriametrics-addr>:8428/api/v1/export/native?match[]=vm_http
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmselect>:8481/select/0/prometheus/api/v1/export/native?match=vm_http_request_errors_total' > filename.txt
```
@ -90,7 +90,7 @@ More information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl --data-binary "@import.txt" -X POST 'http://destination-victoriametrics:8428/api/v1/import'
```
@ -99,7 +99,7 @@ curl --data-binary "@import.txt" -X POST 'http://destination-victoriametrics:842
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl --data-binary "@import.txt" -X POST 'http://<vminsert>:8480/insert/0/prometheus/api/v1/import'
```
@ -107,7 +107,7 @@ curl --data-binary "@import.txt" -X POST 'http://<vminsert>:8480/insert/0/promet
<div class="with-copy" markdown="1">
```bash
```console
curl -d 'metric_name{foo="bar"} 123' -X POST 'http://<vminsert>:8480/insert/0/prometheus/api/v1/import/prometheus'
```
@ -124,7 +124,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl --data-binary "@import.txt" -X POST 'http://localhost:8428/api/v1/import/prometheus'
curl -d "GOOG,1.23,4.56,NYSE" 'http://localhost:8428/api/v1/import/csv?format=2:metric:ask,3:metric:bid,1:label:ticker,4:label:market'
```
@ -134,7 +134,7 @@ curl -d "GOOG,1.23,4.56,NYSE" 'http://localhost:8428/api/v1/import/csv?format=2:
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl --data-binary "@import.txt" -X POST 'http://<vminsert>:8480/insert/0/prometheus/api/v1/import/csv'
curl -d "GOOG,1.23,4.56,NYSE" 'http://<vminsert>:8480/insert/0/prometheus/api/v1/import/csv?format=2:metric:ask,3:metric:bid,1:label:ticker,4:label:market'
```
@ -153,7 +153,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/prometheus/api/v1/labels'
```
@ -162,7 +162,7 @@ curl -G 'http://localhost:8428/prometheus/api/v1/labels'
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmselect>:8481/select/0/prometheus/api/v1/labels'
```
@ -179,7 +179,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/prometheus/api/v1/label/job/values'
```
@ -188,7 +188,7 @@ curl -G 'http://localhost:8428/prometheus/api/v1/label/job/values'
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmselect>:8481/select/0/prometheus/api/v1/label/job/values'
```
@ -204,7 +204,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/prometheus/api/v1/query?query=vm_http_request_errors_total&time=2021-02-22T19:10:30.781Z'
```
@ -213,7 +213,7 @@ curl -G 'http://localhost:8428/prometheus/api/v1/query?query=vm_http_request_err
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmselect>:8481/select/0/prometheus/api/v1/query?query=vm_http_request_errors_total&time=2021-02-22T19:10:30.781Z'
```
@ -231,7 +231,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/prometheus/api/v1/query_range?query=vm_http_request_errors_total&start=2021-02-22T19:10:30.781Z&step=20m'
```
@ -240,7 +240,7 @@ curl -G 'http://localhost:8428/prometheus/api/v1/query_range?query=vm_http_reque
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmselect>:8481/select/0/prometheus/api/v1/query_range?query=vm_http_request_errors_total&start=2021-02-22T19:10:30.781Z&step=20m'
```
@ -248,11 +248,11 @@ curl -G 'http://<vmselect>:8481/select/0/prometheus/api/v1/query_range?query=vm_
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmselect>:8481/select/0/prometheus/api/v1/query_range?query=vm_http_request_errors_total&start=-1h&step=10m'
```
```bash
```console
curl -G http://<vmselect>:8481/select/0/prometheus/api/v1/query_range --data-urlencode 'query=sum(increase(vm_http_request_errors_total{status=""}[5m])) by (status)'
```
@ -270,7 +270,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/prometheus/api/v1/series?match[]=vm_http_request_errors_total&start=-1h'
```
@ -279,7 +279,7 @@ curl -G 'http://localhost:8428/prometheus/api/v1/series?match[]=vm_http_request_
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmselect>:8481/select/0/prometheus/api/v1/series?match[]=vm_http_request_errors_total&start=-1h'
```
@ -296,7 +296,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/prometheus/api/v1/status/tsdb'
```
@ -305,7 +305,7 @@ curl -G 'http://localhost:8428/prometheus/api/v1/status/tsdb'
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmselect>:8481/select/0/prometheus/api/v1/status/tsdb'
```
@ -324,7 +324,7 @@ Should be sent to vmagent/VMsingle
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmsingle>:8428/api/v1/targets'
```
@ -332,7 +332,7 @@ curl -G 'http://<vmsingle>:8428/api/v1/targets'
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmagent>:8429/api/v1/targets'
```
@ -349,7 +349,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
echo '
{
"series": [
@ -376,7 +376,7 @@ echo '
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
echo '
{
"series": [
@ -411,7 +411,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/federate?match[]=vm_http_request_errors_total&start=2021-02-22T19:10:30.781Z'
```
@ -420,7 +420,7 @@ curl -G 'http://localhost:8428/federate?match[]=vm_http_request_errors_total&sta
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmselect>:8481/select/0/prometheus/federate?match[]=vm_http_request_errors_total&start=2021-02-22T19:10:30.781Z'
```
@ -438,7 +438,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://localhost:8428/graphite/metrics/find?query=vm_http_request_errors_total'
```
@ -447,7 +447,7 @@ curl -G 'http://localhost:8428/graphite/metrics/find?query=vm_http_request_error
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -G 'http://<vmselect>:8481/select/0/graphite/metrics/find?query=vm_http_request_errors_total'
```
@ -466,7 +466,7 @@ Additional information:
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/write'
```
@ -475,7 +475,7 @@ curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'ht
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://<vminsert>:8480/insert/0/influx/write'
```
@ -495,7 +495,7 @@ Turned off by default. Enable OpenTSDB receiver in VictoriaMetrics by setting `-
Single:
<div class="with-copy" markdown="1">
```bash
```console
echo "put foo.bar.baz `date +%s` 123 tag1=value1 tag2=value2" | nc -N localhost 4242
```
@ -504,7 +504,7 @@ echo "put foo.bar.baz `date +%s` 123 tag1=value1 tag2=value2" | nc -N localhost
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
echo "put foo.bar.baz `date +%s` 123 tag1=value1 tag2=value2 VictoriaMetrics_AccountID=0" | nc -N http://<vminsert> 4242
```
@ -515,7 +515,7 @@ Enable HTTP server for OpenTSDB /api/put requests by setting `-opentsdbHTTPListe
Single:
<div class="with-copy" markdown="1">
```bash
```console
curl -H 'Content-Type: application/json' -d '[{"metric":"foo","value":45.34},{"metric":"bar","value":43}]' http://localhost:4242/api/put
```
@ -524,7 +524,7 @@ curl -H 'Content-Type: application/json' -d '[{"metric":"foo","value":45.34},{"m
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
curl -H 'Content-Type: application/json' -d '[{"metric":"foo","value":45.34},{"metric":"bar","value":43}]'
'http://<vminsert>:8480/insert/42/opentsdb/api/put'
```
@ -543,7 +543,7 @@ Enable Graphite receiver in VictoriaMetrics by setting `-graphiteListenAddr` com
Single:
<div class="with-copy" markdown="1">
```bash
```console
echo "foo.bar.baz;tag1=value1;tag2=value2 123 `date +%s`" |
nc -N localhost 2003
```
@ -553,7 +553,7 @@ echo "foo.bar.baz;tag1=value1;tag2=value2 123 `date +%s`" |
Cluster:
<div class="with-copy" markdown="1">
```bash
```console
echo "foo.bar.baz;tag1=value1;tag2=value2;VictoriaMetrics_AccountID=42 123 `date +%s`" | nc -N http://<vminsert> 2003
```

59
docs/vmagent.md
View file

@ -77,7 +77,7 @@ Pass `-help` to `vmagent` in order to see [the full list of supported command-li
* Sending `SUGHUP` signal to `vmagent` process:
```bash
```console
kill -SIGHUP `pidof vmagent`
```
@ -256,12 +256,13 @@ Labels can be added to metrics by the following mechanisms:
VictoriaMetrics components (including `vmagent`) support Prometheus-compatible relabeling.
They provide the following additional actions on top of actions from the [Prometheus relabeling](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config):
* `replace_all`: replaces all of the occurences of `regex` in the values of `source_labels` with the `replacement` and stores the results in the `target_label`.
* `labelmap_all`: replaces all of the occurences of `regex` in all the label names with the `replacement`.
* `keep_if_equal`: keeps the entry if all the label values from `source_labels` are equal.
* `drop_if_equal`: drops the entry if all the label values from `source_labels` are equal.
* `keep_metrics`: keeps all the metrics with names matching the given `regex`.
* `drop_metrics`: drops all the metrics with names matching the given `regex`.
* `replace_all`: replaces all of the occurences of `regex` in the values of `source_labels` with the `replacement` and stores the results in the `target_label`
* `labelmap_all`: replaces all of the occurences of `regex` in all the label names with the `replacement`
* `keep_if_equal`: keeps the entry if all the label values from `source_labels` are equal
* `drop_if_equal`: drops the entry if all the label values from `source_labels` are equal
* `keep_metrics`: keeps all the metrics with names matching the given `regex`
* `drop_metrics`: drops all the metrics with names matching the given `regex`
* `graphite`: applies Graphite-style relabeling to metric name. See [these docs](#graphite-relabeling)
The `regex` value can be split into multiple lines for improved readability and maintainability. These lines are automatically joined with `|` char when parsed. For example, the following configs are equivalent:
@ -309,6 +310,38 @@ You can read more about relabeling in the following articles:
* [Extracting labels from legacy metric names](https://www.robustperception.io/extracting-labels-from-legacy-metric-names)
* [relabel_configs vs metric_relabel_configs](https://www.robustperception.io/relabel_configs-vs-metric_relabel_configs)
## Graphite relabeling
VictoriaMetrics components support `action: graphite` relabeling rules, which allow extracting various parts from Graphite-style metrics
into the configured labels with the syntax similar to [Glob matching in statsd_exporter](https://github.com/prometheus/statsd_exporter#glob-matching).
Note that the `name` field must be substituted with explicit `__name__` option under `labels` section.
If `__name__` option is missing under `labels` section, then the original Graphite-style metric name is left unchanged.
For example, the following relabeling rule generates `requests_total{job="app42",instance="host124:8080"}` metric
from "app42.host123.requests.total" Graphite-style metric:
```yaml
- action: graphite
match: "*.*.*.total"
labels:
__name__: "${3}_total"
job: "$1"
instance: "${2}:8080"
```
Important notes about `action: graphite` relabeling rules:
- The relabeling rule is applied only to metrics, which match the given `match` expression. Other metrics remain unchanged.
- The `*` matches the maximum possible number of chars until the next dot or until the next part of the `match` expression whichever comes first.
It may match zero chars if the next char is `.`.
For example, `match: "app*foo.bar"` matches `app42foo.bar` and `42` becomes available to use at `labels` section via `$1` capture group.
- The `$0` capture group matches the original metric name.
- The relabeling rules are executed in order defined in the original config.
The `action: graphite` relabeling rules are easier to write and maintain than `action: replace` for labels extraction from Graphite-style metric names.
Additionally, the `action: graphite` relabeling rules usually work much faster than the equivalent `action: replace` rules.
## Prometheus staleness markers
`vmagent` sends [Prometheus staleness markers](https://www.robustperception.io/staleness-and-promql) to `-remoteWrite.url` in the following cases:
@ -564,7 +597,7 @@ Every Kafka message may contain multiple lines in `influx`, `prometheus`, `graph
The following command starts `vmagent`, which reads metrics in InfluxDB line protocol format from Kafka broker at `localhost:9092` from the topic `metrics-by-telegraf` and sends them to remote storage at `http://localhost:8428/api/v1/write`:
```bash
```console
./bin/vmagent -remoteWrite.url=http://localhost:8428/api/v1/write \
-kafka.consumer.topic.brokers=localhost:9092 \
-kafka.consumer.topic.format=influx \
@ -626,13 +659,13 @@ Two types of auth are supported:
* sasl with username and password:
```bash
```console
./bin/vmagent -remoteWrite.url=kafka://localhost:9092/?topic=prom-rw&security.protocol=SASL_SSL&sasl.mechanisms=PLAIN -remoteWrite.basicAuth.username=user -remoteWrite.basicAuth.password=password
```
* tls certificates:
```bash
```console
./bin/vmagent -remoteWrite.url=kafka://localhost:9092/?topic=prom-rw&security.protocol=SSL -remoteWrite.tlsCAFile=/opt/ca.pem -remoteWrite.tlsCertFile=/opt/cert.pem -remoteWrite.tlsKeyFile=/opt/key.pem
```
@ -661,7 +694,7 @@ The `<PKG_TAG>` may be manually set via `PKG_TAG=foobar make package-vmagent`.
The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
by setting it via `<ROOT_IMAGE>` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-vmagent
```
@ -689,7 +722,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8429/debug/pprof/heap > mem.pprof
```
@ -699,7 +732,7 @@ curl http://0.0.0.0:8429/debug/pprof/heap > mem.pprof
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8429/debug/pprof/profile > cpu.pprof
```

24
docs/vmalert.md
View file

@ -40,7 +40,7 @@ implementation and aims to be compatible with its syntax.
To build `vmalert` from sources:
```bash
```console
git clone https://github.com/VictoriaMetrics/VictoriaMetrics
cd VictoriaMetrics
make vmalert
@ -56,12 +56,13 @@ To start using `vmalert` you will need the following things:
aggregating alerts, and sending notifications. Please note, notifier address also supports Consul and DNS Service Discovery via
[config file](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmalert/notifier/config.go).
* remote write address [optional] - [remote write](https://prometheus.io/docs/prometheus/latest/storage/#remote-storage-integrations)
compatible storage to persist rules and alerts state info;
compatible storage to persist rules and alerts state info. To persist results to multiple destinations use vmagent
configured with multiple remote writes as a proxy;
* remote read address [optional] - MetricsQL compatible datasource to restore alerts state from.
Then configure `vmalert` accordingly:
```bash
```console
./bin/vmalert -rule=alert.rules \ # Path to the file with rules configuration. Supports wildcard
-datasource.url=http://localhost:8428 \ # PromQL compatible datasource
-notifier.url=http://localhost:9093 \ # AlertManager URL (required if alerting rules are used)
@ -428,6 +429,21 @@ Flags `-remoteRead.url` and `-notifier.url` are omitted since we assume only rec
See also [downsampling docs](https://docs.victoriametrics.com/#downsampling).
#### Multiple remote writes
For persisting recording or alerting rule results `vmalert` requires `-remoteWrite.url` to be set.
But this flag supports only one destination. To persist rule results to multiple destinations
we recommend using [vmagent](https://docs.victoriametrics.com/vmagent.html) as fan-out proxy:
<img alt="vmalert multiple remote write destinations" src="vmalert_multiple_rw.png">
In this topology, `vmalert` is configured to persist rule results to `vmagent`. And `vmagent`
is configured to fan-out received data to two or more destinations.
Using `vmagent` as a proxy provides additional benefits such as
[data persisting when storage is unreachable](https://docs.victoriametrics.com/vmagent.html#replication-and-high-availability),
or time series modification via [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling).
### Web
`vmalert` runs a web-server (`-httpListenAddr`) for serving metrics and alerts endpoints:
@ -1026,7 +1042,7 @@ It is recommended using
You can build `vmalert` docker image from source and push it to your own docker repository.
Run the following commands from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics):
```bash
```console
make package-vmalert
docker tag victoria-metrics/vmalert:version my-repo:my-version-name
docker push my-repo:my-version-name

BIN
docs/vmalert_multiple_rw.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 80 KiB

12
docs/vmauth.md
View file

@ -14,7 +14,7 @@ The `-auth.config` can point to either local file or to http url.
Just download `vmutils-*` archive from [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases), unpack it
and pass the following flag to `vmauth` binary in order to start authorizing and routing requests:
```bash
```console
/path/to/vmauth -auth.config=/path/to/auth/config.yml
```
@ -133,7 +133,7 @@ It is expected that all the backend services protected by `vmauth` are located i
Do not transfer Basic Auth headers in plaintext over untrusted networks. Enable https. This can be done by passing the following `-tls*` command-line flags to `vmauth`:
```bash
```console
-tls
Whether to enable TLS (aka HTTPS) for incoming requests. -tlsCertFile and -tlsKeyFile must be set if -tls is set
-tlsCertFile string
@ -185,7 +185,7 @@ The `<PKG_TAG>` may be manually set via `PKG_TAG=foobar make package-vmauth`.
The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
by setting it via `<ROOT_IMAGE>` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-vmauth
```
@ -197,7 +197,7 @@ ROOT_IMAGE=scratch make package-vmauth
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8427/debug/pprof/heap > mem.pprof
```
@ -207,7 +207,7 @@ curl http://0.0.0.0:8427/debug/pprof/heap > mem.pprof
<div class="with-copy" markdown="1">
```bash
```console
curl http://0.0.0.0:8427/debug/pprof/profile > cpu.pprof
```
@ -221,7 +221,7 @@ The collected profiles may be analyzed with [go tool pprof](https://github.com/g
Pass `-help` command-line arg to `vmauth` in order to see all the configuration options:
```bash
```console
./vmauth -help
vmauth authenticates and authorizes incoming requests and proxies them to VictoriaMetrics.

18
docs/vmbackup.md
View file

@ -32,7 +32,7 @@ creation of hourly, daily, weekly and monthly backups.
Regular backup can be performed with the following command:
```bash
```console
vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs://<bucket>/<path/to/new/backup>
```
@ -47,7 +47,7 @@ vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=h
If the destination GCS bucket already contains the previous backup at `-origin` path, then new backup can be sped up
with the following command:
```bash
```console
./vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs://<bucket>/<path/to/new/backup> -origin=gs://<bucket>/<path/to/existing/backup>
```
@ -58,7 +58,7 @@ It saves time and network bandwidth costs by performing server-side copy for the
Incremental backups are performed if `-dst` points to an already existing backup. In this case only new data is uploaded to remote storage.
It saves time and network bandwidth costs when working with big backups:
```bash
```console
./vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs://<bucket>/<path/to/existing/backup>
```
@ -68,7 +68,7 @@ Smart backups mean storing full daily backups into `YYYYMMDD` folders and creati
* Run the following command every hour:
```bash
```console
./vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs://<bucket>/latest
```
@ -77,7 +77,7 @@ The command will upload only changed data to `gs://<bucket>/latest`.
* Run the following command once a day:
```bash
```console
vmbackup -storageDataPath=</path/to/victoria-metrics-data> -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs://<bucket>/<YYYYMMDD> -origin=gs://<bucket>/latest
```
@ -133,7 +133,7 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-
for s3 (aws, minio or other s3 compatible storages):
```bash
```console
[default]
aws_access_key_id=theaccesskey
aws_secret_access_key=thesecretaccesskeyvalue
@ -159,7 +159,7 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-
* Usage with s3 custom url endpoint. It is possible to use `vmbackup` with s3 compatible storages like minio, cloudian, etc.
You have to add a custom url endpoint via flag:
```bash
```console
# for minio
-customS3Endpoint=http://localhost:9000
@ -169,7 +169,7 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-
* Run `vmbackup -help` in order to see all the available options:
```bash
```console
-concurrency int
The number of concurrent workers. Higher concurrency may reduce backup duration (default 10)
-configFilePath string
@ -284,6 +284,6 @@ The `<PKG_TAG>` may be manually set via `PKG_TAG=foobar make package-vmbackup`.
The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
by setting it via `<ROOT_IMAGE>` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-vmbackup
```

6
docs/vmctl.md
View file

@ -19,7 +19,7 @@ Features:
To see the full list of supported modes
run the following command:
```bash
```console
$ ./vmctl --help
NAME:
vmctl - VictoriaMetrics command-line tool
@ -531,7 +531,7 @@ and specify `accountID` param.
In this mode, `vmctl` allows verifying correctness and integrity of data exported via [native format](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-export-data-in-native-format) from VictoriaMetrics.
You can verify exported data at disk before uploading it by `vmctl verify-block` command:
```bash
```console
# export blocks from VictoriaMetrics
curl localhost:8428/api/v1/export/native -g -d 'match[]={__name__!=""}' -o exported_data_block
# verify block content
@ -654,7 +654,7 @@ The `<PKG_TAG>` may be manually set via `PKG_TAG=foobar make package-vmctl`.
The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
by setting it via `<ROOT_IMAGE>` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-vmctl
```

10
docs/vmgateway.md
View file

@ -58,7 +58,7 @@ Where:
Start the single version of VictoriaMetrics
```bash
```console
# single
# start node
./bin/victoria-metrics --selfScrapeInterval=10s
@ -66,19 +66,19 @@ Start the single version of VictoriaMetrics
Start vmgateway
```bash
```console
./bin/vmgateway -eula -enable.auth -read.url http://localhost:8428 --write.url http://localhost:8428
```
Retrieve data from the database
```bash
```console
curl 'http://localhost:8431/api/v1/series/count' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ2bV9hY2Nlc3MiOnsidGVuYW50X2lkIjp7fSwicm9sZSI6MX0sImV4cCI6MTkzOTM0NjIxMH0.5WUxEfdcV9hKo4CtQdtuZYOGpGXWwaqM9VuVivMMrVg'
```
A request with an incorrect token or without any token will be rejected:
```bash
```console
curl 'http://localhost:8431/api/v1/series/count'
curl 'http://localhost:8431/api/v1/series/count' -H 'Authorization: Bearer incorrect-token'
@ -128,7 +128,7 @@ limits:
cluster version of VictoriaMetrics is required for rate limiting.
```bash
```console
# start datasource for cluster metrics
cat << EOF > cluster.yaml

10
docs/vmrestore.md
View file

@ -14,7 +14,7 @@ when restarting `vmrestore` with the same args.
VictoriaMetrics must be stopped during the restore process.
```bash
```console
vmrestore -src=gs://<bucket>/<path/to/backup> -storageDataPath=<local/path/to/restore>
```
@ -40,7 +40,7 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q
for s3 (aws, minio or other s3 compatible storages):
```bash
```console
[default]
aws_access_key_id=theaccesskey
aws_secret_access_key=thesecretaccesskeyvalue
@ -66,7 +66,7 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q
* Usage with s3 custom url endpoint. It is possible to use `vmrestore` with s3 api compatible storages, like minio, cloudian and other.
You have to add custom url endpoint with a flag:
```bash
```console
# for minio:
-customS3Endpoint=http://localhost:9000
@ -76,7 +76,7 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q
* Run `vmrestore -help` in order to see all the available options:
```bash
```console
-concurrency int
The number of concurrent workers. Higher concurrency may reduce restore duration (default 10)
-configFilePath string
@ -184,6 +184,6 @@ The `<PKG_TAG>` may be manually set via `PKG_TAG=foobar make package-vmrestore`.
The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
by setting it via `<ROOT_IMAGE>` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
```bash
```console
ROOT_IMAGE=scratch make package-vmrestore
```

62
lib/promrelabel/config.go
View file

@ -23,6 +23,22 @@ type RelabelConfig struct {
Replacement *string `yaml:"replacement,omitempty"`
Action string `yaml:"action,omitempty"`
If *IfExpression `yaml:"if,omitempty"`
// Match is used together with Labels for `action: graphite`. For example:
// - action: graphite
// match: 'foo.*.*.bar'
// labels:
// job: '$1'
// instance: '${2}:8080'
Match string `yaml:"match,omitempty"`
// Labels is used together with Match for `action: graphite`. For example:
// - action: graphite
// match: 'foo.*.*.bar'
// labels:
// job: '$1'
// instance: '${2}:8080'
Labels map[string]string `yaml:"labels,omitempty"`
}
// MultiLineRegex contains a regex, which can be split into multiple lines.
@ -114,12 +130,12 @@ func (pcs *ParsedConfigs) String() string {
if pcs == nil {
return ""
}
var sb strings.Builder
var a []string
for _, prc := range pcs.prcs {
fmt.Fprintf(&sb, "%s,", prc.String())
s := "[" + prc.String() + "]"
a = append(a, s)
}
fmt.Fprintf(&sb, "relabelDebug=%v", pcs.relabelDebug)
return sb.String()
return fmt.Sprintf("%s, relabelDebug=%v", strings.Join(a, ","), pcs.relabelDebug)
}
// LoadRelabelConfigs loads relabel configs from the given path.
@ -200,11 +216,38 @@ func parseRelabelConfig(rc *RelabelConfig) (*parsedRelabelConfig, error) {
if rc.Replacement != nil {
replacement = *rc.Replacement
}
var graphiteMatchTemplate *graphiteMatchTemplate
if rc.Match != "" {
graphiteMatchTemplate = newGraphiteMatchTemplate(rc.Match)
}
var graphiteLabelRules []graphiteLabelRule
if rc.Labels != nil {
graphiteLabelRules = newGraphiteLabelRules(rc.Labels)
}
action := rc.Action
if action == "" {
action = "replace"
}
switch action {
case "graphite":
if graphiteMatchTemplate == nil {
return nil, fmt.Errorf("missing `match` for `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
}
if len(graphiteLabelRules) == 0 {
return nil, fmt.Errorf("missing `labels` for `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
}
if len(rc.SourceLabels) > 0 {
return nil, fmt.Errorf("`source_labels` cannot be used with `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
}
if rc.TargetLabel != "" {
return nil, fmt.Errorf("`target_label` cannot be used with `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
}
if rc.Replacement != nil {
return nil, fmt.Errorf("`replacement` cannot be used with `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
}
if rc.Regex != nil {
return nil, fmt.Errorf("`regex` cannot be used with `action=graphite`; see https://docs.victoriametrics.com/vmagent.html#graphite-relabeling")
}
case "replace":
if targetLabel == "" {
return nil, fmt.Errorf("missing `target_label` for `action=replace`")
@ -274,6 +317,14 @@ func parseRelabelConfig(rc *RelabelConfig) (*parsedRelabelConfig, error) {
default:
return nil, fmt.Errorf("unknown `action` %q", action)
}
if action != "graphite" {
if graphiteMatchTemplate != nil {
return nil, fmt.Errorf("`match` config cannot be applied to `action=%s`; it is applied only to `action=graphite`", action)
}
if len(graphiteLabelRules) > 0 {
return nil, fmt.Errorf("`labels` config cannot be applied to `action=%s`; it is applied only to `action=graphite`", action)
}
}
return &parsedRelabelConfig{
SourceLabels: sourceLabels,
Separator: separator,
@ -284,6 +335,9 @@ func parseRelabelConfig(rc *RelabelConfig) (*parsedRelabelConfig, error) {
Action: action,
If: rc.If,
graphiteMatchTemplate: graphiteMatchTemplate,
graphiteLabelRules: graphiteLabelRules,
regexOriginal: regexOriginalCompiled,
hasCaptureGroupInTargetLabel: strings.Contains(targetLabel, "$"),
hasCaptureGroupInReplacement: strings.Contains(replacement, "$"),

163
lib/promrelabel/config_test.go
View file

@ -45,6 +45,13 @@ func TestRelabelConfigMarshalUnmarshal(t *testing.T) {
- null
- nan
`, "- regex:\n - \"-1.23\"\n - \"false\"\n - \"null\"\n - nan\n")
f(`
- action: graphite
match: 'foo.*.*.aaa'
labels:
instance: '$1-abc'
job: '${2}'
`, "- action: graphite\n match: foo.*.*.aaa\n labels:\n instance: $1-abc\n job: ${2}\n")
}
func TestLoadRelabelConfigsSuccess(t *testing.T) {
@ -53,8 +60,9 @@ func TestLoadRelabelConfigsSuccess(t *testing.T) {
if err != nil {
t.Fatalf("cannot load relabel configs from %q: %s", path, err)
}
if n := pcs.Len(); n != 14 {
t.Fatalf("unexpected number of relabel configs loaded from %q; got %d; want %d", path, n, 14)
nExpected := 16
if n := pcs.Len(); n != nExpected {
t.Fatalf("unexpected number of relabel configs loaded from %q; got %d; want %d", path, n, nExpected)
}
}
@ -77,6 +85,51 @@ func TestLoadRelabelConfigsFailure(t *testing.T) {
})
}
func TestParsedConfigsString(t *testing.T) {
f := func(rcs []RelabelConfig, sExpected string) {
t.Helper()
pcs, err := ParseRelabelConfigs(rcs, false)
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
s := pcs.String()
if s != sExpected {
t.Fatalf("unexpected string representation for ParsedConfigs;\ngot\n%s\nwant\n%s", s, sExpected)
}
}
f([]RelabelConfig{
{
TargetLabel: "foo",
SourceLabels: []string{"aaa"},
},
}, "[SourceLabels=[aaa], Separator=;, TargetLabel=foo, Regex=^(.*)$, Modulus=0, Replacement=$1, Action=replace, If=, "+
"graphiteMatchTemplate=<nil>, graphiteLabelRules=[]], relabelDebug=false")
var ie IfExpression
if err := ie.Parse("{foo=~'bar'}"); err != nil {
t.Fatalf("unexpected error when parsing if expression: %s", err)
}
f([]RelabelConfig{
{
Action: "graphite",
Match: "foo.*.bar",
Labels: map[string]string{
"job": "$1-zz",
},
If: &ie,
},
}, "[SourceLabels=[], Separator=;, TargetLabel=, Regex=^(.*)$, Modulus=0, Replacement=$1, Action=graphite, If={foo=~'bar'}, "+
"graphiteMatchTemplate=foo.*.bar, graphiteLabelRules=[replaceTemplate=$1-zz, targetLabel=job]], relabelDebug=false")
f([]RelabelConfig{
{
Action: "replace",
SourceLabels: []string{"foo", "bar"},
TargetLabel: "x",
If: &ie,
},
}, "[SourceLabels=[foo bar], Separator=;, TargetLabel=x, Regex=^(.*)$, Modulus=0, Replacement=$1, Action=replace, If={foo=~'bar'}, "+
"graphiteMatchTemplate=<nil>, graphiteLabelRules=[]], relabelDebug=false")
}
func TestParseRelabelConfigsSuccess(t *testing.T) {
f := func(rcs []RelabelConfig, pcsExpected *ParsedConfigs) {
t.Helper()
@ -271,4 +324,110 @@ func TestParseRelabelConfigsFailure(t *testing.T) {
},
})
})
t.Run("uppercase-missing-sourceLabels", func(t *testing.T) {
f([]RelabelConfig{
{
Action: "uppercase",
TargetLabel: "foobar",
},
})
})
t.Run("lowercase-missing-targetLabel", func(t *testing.T) {
f([]RelabelConfig{
{
Action: "lowercase",
SourceLabels: []string{"foobar"},
},
})
})
t.Run("graphite-missing-match", func(t *testing.T) {
f([]RelabelConfig{
{
Action: "graphite",
Labels: map[string]string{
"foo": "bar",
},
},
})
})
t.Run("graphite-missing-labels", func(t *testing.T) {
f([]RelabelConfig{
{
Action: "graphite",
Match: "foo.*.bar",
},
})
})
t.Run("graphite-superflouous-sourceLabels", func(t *testing.T) {
f([]RelabelConfig{
{
Action: "graphite",
Match: "foo.*.bar",
Labels: map[string]string{
"foo": "bar",
},
SourceLabels: []string{"foo"},
},
})
})
t.Run("graphite-superflouous-targetLabel", func(t *testing.T) {
f([]RelabelConfig{
{
Action: "graphite",
Match: "foo.*.bar",
Labels: map[string]string{
"foo": "bar",
},
TargetLabel: "foo",
},
})
})
replacement := "foo"
t.Run("graphite-superflouous-replacement", func(t *testing.T) {
f([]RelabelConfig{
{
Action: "graphite",
Match: "foo.*.bar",
Labels: map[string]string{
"foo": "bar",
},
Replacement: &replacement,
},
})
})
var re MultiLineRegex
t.Run("graphite-superflouous-regex", func(t *testing.T) {
f([]RelabelConfig{
{
Action: "graphite",
Match: "foo.*.bar",
Labels: map[string]string{
"foo": "bar",
},
Regex: &re,
},
})
})
t.Run("non-graphite-superflouos-match", func(t *testing.T) {
f([]RelabelConfig{
{
Action: "uppercase",
SourceLabels: []string{"foo"},
TargetLabel: "foo",
Match: "aaa",
},
})
})
t.Run("non-graphite-superflouos-labels", func(t *testing.T) {
f([]RelabelConfig{
{
Action: "uppercase",
SourceLabels: []string{"foo"},
TargetLabel: "foo",
Labels: map[string]string{
"foo": "Bar",
},
},
})
})
}

212
lib/promrelabel/graphite.go Normal file
View file

@ -0,0 +1,212 @@
package promrelabel
import (
"fmt"
"strconv"
"strings"
"sync"
)
var graphiteMatchesPool = &sync.Pool{
New: func() interface{} {
return &graphiteMatches{}
},
}
type graphiteMatches struct {
a []string
}
type graphiteMatchTemplate struct {
sOrig string
parts []string
}
func (gmt *graphiteMatchTemplate) String() string {
return gmt.sOrig
}
type graphiteLabelRule struct {
grt *graphiteReplaceTemplate
targetLabel string
}
func (glr graphiteLabelRule) String() string {
return fmt.Sprintf("replaceTemplate=%s, targetLabel=%s", glr.grt, glr.targetLabel)
}
func newGraphiteLabelRules(m map[string]string) []graphiteLabelRule {
a := make([]graphiteLabelRule, 0, len(m))
for labelName, replaceTemplate := range m {
a = append(a, graphiteLabelRule{
grt: newGraphiteReplaceTemplate(replaceTemplate),
targetLabel: labelName,
})
}
return a
}
func newGraphiteMatchTemplate(s string) *graphiteMatchTemplate {
sOrig := s
var parts []string
for {
n := strings.IndexByte(s, '*')
if n < 0 {
parts = appendGraphiteMatchTemplateParts(parts, s)
break
}
parts = appendGraphiteMatchTemplateParts(parts, s[:n])
parts = appendGraphiteMatchTemplateParts(parts, "*")
s = s[n+1:]
}
return &graphiteMatchTemplate{
sOrig: sOrig,
parts: parts,
}
}
func appendGraphiteMatchTemplateParts(dst []string, s string) []string {
if len(s) == 0 {
// Skip empty part
return dst
}
return append(dst, s)
}
// Match matches s against gmt.
//
// On success it adds matched captures to dst and returns it with true.
// Of failre it returns false.
func (gmt *graphiteMatchTemplate) Match(dst []string, s string) ([]string, bool) {
dst = append(dst, s)
parts := gmt.parts
if len(parts) > 0 {
if p := parts[len(parts)-1]; p != "*" && !strings.HasSuffix(s, p) {
// fast path - suffix mismatch
return dst, false
}
}
for i := 0; i < len(parts); i++ {
p := parts[i]
if p != "*" {
if !strings.HasPrefix(s, p) {
// Cannot match the current part
return dst, false
}
s = s[len(p):]
continue
}
// Search for the matching substring for '*' part.
if i+1 >= len(parts) {
// Matching the last part.
if strings.IndexByte(s, '.') >= 0 {
// The '*' cannot match string with dots.
return dst, false
}
dst = append(dst, s)
return dst, true
}
// Search for the the start of the next part.
p = parts[i+1]
i++
n := strings.Index(s, p)
if n < 0 {
// Cannot match the next part
return dst, false
}
tmp := s[:n]
if strings.IndexByte(tmp, '.') >= 0 {
// The '*' cannot match string with dots.
return dst, false
}
dst = append(dst, tmp)
s = s[n+len(p):]
}
return dst, len(s) == 0
}
type graphiteReplaceTemplate struct {
sOrig string
parts []graphiteReplaceTemplatePart
}
func (grt *graphiteReplaceTemplate) String() string {
return grt.sOrig
}
type graphiteReplaceTemplatePart struct {
n int
s string
}
func newGraphiteReplaceTemplate(s string) *graphiteReplaceTemplate {
sOrig := s
var parts []graphiteReplaceTemplatePart
for {
n := strings.IndexByte(s, '$')
if n < 0 {
parts = appendGraphiteReplaceTemplateParts(parts, s, -1)
break
}
if n > 0 {
parts = appendGraphiteReplaceTemplateParts(parts, s[:n], -1)
}
s = s[n+1:]
if len(s) > 0 && s[0] == '{' {
// The index in the form ${123}
n = strings.IndexByte(s, '}')
if n < 0 {
parts = appendGraphiteReplaceTemplateParts(parts, "$"+s, -1)
break
}
idxStr := s[1:n]
s = s[n+1:]
idx, err := strconv.Atoi(idxStr)
if err != nil {
parts = appendGraphiteReplaceTemplateParts(parts, "${"+idxStr+"}", -1)
} else {
parts = appendGraphiteReplaceTemplateParts(parts, "${"+idxStr+"}", idx)
}
} else {
// The index in the form $123
n := 0
for n < len(s) && s[n] >= '0' && s[n] <= '9' {
n++
}
idxStr := s[:n]
s = s[n:]
idx, err := strconv.Atoi(idxStr)
if err != nil {
parts = appendGraphiteReplaceTemplateParts(parts, "$"+idxStr, -1)
} else {
parts = appendGraphiteReplaceTemplateParts(parts, "$"+idxStr, idx)
}
}
}
return &graphiteReplaceTemplate{
sOrig: sOrig,
parts: parts,
}
}
// Expand expands grt with the given matches into dst and returns it.
func (grt *graphiteReplaceTemplate) Expand(dst []byte, matches []string) []byte {
for _, part := range grt.parts {
if n := part.n; n >= 0 && n < len(matches) {
dst = append(dst, matches[n]...)
} else {
dst = append(dst, part.s...)
}
}
return dst
}
func appendGraphiteReplaceTemplateParts(dst []graphiteReplaceTemplatePart, s string, n int) []graphiteReplaceTemplatePart {
if len(s) > 0 {
dst = append(dst, graphiteReplaceTemplatePart{
s: s,
n: n,
})
}
return dst
}

93
lib/promrelabel/graphite_test.go Normal file
View file

@ -0,0 +1,93 @@
package promrelabel
import (
"reflect"
"testing"
)
func TestGraphiteTemplateMatchExpand(t *testing.T) {
f := func(matchTpl, s, replaceTpl, resultExpected string) {
t.Helper()
gmt := newGraphiteMatchTemplate(matchTpl)
matches, ok := gmt.Match(nil, s)
if !ok {
matches = nil
}
grt := newGraphiteReplaceTemplate(replaceTpl)
result := grt.Expand(nil, matches)
if string(result) != resultExpected {
t.Fatalf("unexpected result; got %q; want %q", result, resultExpected)
}
}
f("", "", "", "")
f("test.*.*.counter", "test.foo.bar.counter", "${2}_total", "bar_total")
f("test.*.*.counter", "test.foo.bar.counter", "$1_total", "foo_total")
f("test.*.*.counter", "test.foo.bar.counter", "total_$0", "total_test.foo.bar.counter")
f("test.dispatcher.*.*.*", "test.dispatcher.foo.bar.baz", "$3-$2-$1", "baz-bar-foo")
f("*.signup.*.*", "foo.signup.bar.baz", "$1-${3}_$2_total", "foo-baz_bar_total")
}
func TestGraphiteMatchTemplateMatch(t *testing.T) {
f := func(tpl, s string, matchesExpected []string, okExpected bool) {
t.Helper()
gmt := newGraphiteMatchTemplate(tpl)
tplGot := gmt.String()
if tplGot != tpl {
t.Fatalf("unexpected template; got %q; want %q", tplGot, tpl)
}
matches, ok := gmt.Match(nil, s)
if ok != okExpected {
t.Fatalf("unexpected ok result for tpl=%q, s=%q; got %v; want %v", tpl, s, ok, okExpected)
}
if okExpected {
if !reflect.DeepEqual(matches, matchesExpected) {
t.Fatalf("unexpected matches for tpl=%q, s=%q; got\n%q\nwant\n%q\ngraphiteMatchTemplate=%v", tpl, s, matches, matchesExpected, gmt)
}
}
}
f("", "", []string{""}, true)
f("", "foobar", nil, false)
f("foo", "foo", []string{"foo"}, true)
f("foo", "", nil, false)
f("foo.bar.baz", "foo.bar.baz", []string{"foo.bar.baz"}, true)
f("*", "foobar", []string{"foobar", "foobar"}, true)
f("**", "foobar", nil, false)
f("*", "foo.bar", nil, false)
f("*foo", "barfoo", []string{"barfoo", "bar"}, true)
f("*foo", "foo", []string{"foo", ""}, true)
f("*foo", "bar.foo", nil, false)
f("foo*", "foobar", []string{"foobar", "bar"}, true)
f("foo*", "foo", []string{"foo", ""}, true)
f("foo*", "foo.bar", nil, false)
f("foo.*", "foobar", nil, false)
f("foo.*", "foo.bar", []string{"foo.bar", "bar"}, true)
f("foo.*", "foo.bar.baz", nil, false)
f("*.*.baz", "foo.bar.baz", []string{"foo.bar.baz", "foo", "bar"}, true)
f("*.bar", "foo.bar.baz", nil, false)
f("*.bar", "foo.baz", nil, false)
}
func TestGraphiteReplaceTemplateExpand(t *testing.T) {
f := func(tpl string, matches []string, resultExpected string) {
t.Helper()
grt := newGraphiteReplaceTemplate(tpl)
tplGot := grt.String()
if tplGot != tpl {
t.Fatalf("unexpected template; got %q; want %q", tplGot, tpl)
}
result := grt.Expand(nil, matches)
if string(result) != resultExpected {
t.Fatalf("unexpected result for tpl=%q; got\n%q\nwant\n%q\ngraphiteReplaceTemplate=%v", tpl, result, resultExpected, grt)
}
}
f("", nil, "")
f("foo", nil, "foo")
f("$", nil, "$")
f("$1", nil, "$1")
f("${123", nil, "${123")
f("${123}", nil, "${123}")
f("${foo}45$sdf$3", nil, "${foo}45$sdf$3")
f("$1", []string{"foo", "bar"}, "bar")
f("$0-$1", []string{"foo", "bar"}, "foo-bar")
f("x-${0}-$1", []string{"foo", "bar"}, "x-foo-bar")
}

93
lib/promrelabel/graphite_timing_test.go Normal file
View file

@ -0,0 +1,93 @@
package promrelabel
import (
"fmt"
"testing"
)
func BenchmarkGraphiteMatchTemplateMatch(b *testing.B) {
b.Run("match-short", func(b *testing.B) {
tpl := "*.bar.baz"
s := "foo.bar.baz"
benchmarkGraphiteMatchTemplateMatch(b, tpl, s, true)
})
b.Run("mismtach-short", func(b *testing.B) {
tpl := "*.bar.baz"
s := "foo.aaa"
benchmarkGraphiteMatchTemplateMatch(b, tpl, s, false)
})
b.Run("match-long", func(b *testing.B) {
tpl := "*.*.*.bar.*.baz"
s := "foo.bar.baz.bar.aa.baz"
benchmarkGraphiteMatchTemplateMatch(b, tpl, s, true)
})
b.Run("mismatch-long", func(b *testing.B) {
tpl := "*.*.*.bar.*.baz"
s := "foo.bar.baz.bar.aa.bb"
benchmarkGraphiteMatchTemplateMatch(b, tpl, s, false)
})
}
func benchmarkGraphiteMatchTemplateMatch(b *testing.B, tpl, s string, okExpected bool) {
gmt := newGraphiteMatchTemplate(tpl)
b.ReportAllocs()
b.SetBytes(1)
b.RunParallel(func(pb *testing.PB) {
var matches []string
for pb.Next() {
var ok bool
matches, ok = gmt.Match(matches[:0], s)
if ok != okExpected {
panic(fmt.Errorf("unexpected ok=%v for tpl=%q, s=%q", ok, tpl, s))
}
}
})
}
func BenchmarkGraphiteReplaceTemplateExpand(b *testing.B) {
b.Run("one-replacement", func(b *testing.B) {
tpl := "$1"
matches := []string{"", "foo"}
resultExpected := "foo"
benchmarkGraphiteReplaceTemplateExpand(b, tpl, matches, resultExpected)
})
b.Run("one-replacement-with-prefix", func(b *testing.B) {
tpl := "x-$1"
matches := []string{"", "foo"}
resultExpected := "x-foo"
benchmarkGraphiteReplaceTemplateExpand(b, tpl, matches, resultExpected)
})
b.Run("one-replacement-with-prefix-suffix", func(b *testing.B) {
tpl := "x-$1-y"
matches := []string{"", "foo"}
resultExpected := "x-foo-y"
benchmarkGraphiteReplaceTemplateExpand(b, tpl, matches, resultExpected)
})
b.Run("two-replacements", func(b *testing.B) {
tpl := "$1$2"
matches := []string{"", "foo", "bar"}
resultExpected := "foobar"
benchmarkGraphiteReplaceTemplateExpand(b, tpl, matches, resultExpected)
})
b.Run("two-replacements-with-delimiter", func(b *testing.B) {
tpl := "$1-$2"
matches := []string{"", "foo", "bar"}
resultExpected := "foo-bar"
benchmarkGraphiteReplaceTemplateExpand(b, tpl, matches, resultExpected)
})
}
func benchmarkGraphiteReplaceTemplateExpand(b *testing.B, tpl string, matches []string, resultExpected string) {
grt := newGraphiteReplaceTemplate(tpl)
b.ReportAllocs()
b.SetBytes(1)
b.RunParallel(func(pb *testing.PB) {
var b []byte
for pb.Next() {
b = grt.Expand(b[:0], matches)
if string(b) != resultExpected {
panic(fmt.Errorf("unexpected result; got\n%q\nwant\n%q", b, resultExpected))
}
}
})
}

8
lib/promrelabel/if_expression.go
View file

@ -18,6 +18,14 @@ type IfExpression struct {
lfs []*labelFilter
}
// String returns string representation of ie.
func (ie *IfExpression) String() string {
if ie == nil {
return ""
}
return ie.s
}
// Parse parses `if` expression from s and stores it to ie.
func (ie *IfExpression) Parse(s string) error {
expr, err := metricsql.Parse(s)

31
lib/promrelabel/if_expression_test.go
View file

@ -2,6 +2,7 @@ package promrelabel
import (
"bytes"
"encoding/json"
"fmt"
"testing"
@ -36,6 +37,36 @@ func TestIfExpressionParseSuccess(t *testing.T) {
f(`foo{bar=~"baz", x!="y"}`)
}
func TestIfExpressionMarshalUnmarshalJSON(t *testing.T) {
f := func(s, jsonExpected string) {
t.Helper()
var ie IfExpression
if err := ie.Parse(s); err != nil {
t.Fatalf("cannot parse ifExpression %q: %s", s, err)
}
data, err := json.Marshal(&ie)
if err != nil {
t.Fatalf("cannot marshal ifExpression %q: %s", s, err)
}
if string(data) != jsonExpected {
t.Fatalf("unexpected value after json marshaling;\ngot\n%s\nwant\n%s", data, jsonExpected)
}
var ie2 IfExpression
if err := json.Unmarshal(data, &ie2); err != nil {
t.Fatalf("cannot unmarshal ifExpression from json %q: %s", data, err)
}
data2, err := json.Marshal(&ie2)
if err != nil {
t.Fatalf("cannot marshal ifExpression2: %s", err)
}
if string(data2) != jsonExpected {
t.Fatalf("unexpected data after unmarshal/marshal cycle;\ngot\n%s\nwant\n%s", data2, jsonExpected)
}
}
f("foo", `"foo"`)
f(`{foo="bar",baz=~"x.*"}`, `"{foo=\"bar\",baz=~\"x.*\"}"`)
}
func TestIfExpressionUnmarshalFailure(t *testing.T) {
f := func(s string) {
t.Helper()

27
lib/promrelabel/relabel.go
View file

@ -25,6 +25,9 @@ type parsedRelabelConfig struct {
Action string
If *IfExpression
graphiteMatchTemplate *graphiteMatchTemplate
graphiteLabelRules []graphiteLabelRule
regexOriginal *regexp.Regexp
hasCaptureGroupInTargetLabel bool
hasCaptureGroupInReplacement bool
@ -32,8 +35,8 @@ type parsedRelabelConfig struct {
// String returns human-readable representation for prc.
func (prc *parsedRelabelConfig) String() string {
return fmt.Sprintf("SourceLabels=%s, Separator=%s, TargetLabel=%s, Regex=%s, Modulus=%d, Replacement=%s, Action=%s",
prc.SourceLabels, prc.Separator, prc.TargetLabel, prc.Regex.String(), prc.Modulus, prc.Replacement, prc.Action)
return fmt.Sprintf("SourceLabels=%s, Separator=%s, TargetLabel=%s, Regex=%s, Modulus=%d, Replacement=%s, Action=%s, If=%s, graphiteMatchTemplate=%s, graphiteLabelRules=%s",
prc.SourceLabels, prc.Separator, prc.TargetLabel, prc.Regex, prc.Modulus, prc.Replacement, prc.Action, prc.If, prc.graphiteMatchTemplate, prc.graphiteLabelRules)
}
// Apply applies pcs to labels starting from the labelsOffset.
@ -147,6 +150,26 @@ func (prc *parsedRelabelConfig) apply(labels []prompbmarshal.Label, labelsOffset
return labels
}
switch prc.Action {
case "graphite":
metricName := GetLabelValueByName(src, "__name__")
gm := graphiteMatchesPool.Get().(*graphiteMatches)
var ok bool
gm.a, ok = prc.graphiteMatchTemplate.Match(gm.a[:0], metricName)
if !ok {
// Fast path - name mismatch
graphiteMatchesPool.Put(gm)
return labels
}
// Slow path - extract labels from graphite metric name
bb := relabelBufPool.Get()
for _, gl := range prc.graphiteLabelRules {
bb.B = gl.grt.Expand(bb.B[:0], gm.a)
valueStr := string(bb.B)
labels = setLabelValue(labels, labelsOffset, gl.targetLabel, valueStr)
}
relabelBufPool.Put(bb)
graphiteMatchesPool.Put(gm)
return labels
case "replace":
// Store `replacement` at `target_label` if the `regex` matches `source_labels` joined with `separator`
bb := relabelBufPool.Get()

45
lib/promrelabel/relabel_test.go
View file

@ -1580,7 +1580,6 @@ func TestApplyRelabelConfigs(t *testing.T) {
},
})
})
t.Run("upper-lower-case", func(t *testing.T) {
f(`
- action: uppercase
@ -1618,7 +1617,6 @@ func TestApplyRelabelConfigs(t *testing.T) {
Value: "bar;foo",
},
})
})
f(`
- action: lowercase
source_labels: ["foo"]
@ -1637,6 +1635,49 @@ func TestApplyRelabelConfigs(t *testing.T) {
Value: "quux",
},
})
})
t.Run("graphite-match", func(t *testing.T) {
f(`
- action: graphite
match: foo.*.baz
labels:
__name__: aaa
job: ${1}-zz
`, []prompbmarshal.Label{
{
Name: "__name__",
Value: "foo.bar.baz",
},
}, true, []prompbmarshal.Label{
{
Name: "__name__",
Value: "aaa",
},
{
Name: "job",
Value: "bar-zz",
},
})
})
t.Run("graphite-mismatch", func(t *testing.T) {
f(`
- action: graphite
match: foo.*.baz
labels:
__name__: aaa
job: ${1}-zz
`, []prompbmarshal.Label{
{
Name: "__name__",
Value: "foo.bar.bazz",
},
}, true, []prompbmarshal.Label{
{
Name: "__name__",
Value: "foo.bar.bazz",
},
})
})
}
func TestFinalizeLabels(t *testing.T) {

8
lib/promrelabel/testdata/relabel_configs_valid.yml vendored
View file

@ -39,3 +39,11 @@
- source_labels: [__tmp_uppercase]
target_label: lower_aaa
action: lowercase
- if: '{foo=~"bar.*",baz="aa"}'
target_label: aaa
replacement: foobar
- action: graphite
match: 'foo.*.bar'
labels:
instance: 'foo-$1'
job: '${1}-bar'

52
lib/storage/index_db.go
View file

@ -390,13 +390,13 @@ func (db *indexDB) putMetricNameToCache(metricID uint64, metricName []byte) {
db.s.metricNameCache.Set(key[:], metricName)
}
// maybeCreateIndexes probabilistically creates indexes for the given (tsid, metricNameRaw) at db.
// maybeCreateIndexes probabilistically creates global and per-day indexes for the given (tsid, metricNameRaw, date) at db.
//
// The probability increases from 0 to 100% during the first hour since db rotation.
//
// It returns true if new index entry was created, and false if it was skipped.
func (db *indexDB) maybeCreateIndexes(tsid *TSID, metricNameRaw []byte) (bool, error) {
pMin := float64(fasttime.UnixTimestamp()-db.rotationTimestamp) / 3600
func (is *indexSearch) maybeCreateIndexes(tsid *TSID, metricNameRaw []byte, date uint64) (bool, error) {
pMin := float64(fasttime.UnixTimestamp()-is.db.rotationTimestamp) / 3600
if pMin < 1 {
p := float64(uint32(fastHashUint64(tsid.MetricID))) / (1 << 32)
if p > pMin {
@ -410,11 +410,14 @@ func (db *indexDB) maybeCreateIndexes(tsid *TSID, metricNameRaw []byte) (bool, e
return false, fmt.Errorf("cannot unmarshal metricNameRaw %q: %w", metricNameRaw, err)
}
mn.sortTags()
if err := db.createIndexes(tsid, mn); err != nil {
return false, err
if err := is.createGlobalIndexes(tsid, mn); err != nil {
return false, fmt.Errorf("cannot create global indexes: %w", err)
}
if err := is.createPerDayIndexes(date, tsid.MetricID, mn); err != nil {
return false, fmt.Errorf("cannot create per-day indexes for date=%d: %w", date, err)
}
PutMetricName(mn)
atomic.AddUint64(&db.timeseriesRepopulated, 1)
atomic.AddUint64(&is.db.timeseriesRepopulated, 1)
return true, nil
}
@ -515,7 +518,10 @@ type indexSearch struct {
}
// GetOrCreateTSIDByName fills the dst with TSID for the given metricName.
func (is *indexSearch) GetOrCreateTSIDByName(dst *TSID, metricName []byte) error {
//
// It also registers the metricName in global and per-day indexes
// for the given date if the metricName->TSID entry is missing in the index.
func (is *indexSearch) GetOrCreateTSIDByName(dst *TSID, metricName []byte, date uint64) error {
// A hack: skip searching for the TSID after many serial misses.
// This should improve insertion performance for big batches
// of new time series.
@ -540,7 +546,7 @@ func (is *indexSearch) GetOrCreateTSIDByName(dst *TSID, metricName []byte) error
// TSID for the given name wasn't found. Create it.
// It is OK if duplicate TSID for mn is created by concurrent goroutines.
// Metric results will be merged by mn after TableSearch.
if err := is.db.createTSIDByName(dst, metricName); err != nil {
if err := is.createTSIDByName(dst, metricName, date); err != nil {
return fmt.Errorf("cannot create TSID by MetricName %q: %w", metricName, err)
}
return nil
@ -571,19 +577,25 @@ func (db *indexDB) putIndexSearch(is *indexSearch) {
db.indexSearchPool.Put(is)
}
func (db *indexDB) createTSIDByName(dst *TSID, metricName []byte) error {
func (is *indexSearch) createTSIDByName(dst *TSID, metricName []byte, date uint64) error {
mn := GetMetricName()
defer PutMetricName(mn)
if err := mn.Unmarshal(metricName); err != nil {
return fmt.Errorf("cannot unmarshal metricName %q: %w", metricName, err)
}
created, err := db.getOrCreateTSID(dst, metricName, mn)
created, err := is.db.getOrCreateTSID(dst, metricName, mn)
if err != nil {
return fmt.Errorf("cannot generate TSID: %w", err)
}
if err := db.createIndexes(dst, mn); err != nil {
return fmt.Errorf("cannot create indexes: %w", err)
if !is.db.s.registerSeriesCardinality(dst.MetricID, mn) {
return errSeriesCardinalityExceeded
}
if err := is.createGlobalIndexes(dst, mn); err != nil {
return fmt.Errorf("cannot create global indexes: %w", err)
}
if err := is.createPerDayIndexes(date, dst.MetricID, mn); err != nil {
return fmt.Errorf("cannot create per-day indexes for date=%d: %w", date, err)
}
// There is no need in invalidating tag cache, since it is invalidated
@ -591,7 +603,7 @@ func (db *indexDB) createTSIDByName(dst *TSID, metricName []byte) error {
if created {
// Increase the newTimeseriesCreated counter only if tsid wasn't found in indexDB
atomic.AddUint64(&db.newTimeseriesCreated, 1)
atomic.AddUint64(&is.db.newTimeseriesCreated, 1)
if logNewSeries {
logger.Infof("new series created: %s", mn.String())
}
@ -599,6 +611,8 @@ func (db *indexDB) createTSIDByName(dst *TSID, metricName []byte) error {
return nil
}
var errSeriesCardinalityExceeded = fmt.Errorf("cannot create series because series cardinality limit exceeded")
// SetLogNewSeries updates new series logging.
//
// This function must be called before any calling any storage functions.
@ -648,7 +662,7 @@ func generateTSID(dst *TSID, mn *MetricName) {
dst.MetricID = generateUniqueMetricID()
}
func (db *indexDB) createIndexes(tsid *TSID, mn *MetricName) error {
func (is *indexSearch) createGlobalIndexes(tsid *TSID, mn *MetricName) error {
// The order of index items is important.
// It guarantees index consistency.
@ -679,7 +693,7 @@ func (db *indexDB) createIndexes(tsid *TSID, mn *MetricName) error {
ii.registerTagIndexes(prefix.B, mn, tsid.MetricID)
kbPool.Put(prefix)
return db.tb.AddItems(ii.Items)
return is.db.tb.AddItems(ii.Items)
}
type indexItems struct {
@ -2681,11 +2695,11 @@ const (
int64Max = int64((1 << 63) - 1)
)
func (is *indexSearch) storeDateMetricID(date, metricID uint64, mn *MetricName) error {
func (is *indexSearch) createPerDayIndexes(date, metricID uint64, mn *MetricName) error {
ii := getIndexItems()
defer putIndexItems(ii)
ii.B = is.marshalCommonPrefix(ii.B, nsPrefixDateToMetricID)
ii.B = marshalCommonPrefix(ii.B, nsPrefixDateToMetricID)
ii.B = encoding.MarshalUint64(ii.B, date)
ii.B = encoding.MarshalUint64(ii.B, metricID)
ii.Next()
@ -2693,7 +2707,7 @@ func (is *indexSearch) storeDateMetricID(date, metricID uint64, mn *MetricName)
// Create per-day inverted index entries for metricID.
kb := kbPool.Get()
defer kbPool.Put(kb)
kb.B = is.marshalCommonPrefix(kb.B[:0], nsPrefixDateTagToMetricIDs)
kb.B = marshalCommonPrefix(kb.B[:0], nsPrefixDateTagToMetricIDs)
kb.B = encoding.MarshalUint64(kb.B, date)
ii.registerTagIndexes(kb.B, mn, metricID)
if err := is.db.tb.AddItems(ii.Items); err != nil {
@ -2812,7 +2826,7 @@ func reverseBytes(dst, src []byte) []byte {
func (is *indexSearch) hasDateMetricID(date, metricID uint64) (bool, error) {
ts := &is.ts
kb := &is.kb
kb.B = is.marshalCommonPrefix(kb.B[:0], nsPrefixDateToMetricID)
kb.B = marshalCommonPrefix(kb.B[:0], nsPrefixDateToMetricID)
kb.B = encoding.MarshalUint64(kb.B, date)
kb.B = encoding.MarshalUint64(kb.B, metricID)
if err := ts.FirstItemWithPrefix(kb.B); err != nil {

20
lib/storage/index_db_test.go
View file

@ -604,7 +604,7 @@ func testIndexDBBigMetricName(db *indexDB) error {
mn.MetricGroup = append(mn.MetricGroup[:0], bigBytes...)
mn.sortTags()
metricName := mn.Marshal(nil)
if err := is.GetOrCreateTSIDByName(&tsid, metricName); err == nil {
if err := is.GetOrCreateTSIDByName(&tsid, metricName, 0); err == nil {
return fmt.Errorf("expecting non-nil error on an attempt to insert metric with too big MetricGroup")
}
@ -617,7 +617,7 @@ func testIndexDBBigMetricName(db *indexDB) error {
}}
mn.sortTags()
metricName = mn.Marshal(nil)
if err := is.GetOrCreateTSIDByName(&tsid, metricName); err == nil {
if err := is.GetOrCreateTSIDByName(&tsid, metricName, 0); err == nil {
return fmt.Errorf("expecting non-nil error on an attempt to insert metric with too big tag key")
}
@ -630,7 +630,7 @@ func testIndexDBBigMetricName(db *indexDB) error {
}}
mn.sortTags()
metricName = mn.Marshal(nil)
if err := is.GetOrCreateTSIDByName(&tsid, metricName); err == nil {
if err := is.GetOrCreateTSIDByName(&tsid, metricName, 0); err == nil {
return fmt.Errorf("expecting non-nil error on an attempt to insert metric with too big tag value")
}
@ -645,7 +645,7 @@ func testIndexDBBigMetricName(db *indexDB) error {
}
mn.sortTags()
metricName = mn.Marshal(nil)
if err := is.GetOrCreateTSIDByName(&tsid, metricName); err == nil {
if err := is.GetOrCreateTSIDByName(&tsid, metricName, 0); err == nil {
return fmt.Errorf("expecting non-nil error on an attempt to insert metric with too many tags")
}
@ -679,7 +679,7 @@ func testIndexDBGetOrCreateTSIDByName(db *indexDB, metricGroups int) ([]MetricNa
// Create tsid for the metricName.
var tsid TSID
if err := is.GetOrCreateTSIDByName(&tsid, metricNameBuf); err != nil {
if err := is.GetOrCreateTSIDByName(&tsid, metricNameBuf, 0); err != nil {
return nil, nil, fmt.Errorf("unexpected error when creating tsid for mn:\n%s: %w", &mn, err)
}
@ -691,8 +691,8 @@ func testIndexDBGetOrCreateTSIDByName(db *indexDB, metricGroups int) ([]MetricNa
date := uint64(timestampFromTime(time.Now())) / msecPerDay
for i := range tsids {
tsid := &tsids[i]
if err := is.storeDateMetricID(date, tsid.MetricID, &mns[i]); err != nil {
return nil, nil, fmt.Errorf("error in storeDateMetricID(%d, %d): %w", date, tsid.MetricID, err)
if err := is.createPerDayIndexes(date, tsid.MetricID, &mns[i]); err != nil {
return nil, nil, fmt.Errorf("error in createPerDayIndexes(%d, %d): %w", date, tsid.MetricID, err)
}
}
@ -1662,7 +1662,7 @@ func TestSearchTSIDWithTimeRange(t *testing.T) {
metricNameBuf = mn.Marshal(metricNameBuf[:0])
var tsid TSID
if err := is.GetOrCreateTSIDByName(&tsid, metricNameBuf); err != nil {
if err := is.GetOrCreateTSIDByName(&tsid, metricNameBuf, 0); err != nil {
t.Fatalf("unexpected error when creating tsid for mn:\n%s: %s", &mn, err)
}
mns = append(mns, mn)
@ -1675,8 +1675,8 @@ func TestSearchTSIDWithTimeRange(t *testing.T) {
for i := range tsids {
tsid := &tsids[i]
metricIDs.Add(tsid.MetricID)
if err := is.storeDateMetricID(date, tsid.MetricID, &mns[i]); err != nil {
t.Fatalf("error in storeDateMetricID(%d, %d): %s", date, tsid.MetricID, err)
if err := is.createPerDayIndexes(date, tsid.MetricID, &mns[i]); err != nil {
t.Fatalf("error in createPerDayIndexes(%d, %d): %s", date, tsid.MetricID, err)
}
}
allMetricIDs.Union(&metricIDs)

22
lib/storage/index_db_timing_test.go
View file

@ -93,7 +93,7 @@ func benchmarkIndexDBAddTSIDs(db *indexDB, tsid *TSID, mn *MetricName, startOffs
}
mn.sortTags()
metricName = mn.Marshal(metricName[:0])
if err := is.GetOrCreateTSIDByName(tsid, metricName); err != nil {
if err := is.GetOrCreateTSIDByName(tsid, metricName, 0); err != nil {
panic(fmt.Errorf("cannot insert record: %w", err))
}
}
@ -122,6 +122,8 @@ func BenchmarkHeadPostingForMatchers(b *testing.B) {
var mn MetricName
var metricName []byte
var tsid TSID
is := db.getIndexSearch(noDeadline)
defer db.putIndexSearch(is)
addSeries := func(kvs ...string) {
mn.Reset()
for i := 0; i < len(kvs); i += 2 {
@ -129,20 +131,20 @@ func BenchmarkHeadPostingForMatchers(b *testing.B) {
}
mn.sortTags()
metricName = mn.Marshal(metricName[:0])
if err := db.createTSIDByName(&tsid, metricName); err != nil {
if err := is.createTSIDByName(&tsid, metricName, 0); err != nil {
b.Fatalf("cannot insert record: %s", err)
}
}
for n := 0; n < 10; n++ {
ns := strconv.Itoa(n)
for i := 0; i < 100000; i++ {
is := strconv.Itoa(i)
addSeries("i", is, "n", ns, "j", "foo")
ix := strconv.Itoa(i)
addSeries("i", ix, "n", ns, "j", "foo")
// Have some series that won't be matched, to properly test inverted matches.
addSeries("i", is, "n", ns, "j", "bar")
addSeries("i", is, "n", "0_"+ns, "j", "bar")
addSeries("i", is, "n", "1_"+ns, "j", "bar")
addSeries("i", is, "n", "2_"+ns, "j", "foo")
addSeries("i", ix, "n", ns, "j", "bar")
addSeries("i", ix, "n", "0_"+ns, "j", "bar")
addSeries("i", ix, "n", "1_"+ns, "j", "bar")
addSeries("i", ix, "n", "2_"+ns, "j", "foo")
}
}
@ -313,7 +315,7 @@ func BenchmarkIndexDBGetTSIDs(b *testing.B) {
for i := 0; i < recordsCount; i++ {
mn.sortTags()
metricName = mn.Marshal(metricName[:0])
if err := is.GetOrCreateTSIDByName(&tsid, metricName); err != nil {
if err := is.GetOrCreateTSIDByName(&tsid, metricName, 0); err != nil {
b.Fatalf("cannot insert record: %s", err)
}
}
@ -331,7 +333,7 @@ func BenchmarkIndexDBGetTSIDs(b *testing.B) {
for i := 0; i < recordsPerLoop; i++ {
mnLocal.sortTags()
metricNameLocal = mnLocal.Marshal(metricNameLocal[:0])
if err := is.GetOrCreateTSIDByName(&tsidLocal, metricNameLocal); err != nil {
if err := is.GetOrCreateTSIDByName(&tsidLocal, metricNameLocal, 0); err != nil {
panic(fmt.Errorf("cannot obtain tsid: %w", err))
}
}

157
lib/storage/storage.go
View file

@ -2,6 +2,7 @@ package storage
import (
"bytes"
"errors"
"fmt"
"io"
"io/ioutil"
@ -365,7 +366,7 @@ func (s *Storage) CreateSnapshot() (string, error) {
srcMetadataDir := srcDir + "/metadata"
dstMetadataDir := dstDir + "/metadata"
if err := fs.CopyDirectory(srcMetadataDir, dstMetadataDir); err != nil {
return "", fmt.Errorf("cannot copy metadata: %s", err)
return "", fmt.Errorf("cannot copy metadata: %w", err)
}
fs.MustSyncPath(dstDir)
@ -745,6 +746,27 @@ func (s *Storage) mustRotateIndexDB() {
// and slowly re-populate new idb with entries from the cache via maybeCreateIndexes().
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401
// Flush metric id caches for the current and the previous hour,
// since they may contain entries missing in idbNew.
// This should prevent from missing data in queries when
// the following steps are performed for short -retentionPeriod (e.g. 1 day):
//
// 1. Add samples for some series between 3-4 UTC. These series are registered in currHourMetricIDs.
// 2. The indexdb rotation is performed at 4 UTC. currHourMetricIDs is moved to prevHourMetricIDs.
// 3. Continue adding samples for series from step 1 during time range 4-5 UTC.
// These series are already registered in prevHourMetricIDs, so VM doesn't add per-day entries to the current indexdb.
// 4. Stop adding new samples for these series just before 5 UTC.
// 5. The next indexdb rotation is performed at 4 UTC next day.
// The information about the series from step 5 disappears from indexdb, since the old indexdb from step 1 is deleted,
// while the current indexdb doesn't contain information about the series.
// So queries for the last 24 hours stop returning samples added at step 3.
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2698
s.pendingHourEntriesLock.Lock()
s.pendingHourEntries = &uint64set.Set{}
s.pendingHourEntriesLock.Unlock()
s.currHourMetricIDs.Store(&hourMetricIDs{})
s.prevHourMetricIDs.Store(&hourMetricIDs{})
// Flush dateMetricIDCache, so idbNew can be populated with fresh data.
s.dateMetricIDCache.Reset()
@ -1644,10 +1666,7 @@ var (
// The the MetricRow.Timestamp is used for registering the metric name starting from the given timestamp.
// Th MetricRow.Value field is ignored.
func (s *Storage) RegisterMetricNames(mrs []MetricRow) error {
var (
metricName []byte
)
var metricName []byte
var genTSID generationTSID
mn := GetMetricName()
defer PutMetricName(mn)
@ -1658,64 +1677,35 @@ func (s *Storage) RegisterMetricNames(mrs []MetricRow) error {
for i := range mrs {
mr := &mrs[i]
if s.getTSIDFromCache(&genTSID, mr.MetricNameRaw) {
if genTSID.generation != idb.generation {
// The found entry is from the previous cache generation
// so attempt to re-populate the current generation with this entry.
// This is needed for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401
created, err := idb.maybeCreateIndexes(&genTSID.TSID, mr.MetricNameRaw)
if err != nil {
return fmt.Errorf("cannot create indexes in the current indexdb: %w", err)
}
if created {
genTSID.generation = idb.generation
s.putTSIDToCache(&genTSID, mr.MetricNameRaw)
}
}
// Fast path - mr.MetricNameRaw has been already registered.
if genTSID.generation == idb.generation {
// Fast path - mr.MetricNameRaw has been already registered in the current idb.
continue
}
}
// Slow path - register mr.MetricNameRaw.
if err := mn.UnmarshalRaw(mr.MetricNameRaw); err != nil {
return fmt.Errorf("cannot register the metric because cannot unmarshal MetricNameRaw %q: %w", mr.MetricNameRaw, err)
return fmt.Errorf("cannot unmarshal MetricNameRaw %q: %w", mr.MetricNameRaw, err)
}
mn.sortTags()
metricName = mn.Marshal(metricName[:0])
if err := is.GetOrCreateTSIDByName(&genTSID.TSID, metricName); err != nil {
return fmt.Errorf("cannot register the metric because cannot create TSID for metricName %q: %w", metricName, err)
}
s.putTSIDToCache(&genTSID, mr.MetricNameRaw)
// Register the metric in per-day inverted index.
date := uint64(mr.Timestamp) / msecPerDay
metricID := genTSID.TSID.MetricID
if s.dateMetricIDCache.Has(date, metricID) {
// Fast path: the metric has been already registered in per-day inverted index
if err := is.GetOrCreateTSIDByName(&genTSID.TSID, metricName, date); err != nil {
if errors.Is(err, errSeriesCardinalityExceeded) {
continue
}
// Slow path: acutally register the metric in per-day inverted index.
ok, err := is.hasDateMetricID(date, metricID)
if err != nil {
return fmt.Errorf("cannot register the metric in per-date inverted index because of error when locating (date=%d, metricID=%d) in database: %w",
date, metricID, err)
return fmt.Errorf("cannot create TSID for metricName %q: %w", metricName, err)
}
if !ok {
// The (date, metricID) entry is missing in the indexDB. Add it there.
if err := is.storeDateMetricID(date, metricID, mn); err != nil {
return fmt.Errorf("cannot register the metric in per-date inverted index because of error when storing (date=%d, metricID=%d) in database: %w",
date, metricID, err)
}
}
// The metric must be added to cache only after it has been successfully added to indexDB.
s.dateMetricIDCache.Set(date, metricID)
genTSID.generation = idb.generation
s.putTSIDToCache(&genTSID, mr.MetricNameRaw)
s.dateMetricIDCache.Set(date, genTSID.TSID.MetricID)
}
return nil
}
func (s *Storage) add(rows []rawRow, dstMrs []*MetricRow, mrs []MetricRow, precisionBits uint8) error {
idb := s.idb()
j := 0
is := idb.getIndexSearch(noDeadline)
defer idb.putIndexSearch(is)
var (
// These vars are used for speeding up bulk imports of multiple adjacent rows for the same metricName.
prevTSID TSID
@ -1728,6 +1718,7 @@ func (s *Storage) add(rows []rawRow, dstMrs []*MetricRow, mrs []MetricRow, preci
// Return only the first error, since it has no sense in returning all errors.
var firstWarn error
j := 0
for i := range mrs {
mr := &mrs[i]
if math.IsNaN(mr.Value) {
@ -1772,11 +1763,6 @@ func (s *Storage) add(rows []rawRow, dstMrs []*MetricRow, mrs []MetricRow, preci
}
if s.getTSIDFromCache(&genTSID, mr.MetricNameRaw) {
r.TSID = genTSID.TSID
if s.isSeriesCardinalityExceeded(r.TSID.MetricID, mr.MetricNameRaw) {
// Skip the row, since the limit on the number of unique series has been exceeded.
j--
continue
}
// Fast path - the TSID for the given MetricNameRaw has been found in cache and isn't deleted.
// There is no need in checking whether r.TSID.MetricID is deleted, since tsidCache doesn't
// contain MetricName->TSID entries for deleted time series.
@ -1785,16 +1771,18 @@ func (s *Storage) add(rows []rawRow, dstMrs []*MetricRow, mrs []MetricRow, preci
prevMetricNameRaw = mr.MetricNameRaw
if genTSID.generation != idb.generation {
// The found entry is from the previous cache generation
// The found entry is from the previous cache generation,
// so attempt to re-populate the current generation with this entry.
// This is needed for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401
created, err := idb.maybeCreateIndexes(&genTSID.TSID, mr.MetricNameRaw)
date := uint64(r.Timestamp) / msecPerDay
created, err := is.maybeCreateIndexes(&genTSID.TSID, mr.MetricNameRaw, date)
if err != nil {
return fmt.Errorf("cannot create indexes in the current indexdb: %w", err)
return fmt.Errorf("cannot create indexes: %w", err)
}
if created {
genTSID.generation = idb.generation
s.putTSIDToCache(&genTSID, mr.MetricNameRaw)
s.dateMetricIDCache.Set(date, genTSID.TSID.MetricID)
}
}
continue
@ -1822,7 +1810,6 @@ func (s *Storage) add(rows []rawRow, dstMrs []*MetricRow, mrs []MetricRow, preci
sort.Slice(pendingMetricRows, func(i, j int) bool {
return string(pendingMetricRows[i].MetricName) < string(pendingMetricRows[j].MetricName)
})
is := idb.getIndexSearch(noDeadline)
prevMetricNameRaw = nil
var slowInsertsCount uint64
for i := range pendingMetricRows {
@ -1838,36 +1825,31 @@ func (s *Storage) add(rows []rawRow, dstMrs []*MetricRow, mrs []MetricRow, preci
// Fast path - the current mr contains the same metric name as the previous mr, so it contains the same TSID.
// This path should trigger on bulk imports when many rows contain the same MetricNameRaw.
r.TSID = prevTSID
if s.isSeriesCardinalityExceeded(r.TSID.MetricID, mr.MetricNameRaw) {
// Skip the row, since the limit on the number of unique series has been exceeded.
j--
continue
}
continue
}
slowInsertsCount++
if err := is.GetOrCreateTSIDByName(&r.TSID, pmr.MetricName); err != nil {
date := uint64(r.Timestamp) / msecPerDay
if err := is.GetOrCreateTSIDByName(&r.TSID, pmr.MetricName, date); err != nil {
j--
if errors.Is(err, errSeriesCardinalityExceeded) {
continue
}
// Do not stop adding rows on error - just skip invalid row.
// This guarantees that invalid rows don't prevent
// from adding valid rows into the storage.
if firstWarn == nil {
firstWarn = fmt.Errorf("cannot obtain or create TSID for MetricName %q: %w", pmr.MetricName, err)
}
j--
continue
}
genTSID.generation = idb.generation
genTSID.TSID = r.TSID
s.putTSIDToCache(&genTSID, mr.MetricNameRaw)
s.dateMetricIDCache.Set(date, genTSID.TSID.MetricID)
prevTSID = r.TSID
prevMetricNameRaw = mr.MetricNameRaw
if s.isSeriesCardinalityExceeded(r.TSID.MetricID, mr.MetricNameRaw) {
// Skip the row, since the limit on the number of unique series has been exceeded.
j--
continue
}
}
idb.putIndexSearch(is)
putPendingMetricRows(pmrs)
atomic.AddUint64(&s.slowRowInserts, slowInsertsCount)
}
@ -1877,39 +1859,41 @@ func (s *Storage) add(rows []rawRow, dstMrs []*MetricRow, mrs []MetricRow, preci
dstMrs = dstMrs[:j]
rows = rows[:j]
var firstError error
if err := s.tb.AddRows(rows); err != nil {
firstError = fmt.Errorf("cannot add rows to table: %w", err)
err := s.updatePerDateData(rows, dstMrs)
if err != nil {
err = fmt.Errorf("cannot update per-date data: %w", err)
} else {
err = s.tb.AddRows(rows)
if err != nil {
err = fmt.Errorf("cannot add rows to table: %w", err)
}
if err := s.updatePerDateData(rows, dstMrs); err != nil && firstError == nil {
firstError = fmt.Errorf("cannot update per-date data: %w", err)
}
if firstError != nil {
return fmt.Errorf("error occurred during rows addition: %w", firstError)
if err != nil {
return fmt.Errorf("error occurred during rows addition: %w", err)
}
return nil
}
func (s *Storage) isSeriesCardinalityExceeded(metricID uint64, metricNameRaw []byte) bool {
func (s *Storage) registerSeriesCardinality(metricID uint64, mn *MetricName) bool {
if sl := s.hourlySeriesLimiter; sl != nil && !sl.Add(metricID) {
atomic.AddUint64(&s.hourlySeriesLimitRowsDropped, 1)
logSkippedSeries(metricNameRaw, "-storage.maxHourlySeries", sl.MaxItems())
return true
logSkippedSeries(mn, "-storage.maxHourlySeries", sl.MaxItems())
return false
}
if sl := s.dailySeriesLimiter; sl != nil && !sl.Add(metricID) {
atomic.AddUint64(&s.dailySeriesLimitRowsDropped, 1)
logSkippedSeries(metricNameRaw, "-storage.maxDailySeries", sl.MaxItems())
return true
}
logSkippedSeries(mn, "-storage.maxDailySeries", sl.MaxItems())
return false
}
return true
}
func logSkippedSeries(metricNameRaw []byte, flagName string, flagValue int) {
func logSkippedSeries(mn *MetricName, flagName string, flagValue int) {
select {
case <-logSkippedSeriesTicker.C:
// Do not use logger.WithThrottler() here, since this will result in increased CPU load
// because of getUserReadableMetricName() calls per each logSkippedSeries call.
logger.Warnf("skip series %s because %s=%d reached", getUserReadableMetricName(metricNameRaw), flagName, flagValue)
logger.Warnf("skip series %s because %s=%d reached", mn, flagName, flagValue)
default:
}
}
@ -2111,7 +2095,7 @@ func (s *Storage) updatePerDateData(rows []rawRow, mrs []*MetricRow) error {
continue
}
if !ok {
// The (date, metricID) entry is missing in the indexDB. Add it there.
// The (date, metricID) entry is missing in the indexDB. Add it there together with per-day indexes.
// It is OK if the (date, metricID) entry is added multiple times to db
// by concurrent goroutines.
if err := mn.UnmarshalRaw(dmid.mr.MetricNameRaw); err != nil {
@ -2121,9 +2105,9 @@ func (s *Storage) updatePerDateData(rows []rawRow, mrs []*MetricRow) error {
continue
}
mn.sortTags()
if err := is.storeDateMetricID(date, metricID, mn); err != nil {
if err := is.createPerDayIndexes(date, metricID, mn); err != nil {
if firstError == nil {
firstError = fmt.Errorf("error when storing (date=%d, metricID=%d) in database: %w", date, metricID, err)
firstError = fmt.Errorf("error when storing per-date inverted index for (date=%d, metricID=%d): %w", date, metricID, err)
}
continue
}
@ -2374,6 +2358,7 @@ func (s *Storage) updateCurrHourMetricIDs() {
newMetricIDs := s.pendingHourEntries
s.pendingHourEntries = &uint64set.Set{}
s.pendingHourEntriesLock.Unlock()
hour := fasttime.UnixHour()
if newMetricIDs.Len() == 0 && hm.hour == hour {
// Fast path: nothing to update.

8
snap/local/README.md
View file

@ -10,7 +10,7 @@ Install snapcraft or docker
build snap package with command
```bash
```console
make build-snap
```
@ -21,7 +21,7 @@ You can install it with command: `snap install victoriametrics_v1.46.0+git1.1beb
installation and configuration:
```bash
```console
# install
snap install victoriametrics
# logs
@ -34,7 +34,7 @@ Configuration management:
Prometheus scrape config can be edited with your favorite editor, its located at
```bash
```console
vi /var/snap/victoriametrics/current/etc/victoriametrics-scrape-config.yaml
```
@ -42,7 +42,7 @@ after changes, you can trigger config reread with `curl localhost:8248/-/reload`
Configuration tuning is possible with editing extra_flags:
```bash
```console
echo 'FLAGS="-selfScrapeInterval=10s -search.logSlowQueryDuration=20s"' > /var/snap/victoriametrics/current/extra_flags
snap restart victoriametrics
```