github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity

vendor update: updates metrics to v1.34.0 (#6523)

Browse source 
mention change for
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6457

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
This commit is contained in:
Nikolay 2024-06-24 14:51:34 +02:00 committed by GitHub
parent 1106d91741
commit 1af13208c2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 65 additions and 25 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
docs/CHANGELOG.md
View file

@ -4,12 +4,11 @@ weight: 100
title: CHANGELOG title: CHANGELOG
menu: menu:
docs: docs:
parent: 'victoriametrics' parent: victoriametrics
weight: 100 weight: 100
aliases: aliases:
- /CHANGELOG.html - /CHANGELOG.html
--- ---
# CHANGELOG # CHANGELOG
The following `tip` changes can be tested by building VictoriaMetrics components from the latest commits according to the following docs: The following `tip` changes can be tested by building VictoriaMetrics components from the latest commits according to the following docs:
@ -45,6 +44,7 @@ See also [LTS releases](https://docs.victoriametrics.com/lts-releases/).
* FEATURE: [vmsingle](https://docs.victoriametrics.com/single-server-victoriametrics/): validate that rollup expressions has ranged vector arguments passed when `-search.disableImplicitConversion` or `-search.logImplicitConversion` are enabled. For example, `rate(metric)` or `count_over_time(metric)` will fail to execute if these flags are set. * FEATURE: [vmsingle](https://docs.victoriametrics.com/single-server-victoriametrics/): validate that rollup expressions has ranged vector arguments passed when `-search.disableImplicitConversion` or `-search.logImplicitConversion` are enabled. For example, `rate(metric)` or `count_over_time(metric)` will fail to execute if these flags are set.
* FEATURE: [vmalert-tool](https://docs.victoriametrics.com/vmalert-tool/): support file path with hierarchical patterns and regexpes, and http url in unittest cmd-line flag `-files`, e.g. `-files="http://<some-server-addr>/path/to/rules"` or `-files="dir/**/*.yaml"`. * FEATURE: [vmalert-tool](https://docs.victoriametrics.com/vmalert-tool/): support file path with hierarchical patterns and regexpes, and http url in unittest cmd-line flag `-files`, e.g. `-files="http://<some-server-addr>/path/to/rules"` or `-files="dir/**/*.yaml"`.
* BUGFIX: all VictoriaMetrics components: properly calculate `process_resident_memory_bytes` metric for OS with non-default value of memory page size. See this [issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6457) for details.
* BUGFIX: all VictoriaMetrics components: prioritize `-configAuthKey` and `-reloadAuthKey` over `-httpAuth.*` settings. This change aligns behavior of mentioned flags with other auth flags like `-metricsAuthKey`, `-flagsAuthKey`, `-pprofAuthKey`. Check [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6329). * BUGFIX: all VictoriaMetrics components: prioritize `-configAuthKey` and `-reloadAuthKey` over `-httpAuth.*` settings. This change aligns behavior of mentioned flags with other auth flags like `-metricsAuthKey`, `-flagsAuthKey`, `-pprofAuthKey`. Check [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6329).
* BUGFIX: all VictoriaMetrics components: do not trim trailing spaces when reading content from `*.passwordFile` and similar flags. Previously, trailing spaces were trimmed from the content of the password file, which could lead to unexpected authentication errors. * BUGFIX: all VictoriaMetrics components: do not trim trailing spaces when reading content from `*.passwordFile` and similar flags. Previously, trailing spaces were trimmed from the content of the password file, which could lead to unexpected authentication errors.
* BUGFIX: [vmctl](https://docs.victoriametrics.com/vmctl/): add `--disable-progress-bar` global command-line flag. It can be used for disabling dynamic progress bar for all migration modes. `--vm-disable-progress-bar` command-line flag is deprecated and will be removed in the future releases. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6367). * BUGFIX: [vmctl](https://docs.victoriametrics.com/vmctl/): add `--disable-progress-bar` global command-line flag. It can be used for disabling dynamic progress bar for all migration modes. `--vm-disable-progress-bar` command-line flag is deprecated and will be removed in the future releases. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6367).
@ -93,7 +93,7 @@ Released at 2024-06-07
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert/): support reading [DNS SRV](https://en.wikipedia.org/wiki/SRV_record) records in `-datasource.url`, `-remoteWrite.url` and `-remoteRead.url` command-line option. For example, `-remoteWrite.url=http://srv+victoria-metrics` automatically resolves the `victoria-metrics` DNS SRV to a list of hostnames with TCP ports and then sends data to one of the addresses. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6053). * FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert/): support reading [DNS SRV](https://en.wikipedia.org/wiki/SRV_record) records in `-datasource.url`, `-remoteWrite.url` and `-remoteRead.url` command-line option. For example, `-remoteWrite.url=http://srv+victoria-metrics` automatically resolves the `victoria-metrics` DNS SRV to a list of hostnames with TCP ports and then sends data to one of the addresses. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6053).
* FEATURE: [vmbackup](https://docs.victoriametrics.com/vmbackup/), [vmrestore](https://docs.victoriametrics.com/vmrestore/), [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager/): add `-s3TLSInsecureSkipVerify` command-line flag for skipping TLS certificates verification when connecting to S3 endpoint. * FEATURE: [vmbackup](https://docs.victoriametrics.com/vmbackup/), [vmrestore](https://docs.victoriametrics.com/vmrestore/), [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager/): add `-s3TLSInsecureSkipVerify` command-line flag for skipping TLS certificates verification when connecting to S3 endpoint.
* FEATURE: expose metric `vm_indexdb_items_dropped_total` to track the number of IndexDB records that had to be dropped during ingestion. The reason of dropping the record will be annotated in `reason` label of the exposed metric. This change also comes with a new [alerting rule](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/alerts-health.yml) to track changes of this metric. * FEATURE: expose metric `vm_indexdb_items_dropped_total` to track the number of IndexDB records that had to be dropped during ingestion. The reason of dropping the record will be annotated in `reason` label of the exposed metric. This change also comes with a new [alerting rule](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/alerts-health.yml) to track changes of this metric.
* FEATURE: [alerts-health](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/alerts-health.yml): add new alerting rules `TooLongLabelValues` and `TooLongLabelNames` to notify about truncation of label values or names respectively. * FEATURE: [alerts-health](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/alerts-health.yml): add new alerting rules `TooLongLabelValues` and `TooLongLabelNames` to notify about truncation of label values or names respectively.
* FEATURE: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): expose `vm_streamaggr_ignored_samples_total` [counters](https://docs.victoriametrics.com/keyconcepts/#counter) at [`/metrics` page](https://docs.victoriametrics.com/#monitoring), which can be used for detecting amount of too old or NaN valued ignored samples. Expose also `vm_streamaggr_samples_lag_seconds` [histogram](https://docs.victoriametrics.com/keyconcepts/#histogram) to monitor aggregated samples lag. * FEATURE: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): expose `vm_streamaggr_ignored_samples_total` [counters](https://docs.victoriametrics.com/keyconcepts/#counter) at [`/metrics` page](https://docs.victoriametrics.com/#monitoring), which can be used for detecting amount of too old or NaN valued ignored samples. Expose also `vm_streamaggr_samples_lag_seconds` [histogram](https://docs.victoriametrics.com/keyconcepts/#histogram) to monitor aggregated samples lag.
* FEATURE: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): improve filtering speed of the received data samples if [match](https://docs.victoriametrics.com/stream-aggregation/#stream-aggregation-config) field is matching only [metric name](https://docs.victoriametrics.com/keyconcepts/#structure-of-a-metric). * FEATURE: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): improve filtering speed of the received data samples if [match](https://docs.victoriametrics.com/stream-aggregation/#stream-aggregation-config) field is matching only [metric name](https://docs.victoriametrics.com/keyconcepts/#structure-of-a-metric).
* FEATURE: [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/): adds `-relabelConfigCheckInterval` flag and `/-/reload` endpoint for reloading relabeling rules. See this [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3923) for details. * FEATURE: [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/): adds `-relabelConfigCheckInterval` flag and `/-/reload` endpoint for reloading relabeling rules. See this [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3923) for details.
@ -105,12 +105,12 @@ Released at 2024-06-07
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent/): prevent potential panic during [stream aggregation](https://docs.victoriametrics.com/stream-aggregation.html) if more than one `--remoteWrite.streamAggr.dedupInterval` is configured. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6205). * BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent/): prevent potential panic during [stream aggregation](https://docs.victoriametrics.com/stream-aggregation.html) if more than one `--remoteWrite.streamAggr.dedupInterval` is configured. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6205).
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent/): skip empty data blocks before sending to the remote write destination. Thanks to @viperstars for [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6241). * BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent/): skip empty data blocks before sending to the remote write destination. Thanks to @viperstars for [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6241).
* BUGFIX: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): set correct suffix `<output>_prometheus` for aggregation outputs [increase_prometheus](https://docs.victoriametrics.com/stream-aggregation/#increase_prometheus) and [total_prometheus](https://docs.victoriametrics.com/stream-aggregation/#total_prometheus). Before, outputs `total` and `total_prometheus` or `increase` and `increase_prometheus` had the same suffix. * BUGFIX: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): set correct suffix `<output>_prometheus` for aggregation outputs [increase_prometheus](https://docs.victoriametrics.com/stream-aggregation/#increase_prometheus) and [total_prometheus](https://docs.victoriametrics.com/stream-aggregation/#total_prometheus). Before, outputs `total` and `total_prometheus` or `increase` and `increase_prometheus` had the same suffix.
* BUGFIX: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): prevent from excessive resource usage when stream aggregation config file is empty. * BUGFIX: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): prevent from excessive resource usage when stream aggregation config file is empty.
* BUGFIX: properly estimate the needed memory for query execution if it has the format [`aggr_func`](https://docs.victoriametrics.com/metricsql/#aggregate-functions)([`rollup_func[d]`](https://docs.victoriametrics.com/metricsql/#rollup-functions) (for example, `sum(rate(request_duration_seconds_bucket[5m]))`). This should allow performing aggregations over bigger number of time series when VictoriaMetrics runs in environments with small amounts of available memory. The issue has been introduced in [this commit](https://github.com/VictoriaMetrics/VictoriaMetrics/commit/5138eaeea0791caa34bcfab410e0ca9cd253cd8f) in [v1.83.0](https://docs.victoriametrics.com/changelog_2022/#v1830). * BUGFIX: properly estimate the needed memory for query execution if it has the format [`aggr_func`](https://docs.victoriametrics.com/metricsql/#aggregate-functions)([`rollup_func[d]`](https://docs.victoriametrics.com/metricsql/#rollup-functions) (for example, `sum(rate(request_duration_seconds_bucket[5m]))`). This should allow performing aggregations over bigger number of time series when VictoriaMetrics runs in environments with small amounts of available memory. The issue has been introduced in [this commit](https://github.com/VictoriaMetrics/VictoriaMetrics/commit/5138eaeea0791caa34bcfab410e0ca9cd253cd8f) in [v1.83.0](https://docs.victoriametrics.com/changelog_2022/#v1830).
* BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/): correctly apply `-inmemoryDataFlushInterval` when it's set to minimum supported value 1s. * BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/): correctly apply `-inmemoryDataFlushInterval` when it's set to minimum supported value 1s.
* BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/): reduce the default value for `-maxLabelValueLen` command-line flag from `16KiB` to `1KiB`. This should prevent from issues like [this one](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6176) when time series with too long labels are ingested into VictoriaMetrics. * BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and `vminsert` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/): reduce the default value for `-maxLabelValueLen` command-line flag from `16KiB` to `1KiB`. This should prevent from issues like [this one](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6176) when time series with too long labels are ingested into VictoriaMetrics.
* BUGFIX: [vmauth](https://docs.victoriametrics.com/vmauth/): properly release memory used for metrics during config reload. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6247). * BUGFIX: [vmauth](https://docs.victoriametrics.com/vmauth/): properly release memory used for metrics during config reload. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6247).
* BUGFIX: [vmauth](https://docs.victoriametrics.com/vmauth/): fix discovering backend IPs when `url_prefix` contains hostname with srv+ prefix. Thanks to @shichanglin5 for the [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6401). * BUGFIX: [vmauth](https://docs.victoriametrics.com/vmauth/): fix discovering backend IPs when `url_prefix` contains hostname with srv+ prefix. Thanks to @shichanglin5 for the [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6401).
* BUGFIX: [dashboards](https://grafana.com/orgs/victoriametrics): fix `AnnotationQueryRunner` error in Grafana when executing annotations query against Prometheus backend. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6309) for details. * BUGFIX: [dashboards](https://grafana.com/orgs/victoriametrics): fix `AnnotationQueryRunner` error in Grafana when executing annotations query against Prometheus backend. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6309) for details.
* BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/): filter deleted label names and values from [`/api/v1/labels`](https://docs.victoriametrics.com/url-examples/#apiv1labels) and [`/api/v1/label/.../values`](https://docs.victoriametrics.com/url-examples/#apiv1labelvalues) responses when `match[]` filter matches small number of time series. The issue was introduced [v1.81.0](https://docs.victoriametrics.com/changelog_2022/#v1810). * BUGFIX: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/): filter deleted label names and values from [`/api/v1/labels`](https://docs.victoriametrics.com/url-examples/#apiv1labels) and [`/api/v1/label/.../values`](https://docs.victoriametrics.com/url-examples/#apiv1labelvalues) responses when `match[]` filter matches small number of time series. The issue was introduced [v1.81.0](https://docs.victoriametrics.com/changelog_2022/#v1810).
* BUGFIX: [vmalert-tool](https://docs.victoriametrics.com/vmalert-tool/): fix float values template in `input_series`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6391). * BUGFIX: [vmalert-tool](https://docs.victoriametrics.com/vmalert-tool/): fix float values template in `input_series`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6391).
@ -141,7 +141,7 @@ Released at 2024-04-26
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): update error messages on the Query page for enhanced clarity. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6177). * FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): update error messages on the Query page for enhanced clarity. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6177).
* BUGFIX: [downsampling](https://docs.victoriametrics.com/#downsampling): skip unnecessary index lookups if downsampling wasn't set for ENT versions of VictoriaMetrics. Before, users of VictoriaMetrics ENT could have experience elevated CPU usage even if no downsampling was configured. The issue was introduced in [v1.100.0](https://docs.victoriametrics.com/changelog/#v11000). * BUGFIX: [downsampling](https://docs.victoriametrics.com/#downsampling): skip unnecessary index lookups if downsampling wasn't set for ENT versions of VictoriaMetrics. Before, users of VictoriaMetrics ENT could have experience elevated CPU usage even if no downsampling was configured. The issue was introduced in [v1.100.0](https://docs.victoriametrics.com/changelog/#v11000).
* BUGFIX: [downsampling](https://docs.victoriametrics.com/#downsampling): properly populate downsampling metadata for data parts created by VictoriaMetrics ENT versions lower than v1.100.0. The bug could trigger the downsampling actions for parts that were downsampled already. This bug doesn't have any negative effect apart from spending extra CPU resources on the repeated downsampling. The issue was introduced in [v1.100.0](https://docs.victoriametrics.com/changelog/#v11000). * BUGFIX: [downsampling](https://docs.victoriametrics.com/#downsampling): properly populate downsampling metadata for data parts created by VictoriaMetrics ENT versions lower than v1.100.0. The bug could trigger the downsampling actions for parts that were downsampled already. This bug doesn't have any negative effect apart from spending extra CPU resources on the repeated downsampling. The issue was introduced in [v1.100.0](https://docs.victoriametrics.com/changelog/#v11000).
* BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert/): supported any status codes from the range 200-299 from alertmanager. Previously, only 200 status code considered a successful action. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6110). * BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert/): supported any status codes from the range 200-299 from alertmanager. Previously, only 200 status code considered a successful action. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6110).
* BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert/): avoid blocking `/api/v1/rules`, `/api/v1/alerts`, `/metrics` APIs when alerting rule uses template functions `query`, which could takes a while to execute. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6079). * BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert/): avoid blocking `/api/v1/rules`, `/api/v1/alerts`, `/metrics` APIs when alerting rule uses template functions `query`, which could takes a while to execute. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6079).
* BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert/): fix links with anchors in vmalert's UI. Starting from [v1.99.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.99.0) vmalert could ignore anchors pointing to specific rule groups if `search` param was present in URL. * BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert/): fix links with anchors in vmalert's UI. Starting from [v1.99.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.99.0) vmalert could ignore anchors pointing to specific rule groups if `search` param was present in URL.
@ -189,7 +189,7 @@ Released at 2024-04-04
* FEATURE: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): add [increase_prometheus](https://docs.victoriametrics.com/stream-aggregation/#increase_prometheus) and [total_prometheus](https://docs.victoriametrics.com/stream-aggregation/#total_prometheus) outputs, which can be used for `increase` and `total` aggregations when the first sample of every new [time series](https://docs.victoriametrics.com/keyconcepts/#time-series) must be ignored. * FEATURE: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): add [increase_prometheus](https://docs.victoriametrics.com/stream-aggregation/#increase_prometheus) and [total_prometheus](https://docs.victoriametrics.com/stream-aggregation/#total_prometheus) outputs, which can be used for `increase` and `total` aggregations when the first sample of every new [time series](https://docs.victoriametrics.com/keyconcepts/#time-series) must be ignored.
* FEATURE: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): expose `vm_streamaggr_flush_timeouts_total` and `vm_streamaggr_dedup_flush_timeouts_total` [counters](https://docs.victoriametrics.com/keyconcepts/#counter) at [`/metrics` page](https://docs.victoriametrics.com/#monitoring), which can be used for detecting flush timeouts for stream aggregation states. Expose also `vm_streamaggr_flush_duration_seconds` and `vm_streamaggr_dedup_flush_duration_seconds` [histograms](https://docs.victoriametrics.com/keyconcepts/#histogram) for monitoring the real flush durations of stream aggregation states. * FEATURE: [stream aggregation](https://docs.victoriametrics.com/stream-aggregation/): expose `vm_streamaggr_flush_timeouts_total` and `vm_streamaggr_dedup_flush_timeouts_total` [counters](https://docs.victoriametrics.com/keyconcepts/#counter) at [`/metrics` page](https://docs.victoriametrics.com/#monitoring), which can be used for detecting flush timeouts for stream aggregation states. Expose also `vm_streamaggr_flush_duration_seconds` and `vm_streamaggr_dedup_flush_duration_seconds` [histograms](https://docs.victoriametrics.com/keyconcepts/#histogram) for monitoring the real flush durations of stream aggregation states.
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): improve trace display for better visual separation of branches. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5926). * FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): improve trace display for better visual separation of branches. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5926).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/): ability to limit the ingestion rate via `-maxIngestionRate` command-line flag. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5900). * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/): ability to limit the ingestion rate via `-maxIngestionRate` command-line flag. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5900).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/): use the provided `-remoteWrite.tlsServerName` as `Host` header in requests to `-remoteWrite.url`. This allows sending data to https remote storage by IP address instead of hostname. Thanks to @minor-fixes for initial idea and [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5802). * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/): use the provided `-remoteWrite.tlsServerName` as `Host` header in requests to `-remoteWrite.url`. This allows sending data to https remote storage by IP address instead of hostname. Thanks to @minor-fixes for initial idea and [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5802).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/): add `-remoteWrite.shardByURL.ignoreLabels` command-line flag, which can be used for specifying the ignored list of labels when [sharding by `-remoteWrite.url` is enabled](https://docs.victoriametrics.com/vmagent/#sharding-among-remote-storages). Thanks to @edma2 for the idea and [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5938). * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/): add `-remoteWrite.shardByURL.ignoreLabels` command-line flag, which can be used for specifying the ignored list of labels when [sharding by `-remoteWrite.url` is enabled](https://docs.victoriametrics.com/vmagent/#sharding-among-remote-storages). Thanks to @edma2 for the idea and [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5938).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/): automatically reload updated root CA certificates from files without the need to restart `vmagent`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5526). * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/): automatically reload updated root CA certificates from files without the need to restart `vmagent`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5526).
@ -410,10 +410,10 @@ The v1.97.x line will be supported for at least 12 months since [v1.97.0](https:
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/): do not send unfinished [aggregation state](https://docs.victoriametrics.com/stream-aggregation/) on shutdown or [hot config reload](https://docs.victoriametrics.com/stream-aggregation/#configuration-update) by default, as it tend to produce unexpected anomalies with lower values. The old behavior can be restored by specifying `flush_on_shutdown: true` setting in streaming aggregation config. See more details [here](https://docs.victoriametrics.com/stream-aggregation/#aggregation-outputs). * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent/): do not send unfinished [aggregation state](https://docs.victoriametrics.com/stream-aggregation/) on shutdown or [hot config reload](https://docs.victoriametrics.com/stream-aggregation/#configuration-update) by default, as it tend to produce unexpected anomalies with lower values. The old behavior can be restored by specifying `flush_on_shutdown: true` setting in streaming aggregation config. See more details [here](https://docs.victoriametrics.com/stream-aggregation/#aggregation-outputs).
* FEATURE: [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation/): expand `%{ENV_VAR}` placeholders in config files with the corresponding environment variable values. * FEATURE: [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation/): expand `%{ENV_VAR}` placeholders in config files with the corresponding environment variable values.
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmagent/): expose ability to set OAuth2 endpoint parameters via the following command-line flags: * FEATURE: [vmalert](https://docs.victoriametrics.com/vmagent/): expose ability to set OAuth2 endpoint parameters via the following command-line flags:
- `-datasource.oauth2.endpointParams` for `-datasource.url` * `-datasource.oauth2.endpointParams` for `-datasource.url`
- `-notifier.oauth2.endpointParams` for `-notifier.url` * `-notifier.oauth2.endpointParams` for `-notifier.url`
- `-remoteRead.oauth2.endpointParams` for `-remoteRead.url` * `-remoteRead.oauth2.endpointParams` for `-remoteRead.url`
- `-remoteWrite.oauth2.endpointParams` for `-remoteWrite.url` * `-remoteWrite.oauth2.endpointParams` for `-remoteWrite.url`
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): add ability to proxy incoming requests to different backends based on the requested host via `src_hosts` option at `url_map`. See [these docs](https://docs.victoriametrics.com/vmauth/#generic-http-proxy-for-different-backends). * FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): add ability to proxy incoming requests to different backends based on the requested host via `src_hosts` option at `url_map`. See [these docs](https://docs.victoriametrics.com/vmauth/#generic-http-proxy-for-different-backends).
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): expose `vmauth_user_request_backend_errors_total` and `vmauth_unauthorized_user_request_backend_errors_total` [metrics](https://docs.victoriametrics.com/vmauth/#monitoring), which track the number of failed requests because of backend errors. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5565). * FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): expose `vmauth_user_request_backend_errors_total` and `vmauth_unauthorized_user_request_backend_errors_total` [metrics](https://docs.victoriametrics.com/vmauth/#monitoring), which track the number of failed requests because of backend errors. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5565).
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): add an ability to specify additional labels for [per-user metrics](https://docs.victoriametrics.com/vmauth/#monitoring) via `metric_labels` section. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5565). * FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth/): add an ability to specify additional labels for [per-user metrics](https://docs.victoriametrics.com/vmauth/#monitoring) via `metric_labels` section. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5565).
@ -429,8 +429,8 @@ The v1.97.x line will be supported for at least 12 months since [v1.97.0](https:
* FEATURE: [Alerting rules for VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#alerts): add `job` label to `DiskRunsOutOfSpace` alerting rule, so it is easier to understand to which installation the triggered instance belongs. * FEATURE: [Alerting rules for VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#alerts): add `job` label to `DiskRunsOutOfSpace` alerting rule, so it is easier to understand to which installation the triggered instance belongs.
* FEATURE: `vmstorage`: add tenant identifier for log messages regarding dropping excessive labels due to limits defined by `-maxLabelsPerTimeseries` or `-maxLabelValueLen` command-line flags. Previously, it was hard to understand to which tenant the dropped labels belong. * FEATURE: `vmstorage`: add tenant identifier for log messages regarding dropping excessive labels due to limits defined by `-maxLabelsPerTimeseries` or `-maxLabelValueLen` command-line flags. Previously, it was hard to understand to which tenant the dropped labels belong.
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add the ability to export and import query reports: * FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add the ability to export and import query reports:
- add a `Query Analyzer` page that allows you to build graphs from `JSON` data containing the results of executing a query request. * add a `Query Analyzer` page that allows you to build graphs from `JSON` data containing the results of executing a query request.
- add an `Export query` button to the graph that saves the result of executing the query in `JSON`. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5497). * add an `Export query` button to the graph that saves the result of executing the query in `JSON`. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5497).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add `-vmui.defaultTimezone` flag to set a default timezone. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5375) and [these docs](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmui#timezone-configuration). * FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add `-vmui.defaultTimezone` flag to set a default timezone. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5375) and [these docs](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmui#timezone-configuration).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): include UTC in the timezone selection dropdown for standardized time referencing. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5375). * FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): include UTC in the timezone selection dropdown for standardized time referencing. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5375).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add the ability to expand/collapse all tracing entries and download tracing data in .json format. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5677). * FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add the ability to expand/collapse all tracing entries and download tracing data in .json format. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5677).
@ -1083,3 +1083,4 @@ See changes [here](https://docs.victoriametrics.com/changelog_2020/#v1420)
## Previous releases ## Previous releases
See [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases). See [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).

2
go.mod
View file

@ -8,7 +8,7 @@ require (
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.3.2 github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.3.2
github.com/VictoriaMetrics/easyproto v0.1.4 github.com/VictoriaMetrics/easyproto v0.1.4
github.com/VictoriaMetrics/fastcache v1.12.2 github.com/VictoriaMetrics/fastcache v1.12.2
github.com/VictoriaMetrics/metrics v1.33.1 github.com/VictoriaMetrics/metrics v1.34.0
github.com/VictoriaMetrics/metricsql v0.75.1 github.com/VictoriaMetrics/metricsql v0.75.1
github.com/aws/aws-sdk-go-v2 v1.27.1 github.com/aws/aws-sdk-go-v2 v1.27.1
github.com/aws/aws-sdk-go-v2/config v1.27.17 github.com/aws/aws-sdk-go-v2/config v1.27.17

2
go.sum
View file

@ -71,6 +71,8 @@ github.com/VictoriaMetrics/fastcache v1.12.2 h1:N0y9ASrJ0F6h0QaC3o6uJb3NIZ9VKLjC
github.com/VictoriaMetrics/fastcache v1.12.2/go.mod h1:AmC+Nzz1+3G2eCPapF6UcsnkThDcMsQicp4xDukwJYI= github.com/VictoriaMetrics/fastcache v1.12.2/go.mod h1:AmC+Nzz1+3G2eCPapF6UcsnkThDcMsQicp4xDukwJYI=
github.com/VictoriaMetrics/metrics v1.33.1 h1:CNV3tfm2Kpv7Y9W3ohmvqgFWPR55tV2c7M2U6OIo+UM= github.com/VictoriaMetrics/metrics v1.33.1 h1:CNV3tfm2Kpv7Y9W3ohmvqgFWPR55tV2c7M2U6OIo+UM=
github.com/VictoriaMetrics/metrics v1.33.1/go.mod h1:r7hveu6xMdUACXvB8TYdAj8WEsKzWB0EkpJN+RDtOf8= github.com/VictoriaMetrics/metrics v1.33.1/go.mod h1:r7hveu6xMdUACXvB8TYdAj8WEsKzWB0EkpJN+RDtOf8=
github.com/VictoriaMetrics/metrics v1.34.0 h1:0i8k/gdOJdSoZB4Z9pikVnVQXfhcIvnG7M7h2WaQW2w=
github.com/VictoriaMetrics/metrics v1.34.0/go.mod h1:r7hveu6xMdUACXvB8TYdAj8WEsKzWB0EkpJN+RDtOf8=
github.com/VictoriaMetrics/metricsql v0.75.1 h1:cE5Ex6qSdI9vVT2BnsO6GpepB/8LPoSPKQmrM+fuQ84= github.com/VictoriaMetrics/metricsql v0.75.1 h1:cE5Ex6qSdI9vVT2BnsO6GpepB/8LPoSPKQmrM+fuQ84=
github.com/VictoriaMetrics/metricsql v0.75.1/go.mod h1:bEC8gqV+7kjnp97a8Gd6JbV1TraeZhfhvYAuaDuNR/U= github.com/VictoriaMetrics/metricsql v0.75.1/go.mod h1:bEC8gqV+7kjnp97a8Gd6JbV1TraeZhfhvYAuaDuNR/U=
github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1ow= github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1ow=

36
vendor/github.com/VictoriaMetrics/metrics/histogram.go generated vendored
View file

@ -47,7 +47,7 @@ var bucketMultiplier = math.Pow(10, 1.0/bucketsPerDecimal)
// Zero histogram is usable. // Zero histogram is usable.
type Histogram struct { type Histogram struct {
// Mu gurantees synchronous update for all the counters and sum. // Mu gurantees synchronous update for all the counters and sum.
mu sync.Mutex mu sync.RWMutex
decimalBuckets [decimalBucketsCount]*[bucketsPerDecimal]uint64 decimalBuckets [decimalBucketsCount]*[bucketsPerDecimal]uint64
@ -109,6 +109,32 @@ func (h *Histogram) Update(v float64) {
h.mu.Unlock() h.mu.Unlock()
} }
// Merge merges histograms
func (h *Histogram) Merge(b *Histogram) {
h.mu.Lock()
defer h.mu.Unlock()
b.mu.RLock()
defer b.mu.RUnlock()
h.lower += b.lower
h.upper += b.upper
h.sum += b.sum
for i, db := range b.decimalBuckets {
if db == nil {
continue
}
if h.decimalBuckets[i] == nil {
var b [bucketsPerDecimal]uint64
h.decimalBuckets[i] = &b
}
for j := range db {
h.decimalBuckets[i][j] = db[j]
}
}
}
// VisitNonZeroBuckets calls f for all buckets with non-zero counters. // VisitNonZeroBuckets calls f for all buckets with non-zero counters.
// //
// vmrange contains "<start>...<end>" string with bucket bounds. The lower bound // vmrange contains "<start>...<end>" string with bucket bounds. The lower bound
@ -116,7 +142,7 @@ func (h *Histogram) Update(v float64) {
// This is required to be compatible with Prometheus-style histogram buckets // This is required to be compatible with Prometheus-style histogram buckets
// with `le` (less or equal) labels. // with `le` (less or equal) labels.
func (h *Histogram) VisitNonZeroBuckets(f func(vmrange string, count uint64)) { func (h *Histogram) VisitNonZeroBuckets(f func(vmrange string, count uint64)) {
h.mu.Lock() h.mu.RLock()
if h.lower > 0 { if h.lower > 0 {
f(lowerBucketRange, h.lower) f(lowerBucketRange, h.lower)
} }
@ -135,7 +161,7 @@ func (h *Histogram) VisitNonZeroBuckets(f func(vmrange string, count uint64)) {
if h.upper > 0 { if h.upper > 0 {
f(upperBucketRange, h.upper) f(upperBucketRange, h.upper)
} }
h.mu.Unlock() h.mu.RUnlock()
} }
// NewHistogram creates and returns new histogram with the given name. // NewHistogram creates and returns new histogram with the given name.
@ -223,9 +249,9 @@ func (h *Histogram) marshalTo(prefix string, w io.Writer) {
} }
func (h *Histogram) getSum() float64 { func (h *Histogram) getSum() float64 {
h.mu.Lock() h.mu.RLock()
sum := h.sum sum := h.sum
h.mu.Unlock() h.mu.RUnlock()
return sum return sum
} }

2
vendor/github.com/VictoriaMetrics/metrics/process_metrics_linux.go generated vendored
View file

@ -80,7 +80,7 @@ func writeProcessMetrics(w io.Writer) {
WriteCounterUint64(w, "process_major_pagefaults_total", uint64(p.Majflt)) WriteCounterUint64(w, "process_major_pagefaults_total", uint64(p.Majflt))
WriteCounterUint64(w, "process_minor_pagefaults_total", uint64(p.Minflt)) WriteCounterUint64(w, "process_minor_pagefaults_total", uint64(p.Minflt))
WriteGaugeUint64(w, "process_num_threads", uint64(p.NumThreads)) WriteGaugeUint64(w, "process_num_threads", uint64(p.NumThreads))
WriteGaugeUint64(w, "process_resident_memory_bytes", uint64(p.Rss)*4096) WriteGaugeUint64(w, "process_resident_memory_bytes", uint64(p.Rss)*uint64(os.Getpagesize()))
WriteGaugeUint64(w, "process_start_time_seconds", uint64(startTimeSeconds)) WriteGaugeUint64(w, "process_start_time_seconds", uint64(startTimeSeconds))
WriteGaugeUint64(w, "process_virtual_memory_bytes", uint64(p.Vsize)) WriteGaugeUint64(w, "process_virtual_memory_bytes", uint64(p.Vsize))
writeProcessMemMetrics(w) writeProcessMemMetrics(w)

17
vendor/github.com/VictoriaMetrics/metrics/push.go generated vendored
View file

@ -32,6 +32,10 @@ type PushOptions struct {
// By default the compression is enabled. // By default the compression is enabled.
DisableCompression bool DisableCompression bool
// Method is an optional of HTTP request method.
// By default the Method is GET.
Method string
// Optional WaitGroup for waiting until all the push workers created with this WaitGroup are stopped. // Optional WaitGroup for waiting until all the push workers created with this WaitGroup are stopped.
WaitGroup *sync.WaitGroup WaitGroup *sync.WaitGroup
} }
@ -264,6 +268,7 @@ func PushMetricsExt(ctx context.Context, pushURL string, writeMetrics func(w io.
type pushContext struct { type pushContext struct {
pushURL *url.URL pushURL *url.URL
method string
pushURLRedacted string pushURLRedacted string
extraLabels string extraLabels string
headers http.Header headers http.Header
@ -295,6 +300,11 @@ func newPushContext(pushURL string, opts *PushOptions) (*pushContext, error) {
return nil, fmt.Errorf("missing host in pushURL=%q", pushURL) return nil, fmt.Errorf("missing host in pushURL=%q", pushURL)
} }
method := opts.Method
if len(method) == 0 {
method = http.MethodGet
}
// validate ExtraLabels // validate ExtraLabels
extraLabels := opts.ExtraLabels extraLabels := opts.ExtraLabels
if err := validateTags(extraLabels); err != nil { if err := validateTags(extraLabels); err != nil {
@ -317,6 +327,7 @@ func newPushContext(pushURL string, opts *PushOptions) (*pushContext, error) {
client := &http.Client{} client := &http.Client{}
return &pushContext{ return &pushContext{
pushURL: pu, pushURL: pu,
method: method,
pushURLRedacted: pushURLRedacted, pushURLRedacted: pushURLRedacted,
extraLabels: extraLabels, extraLabels: extraLabels,
headers: headers, headers: headers,
@ -367,18 +378,18 @@ func (pc *pushContext) pushMetrics(ctx context.Context, writeMetrics func(w io.W
// Prepare the request to sent to pc.pushURL // Prepare the request to sent to pc.pushURL
reqBody := bytes.NewReader(bb.B) reqBody := bytes.NewReader(bb.B)
req, err := http.NewRequestWithContext(ctx, "GET", pc.pushURL.String(), reqBody) req, err := http.NewRequestWithContext(ctx, pc.method, pc.pushURL.String(), reqBody)
if err != nil { if err != nil {
panic(fmt.Errorf("BUG: metrics.push: cannot initialize request for metrics push to %q: %w", pc.pushURLRedacted, err)) panic(fmt.Errorf("BUG: metrics.push: cannot initialize request for metrics push to %q: %w", pc.pushURLRedacted, err))
} }
// Set the needed headers req.Header.Set("Content-Type", "text/plain")
// Set the needed headers, and `Content-Type` allowed be overwrited.
for name, values := range pc.headers { for name, values := range pc.headers {
for _, value := range values { for _, value := range values {
req.Header.Add(name, value) req.Header.Add(name, value)
} }
} }
req.Header.Set("Content-Type", "text/plain")
if !pc.disableCompression { if !pc.disableCompression {
req.Header.Set("Content-Encoding", "gzip") req.Header.Set("Content-Encoding", "gzip")
} }

2
vendor/modules.txt vendored
View file

@ -115,7 +115,7 @@ github.com/VictoriaMetrics/easyproto
# github.com/VictoriaMetrics/fastcache v1.12.2 # github.com/VictoriaMetrics/fastcache v1.12.2
## explicit; go 1.13 ## explicit; go 1.13
github.com/VictoriaMetrics/fastcache github.com/VictoriaMetrics/fastcache
# github.com/VictoriaMetrics/metrics v1.33.1 # github.com/VictoriaMetrics/metrics v1.34.0
## explicit; go 1.17 ## explicit; go 1.17
github.com/VictoriaMetrics/metrics github.com/VictoriaMetrics/metrics
# github.com/VictoriaMetrics/metricsql v0.75.1 # github.com/VictoriaMetrics/metricsql v0.75.1