Commit graph

6200 commits

Author SHA1 Message Date
Aliaksandr Valialkin
a32a9070c1
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2023-02-27 19:25:57 -08:00
Aliaksandr Valialkin
06854738b6
.github/workflows/check-licenses.yml: use the correct version of Go - 1.20.1 - instead of 1.21.0 2023-02-27 19:25:13 -08:00
Aliaksandr Valialkin
b596228765
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2023-02-27 17:36:37 -08:00
Aliaksandr Valialkin
95e1173423
docs/CHANGELOG.md: document v1.79.10 release 2023-02-27 17:35:48 -08:00
Aliaksandr Valialkin
d0f9a5d4c4
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2023-02-27 15:39:19 -08:00
Aliaksandr Valialkin
8937de5f99
vendor: make vendor-update 2023-02-27 15:32:45 -08:00
Aliaksandr Valialkin
8288e327ee
docs/CHANGELOG.md: cut v1.88.1 2023-02-27 15:28:18 -08:00
Aliaksandr Valialkin
dfe3939665
docs/CHANGELOG.md: link to the issue, which may benefit from -internStringDisableCache command-line flag
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3863
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3692
2023-02-27 14:55:40 -08:00
Aliaksandr Valialkin
472a9360e6
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2023-02-27 14:19:26 -08:00
Aliaksandr Valialkin
46127b432d
lib/bytesutil: add -internStringDisableCache and -internStringCacheExpireDuration command-line flags
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3872
2023-02-27 14:16:49 -08:00
Aliaksandr Valialkin
0d3f31f60e
lib/storage: follow-up for 39cdc546dd
- Use flag.Duration instead of flagutil.Duration for -snapshotCreateTimeout,
  since the flagutil.Duration is intended mostly for big durations, e.g. days, months and years,
  while the -snapshotCreateTimeout is usually smaller than one hour.
- Add links to https://docs.victoriametrics.com/#how-to-work-with-snapshots in docs/CHANGELOG.md,
  so readers could easily find the corresponding docs when reading the changelog.
- Properly remove all the created directories on unsuccessful attempt to create
  snapshot in Storage.CreateSnapshot().

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3551
2023-02-27 13:07:38 -08:00
Zakhar Bessarab
39cdc546dd
lib/storage: enhancements for snapshots process (#3873)
* lib/{fs,mergeset,storage}: skip `.must-remove.` dirs when creating snapshot (#3858)

* lib/{mergeset,storage}: add timeout configuration for snapshots creation, remove incomplete snapshots from storage

* docs: fix formatting

* app/vmstorage: add metrics to track status of snapshots

* app/vmstorage: use `vm_http_requests_total` metric for snapshot endpoints metrics, rename new flag to make name more clear

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vmstorage: update flag name in docs

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vmstorage: reflect new metrics names change in docs

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-27 12:12:03 -08:00
Zakhar Bessarab
5fadd58cf6
lib/promscrape: correctly register vm_promscrape_config_* metrics (#3876)
* lib/promscrape: set `vm_promscrape_config_last_reload_successful` to 1 if there was no promscrape config provided

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape: register `vm_promscrape_config_*` metrics only in case promscrape config is used

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-27 11:53:53 -08:00
Zakhar Bessarab
ac6d937372
doc: add changelog reference for vmgateway OpenID discovery (#3877)
* doc: add changelog reference for vmgateway OpenID discovery

* doc: add vmgateway docs

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-27 11:50:22 -08:00
Aliaksandr Valialkin
b3bb18d674
app/vmselect/promql: fix panic when calculating aggr_func(rollup*())
The panic has been introduced in dac21d874b
2023-02-27 11:48:27 -08:00
Zakhar Bessarab
ac4c7adec6
app/vmgateway: add new flag doc
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-02-27 11:18:34 -08:00
Zakhar Bessarab
5d7da8479f
app/vmgateway: fix typo in docs
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-02-27 11:18:08 -08:00
Zakhar Bessarab
4ee73f54a6
app/vmgateway: add OpenID discovery of JWKS endpoints
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-02-27 11:17:39 -08:00
Aliaksandr Valialkin
23871fb0bf
app/vmagent: add -remoteWrite.vmProtoCompressLevel command-line flag for tuning the compression level for VictoriaMetrics remote write protocol 2023-02-27 11:03:49 -08:00
Aliaksandr Valialkin
1a6f2f07fd
lib/httpserver: use github.com/klauspost/compress/gzhttp for compressing http responses
This allows removing gzip-related code from lib/httpserver.
2023-02-27 10:33:43 -08:00
Dmytro Kozlov
27c9446520
app/vmctl: skip series if measurement not found (#3869)
app/vmctl: skip measurements with no fields for influxdb mode
2023-02-27 14:28:47 +01:00
Dmytro Kozlov
bcbf73225e
app/vmctl: enable version flag (#3868) 2023-02-27 10:30:53 +01:00
Aliaksandr Valialkin
255a0cf635
all: add makefile rules for GOARCH=s390x for all the VictoriaMetrics components
This is a follow-up for 007530f882
2023-02-26 12:36:51 -08:00
v1gnesh
007530f882
Option to build for s390x (Linux on IBM Z) (#3870)
Added `victoria-metrics-linux-s390x` to allow single node builds for `s390x` platform.

Leaving other packaging options at the moment, as on this platform, they're mostly going to be built from/with container images hosted within the company as a base, and not alpine.
2023-02-26 12:29:50 -08:00
Aliaksandr Valialkin
b00fcad604
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2023-02-26 12:28:11 -08:00
Aliaksandr Valialkin
f7ef80aaad
.golangci.yml: properly enable revive linter and fix all the warnings it detects 2023-02-26 12:18:59 -08:00
Aliaksandr Valialkin
ffa327d6d1
app/vmagent: use the provided auth options when checking whether the remote storage supports VictoriaMetrics remote write protocol
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3847
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225
2023-02-26 12:07:47 -08:00
Aliaksandr Valialkin
3d755041c3
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2023-02-24 18:59:30 -08:00
Aliaksandr Valialkin
eece61a611
deployment/marketplace: update VictoriaMetrics release from v1.87.1 to v1.88.0 2023-02-24 18:58:39 -08:00
Aliaksandr Valialkin
0c016279d5
deployment/docker: update VictoriaMetrics docker tag from v1.87.1 to v1.88.0 2023-02-24 18:57:31 -08:00
Aliaksandr Valialkin
e22a9d6ba6
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2023-02-24 17:55:04 -08:00
Aliaksandr Valialkin
2d36dbcfa9
docs/CHANGELOG.md: cut v1.88.0 2023-02-24 17:54:21 -08:00
Aliaksandr Valialkin
9d7dc73038
vendor: make vendor-update 2023-02-24 17:33:28 -08:00
Aliaksandr Valialkin
63d9048990
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2023-02-24 17:32:15 -08:00
Aliaksandr Valialkin
8cfe4064b5
vendor: make vendor-update 2023-02-24 17:26:51 -08:00
Aliaksandr Valialkin
8db1fd2f78
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2023-02-24 17:17:34 -08:00
Aliaksandr Valialkin
b4bf8dc2f0
docs: update -help output after e1c3267e34 2023-02-24 17:16:13 -08:00
Roman Khavronenko
e1c3267e34
vmselect/promql: check for deadline in count_values fn (#3806)
* vmselect/promql: check for deadline in `count_values` fn

`count_values` could be very slow during the data processing.
Checking for deadline between iterations supposed to reduce
probability of exceeding `search.maxQueryDuration`.

The change also adds a new trace record, which captures the time
spent in aggregation function. Before that, the trace for aggr funcs
could be confusing since it doesn't account for all the places where
time was spent.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-24 16:59:26 -08:00
Aliaksandr Valialkin
c33cc4322c
docs/CHANGELOG.md: document v1.87.2 release 2023-02-24 16:14:28 -08:00
Aliaksandr Valialkin
6a88d5402d
docs/CHANGELOG.md: document v1.79.9 release 2023-02-24 15:10:48 -08:00
Aliaksandr Valialkin
8f0afc656e
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2023-02-24 13:49:32 -08:00
Roman Khavronenko
dac21d874b
metricsql: support optional 2nd argument for rollup functions (#3841)
* metricsql: support optional 2nd argument for rollup functions

Support optional 2nd argument `min`, `max` or `avg` for rollup functions:
 * rollup
 * rollup_delta
 * rollup_deriv
 * rollup_increase
 * rollup_rate
 * rollup_scrape_interval

 If second argument is passed, then rollup function will return only the selected aggregation type.
 This change can be useful for situations where only one type of rollup calculation is needed.
 For example, `rollup_rate(requests_total[5m], "max")`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-24 13:47:52 -08:00
Aliaksandr Valialkin
87aeeec3e8
docs/CHANGELOG.md: document d8eaa511b0 2023-02-24 12:42:02 -08:00
Zakhar Bessarab
d8eaa511b0
lib/{fs,mergeset,storage}: skip .must-remove. dirs when creating snapshot (#3858) (#3867) 2023-02-24 12:38:42 -08:00
Aliaksandr Valialkin
a2340e6c95
docs/CHANGELOG.md: typo fix: scrape scrape -> scrape 2023-02-24 12:33:29 -08:00
Aliaksandr Valialkin
c6ad3692ad
lib/promscrape: follow-up for 43e104a83f
- Return immediately on context cancel during the backoff sleep.
  This should help with https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3747

- Add a comment describing why the second attempt to obtain the response from remote side
  is perfromed immediately after the first attempt.

- Remove fasthttp dependency from lib/promscrape/discoveryutils

- Set context deadline before calling doRequestWithPossibleRetry().
  This simplifies the doRequestWithPossibleRetry() a bit.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3293
2023-02-24 12:20:42 -08:00
Zakhar Bessarab
43e104a83f
fix: do not use exponential backoff for first retry of scrape request (#3824)
* fix: do not use exponential backoff for first retry of scrape request (#3293)

* lib/promscrape: refactor `doRequestWithPossibleRetry` backoff to simplify logic

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Update lib/promscrape/client.go

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* lib/promscrape: refactor `doRequestWithPossibleRetry` to make it more straightforward

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-02-24 11:39:56 -08:00
Aliaksandr Valialkin
c87c7d1e29
app/vmselect/promql: measure the time required for calculating the aggregate function from the prepared source time series 2023-02-23 20:05:14 -08:00
Aliaksandr Valialkin
be94882ada
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2023-02-23 19:27:31 -08:00
Aliaksandr Valialkin
8b7a828c65
app/vmselect/vmui: make vmui-update after d4fc0ed874 2023-02-23 19:25:52 -08:00