Commit graph

5777 commits

Author SHA1 Message Date
Aliaksandr Valialkin
9b5245a836
app/vmselect/promql: fix TestIncrementalAggr test on systems less than 3 CPU cores
This is a follow-up for 4856a4cf5a
2023-03-25 01:21:43 -07:00
Aliaksandr Valialkin
c780c6a280
app/vmselect: optimize incremental aggregates a bit
Substitute sync.Map with an ordinary slice indexed by workerID.
This should reduce the overhead when updating the incremental aggregate state
2023-03-24 23:50:31 -07:00
oliverpool
a3bc64e7ed
app/vmselect/promql: add test to ensure 8-byte alignment (#3948)
See 0af9e2b693
2023-03-24 23:49:04 -07:00
Aliaksandr Valialkin
0dcd796323
vendor: run make vendor-update 2023-03-24 22:29:03 -07:00
Alexander Marshalov
95c67d8309
allowed using dashes and dots in environment variables names (#4009)
* allowed using dashes and dots in environment variables names for templating config files with envtemplate (#3999)

Signed-off-by: Alexander Marshalov <_@marshalov.org>

* Apply suggestions from code review

---------

Signed-off-by: Alexander Marshalov <_@marshalov.org>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-03-24 22:23:52 -07:00
Aliaksandr Valialkin
8a878fa169
app/vmbackup: simplify code a bit after 5ba347bd2c
Unconditionally call deleteSnapshot() func just after making the snapshot, either successful or unsuccessful

Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2055
2023-03-24 22:20:20 -07:00
Zakhar Bessarab
91a5661b86
app/vmbackup: delete created snapshot in case of error during backup (#4008)
Related issue: #2055

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-03-24 22:20:20 -07:00
Nikolay
c2380f73e6
lib/netutil: log only parsing errors for proxy-protocol (#3985)
* lib/netutil: log only parsing errors for proxy-protocol

Previosly every error was logged. With configured TCP health checks at load-balancer or kubernetes, vmauth spams a lot of false positive error message into logs

* Update docs/CHANGELOG.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Update lib/netutil/tcplistener.go

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-03-24 22:14:32 -07:00
Aliaksandr Valialkin
e7c20f0c75
lib/{mergeset,storage}: prevent from long wait time when creating a snapshot under high data ingestion rate
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3551
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3873
2023-03-24 22:12:08 -07:00
Dmytro Kozlov
67a526f5f7
lib/storage: fix collect downsampling metrics (#489)
* lib/storage: fix downsampling

* lib/storage: update logic

* lib/storage: fix comments, removed unneeded check
2023-03-19 23:34:33 -07:00
Aliaksandr Valialkin
4793665c4f
docs/CHANGELOG.md: cut v1.87.3 2023-03-12 23:24:37 -07:00
Aliaksandr Valialkin
1050d6bc0f
app/vmselect/promql: prevent from cannot unmarshal timeseries from rollupResultCache panic after the upgrade to v1.89.0
The issue has been introduced in 0af9e2b693
2023-03-12 19:10:48 -07:00
Aliaksandr Valialkin
dedac538c6
Makefile: update golangci-lint from v1.51.1 to v1.51.2
See https://github.com/golangci/golangci-lint/releases/tag/v1.51.2
2023-03-12 17:12:49 -07:00
Aliaksandr Valialkin
5a9300be13
app/vmselect: remove data race on updating EvalConfig.IsPartialResponse from concurrently running goroutines
This properly returns `is_partial: true` for partial responses.
2023-03-12 16:57:28 -07:00
Aliaksandr Valialkin
c8f8aa85a2
app/vmselect/promql: prevent from SIGBUS crash on architecures, which deny unaligned access to 8-byte words (e.g. ARM)
Thanks to @oliverpool for nailing down the root cause of the issue and for the initial attempt to fix it
at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3927
2023-03-12 16:56:37 -07:00
Aliaksandr Valialkin
fe1d959ca5
docs/CHANGELOG.md: document 113a89904d 2023-03-12 03:00:41 -07:00
Nikolay
ed2e4685d9
lib/vmselectapi: fixes regression for disable compression setting (#3932)
after vmselect api refactoring it wasn't possible to disable response cache.
This patch restores correct behavior for rpc.disableCompression flag
2023-03-12 01:59:38 -08:00
Aliaksandr Valialkin
60fcf88e45
vendor: make vendor-update 2023-03-12 01:49:22 -08:00
Roman Khavronenko
3d2cfc4f33
security: bump go version to 1.20.2 (#3935)
upgrade Go builder from Go1.20.1 to Go1.20.2
See the list of issues addressed in Go1.20.2 here (https://github.com/golang/go/issues?q=milestone%3AGo1.20.2+label%3ACherryPickApproved).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-03-12 01:38:51 -08:00
Aliaksandr Valialkin
7f34158db5
app/vmselect/netstorage: do not intern string representation of MetricName for time series received from vmstorage
It has been appeared that this interning may lead to increased memory usage and increased CPU usage
when vmselect performs queries, which select big number of time series.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3692
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3863
2023-03-12 01:36:07 -08:00
Aliaksandr Valialkin
f5d50c4bc5
docs/CHANGELOG.md: document 927d9da270 2023-03-12 01:33:53 -08:00
Nikolay
ea038a8940
lib/storage: correctly handle io.EOF error for pre-fetched metrics (#3946)
io.EOF shouldn't be returned from this function. It breaks all search
API logic and may result in empty query results.
2023-03-12 01:29:05 -08:00
Aliaksandr Valialkin
7feec58f91
docs/CHANGELOG.md: clarify the description for 6bfe9cc733
- Add the panic message to the description, so it is easier to google
- Add a link to the corresponding bugreport

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3897
2023-03-12 01:26:28 -08:00
Nikolay
ce950d3b06
lib{mergset,storage}: prevent possible race condition with logging st… (#3900)
lib{mergset,storage}: prevent possible race condition with logging stats for merges

Previously partwrapper could be release by background process and reference for part may be invalid
during logging stats. It will lead to panic at vmstorage
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3897
2023-03-12 01:23:55 -08:00
Aliaksandr Valialkin
bc144e2b05
all: follow-up for 7a3e16e774
- Sync the description for -httpListenAddr.useProxyProtocol command-line flag at vmagent and vmauth,
  so it is consistent with the description at vmauth and victoria-metrics
- Add a sample of panic text to docs/CHANGELOG.md, so it could be googled
- Mention the -httpListenAddr.useProxyProtocol command-line flag in the description for the bugfix

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3335
2023-03-12 01:19:55 -08:00
Nikolay
c80d0aaaf0
lib/netutil: fixes panic at proxy protocol (#3905)
it may occur if non proxy protocol message received by tcp server.
Listener Accept method must return only non-recoverable errors.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3335
2023-03-12 01:12:53 -08:00
Zakhar Bessarab
3f57d4d489
lib/promscrape: correctly register vm_promscrape_config_* metrics (#3876)
* lib/promscrape: set `vm_promscrape_config_last_reload_successful` to 1 if there was no promscrape config provided

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape: register `vm_promscrape_config_*` metrics only in case promscrape config is used

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-27 12:06:22 -08:00
Dmytro Kozlov
d161873857
app/vmctl: skip series if measurement not found (#3869)
app/vmctl: skip measurements with no fields for influxdb mode
2023-02-27 12:04:48 -08:00
Aliaksandr Valialkin
bd319ea7e5
docs/CHANGELOG.md: mention that v1.87.2 is an LTS release 2023-02-24 16:12:56 -08:00
Aliaksandr Valialkin
27afb0ac37
docs/CHANGELOG.md: document v1.79.9 release 2023-02-24 15:11:48 -08:00
Aliaksandr Valialkin
16578b58b6
docs/CHANGELOG.md: cut v1.87.2 release 2023-02-24 15:09:40 -08:00
Aliaksandr Valialkin
5816678a86
docs/CHANGELOG.md: document d8eaa511b0 2023-02-24 12:44:36 -08:00
Zakhar Bessarab
c9547f4fba
lib/{fs,mergeset,storage}: skip .must-remove. dirs when creating snapshot (#3858) (#3867) 2023-02-24 12:44:35 -08:00
Aliaksandr Valialkin
1026e25f33
docs/CHANGELOG.md: typo fix: scrape scrape -> scrape 2023-02-24 12:34:07 -08:00
Aliaksandr Valialkin
95b66f3de5
lib/promscrape: follow-up for 43e104a83f
- Return immediately on context cancel during the backoff sleep.
  This should help with https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3747

- Add a comment describing why the second attempt to obtain the response from remote side
  is perfromed immediately after the first attempt.

- Remove fasthttp dependency from lib/promscrape/discoveryutils

- Set context deadline before calling doRequestWithPossibleRetry().
  This simplifies the doRequestWithPossibleRetry() a bit.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3293
2023-02-24 12:26:33 -08:00
Zakhar Bessarab
3f64dae46b
fix: do not use exponential backoff for first retry of scrape request (#3824)
* fix: do not use exponential backoff for first retry of scrape request (#3293)

* lib/promscrape: refactor `doRequestWithPossibleRetry` backoff to simplify logic

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Update lib/promscrape/client.go

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* lib/promscrape: refactor `doRequestWithPossibleRetry` to make it more straightforward

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-02-24 12:26:32 -08:00
Alexander Marshalov
808a06bfd4
fix interpolate function for filling only intermediate gaps (#3816) (#3857)
* fix interpolate function for filling only intermediate gaps (#3816)

* Update docs/CHANGELOG.md

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-23 19:30:23 -08:00
Aliaksandr Valialkin
8dd8972a7a
docs/CHANGELOG.md: document 6d019a3c37
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3830
2023-02-23 19:30:23 -08:00
Mattias Ängehov
13d0a2a3e7
Azure Service Discovery - Fix token fetch for Container Apps/App Services (#3832)
* Modify API version when running in Container App

* Handle expires on from token response

Response from IMDS does not always contain expires in value which is
currently used to get the token expiry time. An example resources that
doesn't provide it are Container Apps and App Service.

Signed-off-by: Mattias Ängehov <mattias.angehov@castoredc.com>

* Fix client id parameter for user assigned identity

* Apply suggestions from code review

---------

Signed-off-by: Mattias Ängehov <mattias.angehov@castoredc.com>
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2023-02-23 19:30:23 -08:00
Aliaksandr Valialkin
6ac2f62577
docs/CHANGELOG.md: document d2b92d3264
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3747
2023-02-22 18:29:28 -08:00
Zakhar Bessarab
3dfb9e732c
lib/promscrape: fix cancelling in-flight scrape requests during configuration reload (#3853)
* lib/promscrape: fix cancelling in-flight scrape requests during configuration reload (see #3747)

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape: fix order of params for `doRequestWithPossibleRetry` to follow codestyle

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape: accept deadline explicitly and extend passed context for local use

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-02-22 18:28:28 -08:00
Aliaksandr Valialkin
2961a90f7d
vendor: update github.com/VictoriaMetrics/fasthttp from v1.1.0 to v1.2.0
The v1.2.0 adds HostClient.DoCtx() function, which is needed by https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3747
for implementing fast canceling of pending requests to scrape targets on config update
2023-02-22 18:27:47 -08:00
Aliaksandr Valialkin
6353aead68
vendor: make vendor-update 2023-02-18 23:16:34 -08:00
Aliaksandr Valialkin
cc2a82a7dc
all: update Go builder from Go1.20.0 to Go1.20.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.20.1+label%3ACherryPickApproved
2023-02-14 23:13:32 -08:00
Aliaksandr Valialkin
68e86a5d49
docs/CHANGELOG.md: improve the docs for 8ea02eaa8e 2023-02-14 22:47:46 -08:00
Droxenator
732dd5357a
fixed opentsdbListenAddr timestamp conversion (#3810)
Co-authored-by: Andrei Ivanov <a.ivanov@corp.mail.ru>
2023-02-14 22:47:45 -08:00
Aliaksandr Valialkin
ddd8d01cf5
lib/{mergeset,storage}: allow at least 3 concurrent flushes during background merges on systems with 1 or 2 CPU cores
This should prevent from data ingestion slowdown and query performance degradation
on systems with small number of CPU cores (1 or 2), when big merge is performed.

This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3790

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
2023-02-11 12:17:46 -08:00
Aliaksandr Valialkin
49f69a7068
all: update alpine base docker image from 1.17.1 to 1.17.2
See https://alpinelinux.org/posts/Alpine-3.17.2-released.html
2023-02-11 12:16:14 -08:00
Aliaksandr Valialkin
a68bb879d4
vendor: make vendor-update 2023-02-09 14:48:38 -08:00
Aliaksandr Valialkin
6ee15f94f8
docs/CHANGELOG.md: document d621d50d4fb3b43a0bcb4419bee979f0192d38fe 2023-02-09 14:40:15 -08:00