Commit graph

6912 commits

Author SHA1 Message Date
Aliaksandr Valialkin
5827ff7e30
vendor: run make vendor-update 2024-03-01 01:19:32 +02:00
Aliaksandr Valialkin
3061790d85
app/{vmagent,vminsert}: follow-up for 434a5803e7
Document the /opentelemetry/v1/metrics endpoint instead of /opentelemetry/api/v1/push,
since the /v1/metrics suffix is hardcoded in OpenTelemetry protocol specification.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5871
2024-02-29 18:09:15 +02:00
Nikolay
003d1fecd6
app/{vmagent,vminsert}: adds /v1/metrics suffix for opentelemetry route path (#5871)
* app/{vmagent,vminsert}: adds /v1/metrics suffix for opentelemetry route path
it must fix compatibility with opentemetry-collector [spec](https://opentelemetry.io/docs/specs/otlp/\#otlphttp-request)
this suffix is hard-coded and cannot be changed with collector configuration

* Apply suggestions from code review

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-29 18:08:34 +02:00
Hui Wang
3bb36bb510
metricsql: fix label_join() when dst_label is equal to one of the `… (#5886)
* metricsql: fix label_join() when `dst_label` is equal to one of the `src_label`

* Update app/vmselect/promql/transform.go

* Update docs/CHANGELOG.md

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-29 16:04:31 +02:00
Aliaksandr Valialkin
1258c22881
deployment/docker: downgrade Go builder from 1.22.0 to 1.21.7
Go1.22.0 has the bug https://github.com/golang/go/issues/65705 ,
which prevents vmagent from normal operation.
2024-02-29 13:49:12 +02:00
Aliaksandr Valialkin
196b5ffa65
docs/CHANGELOG.md: document d68bb658ce
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5833
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5834
2024-02-23 01:27:28 +02:00
Anton L
11fb95f25b
5833 Fix Deadlock when using shardByURL of VMAgent (#5834) 2024-02-23 01:25:08 +02:00
Alexander Marshalov
708551399c
[lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5814)
* [lib/promutils, lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5801)

* fixed tests

* fixed test

* Revert "fixed test"

This reverts commit 8a29764806.

* Revert "fixed tests"

This reverts commit 9ce13d1042.

* Revert "[lib/promutils, lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5801)"

This reverts commit a7a04bd4

* [lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5801)

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-02-23 00:56:37 +02:00
hagen1778
5f028e54b5
app/vmalert: consistently sort groups by name and filename on /groups page
This should prevent non-deterministic sorting for groups with identical names.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-23 00:30:23 +02:00
Aliaksandr Valialkin
83cf46719e
docs/CHANGELOG.md: document f8207e33a2 2024-02-17 17:59:01 +02:00
Alexander Marshalov
1cb512fd0a
lib/httputils: fixed error message for getting zero duration (#5795) (#5812) 2024-02-17 17:57:39 +02:00
Aliaksandr Valialkin
cf60be4a0b
vendor: run make vendor-update 2024-02-14 19:00:11 +02:00
Aliaksandr Valialkin
b08ca0b9ae
docs/CHANGELOG.md: cut v1.93.12 LTS release 2024-02-14 18:57:46 +02:00
Aliaksandr Valialkin
cc4aff863d
Makefile: run go mod tidy with -compat=1.22 after 95222b2079 2024-02-12 22:47:43 +02:00
Aliaksandr Valialkin
95407d8e9b
all: upgrade Go builder from Go1.21.7 to Go1.22.0
See https://go.dev/doc/go1.22
2024-02-12 22:41:50 +02:00
Aliaksandr Valialkin
3a831da481
lib/promscrape: go fmt after d918a7267d 2024-02-12 20:03:19 +02:00
Aliaksandr Valialkin
d918a7267d
lib/promscrape: fix TestGetStaticScrapeWorkSuccess test after ca67926952 2024-02-12 20:00:29 +02:00
Aliaksandr Valialkin
c07c9ad028
lib/mergeset: do not panic on too long items passed to Table.AddItems()
Instead, log a sample of these long items once per 5 seconds into error log,
so users could notice and fix the issue with too long labels or too many labels.

Previously this panic could occur in production when ingesting samples with too long labels.
2024-02-12 19:45:13 +02:00
Aliaksandr Valialkin
7fef8ba908
app/victoria-metrics: properly send staleness markers on victoriametrics shutdown if -selfScrapeInterval > 0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/943
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
2024-02-08 15:51:40 +02:00
Aliaksandr Valialkin
70eead1dbc
docs/CHANGELOG.md: clarify the bugfix description for e1bf8440eb 2024-02-08 13:02:48 +02:00
Aliaksandr Valialkin
246a7758ac
lib/mergeset: prevent from possible too big indexBlockSize panic
This panic could occur when samples with too long label values are ingested into VictoriaMetrics.
This could result in too long fistItem and commonPrefix values at blockHeader (up to 64kb each).
This may inflate the maximum index block size by 4 * maxIndexBlockSize.
2024-02-08 12:58:57 +02:00
Aliaksandr Valialkin
0a4654af3d
app/vmselect/promql: properly handle precision errors in rollup functions
changes(), increases_over_time() and resets() shouldn't take into account
value changes, which may occur because of precision errors.

The maximum guaranteed precision for raw samples stored in VictoriaMetrics is 12 decimal digits.
So do not count relative changes for values if they are smaller than 1e-12 comparing to the value.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/767
2024-02-08 02:34:24 +02:00
Aliaksandr Valialkin
b3f057f75a
all: update Go builder from Go1.21.6 to Go1.21.7
See https://github.com/golang/go/issues?q=milestone%3AGo1.21.7+label%3ACherryPickApproved
2024-02-07 04:03:03 +02:00
Aliaksandr Valialkin
c7b3969b43
lib/storage: move fixupTimestamps() call to Block.Init()
This is a follow-up for 0bf7921721
2024-02-06 22:45:20 +02:00
Zakhar Bessarab
25f83e6c54
lib/storage/raw_row: properly initialize TS for tmp blocks (#5762)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-02-06 22:45:18 +02:00
Aliaksandr Valialkin
442bd6dc7a
vendor: update github.com/VictoriaMetrics/metricsql from v0.70.0 to v0.70.1
This should help with the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5604
2024-02-06 22:15:46 +02:00
Aliaksandr Valialkin
4d08a0327f
docs/CHANGELOG.md: cut v1.93.11 LTS release 2024-02-01 17:14:58 +02:00
Aliaksandr Valialkin
248e84754d
vendor: run make vendor-update 2024-02-01 17:10:39 +02:00
Aliaksandr Valialkin
be959a5c01
deployment/docker: upgrade base Docker image from Alpine 3.19.0 to 3.19.1
See https://www.alpinelinux.org/posts/Alpine-3.19.1-released.html
2024-01-30 22:48:18 +02:00
Roman Khavronenko
ca67926952
lib/promscrape: respect 0 value for series_limit param (#5663)
* lib/promscrape: respect `0` value for `series_limit` param

Respect `0` value for `series_limit` param in `scrape_config`
even if global limit was set via `-promscrape.seriesLimitPerTarget`.
Previously, `0` value will be ignored in favor of `-promscrape.seriesLimitPerTarget`.

This behavior aligns with possibility to override `series_limit` value via
relabeling with `__series_limit__` label.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update docs/CHANGELOG.md

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-23 13:17:13 +02:00
Aliaksandr Valialkin
48a851d7d1
app/vmselect: handle negative time range start in a generic manner inside NewSearchQuery()
This is a follow-up for cf03e11d89

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5553
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5630
2024-01-22 01:41:24 +02:00
Hui Wang
26980cd76d
lib/promscrape/discovery/kubernetes: fix watcher start order for roles endpoints and endpointslice (#5557)
* lib/promscrape/discovery/kubernetes: fix watcher start order for roles endpoints and endpointslice

Previously the groupWatcher could be mistakenly stopped when requests for pod or services resources take too long.

* remove mislead comment

* docs/sd_configs.md: mention -promscrape.kubernetes.attachNodeMetadataAll flag in the description for attach_metadata section

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4640

* wip

* lib/promscrape/kubernetes: prevent from stopping groupWatcher when there are in-flight apiWatcher.mustStart() calls

groupWatcher is stopped if it has zero registered apiWatchers during 14 seconds.
But such a groupWatcher can be still in use if apiWatcher for `role: endpoints` or `role: endpointslice`
is being registered and the discovery of the associated `pod` and/or `service` objects takes longer
than 14 seconds - see the beginning of groupWatcher.startWatchersForRole() function for details.

Track the number of in-flight calls to apiWatcher.mustStart() and prevent from stopping the associated groupWatcher
if the number of in-flight calls is non-zero.

P.S. postponing the discovery of `pod` and/or `service` objects associated with `endpoints` or `endpointslice` roles
isn't the best solution, since it slows down initial discovery of `endpoints` and `endpointslice` targets.

* typo fix

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-22 01:35:43 +02:00
Hui Wang
18f0776ade
app/vmselect/promql: properly handle possible negative results caused… (#5608)
* app/vmselect/promql: properly handle possible negative results caused by float operations precision error in rollup functions like rate() or increase()

* fix test
2024-01-22 01:05:39 +02:00
Nikolay
2d4755cc1b
app/vmselect/netstorage (#5649)
* app/vmselect/netstorage

correctly handle errGlobal set

* wip

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5649

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-22 01:03:05 +02:00
Nikolay
c79ef7fbde
app/vmselect: abort streaming connections for vmselect (#5650)
* app/vmselect: abort streaming connections for vmselect
due to streaming nature of export APIs, curl and simmilr tools cannot
detect errors that happened after http.Header with status 200 was
written to it.

This PR tracks if body write was already started and closes connection.

It allows client to detect not expected chunk sequence and return error
to the caller.

Mostly it affects vmselect at cluster version

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5645

* wip

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5645
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5650

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-22 00:56:36 +02:00
Roman Khavronenko
30284e9fd4
app/vmselect: properly calculate start param for queries with too big look-behind window (#5630)
Properly determine time range search for instant queries with too big look-behind window like `foo[100y]`.
 Previously, such queries could return empty responses even if `foo` is present in database.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5553

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-21 23:49:53 +02:00
Aliaksandr Valialkin
abd3641328
lib/storage: do not prefetch metric names for small number of metricIDs
This eliminates prefetchedMetricIDsLock lock contention for queries, which return less than 500 time series.

This is a follow-up for 9d886a2eb0
2024-01-17 13:52:51 +02:00
Aliaksandr Valialkin
3e80ace2f8
LICENSE: update the current year from 2023 to 2024 2024-01-17 01:48:36 +02:00
Aliaksandr Valialkin
8777864167
docs/CHANGELOG.md: typo fix in release date for v1.93.10 2024-01-17 01:46:14 +02:00
Aliaksandr Valialkin
d277977e7e
docs/CHANGELOG.md: cut v1.97.10 LTS release 2024-01-17 01:12:34 +02:00
Aliaksandr Valialkin
25564e66e6
lib/httputils: handle step=undefined query arg as an empty value
This is needed for Grafana, which may send step=undefined
when working with alerting rules and instant queries.
2024-01-17 00:13:25 +02:00
Aliaksandr Valialkin
94a17d792d
deployment/docker: update Go builder from Go1.21.5 to Go1.21.6 2024-01-17 00:09:35 +02:00
Aliaksandr Valialkin
4b59047c8a
app/vmstorage: deregister storage metrics before stopping the storage
This prevents from possible nil pointer dereference issues when the storage metrics
are read after the storage is stopped.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
2024-01-16 23:30:11 +02:00
Aliaksandr Valialkin
fc42900eca
lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop()
Previously the was a race condition when the background goroutine still could try collecting metrics
from already stopped resources after returning from pushmetrics.Stop().
Now the pushmetrics.Stop() waits until the background goroutine is stopped before returning.

This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5549
and the commit fe2d9f6646 .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
2024-01-16 23:18:29 +02:00
hagen1778
07fe834f54
app/all: follow-up after 84d710beab
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-16 23:08:21 +02:00
zhdd99
66bafee651
lib/pushmetrics: fix a panic caused by pushing metrics during the graceful shutdown process of vmstorage nodes. (#5549)
Co-authored-by: zhangdongdong <zhangdongdong@kuaishou.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-16 23:06:51 +02:00
Aliaksandr Valialkin
4542d86819
vendor: run make vendor-update 2024-01-16 22:57:10 +02:00
Aliaksandr Valialkin
30ae1b7901
lib/storage: follow-up for 4b8088e377
- Clarify the bugfix description at docs/CHANGELOG.md
- Simplify the code by accessing prefetchedMetricIDs struct under the lock
  instead of using lockless access to immutable struct.
  This shouldn't worsen code scalability too much on busy systems with many CPU cores,
  since the code executed under the lock is quite small and fast.
  This allows removing cloning of prefetchedMetricIDs struct every time
  new metric names are pre-fetched. This should reduce load on Go GC,
  since the cloning of uin64set.Set struct allocates many new objects.
2024-01-16 22:41:55 +02:00
Roman Khavronenko
3240fed7d7
lib/storage: properly check for storage/prefetchedMetricIDs cache expiration deadline (#5607)
Before, this cache was limited only by size.
Cache invalidation by time happens with jitter to prevent thundering herd problem.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-16 22:39:20 +02:00
Aliaksandr Valialkin
f043908156
app/vmselect/promql: simplify the code after 388d020b7c
Add a test, which verifies the correct sorting of float64 slices with NaNs.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5506
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509
2024-01-16 22:36:11 +02:00