Commit graph

556 commits

Author SHA1 Message Date
Nikolay
8357c22cc8
app/vmselect: properly return binary pow function result (#7619)
Previously, for `^` aka pow function calls, VictoriaMetrics returned `1`
if left arg was Nan. For example, given query=`(hour()==2)^1` returns 1
for NaN produced by hour() == 2 function. It added additional non-exist
datapoints to the timeseries.

This commit port bugfix from `metricql` package and adds test for it.
Now, VictoriaMetrics
correctly returns `NaN` for such cases.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7359

Signed-off-by: f41gh7 <nik@victoriametrics.com>
(cherry picked from commit bb399518db)
2024-11-21 15:23:49 +01:00
hagen1778
790eab3026
tests: integration tests for vmsingle (#7434)
### Describe Your Changes

This PR continues the implementation of integration tests (#7199). It
adds the support for vm-single:

-    A vmsingle app wrapper has been added
- Sample vmsingle tests that test the VM documentation related to
querying data (#7435)
- The tests use the go-cmp/{cmp,/cmpopts} packages, therefore they have
been added to ./vendor
-    Minor refactoring: data objects have been moved to model.go

Advice on porting things to cluster branch:

- The build rule must include tests that start with TestVmsingle
(similarly to how TestCluster tests are skipped in master branch)
- The build rule must depend on `vmstorage vminsert vmselect` instead of
`victoria-metrics`
- The query_test.go can actually be implemented for cluster as well. To
do this the tests need to be renamed to start with TestCluster and the
tests must instantiace vm{storage,insert,select} instead of vmsingle.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-07 13:25:01 +01:00
Andrii Chubatiuk
da14c7b26e
app/vlinsert: adds journald ingestion support
This commit allows to ingest logs with journald format. 

https://www.freedesktop.org/software/systemd/man/latest/systemd-journal-remote.service.html

related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4618
2024-10-27 20:42:41 +01:00
Aliaksandr Valialkin
8c55b699f4
app/vlogscli: add interactive command-line tool for querying VictoriaLogs 2024-10-01 12:24:53 +02:00
Aliaksandr Valialkin
2c477baf86
vendor: run make vendor-update 2024-09-26 22:33:13 +02:00
Aliaksandr Valialkin
4e00e4428e
app/vmselect/promql: properly calculate c1 and c2 and c1 or c2 by upgrading github.com/VictoriaMetrics/metricsql to v0.79.0
The fix is in the https://github.com/VictoriaMetrics/metricsql/pull/34
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6637

(cherry picked from commit b82e2cabc5)
2024-09-19 15:48:06 +02:00
Aliaksandr Valialkin
21dfc36398
vendor: run make vendor-update 2024-09-08 21:05:23 +02:00
Nikolay
08cbbf8134
lib/promscrape: fixes proxy autorization (#6783)
* Adds custom dial func for HTTP-Connect and socks5 proxy tunnels.
  Standard golang http.transport exposes GetProxyConnectHeader function,
  but it doesn't allow to use separate tls config for proxy.
  It also not possible to enforce HTTP-Connect with standard http lib.
* For http scrape targets, by default http.Transport.Proxy function must
  be used. Since it has special case with full uri forward.
* Adds proxy.URL json methods that allow to properly copy internal
fields, like User/Password.
It should fix bug with proxy_url. When credentials specified at URL was
ignored.
* Adds tests for scrape client proxy requests

related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6771
2024-08-19 22:50:39 +02:00
f41gh7
3c8c45b41b
vendor: updates metricsql to v0.77.0 with bugfix
Fixes panic if incorrect metricsql expression passed to the prettifier API.

Prettify function had misleading panic for duration expression formatting. It expected all WITH templates to be already parsed.
But WITH expression expand was removed.

Bug was introduced at e712a49898
and present at v1.98.0+ releases

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6736
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-01 12:39:06 +02:00
Aliaksandr Valialkin
00b1ee6b5c
vendor: run make vendor-update 2024-07-27 13:52:55 +02:00
Aliaksandr Valialkin
6eda5fd098
vendor: run make vendor-update 2024-07-17 20:47:33 +02:00
Aliaksandr Valialkin
06f7f23ebe
vendor: update github.com/VictoriaMetrics/metrics from v1.35.0 to v1.35.1 2024-07-16 13:18:49 +02:00
Aliaksandr Valialkin
a8356f3a26
vendor: update github.com/VictoriaMetrics/metrics from v1.34.1 to v1.35.0
Fix potential memory leaks across VictoriaMetrics codebase after metrics.UnregisterSet(s) call
because of missing s.UnregisterAllMetrics() call.

This is a follow-up for 6a6e34ab8e . It is OK if some vmauth metrics
aren't visible for a few microseconds when the previous metrics are unregistered and new metrics
weren't registered yet.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6247
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4690
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6252
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5805
2024-07-15 10:45:39 +02:00
Nikolay
d136bfcdbf
make vendor-update (#6627)
make vendor-update
2024-07-10 17:17:08 +02:00
Aliaksandr Valialkin
73d286847b
vendor: update github.com/valyala/quicktemplate from v1.7.0 to v1.8.0 2024-07-05 01:22:49 +02:00
Aliaksandr Valialkin
7c9212afeb
vendor: run make vendor-update 2024-07-03 00:00:23 +02:00
Aliaksandr Valialkin
14d52a749b
vendor: run make vendor-update 2024-06-24 23:09:16 +02:00
Nikolay
7ab046cf88
vendor update: updates metrics to v1.34.0 (#6523)
mention change for
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6457

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 1af13208c2)
2024-06-24 14:52:32 +02:00
Aliaksandr Valialkin
abb71b7dfc
vendor: run make vendor-update 2024-06-07 16:40:50 +02:00
Aliaksandr Valialkin
80c4fad8a9
vendor: run make vendor-update 2024-05-22 21:58:53 +02:00
Aliaksandr Valialkin
1bca5e37c1
vendor: run make vendor-update 2024-05-12 16:18:53 +02:00
Aliaksandr Valialkin
104cb56a80
vendor: run make vendor-update 2024-04-17 22:55:08 +02:00
Aliaksandr Valialkin
bb04943a2a
vendor: run make vendor-update 2024-04-17 21:01:21 +02:00
Aliaksandr Valialkin
c8a46b0ad2
vendor: run make vendor-update 2024-04-11 09:46:29 +02:00
Aliaksandr Valialkin
2f07043e74
vendor: run make vendor-update 2024-04-04 01:42:59 +03:00
Aliaksandr Valialkin
968c5e5dec
vendor: update github.com/VictoriaMetrics/metrics and github.com/VictoriaMetrics/metricsql to newer versions
This is needed for updating broken links to MetricsQL docs:

https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/MetricsQL -> https://docs.victoriametrics.com/metricsql/

This is a follow-up for 7e3511ffbd
2024-04-03 02:40:08 +03:00
Aliaksandr Valialkin
5b90106c4b
vendor: run make vendor-update 2024-03-18 00:51:52 +02:00
Aliaksandr Valialkin
f5dd9b423e
vendor: run make vendor-update 2024-03-06 21:24:55 +02:00
Aliaksandr Valialkin
7675bc8f27
vendor: run make vendor-update 2024-03-01 00:55:54 +02:00
Aliaksandr Valialkin
1109521806
vendor: retain v0.46.0 for github.com/prometheus/common , since v0.48.0 leads to build error
The error is:

vendor/github.com/prometheus/client_golang/prometheus/testutil/promlint/promlint.go:71:38: undefined: expfmt.FmtText
2024-02-25 02:20:45 +02:00
Aliaksandr Valialkin
b3dbbc22b9
app/vmselect/promql: properly handle args in count_values_over_time() function
Prevsiously they were swapped - the first arg should be the label name and the second arg should be label filters

This is a follow-up for e389b7b959e8144fdff5075bf7a5a39b2b0c6dd3

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5847
2024-02-25 01:48:37 +02:00
Aliaksandr Valialkin
3519c055d5
vendor: run make vendor-update 2024-02-24 03:23:56 +02:00
Aliaksandr Valialkin
d845edc24b
lib: consistently use atomic.* types instead of atomic.* functions
See ea9e2b19a5
2024-02-24 02:10:04 +02:00
Aliaksandr Valialkin
477fdc21aa
app/vmselect/promql: add count_values_over_time() MetricsQL function
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5847
2024-02-23 01:05:31 +02:00
Aliaksandr Valialkin
690f6c833c
vendor: update github.com/VictoriaMetrics/metricsql from v0.72.1 to v0.73.0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5383
2024-02-18 18:43:41 +02:00
Aliaksandr Valialkin
689c0b9817
vendor: update github.com/VictoriaMetrics/metrics from v1.31.0 to v1.32.0 2024-02-18 12:42:29 +02:00
Aliaksandr Valialkin
e50ebe467a
vendor: run make vendor-update 2024-02-14 15:45:05 +02:00
Aliaksandr Valialkin
5f2905d120
app/vmselect: add sum_eq_over_time, sum_gt_over_time and sum_le_over_time functions to MetricsQL
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4641
2024-02-13 23:40:30 +02:00
Aliaksandr Valialkin
dcacf209d9
vendor: update github.com/VictoriaMetrics/metricsql from v0.70.1 to v0.71.0
This adds an ability to propagate label filters across label_set() and alias() functions.

This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827#issuecomment-1654095358
2024-02-13 06:37:09 +02:00
Aliaksandr Valialkin
e75f64e993
vendor: run make vendor-update 2024-02-12 22:31:41 +02:00
Aliaksandr Valialkin
f5680a6857
all: upgrade Go builder from Go1.21.7 to Go1.22.0
See https://go.dev/doc/go1.22
2024-02-12 22:14:00 +02:00
Aliaksandr Valialkin
ebd0f1b72e
vendor: update github.com/VictoriaMetrics/metricsql from v0.70.0 to v0.70.1
This should help with the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5604
2024-02-06 21:52:55 +02:00
Aliaksandr Valialkin
765917ae6a
vendor: run make vendor-update 2024-02-01 15:24:53 +02:00
Aliaksandr Valialkin
ccb1143823
vendor: run make vendor-update 2024-01-30 18:47:30 +02:00
Aliaksandr Valialkin
09c388a8e4
lib/promscrape: use the standard net/http.Client instead of fasthttp.Client for scraping targets in non-streaming mode
While fasthttp.Client uses less CPU and RAM when scraping targets with small responses (up to 10K metrics),
it doesn't work well when scraping targets with big responses such as kube-state-metrics.
In this case it could use big amounts of additional memory comparing to net/http.Client,
since fasthttp.Client reads the full response in memory and then tries re-using the large buffer
for further scrapes.

Additionally, fasthttp.Client-based scraping had various issues with proxying, redirects
and scrape timeouts like the following ones:

- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5425
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2794
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1017

This should help reducing memory usage for the case when target returns big response
and this response is scraped by fasthttp.Client at first before switching to stream parsing mode
for subsequent scrapes. Now the switch to stream parsing mode is performed on the first scrape
after reading the response body in memory and noticing that its size exceeds the value passed
to -promscrape.minResponseSizeForStreamParse command-line flag.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5567

Overrides https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4931
2024-01-30 18:39:55 +02:00
Aliaksandr Valialkin
66a1541ce6
vendor: run make vendor-update 2024-01-26 22:59:59 +01:00
Aliaksandr Valialkin
7c737846a6
vendor: run make vendor-update 2024-01-16 22:48:46 +02:00
Aliaksandr Valialkin
825cfdb5ef
app/vmstorage: expose proper types for storage metrics when -metrics.exposeMetadata command-line flag is set
This is a follow-up for 326a77c697
2024-01-16 22:03:31 +02:00
Aliaksandr Valialkin
9e5e514faf
lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop()
Previously the was a race condition when the background goroutine still could try collecting metrics
from already stopped resources after returning from pushmetrics.Stop().
Now the pushmetrics.Stop() waits until the background goroutine is stopped before returning.

This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5549
and the commit fe2d9f6646 .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
2024-01-16 21:18:22 +02:00
Aliaksandr Valialkin
fce76b2fb0
vendor/github.com/VictoriaMetrics/easyproto: update from v0.1.3 to v0.1.4
This fixes vet error for 32-bit architectures:

https://github.com/VictoriaMetrics/VictoriaMetrics/actions/runs/7521709384/job/20472882877
2024-01-16 21:14:03 +02:00