Commit graph

548 commits

Author SHA1 Message Date
Aliaksandr Valialkin
744f8c3fe7
app/vmselect/promql: add outliers_iqr(q) and outlier_iqr_over_time(m[d]) functions
These functions allow detecting anomalies in series and samples using Interquartile range method.
See Outliers section at https://en.wikipedia.org/wiki/Interquartile_range for more details.
2023-10-31 22:14:14 +01:00
Aliaksandr Valialkin
48b842d2ad
vendor: run make vendor-update 2023-10-31 20:20:07 +01:00
Aliaksandr Valialkin
627a4e9330
app/vmselect/promql: add labels_equal(q, "label1", "label2", ...) function
This function returns q series, which have identical values for the listed labels
"label1", "label2", ...

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5148
2023-10-16 21:51:13 +02:00
Aliaksandr Valialkin
b2f9b9d634
app/vmselect/promql: add drop_empty_series() function for dropping empty series before performing additional calculations
This can be useful in the following queries:

   drop_empty_series(temperature <= 30) default 40

This query drops temperature series with all the values bigger than 30 on the selected time range,
while replacing gaps in the remaining series with 40.

The query without drop_empty_series:

  (temperature <= 30) default 40

would leave all the temperature series with all the values bigger than 30 on the selected time range,
and replace all their values with 40. This is not what could be epxected in some cases
like here - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5071
2023-10-16 20:59:21 +02:00
Aliaksandr Valialkin
fb1fcbefda
vendor: run make vendor-update 2023-10-16 14:33:16 +02:00
Aliaksandr Valialkin
cd1169618f
vendor: run make vendor-update 2023-10-02 21:53:35 +02:00
Aliaksandr Valialkin
c954019e43
app/vmselect/promql: completely substitute median_over_time() WITH template with regular median_over_time() rollup function
This is a follow-up for 34d7a670d0

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5034
2023-09-25 15:31:25 +02:00
Dmytro Kozlov
5477b52991
vmagent: add validation of MetricsQL functions (#4991)
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-09-15 13:16:22 +02:00
Aliaksandr Valialkin
1dd19df849
vendor: return back the latest version of golang.org/x/exp/slices, which works correctly with github.com/prometheus/prometheus/model/labels 2023-09-07 12:40:00 +02:00
Aliaksandr Valialkin
5cbb330555
vendor: run make vendor-update 2023-09-07 12:35:42 +02:00
Aliaksandr Valialkin
1ca3b660f0
app/vmselect/promql: add support for _ delimiters in numeric values
For example, 1_234_567_890 is equivalent to 1234567890,
while 1.234_567_890 is equivalent to 1.234567890
2023-08-30 14:35:58 +02:00
Aliaksandr Valialkin
503798da53
vendor: run make vendor-update 2023-08-29 13:16:20 +02:00
Aliaksandr Valialkin
5c80b11c15
app/vmselect: prevent from panic when lookbehind window inside rollup function is parsed into negative value
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4795
2023-08-12 04:49:56 -07:00
Aliaksandr Valialkin
6092b98849
vendor: run make vendor-update 2023-08-11 08:38:36 -07:00
hagen1778
9d4c838412
vendor: make vendor-update
Follow-up after 3f6efab6ae
Related to https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4441

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-31 07:55:01 -07:00
Aliaksandr Valialkin
2e39070f95
vendor: run make vendor-update 2023-07-26 15:09:26 -07:00
Aliaksandr Valialkin
a3c8f902c1
vendor: update github.com/VictoriaMetrics/metricsql from v0.60.0 to v0.61.1
This adds support for passing durations via WITH template vars:

- `WITH (w = 5m) m[w]` is transformed to `m[5m]`
- `WITH (f(w, step, off) = m[w:step] offset off) f(5m, 10s, 1h)` is transformed to `m[5m:10s] offset 1h`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4025
Updates https://github.com/VictoriaMetrics/metricsql/issues/12

See also the initial implementation by @lujiajing1126 at https://github.com/VictoriaMetrics/metricsql/pull/13
2023-07-19 15:04:38 -07:00
Aliaksandr Valialkin
4fb505dfe9
vendor: update github.com/VictoriaMetrics/metricsql from v0.59.1 to v0.60.0
This update adds support for metricsql.Prettify() function, which is going to be used
for prettifying long MetricsQL queries in VMUI
2023-07-18 19:19:33 -07:00
Aliaksandr Valialkin
31b7c947ef
vendor: make vendor-update 2023-07-18 16:14:07 -07:00
Aliaksandr Valialkin
798ff17c42
properly encode the right part of binary operation if it starts with prefix series selector 2023-07-18 16:03:57 -07:00
dependabot[bot]
6508659252
build(deps): bump github.com/cheggaaa/pb/v3 from 3.1.3 to 3.1.4 (#4645)
Bumps [github.com/cheggaaa/pb/v3](https://github.com/cheggaaa/pb) from 3.1.3 to 3.1.4.
- [Commits](https://github.com/cheggaaa/pb/compare/v3.1.3...v3.1.4)

---
updated-dependencies:
- dependency-name: github.com/cheggaaa/pb/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-17 19:08:11 -07:00
Aliaksandr Valialkin
5ace0701d3
app/vmselect/promql: add the ability to copy all the labels from one side of group_left()/group_right() operation
This is performed by specifying `*` inside group_left()/group_right().
Also allow specifying prefix for the copied labels via `group_left(...) prefix "..."` and `group_right(...) prefix "..."` syntax.
For example, the following query adds all the namespace-related labels to pod info, and prefixes all the copied label names with "ns_" prefix:

  kube_pod_info * on(namespace) group_left(*) prefix "ns_" kube_namespace_labels

This resolves the following StackOverflow questions:

- https://stackoverflow.com/questions/76661818/how-to-add-namespace-labels-to-pod-labels-in-prometheus
- https://stackoverflow.com/questions/76653997/how-can-i-make-a-new-copy-of-kube-namespace-labels-metric-with-a-different-name
2023-07-17 16:58:30 -07:00
Aliaksandr Valialkin
cc54fa2a56
app/vmselect/promql: recommend to use (a op b) keep_metric_names instead of a op b keep_metric_names
The `a op b keep_metric_names` is ambigouos to `a op (b keep_metric_names)` when `b` is a transform or rollup function.
For example, `a + rate(b) keep_metric_names`. So it is better to use more clear syntax: `(a op b) keep_metric_names`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3710
2023-07-16 23:47:15 -07:00
Zakhar Bessarab
781947a7e2
metricsql: add support of using keep_metric_names for binary operations (#4109)
* metricsql: add support of using keep_metric_names for binary operations

This should help to avoid confusion with queries like one in the issue #3710.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* wip

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-07-16 03:01:27 -07:00
Aliaksandr Valialkin
a7fdc3fcc7
all: add support for or filters in series selectors
This commit adds ability to select series matching distinct filters via a single series selector.
For example, the following selector selects series with either {env="prod",job="a"}
or {env="dev",job="b"} labels:

  {env="prod",job="a" or env="dev",job="b"}

The `or` filter is supported in all the VictoriaMetrics tools now.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3997
Uses https://github.com/VictoriaMetrics/metricsql/pull/14
2023-07-15 23:56:18 -07:00
Dmytro Kozlov
555a0a9d57
app/vmctl: fix issue with adding many seconds (#4617)
* app/vmctl: fix issue with adding many seconds

* app/vmagent: add CHANGELOG.md
2023-07-13 12:09:54 -07:00
Aliaksandr Valialkin
e2a2d64053
vendor: run make vendor-update 2023-07-07 00:05:50 -07:00
Aliaksandr Valialkin
131a1146d9
vendor: make vendor-update 2023-05-18 12:22:16 -07:00
Aliaksandr Valialkin
9af04e4140
vendor: update github.com/VictoriaMetrics/metrics from v1.23.1 to v1.24.0
This change adds process_* metrics to VictoriaMetrics components under Windows OS

See https://github.com/VictoriaMetrics/metrics/pull/47
2023-05-16 11:37:18 -07:00
Aliaksandr Valialkin
64fa95dc92
vendor: update github.com/valyala/gozstd from v1.19.1 to v1.20.1 2023-05-10 02:15:22 -07:00
Aliaksandr Valialkin
1fa5223237
vendor: make vendor-update 2023-05-09 23:16:43 -07:00
Zakhar Bessarab
19eaf17e11
app/vmalert: add support of recursive path globs for rules and templates (#4148)
Supports using `**` for `-rule` and `-rule.templates`: `dir/**/*.tpl` loads contents of dir and all subdirectories recursively.

See: #4041

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-05-08 16:22:30 -07:00
Aliaksandr Valialkin
d544bfd73e
vendor: update github.com/VictoriaMetrics/metricsql from v0.56.1 to v0.56.2
This fixes panic when the duration in the query contains `M` suffix.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4120
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3589
2023-04-14 14:06:47 -07:00
Aliaksandr Valialkin
61839d4807
vendor: update github.com/valyala/gozstd from v1.19.0 to v1.19.1 2023-04-10 11:30:26 -07:00
Aliaksandr Valialkin
4077b0fcb1
vendor: make vendor-update 2023-04-10 10:21:39 -07:00
Aliaksandr Valialkin
180a5be623
vendor: update github.com/klauspost/compress from v1.16.3 to v1.16.4
See https://github.com/klauspost/compress/releases/tag/v1.16.4
2023-04-05 21:27:13 -07:00
Aliaksandr Valialkin
93cfb448c7
vendor: update github.com/valyala/gozstd from v1.18.0 to v1.19.0 2023-04-05 20:55:57 -07:00
Aliaksandr Valialkin
0275acc43c
vendor: make vendor-update 2023-04-01 00:00:00 -07:00
Aliaksandr Valialkin
a3ae3268a1
vendor: make vendor-update 2023-03-27 15:28:11 -07:00
Aliaksandr Valialkin
196f7398e0
vendor: run make vendor-update 2023-03-24 18:08:52 -07:00
Aliaksandr Valialkin
46521e20c8
vendor: make vendor-update 2023-03-15 13:24:22 -07:00
Aliaksandr Valialkin
5375e789fd
vendor: make vendor-update 2023-03-14 16:20:05 -07:00
Aliaksandr Valialkin
8be6c85eb3
vendor: update github.com/klauspost/compress from v1.16.0 to v1.16.3 2023-03-14 16:20:03 -07:00
Aliaksandr Valialkin
9cd0a2e875
vendor: make vendor-update 2023-03-12 03:14:09 -07:00
Aliaksandr Valialkin
73514d04b0
vendor: make vendor-update 2023-02-27 15:33:02 -08:00
Aliaksandr Valialkin
06ac40aafa
lib/httpserver: use github.com/klauspost/compress/gzhttp for compressing http responses
This allows removing gzip-related code from lib/httpserver.
2023-02-27 10:35:26 -08:00
Aliaksandr Valialkin
328eb3cf86
vendor: make vendor-update 2023-02-24 17:27:04 -08:00
Aliaksandr Valialkin
d409d3cadb
vendor: make vendor-update 2023-02-21 18:07:26 -08:00
Aliaksandr Valialkin
9d6eeb0c07
go.mod: update github.com/VictoriaMetrics/fastcache from v1.12.0 to v1.12.1 2023-02-21 17:51:47 -08:00
Aliaksandr Valialkin
013e2bacaf
vendor: update github.com/VictoriaMetrics/fasthttp from v1.1.0 to v1.2.0
The v1.2.0 adds HostClient.DoCtx() function, which is needed by https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3747
for implementing fast canceling of pending requests to scrape targets on config update
2023-02-21 17:50:25 -08:00
Aliaksandr Valialkin
a15da5ff73
app/vmselect/promql: add share(q) aggregate function for normalizing results across multiple time series in [0..1] value range per each timestamp and aggregation group 2023-02-18 22:43:54 -08:00
Aliaksandr Valialkin
84b5532bc1
app/vmselect/promql: add range_zscore(q) and range_trim_zscore(z, q) functions
These functions may be useful for dropping outliers at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3759
2023-02-18 22:43:53 -08:00
Aliaksandr Valialkin
94334ed0bb
vendor: make vendor-update 2023-02-18 15:37:05 -08:00
Aliaksandr Valialkin
450b6f6d39
app/vmselect/promql: add range_mad(q) and range_trim_outliers(k, q) functions
These functions may help trimming outliers during query time
for the use case described at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3759
2023-02-18 15:18:47 -08:00
Aliaksandr Valialkin
b572512dbd
vendor: update github.com/valyala/gozstd from v1.17.0 to v1.18.0 2023-02-18 13:55:27 -08:00
Aliaksandr Valialkin
d63862b3ab
vendor: make vendor-update 2023-02-13 11:14:30 -08:00
Aliaksandr Valialkin
db7f237da9
app/vmselect/promql: add mad_over_time(m[d]) function
See https://github.com/prometheus/prometheus/issues/5514
2023-02-11 01:06:39 -08:00
Aliaksandr Valialkin
a68bb879d4
vendor: make vendor-update 2023-02-09 14:48:38 -08:00
Aliaksandr Valialkin
c7a4751c70
vendor: make vendor-update 2023-02-08 08:55:53 -08:00
Aliaksandr Valialkin
2d1ea90bcb
vendor: make vendor-update 2023-02-01 12:48:10 -08:00
Aliaksandr Valialkin
aae7b0dc97
vendor: make vendor-update 2023-01-31 11:05:11 -08:00
Aliaksandr Valialkin
80fa0e2df4
vendor: make vendor-update 2023-01-27 15:59:29 -08:00
Aliaksandr Valialkin
bb1b87e5af
vendor: make vendor-update 2023-01-23 18:43:43 -08:00
Aliaksandr Valialkin
42fa6cee19
vendor: update github.com/VictoriaMetrics/metricsql from v0.51.1 to v0.51.2
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3664
2023-01-17 11:26:11 -08:00
Aliaksandr Valialkin
4fc8cacd1a
vendor: make vendor-update 2023-01-15 14:16:45 -08:00
Aliaksandr Valialkin
4294c9b720
github.com/VictoriaMetrics/metrics: update from v1.23.0 to v1.23.1
See https://github.com/VictoriaMetrics/metrics/issues/42
2023-01-15 14:07:14 -08:00
Aliaksandr Valialkin
c6dbc4eb23
vendor: make vendor-update 2023-01-10 18:58:50 -08:00
Aliaksandr Valialkin
3d2cae9d75
vendor: make vendor-update 2023-01-09 21:34:44 -08:00
Aliaksandr Valialkin
78114e85d6
vendor: update github.com/VictoriaMetrics/metricsql from v0.50.0 to v0.51.0
Updates https://github.com/VictoriaMetrics/metricsql/pull/7
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3589
2023-01-05 01:50:26 -08:00
Aliaksandr Valialkin
0c7d161858
vendor: make vendor-update 2023-01-03 23:36:36 -08:00
Aliaksandr Valialkin
5e87ebabc7
vendor: update github.com/valyala/fastjson/fastfloat from v1.6.3 to v1.6.4
This should properly parse floating-point numbers with missing integer or fractional parts.
For example, 123. or .123

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3544
2022-12-29 10:41:51 -08:00
Aliaksandr Valialkin
7e49818c6d
vendor: make vendor-update 2022-12-28 15:00:25 -08:00
Aliaksandr Valialkin
874a925310
vendor: make vendor-update 2022-12-19 13:07:52 -08:00
Aliaksandr Valialkin
5da63d1ae7
vendor: make vendor-update 2022-12-14 12:14:01 -08:00
Aliaksandr Valialkin
87723ef0e1
vendor: update github.com/klauspost/compress from v1.15.12 to v1.15.13 2022-12-11 02:12:07 -08:00
Aliaksandr Valialkin
b472fbc737
vendor: make vendor-update 2022-12-10 21:47:18 -08:00
Aliaksandr Valialkin
b6ffdde969
vendor: make vendor-update 2022-12-05 23:28:23 -08:00
Aliaksandr Valialkin
c902500efc
app/vmselect/promql: add range_trim_spikes(phi, q) function for trimming phi percent of largest spikes per each time series returned by q 2022-12-05 21:55:24 -08:00
Aliaksandr Valialkin
020e3093a6
vendor: make vendor-update 2022-12-05 01:02:06 -08:00
Aliaksandr Valialkin
8440c822fa
all: follow-up after 05cf8a6ecc 2022-11-29 21:11:42 -08:00
Dmytro Kozlov
002c028f22
vmctl: support of the remote read protocol (#3232)
vmctl: support of the remote read protocol

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2022-11-29 21:08:47 -08:00
Aliaksandr Valialkin
154de24e60
vendor: make vendor-update 2022-11-25 19:53:38 -08:00
Aliaksandr Valialkin
e45022c344
vendor: make vendor-update 2022-11-22 11:26:34 +02:00
Aliaksandr Valialkin
ad548abd6a
app/vmselect/promql: add range_normalize(q1, ..., qN) function for normalizing query results into [0..1] value range
This may be useful for analyzing correlation between time series with different value ranges
2022-11-21 23:25:35 +02:00
Aliaksandr Valialkin
9cb76c9cd3
app/vmselect/promql: allow using SI and IEC suffixes in numeric values inside queries
For example, 10Ki is equivalent to 10*1024, while 5.3M is equivalent to 5.3*1000*1000
2022-11-21 21:28:34 +02:00
Aliaksandr Valialkin
478a295208
vendor: make vendor-update 2022-11-17 01:38:59 +02:00
Aliaksandr Valialkin
0cc59b9f95
app/vmselect/promql: add range_stdvar() and range_stddev() functions for calculating variance and deviation over time series on the selected time range 2022-11-17 01:03:02 +02:00
Aliaksandr Valialkin
facf7efa3a
app/vmselect/promql: add range_linear_regression(q) function for calculating simple linear regression for the selected time series on the selected time range 2022-11-17 00:39:17 +02:00
Aliaksandr Valialkin
372b449f94
vendor: make vendor-update 2022-11-10 13:47:30 +02:00
Aliaksandr Valialkin
9108a1d33f
vendor: update github.com/urfave/cli/v2 from v2.23.2 to v2.23.4 2022-11-07 14:59:05 +02:00
Aliaksandr Valialkin
4c2963e015
vendor: make vendor-update 2022-11-05 10:40:30 +02:00
Aliaksandr Valialkin
be0aaa1e93
vendor: update github.com/urfave/cli/v2 from 2.20.3 to 2.23.0 2022-11-01 10:53:19 +02:00
Aliaksandr Valialkin
b259c55c1b
vendor: make vendor-update 2022-10-29 02:54:08 +03:00
Aliaksandr Valialkin
1386b6bd69
vendor: make vendor-update 2022-10-28 00:19:05 +03:00
Aliaksandr Valialkin
f4e9393076
vendor: make vendor-update 2022-10-28 00:08:50 +03:00
Aliaksandr Valialkin
4f53147ed4
app/{vminsert,vmselect}/netstorage: allow calling Init()+MustStop() in a loop
Previously netstorage.MustStop() call didn't free up all the resources,
so the subsequent call to nestorage.Init() would panic.

This allows writing tests, which call nestorage.Init() + nestorage.MustStop() in a loop.
2022-10-25 14:43:05 +03:00
Aliaksandr Valialkin
8946371466
vendor: make vendor-update 2022-10-24 15:50:57 +03:00
Aliaksandr Valialkin
a408223e6d
vendor: make vendor-update 2022-10-18 10:57:06 +03:00
Aliaksandr Valialkin
d0288ea417
all: log error when environment variables referred from -promscrape.config are missing
This should prevent from using incorrect config files
2022-10-18 10:29:59 +03:00
Aliaksandr Valialkin
00e4c7f265
vendor: make vendor-update 2022-10-14 15:20:48 +03:00