Commit graph

222 commits

Author SHA1 Message Date
Roman Khavronenko
273472dc20
app/vmselect: ignore empty series for limit_offset (#3178)
* app/vmselect: ignore empty series for `limit_offset`

VictoriaMetrics doesn't return empty series (with all NaN values) to
the user. But such series are filtered after transform functions.
It means `limit_offset` will account for empty series as well.

For example, let's consider following data set:
```
time series:
foo{label="1"} NaN, NaN, NaN, NaN // empty series
foo{label="2"} 1, 2, 3, 4
foo{label="3"} 4, 3, 2, 1
```

When user requests all series for metric `foo` the empty series
will be filtered out:
```
/query=foo:
foo{label="v2"} 1, 2, 3, 4
foo{label="v3"} 4, 3, 2, 1
```

But `limit_offset(1, 1, foo)` is applied to original series, not filtered yet.
So it will return `foo{label="v2"}` (skips the first in list)
```
/query=limit_offset(1, 1, foo):
foo{label="v2"} 1, 2, 3, 4
```

Expected result would be to apply `limit_offset` to already filtered list,
so in result we receive `foo{label="v3"}`:
```
/query=limit_offset(1, 1, foo):
foo{label="v3"} 4, 3, 2, 1
```

The change does exactly that - filters empty series before applying `limit_offset`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* app/vmselect: ignore empty series for `limit_offset`

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-30 08:44:05 +03:00
Dmytro Kozlov
36ea8537d9
vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric) (#2982)
* vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric)

* vmselect/promql: fix tests, add documentation

* vmselect/promql: update test

* vmselect/promql: update for alphanumeric sorting, fix tests

* vmselect/promql: remove comments

* vmselect/promql: cleanup

* vmselect/promql: avoid memory allocations, update functions descriptions

* vmselect/promql: make linter happy (remove ineffectual assigment)

* vmselect/promql: add test case, fix behavior when strings are equal

* vendor: update github.com/VictoriaMetrics/metricsql from v0.44.1 to v0.45.0

this adds support for sort_by_label_numeric and sort_by_label_numeric_desc functions

* wip

* lib/promscrape: read response body into memory in stream parsing mode before parsing it

This reduces scrape duration for targets returning big responses.

The response body was already read into memory in stream parsing mode before this change,
so this commit shouldn't increase memory usage.

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-14 17:42:07 +03:00
Aliaksandr Valialkin
8aaaf221cc
app/vmselect/promql: follow-up after 2d71b4859c
- Use getScalar() function for obtaining the expected scalar from phi arg
- Reduce the error message returned to the user when incorrect phi is passed to histogram_quantiles
- Improve the description of this bugfix in the docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3026
2022-08-27 01:38:17 +03:00
Dmytro Kozlov
2cf29cc4d1
vmselect/promql: fix panic in histogram_quantiles function (#3029)
* vmselect/promql: fix panic in histogram_quantiles function

* Update docs/MetricsQL.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-08-27 01:34:54 +03:00
Dmytro Kozlov
d32a6359b0
vmselect/promql: enable search.maxPointsSubqueryPerTimeseries for sub-queries (#2963)
* vmselect/promql: enable search.maxPointsPerTimeSeriesSubquery for sub-queries

* vmselect/promql: cleanup

* vmselect/promql: rename config flag

* vmselect/promql: add tests

* vmselect/promql: use test object instead of log

* vmselect/promql: fix posible panic is subquery has more points. add description

* vmselect/promql: update tests descriptions

* vmselect/promql: update doInternal validation

* vmselect/promql: fix linter

* vmselect/promql: fix linter

* vmselect/promql: update documentation and release notes

* wip

- Properly apply -search.maxPointsSubqueryPerTimeseries limit to subqueries.
  Previously the -search.maxPointsPerTimeseries limit was unexpectedly applied to subqueries
  if it was smaller than the -search.maxPointsSubqueryPerTimeseries .
- Clarify docs for -search.maxPointsSubqueryPerTimeseries command-line flag .
- Document -search.maxPointsPerTimeseries and -search.maxPointsSubqueryPerTimeseries flags at https://docs.victoriametrics.com/#resource-usage-limits .
- Update docs/CHANGELOG.md .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2922

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-24 15:27:41 +03:00
Dmytro Kozlov
ab645b5fab
vmselect/promql: add tests for vmrangeBucketsToLE (#2907)
* vmselect/promql: add tests for vmrangeBucketsToLE

* vmselect/promql: cleanup

* vmselect/promql: cleanup

* vmselect/promql: fix panic tests want result

* vmselect/promql: cleanup

* vmselect/promql: update test name

* vmselect/promql: fix linter error

* vmselect/promql: refactor testcases

* vmselect/promql: cleanup

* vmselect/promql: remove unused reassign to workers, fix typo

* wip

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-26 20:43:43 +03:00
Aliaksandr Valialkin
d920b0afec
app/vmselect/promql: properly return q1 series from q1 ifnot q2 when q2 returns nothing 2022-07-18 14:24:23 +03:00
Roman Khavronenko
78da074ad3
metricsql: properly evaluate timezone_offset over time interval (#2842)
* metricsql: properly evaluate `timezone_offset` over time interval

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update docs/CHANGELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-08 14:04:40 +03:00
Aliaksandr Valialkin
8516670582
app/vmselect/promql: properly calculate histogram_quantile over unexpected le buckets
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819
2022-07-05 13:20:41 +03:00
Aliaksandr Valialkin
fa08220d27
app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2787
2022-06-30 22:40:26 +03:00
Aliaksandr Valialkin
afced37c0b
all: add initial support for query tracing
See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
2022-06-01 02:31:44 +03:00
Aliaksandr Valialkin
68bfa05437
app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps 2022-05-09 17:58:39 +03:00
Aliaksandr Valialkin
6eb1580158
app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d 2022-04-27 10:58:40 +03:00
Aliaksandr Valialkin
dac24aa342
app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries
Previously `vector` time series could be unexpectedly returned from such queries
2022-04-21 15:34:14 +03:00
Aliaksandr Valialkin
bc54ae9608
app/vmselect/promql: fix comparison to nan
The comparison to nan has been broken in d335cc886c

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150
2022-04-21 14:56:41 +03:00
Aliaksandr Valialkin
a5cfe5d13e
app/vmselect/promql: add drop_common_labels() function 2022-04-21 14:20:36 +03:00
Aliaksandr Valialkin
25b841c6ed
app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values
This should prevent from `duplicate time series` errors when executing the following query:

kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1)

where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series
2022-04-20 22:21:20 +03:00
Aliaksandr Valialkin
b843f0e229
app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs 2022-03-26 11:28:14 +02:00
Aliaksandr Valialkin
989668beba
app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130
2022-02-12 15:46:43 +02:00
Aliaksandr Valialkin
cc76174387
app/vmselect/promql: properly keep metric names when optimized path is used for aggregate function calculations
For example, `sum(rate(...) keep_metric_names) by (__name__)` didn't leave the original metric name because of this issue.

This is a follup-up for 1bdc71d917

Udates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
2022-01-17 15:33:39 +02:00
Aliaksandr Valialkin
5a1e4a140f
app/vmselect/promql: implement keep_metric_names modifier for transform and rollup functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
2022-01-14 04:15:35 +02:00
Aliaksandr Valialkin
0580a58feb
app/vmselect/promql: add support for @ modifier
Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier

Extend the support with the following features:
* Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()`
* Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)`
  returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348
2022-01-13 22:15:19 +02:00
Aliaksandr Valialkin
7d635a2311
app/vmselect/promql: fix limit_offset() test
The test has been broken in addae7fc6a
2022-01-13 16:58:47 +02:00
Aliaksandr Valialkin
3396f08874
app/vmselect/promql: follow-up after 177e345d8a
* Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions.
* Simplify their implementation
* Mention these functions in docs/CHANGELOG.md
2021-12-20 13:15:06 +02:00
Aliaksandr Valialkin
c170841951
app/vmselect/promql: add timestamp_with_name(m[d]) function
This function works the same as `timestamp()`, but doesn't remove source time series names.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388
2021-12-15 23:38:09 +02:00
Aliaksandr Valialkin
addae7fc6a
app/vmselect/promql: preserve the order of time series passed to limit_offset() function
Previously time series passed to `limit_offset()` were shuffled according to hash for their labels.
This was unexpected behaviour for most users.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1920 and https://github.com/VictoriaMetrics/VictoriaMetrics/issues/951
2021-12-12 18:07:15 +02:00
Aliaksandr Valialkin
84decc4254
app/vmselect/promql: consistently return zero from deriv(const) 2021-11-17 18:01:58 +02:00
Aliaksandr Valialkin
aa6d11a33f
app/vmselect/promql: add now() function, which returns the current timestamp as a floating-point value in seconds 2021-11-17 16:42:11 +02:00
Aliaksandr Valialkin
3a1ac218ba
app/vmselect/promql: add duration_over_time(m[d], max_interval) function
This function calculates the actual lifetime of the time series on the given lookbehdind window `d`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780
2021-11-08 13:14:57 +02:00
Aliaksandr Valialkin
a102fca4ac
app/vmselect/promql: add limit_offset(limit, offset, q) function, which can be used for paging over big number of time series
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1778
2021-11-03 16:03:10 +02:00
Aliaksandr Valialkin
43bf0333eb
app/vmselect/promql: add label_graphite_group() function for extracting groups from Graphite metric names 2021-11-03 13:20:34 +02:00
Aliaksandr Valialkin
48c3668b3d
app/vmselect/promql: add atan2 binary operator, which is going to be added in Prometheus 2.31
See https://github.com/prometheus/prometheus/pull/9248
2021-10-11 21:17:23 +03:00
Aliaksandr Valialkin
b3d28f3872
app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31
See https://github.com/prometheus/prometheus/issues/9233
2021-10-11 21:17:21 +03:00
Aliaksandr Valialkin
f1e1d20ac4
app/vmselect/promql: consistently return the same set of time series from limitk() function
This is the expected behaviour by most users.
2021-10-08 19:55:29 +03:00
Aliaksandr Valialkin
7e6ec0bbe9
app/vmselect/promql: return back the behaviour for deriv() function when the lookbehind window doesnt contain enough points
It is expected that the `deriv(m[d])` returns non-empty value if the lookbehind window `d`
contains less than 2 samples in the same way as `rate()` does.

This is a follow-up after 3e084be06b .
2021-10-07 12:55:58 +03:00
Roman Khavronenko
0a8804d6aa
app/vmselect: make predict_linear and deriv compatible with Prometheus (#1681)
Previously, `predict_linear` returned slightly different results comparing
to Prometheus. The change makes linear regression algorithm compatible
with Prometheus.

`deriv` was excluded from the list of functions which can adjust the time
window for the same reasons.
2021-10-07 12:55:56 +03:00
Aliaksandr Valialkin
ec6eb03d65
app/vmselect/promql: add topk_last and bottomk_last functions 2021-09-30 13:23:27 +03:00
Roman Khavronenko
95ddfda894
app/vmselect: fix binary comparison func (#1667)
The fix makes the binary comparison func to check for NaNs
before executing the actual comparison. This prevents VM
to return values for non-existing samples for expressions
which contain bool comparisons. Please see added test
for example.
2021-09-30 13:04:10 +03:00
Roman Khavronenko
57eb5ef194
app/vmselect: fix testRowsEqual func NaN checks (#1666)
It appeared, that `testRowsEqual` NaN comparison was incorrect.
The fix caused some tests to fail. Please see the change and
tests updated.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-30 13:04:08 +03:00
Aliaksandr Valialkin
4b3951fd86
app/vmselect/promql: add rollup_scrape_interval(m[d]) function
It calculates the min, max and avg scrape intervals for m over the given lookbehind window d
2021-09-27 19:21:55 +03:00
Roman Khavronenko
6061464d80
app/vmselect: quantile func compatiblity with Prometheus (#1646)
* app/vmselect: `quantile` func compatiblity with Prometheus

The `quantile` func was previously calculated by https://github.com/valyala/histogram
package. The result of such calculation was always the closest real value to
requested quantile. While in Prometheus implementation interpolation is used.
Such difference may result into discrepancy in output between Prometheus and
VictoriaMetrics.

This commit adds a Prometheus-like `quantile` function. It also used by other
functions which depend on it, such as `quantiles`, `quantile_over_time`, `median` etc.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* app/vmselect: `quantile` review fixes

* quantile functions were split into multiple to provide
different API for already sorted data;
* float64sPool is used for reducing allocations. Items in pool may have
different sizes, but defining a new pool was complicates due to name collisions;

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-27 18:57:02 +03:00
Aliaksandr Valialkin
a746fe8778
app/vmselect/promql: follow-up after 57b3320478 2021-09-24 01:25:04 +03:00
Roman Khavronenko
7ac0559d4f
app/vmselect: make sorting for query result similar to Prometheus (#1647)
* app/vmselect: make sorting for query result similar to Prometheus

Updated sorting allows to get the order of series in result similar or equal
to what Prometheus returns.
The change is needed for compatibility reasons.

* Update app/vmselect/promql/exec_test.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-24 01:25:03 +03:00
Aliaksandr Valialkin
76c650bbda app/vmselect/promql: add quantiles_over_time("phiLabel", phi1, ..., phiN, m[d]) function for calculating multiple quantiles at once 2021-09-17 23:35:04 +03:00
Aliaksandr Valialkin
81d24045c2 app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function
This function calculates multiple quantiles over the given buckets at once

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
2021-09-17 13:39:06 +03:00
Aliaksandr Valialkin
3f8de4f82f app/vmselect/promql: add mad(q) and outliers_mad(tolerance, q) functions to MetricsQL 2021-09-16 13:35:38 +03:00
Aliaksandr Valialkin
19516aedb5 app/vmselect/promql: use Prometheus-compatible label value formatting for count_values function 2021-09-13 13:48:18 +03:00
Aliaksandr Valialkin
9b07cb2988 app/vmselect/promql: keep metric name in rollup_candlestick results, since they don't change the original series meaning
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1600
2021-09-09 19:24:10 +03:00
Aliaksandr Valialkin
39bb6bdd79 app/vmselect/promql: add quantile("phiLabel", phi1, ..., phiN, q) aggregate function to MetricsQL
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
2021-08-27 18:40:25 +03:00
Aliaksandr Valialkin
40b06e84f8 app/vmselect/promql: rename sign() function to sgn() in order to be consistent with Prometheus
See https://github.com/prometheus/prometheus/pull/8457 for details.
2021-08-23 11:46:29 +03:00
Aliaksandr Valialkin
49886ecbc8 app/vmselect/promql: add bitmap_and(), bitmap_or() and bitmap_xor() functions to MetricsQL
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1541
2021-08-17 13:22:15 +03:00
Aliaksandr Valialkin
60cfa5f100 app/vmselect/promql: add present_over_time(m[d]) function, which will be available starting from Prometheus 2.29.0
See https://github.com/prometheus/prometheus/releases/tag/v2.29.0-rc.0 and https://github.com/prometheus/prometheus/pull/9097
2021-08-03 12:21:53 +03:00
Aliaksandr Valialkin
b047feeb8b app/vmselect/promql: properly handle (a op b) default N if (a op b) returns NaN series
The result should be a series with `N` values and `a op b` labels. Previously such series has been removed from the result.
2021-07-16 01:44:24 +03:00
Aliaksandr Valialkin
9add9d86a6 app/vmselect/promql: duration handling improvements in MetricsQL queries
- Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600
- Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])
2021-07-12 17:19:32 +03:00
Aliaksandr Valialkin
180829b8c2 app/vmselect/promql: add timezone_offset(tz) function
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1306
2021-05-20 11:54:06 +03:00
Aliaksandr Valialkin
e48fd5776b app/vmselect/promql: fix tests after d3fa0ccabd 2021-04-08 00:17:40 +03:00
Aliaksandr Valialkin
4aa59cae7c app/vmselect/promql: do not delete dst_label if src_label is empty in label_copy(q, src_label, dst_label) and label_move(q, src_label, dst_label) 2021-04-03 22:06:50 +03:00
Aliaksandr Valialkin
b9469de410 app/vmselect/promql: add ability to set label value additionally to label name for the remaining sum of time series returned from topk_* and bottomk_* functions in the form: topk_min(N, m, "label=value") 2021-04-02 23:56:30 +03:00
Aliaksandr Valialkin
f54ece438d app/vmselect/promql: do not crash if histogram_over_time() function name contains uppercase letters such as Histogram_over_time() 2021-03-16 12:23:55 +02:00
Aliaksandr Valialkin
8844b38745 app/vmselect/promql: fix tests after 2dae0a2c47 2021-03-15 20:20:27 +02:00
Nikolay
28e450cd7c duplicate timeseries fix for prometheus_buckets function (#1119)
* try fix for prometheus_buckets

* merge possible end of the bucket collision
2021-03-09 12:54:21 +02:00
Aliaksandr Valialkin
5277507932 app/vmselect/promql: add histogram_avg(), histogram_stddev() and histogram_stdvar() functions to MetricsQL 2021-03-04 14:12:52 +02:00
Aliaksandr Valialkin
e38ca28d99 app/vmselect/promql: increase accuracy for buckets_limit() function for small limits by skipping the first and the last buckets during merge
The first and the last buckets are usually `[0 ... leMin]` and `(leMax ... +Inf)`. If they are merged with adjancent buckets,
then the resulting accuracy can suffer.
2021-02-26 22:56:04 +02:00
Aliaksandr Valialkin
0144b164c7 app/vmselect: add sign(q) and clamp(q, min, max) functions, which will be added in the upcoming Prometheus release
See https://twitter.com/roidelapluie/status/1363428376162295811

The `last_over_time(m[d])` function already exists in MetricsQL.
2021-02-24 17:26:27 +02:00
Aliaksandr Valialkin
d16effc29e app/vmselect/promql: properly calculate histogram_quantile() over zero buckets and only a single non-zero le="+Inf"` bucket like Prometheus does 2021-02-24 00:41:30 +02:00
Aliaksandr Valialkin
5dbe88a1c6 app/vmselect/promql: add increase_pure() function to MetricsQL
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962
2021-02-22 19:15:31 +02:00
Aliaksandr Valialkin
46e98ed490 vendor: update github.com/VictoriaMetrics/metrics from v1.13.1 to v1.14.0
The new version switches from log-linear histograms to log-based histograms,
which provide up to 3.6 times better accuracy.
2021-02-15 15:11:15 +02:00
Aliaksandr Valialkin
c5bdab5a4c app/vmselect/promql: add ability to pass multiple labels to sort_by_label and sort_by_label_desc functions
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992
2021-01-13 12:43:47 +02:00
Aliaksandr Valialkin
262cf81757 app/vmselect: properly parse negative combined offsets such as -1h2m3s
Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.
2020-12-19 01:25:03 +02:00
Aliaksandr Valialkin
d242c2f2bd app/vmselect/promql: add count_eq_over_time(m[d], N) and count_ne_over_time(m[d], N) for calculating the number of samples in m over d that are equal / not equal to N 2020-12-05 12:31:01 +02:00
Aliaksandr Valialkin
8cf76d8747 app/vmselect/promql: add label_uppercase(q, label1, ... labelN) and label_lowercase(q, label1, ... labelN) functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/936
2020-12-03 21:46:18 +02:00
Aliaksandr Valialkin
11bbb3552d app/vmselect/promql: make fmt 2020-12-02 21:34:15 +02:00
Aliaksandr Valialkin
9e98a8f3d3 app/vmselect/promql: return nan from minute(m) when m equals to nan
This aligns VictoriaMetrics behaviour with Prometheus behaviour.

The issue has been spotted in https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/
2020-12-02 20:16:40 +02:00
Aliaksandr Valialkin
4ef7158e89 app/vmselect/promql: return nan from a >bool b if a is nan in the same way as Prometheus does 2020-12-02 00:28:56 +02:00
Aliaksandr Valialkin
4f2c5877db app/vmselect: add -search.treatDotsAsIsInRegexps command-line flag for automatic escaping of dots in regexp label filters 2020-11-11 12:40:28 +02:00
Aliaksandr Valialkin
c4464594b7 app/vmselect/promql: allow passing optional third argument to topk_* and bottomk_* functions in order to obtain sum of time series outside top/bottom K 2020-10-20 20:09:55 +03:00
Aliaksandr Valialkin
b9a4601c97 app/vmselect/promql: return a single time series at max from absent() function like Prometheus does 2020-10-13 15:56:10 +03:00
Aliaksandr Valialkin
217c192c88 app/vmselect/promql: improve time series staleness detection
This should prevent from double counting for time series at the time when it changes label.
The most common case is in K8S, which changes pod uid label with each new deployment.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
2020-10-13 12:20:08 +03:00
Aliaksandr Valialkin
79d70480b7 Revert "app/vmselect/promql: remove metric name after applying clamp_min and clamp_max functions in order to be consistent with Prometheus"
This reverts commit bb61a4769b.

Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users -
functions, which don't change the meaning of the time series shouldn't drop metric name.

Now the following functions do not drop metric name:

* clamp_min
* clamp_max

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
2020-10-12 11:48:38 +03:00
Aliaksandr Valialkin
8c37b63ea9 Revert "app/vmselect/promql: remove metric name from results of certain rollup functions in order to be consistent with Prometheus"
This reverts commit e5202a4eae.

Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users -
functions, which don't change the meaning of the time series shouldn't drop metric name.

Now the following functions do not drop metric name:

* max_over_time
* min_over_time
* avg_over_time
* quantile_over_time
* geomean_over_time
* mode_over_time
* holt_winters
* predict_linear

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
2020-10-12 11:48:38 +03:00
Aliaksandr Valialkin
b776a93608 app/vmselect/promql: support composite durations like Prometheus 2.21 does
The following durations are supported now: `1h5m35s` or `1s543ms`

See https://github.com/prometheus/prometheus/releases/tag/v2.21.0
and https://github.com/prometheus/prometheus/pull/7713
2020-09-11 22:17:24 +03:00
Aliaksandr Valialkin
d3ad0d365e app/vmselect: move Deadline from netstorage to searchutils
This removes dependency on netstorage from searchutils.
2020-09-11 13:39:13 +03:00
Aliaksandr Valialkin
0d0b606455 app/vmselect/promql: adjust integrate() calculations to be more similar to calculations from InfluxDB
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/701
2020-09-08 14:24:02 +03:00
Aliaksandr Valialkin
db91045348 app/vmselect/promql: increase floating point calculations accuracy by dividing by 1e3 instead of multiplying by 1e-3 2020-09-08 14:01:02 +03:00
Aliaksandr Valialkin
478d8f8393 app/vmselect/promql: add count_le_over_time(m[d], le) and count_gt_over_time(m[d], gt) functions
These functions returns the number of raw samples that don't exceed `le` or are bigger than `gt`.
These functions are complement to already existing `share_le_over_time(m[d], le)` and `share_gt_over_time(m[d], gt)`.
2020-09-03 15:28:58 +03:00
Aliaksandr Valialkin
3490160fd0 app/vmselect: unconditionally align time range boundaries to step for subqueries as Prometheus does 2020-09-03 13:22:06 +03:00
Aliaksandr Valialkin
285665e93b app/vmselect/promql: allow passing multiple args to aggregate functions such as avg(q1, q2, q3) 2020-08-15 01:15:16 +03:00
Aliaksandr Valialkin
95a8c492ef app/vmselect/promql: properly handle -n^m like Prometheus does
`-n^m` must be handled as `-(n^m)` instead of `(-n)^m`.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/675
2020-08-07 07:42:42 +03:00
Aliaksandr Valialkin
7f93d61a56 app/vmselect/promql: remove metric name after applying clamp_min and clamp_max functions in order to be consistent with Prometheus
This improves VictoriaMetrics score at https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:42:55 +03:00
Aliaksandr Valialkin
75bff1a567 app/vmselect/promql: remove metric name from results of certain rollup functions in order to be consistent with Prometheus
Rollup functions:

  - avg_over_time
  - min_over_time
  - max_over_time
  - quantile_over_time

This improves VictoriaMetrics results at https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:29:18 +03:00
Aliaksandr Valialkin
8835004a4c app/vmselect: properly handle PromQL queries like scalar1 < metric < scalar2 like Prometheus does
This fixes some cases from https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:21:14 +03:00
Aliaksandr Valialkin
bdb881c43b app/vmselect/promql: add zscore-related functions: zscore_over_time(m[d]) and zscore(q) by (...) 2020-08-03 21:52:15 +03:00
Aliaksandr Valialkin
b191e425b3 app/vmselect/promql: improve further the accuracy of buckets_limit() function
The accuracy is increased by mergin the smallest bucket with the smallest adjacent bucket.
2020-07-26 12:10:56 +03:00
Aliaksandr Valialkin
cf69b1ea6f app/vmselect/promql: add buckets_limit(k, buckets) function, which limits the number of buckets per time series to k
This function works with both Prometheus-style and VictoriaMetrics-style buckets.
The function removes buckets with the lowest values in order to reserve the highest precision.
The function is useful for building heatmaps in Grafana from too big number of buckets.
2020-07-24 16:14:12 +03:00
Aliaksandr Valialkin
8d1721d128 app/vmselect/promql: add rate_over_sum(m[d]) function to MetricsQL, which returns rate over sum of m values over d duration
Something like `sum_over_time(m[d]) / d`, but more accurate.
2020-07-24 01:17:15 +03:00
Aliaksandr Valialkin
5bb4fe1ba4 app/vmselect: take into account the time spent in wait queue before query execution as time spent on the query 2020-07-21 19:00:00 +03:00
Aliaksandr Valialkin
4d2011a87d app/vmselect/promql: add mode() aggregate function 2020-07-20 15:30:11 +03:00
Aliaksandr Valialkin
427fa43ce2 app/vmselect/promql: add mode_over_time(m[d]) function
See https://en.wikipedia.org/wiki/Mode_(statistics) and https://stackoverflow.com/questions/61134078/promql-query-to-return-the-value-from-a-range-vector-which-occurs-maximum-no-of
2020-07-17 18:29:10 +03:00
Aliaksandr Valialkin
ea8dc85ba8 app/vmselect/promql: check that any() doesn't touch metric name 2020-07-17 16:23:11 +03:00
Aliaksandr Valialkin
fc8fe38a82 app/vmselect/promql: add group() aggregate function to MetricsQL
This function has been added in Prometheus 2.20. See https://github.com/prometheus/prometheus/pull/7480
2020-07-17 15:17:38 +03:00