Aliaksandr Valialkin
b1f94f7f0e
app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130
2022-02-12 15:45:09 +02:00
Aliaksandr Valialkin
96b7de6736
app/vmselect/promql: clarify comments on why the right side of if and and operators are executed at first
2022-02-03 00:26:14 +02:00
Aliaksandr Valialkin
4b850c2a59
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values
...
Such filters may slow down time series search, so just skip them.
This is a follow-up for e7f1ceeb84https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
2022-02-02 23:40:02 +02:00
Aliaksandr Valialkin
2016a2c899
app/vmselect/promql: properly handle foo or bar queries
...
Such queries may miss `bar` results after the commit e7f1ceeb84
2022-02-01 17:40:51 +02:00
Aliaksandr Valialkin
a8d22e1223
app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first
...
PromQL and MetricsQL operators are case-insensitive
2022-01-31 22:06:15 +02:00
Aliaksandr Valialkin
e7f1ceeb84
app/vmselect/promql: optimize queries, which join on _info metrics.
...
Automatically add common filters from one side of binary operation
to the other side before sending the query to storage subsystem.
See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/
and https://www.robustperception.io/exposing-the-software-version-to-prometheus
2022-01-31 19:32:36 +02:00
Aliaksandr Valialkin
dc7b63a793
app/vmselect/promql: properly keep metric names when optimized path is used for aggregate function calculations
...
For example, `sum(rate(...) keep_metric_names) by (__name__)` didn't leave the original metric name because of this issue.
This is a follup-up for 1bdc71d917https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
2022-01-17 15:30:30 +02:00
Aliaksandr Valialkin
1bdc71d917
app/vmselect/promql: implement keep_metric_names modifier for transform and rollup functions
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
2022-01-14 04:14:59 +02:00
Aliaksandr Valialkin
f41846d002
app/vmselect/promql: add stale_samples_over_time() function
2022-01-14 01:48:04 +02:00
Aliaksandr Valialkin
c883c15878
app/vmselect/promql: add support for @ modifier
...
Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier
Extend the support with the following features:
* Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()`
* Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)`
returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]`
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348
2022-01-13 22:12:06 +02:00
Aliaksandr Valialkin
ff15a752c1
app/vmselect: accept optional extra_filters[] query args for all the supported Prometheus querying APIs
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863
2021-12-06 17:07:09 +02:00
Michael Fuller
cf8c171f85
vmselect: in promql evaluation, return bytes requested when rollup memory limiter is unable to satisfy the request ( #1838 )
...
Co-authored-by: Michael Fuller <mfuller@digitalocean.com>
2021-11-22 13:20:42 +03:00
Aliaksandr Valialkin
9a3d0c43b5
app/vmselect/promql: add quantiles_over_time("phiLabel", phi1, ..., phiN, m[d]) function for calculating multiple quantiles at once
2021-09-17 23:35:10 +03:00
Aliaksandr Valialkin
cdc372bb98
app/vmselect: add -search.noStaleMarkers command-line flag for disabling stale markers handling in queries
...
This option allows reducing CPU usage a bit when VictoriaMetrics is used
for collecting and processing non-Prometheus data. For example, InfluxDB line protocol, Graphite, OpenTSDB, CSV, etc.
2021-08-18 13:59:02 +03:00
Aliaksandr Valialkin
bd14b0887e
app/vmselect/promql: move common condition to dropStaleNaNs in order to improve code maintainability
2021-08-17 11:01:16 +03:00
Aliaksandr Valialkin
113f0a8a07
app/vmselect/promql: drop staleness marks before calling rollupConfig.Do
...
This allows dropping staleness marks only once and then calculate multiple rollup functions on the result.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
2021-08-15 13:21:10 +03:00
Aliaksandr Valialkin
a925d5a3e1
app/vmselect/promql: duration handling improvements in MetricsQL queries
...
- Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600
- Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])
2021-07-12 17:16:41 +03:00
Aliaksandr Valialkin
71b72304ae
app/vmselect: improve description for -search.maxPointsPerTimeseries command-line flag
2021-03-22 16:45:34 +02:00
Aliaksandr Valialkin
2dae0a2c47
app/vmselect: add round_digits query arg to /api/v1/query and /api/v1/query_range handlers for limiting the number of decimal digits after the point
2021-03-15 12:36:33 +02:00
Aliaksandr Valialkin
2d33230793
app/vmselect/promql: properly make copies of EvalConfig
2021-02-11 12:41:15 +02:00
Aliaksandr Valialkin
7a3a9421f3
app/vmselect/promql: make a copy of EvalConfig when executing q1 and q2 in parallel for q1 binary_op q2
...
This should prevent from data races if the underlying functions modify EvalConfig contents.
2021-02-10 23:05:55 +02:00
Aliaksandr Valialkin
04faea8b45
app/vmselect: parallelize q1 <binary_op> q2 queries by running q1 and q2 in parallel
...
This should reduce query execution times.
2021-02-10 22:59:39 +02:00
Aliaksandr Valialkin
b43b498fd8
app/vmselect: add ability to pass extra_label=<label>=<value> query arg to Prometheus Querying API
...
This enforced `{label="value"}` label filter to the query.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1021
2021-02-01 18:04:17 +02:00
Aliaksandr Valialkin
c164a8d231
app/vmselect/promql: improve documentation for -search.maxPointsPertimeseries command-line flag
...
This should reduce incorrect usage and assumptions for this flag.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1020
2021-01-22 13:00:10 +02:00
Aliaksandr Valialkin
c86286ec1d
app/vmselect/promql: do not ajdust offset value provided in the query
...
Previously it could be modified in order to improve response cache hit ratio.
This is unneeded, since cache hit ratio should remain good because the query time range
should be already aligned to multiple of `step` values.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976
2020-12-27 14:09:25 +02:00
Aliaksandr Valialkin
4146fc4668
all: properly handle CPU limits set on the host system/container
...
This can reduce memory usage on systems with enabled CPU limits.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin
266788be14
app/vmselect: use storage.NewSearchQuery() instead of constructing storage.SearchQuery in-place
...
This should prevent from bugs when AccountID and ProjectID aren't set in storage.SearchQuery.
2020-11-16 18:24:00 +02:00
Aliaksandr Valialkin
1b3efccb24
app/vmselect: stop /api/v1/export/* execution if client disconnects
2020-09-27 23:53:13 +03:00
Aliaksandr Valialkin
f0005c3007
app/vmselect: move Deadline from netstorage to searchutils
...
This removes dependency on netstorage from searchutils.
2020-09-11 13:27:13 +03:00
Aliaksandr Valialkin
c0343a661b
app/vmselect/promql: increase floating point calculations accuracy by dividing by 1e3 instead of multiplying by 1e-3
2020-09-08 14:00:47 +03:00
Aliaksandr Valialkin
e706e59d49
app/vmselect: unconditionally align time range boundaries to step for subqueries as Prometheus does
2020-09-03 13:29:50 +03:00
Aliaksandr Valialkin
509d12643b
app/vmselect: show X-Forwarded-For contents on /api/v1/status/active_queries page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/659
2020-07-31 20:05:18 +03:00
Aliaksandr Valialkin
f0c678c41b
app/vmselect: do not adjust start and end query args passed to /api/v1/query_range when -search.disableCache command-line flag is set
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/563
2020-07-30 23:14:37 +03:00
Aliaksandr Valialkin
79c30cf4cb
app/vmselect: show query origin (aka remote_addr or client address) on the /api/v1/status/active_queries page for every query
2020-07-28 15:13:08 +03:00
Aliaksandr Valialkin
b9c8f6bf34
app/vmselect: log the total available memory for concurrent requests on not enough memory errors
...
This should simplify root cause analysis
2020-07-20 19:51:40 +03:00
Aliaksandr Valialkin
d5dddb0953
all: use %w instead of %s for wrapping errors in fmt.Errorf
...
This will simplify examining the returned errors such as httpserver.ErrorWithStatusCode .
See https://blog.golang.org/go1.13-errors for details.
2020-06-30 23:05:11 +03:00
Aliaksandr Valialkin
574289c3fb
app/vmselect/promql: support for sum(x) by (y) limit N syntax in order to limit the number of output time series after aggregation
2020-05-12 19:45:54 +03:00
Aliaksandr Valialkin
57407cca83
app/vmselect/promql: remove -search.maxPointsPerTimeseries command-line flag
...
Limit the estimated time series count after aggregation with grouping by the number of source time series.
2020-04-29 00:20:04 +03:00
Aliaksandr Valialkin
4e4f57b121
lib/metricsql: move it to a separate repository - github.com/VictoriaMetrics/metrics
2020-04-28 15:28:22 +03:00
Aliaksandr Valialkin
17d96e4503
app/vmselect: add -search.estimatedSeriesCountAfterAggregation command-line flag for tuning the probability of OOMs vs false-positive not enough memory errors
2020-04-28 12:52:37 +03:00
Aliaksandr Valialkin
78b62dee87
app/vmselect/promql: properly adjust time range for data to select
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/309
2020-02-05 21:22:18 +02:00
Aliaksandr Valialkin
366693b9f1
app/vmselect: unconditionally offset -step to rollup_candlestick. This makes results more consistent
2020-02-04 23:32:12 +02:00
Aliaksandr Valialkin
525101339e
app/vmselect/promql: automatically apply offset -step to rollup_candlestick function in order to obtain the expected OHLC results
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/309
2020-02-04 23:24:35 +02:00
Aliaksandr Valialkin
2839f4688a
app/vmselect/promql: fix panic on sum(aggr_over_time(...)) with incorrect number of args
2020-01-15 16:26:09 +02:00
Aliaksandr Valialkin
8b14572f70
app/vmselect/promql: add hoeffding_bound_upper(phi, m[d]) and hoeffding_bound_lower(phi, m[d]) functions
...
These functions can be used for calculating Hoeffding bounds
for `m` over `d` time range and for the given `phi` in the range `[0..1]`.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/283
2020-01-11 14:46:23 +02:00
Aliaksandr Valialkin
164278151f
app/vmselect/promql: add aggr_over_time(("aggr_func1", "aggr_func2", ...), m[d]) function
...
This function can be used for simultaneous calculating of multiple `aggr_func*` functions
that accept range vector. For example, `aggr_over_time(("min_over_time", "max_over_time"), m[d])`
would calculate `min_over_time` and `max_over_time` for `m[d]`.
2020-01-10 21:18:06 +02:00
Aliaksandr Valialkin
b1ded7cf9a
app/vmselect/promql: add absent_over_time(m[d]) func similar to the function in Prometheus 2.16
...
See https://github.com/prometheus/prometheus/issues/2882
2020-01-04 12:45:07 +02:00
Aliaksandr Valialkin
a8360d04c0
app/vmselect/promql: add histogram_over_time(m[d]) rollup function
2020-01-04 12:44:56 +02:00
Aliaksandr Valialkin
3e09d38f29
app/vmselect/promql: fix results caching for multi-arg rollup functions such as quantile_over_time
...
Previosly only a single arg was taken into account, so caching didn't work properly for multi-arg rollup funcs.
2020-01-03 20:49:08 +02:00
Aliaksandr Valialkin
b5645ccbdf
app/vmselect/promql: increase the estimated number of time series returned by aggr() by (something) from 100 to 1K, since 100 may result in OOM for high number of time series
2020-01-03 01:02:21 +02:00
