516 commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	b285207aa7	app/vmselect: add -search.logQueryMemoryUsage command-line flag for logging queries, which take big amounts of memory ... Thanks to @michal-kralik for initial attempts for this feature: - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3651 - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3715 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3553	2023-02-23 18:47:08 -08:00
Alexander Marshalov	aaef0bac00	fix interpolate function for filling only intermediate gaps (#3816) (#3857) ... * fix interpolate function for filling only intermediate gaps (#3816) * Update docs/CHANGELOG.md --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-22 18:38:43 -08:00
Aliaksandr Valialkin	2cea923ff7	app/vmselect/promql: add share(q) aggregate function for normalizing results across multiple time series in [0..1] value range per each timestamp and aggregation group	2023-02-18 22:42:01 -08:00
Aliaksandr Valialkin	c86f1f1d1b	app/vmselect/promql: add range_zscore(q) and range_trim_zscore(z, q) functions ... These functions may be useful for dropping outliers at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3759	2023-02-18 22:41:59 -08:00
Aliaksandr Valialkin	406822a16c	app/vmselect/promql: add range_mad(q) and range_trim_outliers(k, q) functions ... These functions may help trimming outliers during query time for the use case described at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3759	2023-02-18 15:19:10 -08:00
Oleksandr Redko	9fff48c3e3	app,lib: fix typos in comments (#3804)	2023-02-13 13:27:13 +01:00
Aliaksandr Valialkin	b11bdc46be	app/vmselect/promql: add mad_over_time(m[d]) function ... See https://github.com/prometheus/prometheus/issues/5514	2023-02-11 01:06:20 -08:00
Aliaksandr Valialkin	30e968df6d	app/vmselect: use consistent randomizer in tests ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3683	2023-01-23 19:27:25 -08:00
Aliaksandr Valialkin	0c625185cb	app/vmselect/promql: updates tests for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3664	2023-01-17 23:25:45 -08:00
Aliaksandr Valialkin	207a62a3c2	app/vmselect/promql: reduce memory allocations when searching for time series pairs with identical labelsets in q1 op q2 queries ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3641	2023-01-15 13:03:23 -08:00
Aliaksandr Valialkin	27afe7bc38	app/vmselect/promql: reduce the number of memory allocations inside getCommonLabelFilters() ... This should improve performance a bit for `q1 op q2` queries	2023-01-15 13:03:23 -08:00
Aliaksandr Valialkin	7067e8206c	app/vmselect/promql: reduce memory allocations at getCommonLabelFilters() function ... Intern tag keys and values there	2023-01-12 01:27:41 -08:00
Aliaksandr Valialkin	31fc29599f	app/vmselect/promql: move the eval function args in parallel query trace outside the loop	2023-01-10 22:23:30 -08:00
Aliaksandr Valialkin	9a563a6aef	app/vmselect/promql: eliminate memory allocation when sorting values inside float64s	2023-01-09 23:06:46 -08:00
Aliaksandr Valialkin	30ed33fae0	app/vmselect/promql: pre-allocate memory for values to be merged in mergeTimeseries() ... This should reduce the number of memory re-allocations	2023-01-09 22:51:17 -08:00
Aliaksandr Valialkin	645c24dc5f	app/vmselect/promql: consistently intern series names obtained from marshalMetricNameSorted ... This reduces memory allocations when the returned series names are used as map keys later	2023-01-09 22:45:40 -08:00
Aliaksandr Valialkin	2f3ddd4884	app/vmselect/promql: avoid memory allocations and copying from source timeseries to the returned result at timeseriesToResult()	2023-01-09 22:38:59 -08:00
Aliaksandr Valialkin	26cf680468	app/vmselect/promql: remove memory allocations from sortMetricTags()	2023-01-09 22:22:15 -08:00
Aliaksandr Valialkin	4f0c11ee93	app/vmselect/promql: intern output series names inside timeseriesToResult() ... This reduces the number of memory allocations for repeated queries, which return (almost) the same set of time series.	2023-01-09 22:19:56 -08:00
Aliaksandr Valialkin	562d6bca08	app/vmselect/promql: intern output series names during normal aggregation	2023-01-09 22:15:24 -08:00
Aliaksandr Valialkin	21ee9a1fab	app/vmselect/promql: intern output series names during incremental aggregation ... This should reduce the number of memory allocations for repeated queries	2023-01-09 22:11:36 -08:00
Aliaksandr Valialkin	04dff34de4	vendor: update github.com/VictoriaMetrics/metricsql from v0.50.0 to v0.51.0 ... Updates https://github.com/VictoriaMetrics/metricsql/pull/7 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3589	2023-01-05 01:50:10 -08:00
Aliaksandr Valialkin	f1441a598f	app/vmselect/promql: add tests for d3de110070	2022-12-21 20:25:21 -08:00
Aliaksandr Valialkin	d3de110070	app/vmselect/promql: make sure that label_replace() doesn't create an empty dst_label if the src_label doesn't match regex	2022-12-21 20:20:01 -08:00
Aliaksandr Valialkin	9fa3f1dc57	app/vmselect/promql: do not extend too short lookbehind window for rate() function if it is set explicitly ... Previously too short lookbehind window d for rate(m[d]) could be automatically extended if it didn't cover at least two raw samples. This was needed in order to guarantee non-empty results from rate(m[d]) on short time ranges. Now the lookbehind window isn't extended if it is set explicitly, since it is expected that the user knows what he is doing. The lookbehind window continues to be extended when needed if it isn't set explicitly. For example, in the case of rate(m). Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3483	2022-12-20 00:18:20 -08:00
Aliaksandr Valialkin	e272a0ec78	app/vmselect/promql: allow passing inf arg into functions, which accept numeric limit on the number of output time series ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3461	2022-12-10 22:47:47 -08:00
Aliaksandr Valialkin	5eae9a9914	app/vmselect/promql: add range_trim_spikes(phi, q) function for trimming phi percent of largest spikes per each time series returned by q	2022-12-05 21:55:01 -08:00
Aliaksandr Valialkin	d9c3a2b605	app/vmselect/promql: add range_normalize(q1, ..., qN) function for normalizing query results into [0..1] value range ... This may be useful for analyzing correlation between time series with different value ranges	2022-11-21 23:25:00 +02:00
Aliaksandr Valialkin	2c9e403d5f	app/vmselect/promql: properly return an empty result from limit_offset() if offset exceeds the number of inner time series ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3312	2022-11-21 16:47:37 +02:00
Aliaksandr Valialkin	b796a0dc3f	app/vmselect/promql: optimize e1 op e2 when e1 returns an empty result ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3349	2022-11-21 16:09:10 +02:00
Aliaksandr Valialkin	a260e2659e	app/vmselect/promql: add range_stdvar() and range_stddev() functions for calculating variance and deviation over time series on the selected time range	2022-11-17 01:03:40 +02:00
Aliaksandr Valialkin	c1a3192d8b	app/vmselect/promql: add range_linear_regression(q) function for calculating simple linear regression for the selected time series on the selected time range	2022-11-17 00:38:48 +02:00
Aliaksandr Valialkin	c4265322f4	lib/fs: add canOverwrite arg to WriteFileAtomically when it is allowed to overwrite the file atomically if it already exists	2022-10-26 01:07:34 +03:00
Aliaksandr Valialkin	a10647e0bf	app/vmselect/promql: expose missing metric vm_cache_size_max_bytes{type="promql/rollupResult"}	2022-10-23 12:13:47 +03:00
Aliaksandr Valialkin	e6fd33044f	app/vmselect/promql: follow-up for 930f1ee153 ... Document the change at docs/CHANGELOG.md Apply it to histogram_quantile() in the same way as to histogram_share() Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3225	2022-10-13 12:03:08 +03:00
Siqi Liu	930f1ee153	BUGFIX: properly calculate histogram_quantile with the same value and different string le (#3225) ... Co-authored-by: 647(siki.liu) <siki.liu@huolala.cn>	2022-10-13 11:57:16 +03:00
Aliaksandr Valialkin	b8da90b893	app/vmselect/promql: properly handle zero and negative values for -search.maxMemoryPerQuery ... This is a follow-up for 04a05f161c Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203	2022-10-12 09:25:17 +03:00
Aliaksandr Valialkin	04a05f161c	app/vmselect: return back the logic for limits the amounts of memory occupied by concurrently executed queries if -search.maxMemoryPerQuery isn't set ... This is needed for preserving backwards compatibility with the previous releases of VictoriaMetrics. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203	2022-10-10 21:45:13 +03:00
Aliaksandr Valialkin	5138eaeea0	app/vmselect: allow limiting per-query memory usage via -search.maxMemoryPerQuery command-line flag ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203	2022-10-08 01:08:05 +03:00
Aliaksandr Valialkin	f452c84579	app/vmselect/promql: properly calculate vm_rows_scanned_per_query histogram for rollup functions, which take into account only a few samples on the provided lookbehind window	2022-10-06 23:22:24 +03:00
Aliaksandr Valialkin	40e899fd67	app/vmselect/promql: properly calculate quantiles_over_time() over a single raw sample	2022-10-06 22:37:21 +03:00
Aliaksandr Valialkin	fb1cc3cc94	app/vmselect/promql: increase scalability of incremental aggregate calculations on systems with many CPU cores ... Use sync.Map instead of a global mutex there. This should lift scalability limits on systems with many CPU cores.	2022-10-01 20:00:03 +03:00
Aliaksandr Valialkin	b70f815dc4	app/vmselect/promql: remove empty series before applying aggregate function ... Previously empty series (e.g. series with all NaN samples) were passed to aggregate functions. Such series must be ingored by all the aggregate functions. So it is better from consistency PoV filtering out empty series before applying aggregate functions.	2022-09-30 08:39:54 +03:00
Roman Khavronenko	b64b9b9fec	app/vmselect: ignore empty series for limit_offset (#3178) ... * app/vmselect: ignore empty series for `limit_offset` VictoriaMetrics doesn't return empty series (with all NaN values) to the user. But such series are filtered after transform functions. It means `limit_offset` will account for empty series as well. For example, let's consider following data set: ``` time series: foo{label="1"} NaN, NaN, NaN, NaN // empty series foo{label="2"} 1, 2, 3, 4 foo{label="3"} 4, 3, 2, 1 ``` When user requests all series for metric `foo` the empty series will be filtered out: ``` /query=foo: foo{label="v2"} 1, 2, 3, 4 foo{label="v3"} 4, 3, 2, 1 ``` But `limit_offset(1, 1, foo)` is applied to original series, not filtered yet. So it will return `foo{label="v2"}` (skips the first in list) ``` /query=limit_offset(1, 1, foo): foo{label="v2"} 1, 2, 3, 4 ``` Expected result would be to apply `limit_offset` to already filtered list, so in result we receive `foo{label="v3"}`: ``` /query=limit_offset(1, 1, foo): foo{label="v3"} 4, 3, 2, 1 ``` The change does exactly that - filters empty series before applying `limit_offset`. Signed-off-by: hagen1778 <roman@victoriametrics.com> * app/vmselect: ignore empty series for `limit_offset` Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-09-30 08:20:34 +03:00
Roman Khavronenko	166d444159	vmselect/rollup: rm workaround for slow-changing counters (#3163) ... The workaround was introduced to fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962. However, it didn't prove itself useful. Instead, it is recommended using `increase_pure` function. Removing the workaround makes VM to produce accurate results when calculating `delta` or `increase` functions over slow-changing counters with vary intervals between data points. Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-09-26 15:33:25 +03:00
Dmytro Kozlov	b75f1854c5	vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric) (#2982) ... * vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric) * vmselect/promql: fix tests, add documentation * vmselect/promql: update test * vmselect/promql: update for alphanumeric sorting, fix tests * vmselect/promql: remove comments * vmselect/promql: cleanup * vmselect/promql: avoid memory allocations, update functions descriptions * vmselect/promql: make linter happy (remove ineffectual assigment) * vmselect/promql: add test case, fix behavior when strings are equal * vendor: update github.com/VictoriaMetrics/metricsql from v0.44.1 to v0.45.0 this adds support for sort_by_label_numeric and sort_by_label_numeric_desc functions * wip * lib/promscrape: read response body into memory in stream parsing mode before parsing it This reduces scrape duration for targets returning big responses. The response body was already read into memory in stream parsing mode before this change, so this commit shouldn't increase memory usage. * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-09-14 17:41:09 +03:00
Dmytro Kozlov	4415c71a2b	vmselect/{promql, prometheus}: show flag names which user can update in error message (#3049) ... * vmselect/{promql, prometheus}: show flag names which user can update in error message * vmselect/{promql, prometheus}: fix typo	2022-09-06 13:25:59 +03:00
Aliaksandr Valialkin	7dc632719d	app/vmselect/promql: consistently calculate rate_over_sum(m[d]) as sum_over_time(m[d])/d ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3045	2022-09-02 23:18:04 +03:00
Aliaksandr Valialkin	4076277cf0	app/vmselect/promql: evaluate union() args in parallel in order to increase query performance ... Note that the parallel execution of `union()` args may take more memory and CPU time than the sequential execution if args contain heavy queries, which may load all the available CPU, disk and memory resources and vmselect and vmstorage levels.	2022-09-02 19:46:27 +03:00
Aliaksandr Valialkin	ad11b8d83d	app/vmselect/promql: follow-up after 2d71b4859c ... - Use getScalar() function for obtaining the expected scalar from phi arg - Reduce the error message returned to the user when incorrect phi is passed to histogram_quantiles - Improve the description of this bugfix in the docs/CHANGELOG.md Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3026	2022-08-27 01:35:49 +03:00
Dmytro Kozlov	2d71b4859c	vmselect/promql: fix panic in histogram_quantiles function (#3029) ... * vmselect/promql: fix panic in histogram_quantiles function * Update docs/MetricsQL.md Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com> Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>	2022-08-27 01:33:56 +03:00
Dmytro Kozlov	463ea6897b	vmselect/promql: enable search.maxPointsSubqueryPerTimeseries for sub-queries (#2963) ... * vmselect/promql: enable search.maxPointsPerTimeSeriesSubquery for sub-queries * vmselect/promql: cleanup * vmselect/promql: rename config flag * vmselect/promql: add tests * vmselect/promql: use test object instead of log * vmselect/promql: fix posible panic is subquery has more points. add description * vmselect/promql: update tests descriptions * vmselect/promql: update doInternal validation * vmselect/promql: fix linter * vmselect/promql: fix linter * vmselect/promql: update documentation and release notes * wip - Properly apply -search.maxPointsSubqueryPerTimeseries limit to subqueries. Previously the -search.maxPointsPerTimeseries limit was unexpectedly applied to subqueries if it was smaller than the -search.maxPointsSubqueryPerTimeseries . - Clarify docs for -search.maxPointsSubqueryPerTimeseries command-line flag . - Document -search.maxPointsPerTimeseries and -search.maxPointsSubqueryPerTimeseries flags at https://docs.victoriametrics.com/#resource-usage-limits . - Update docs/CHANGELOG.md . Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2922 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-08-24 15:25:18 +03:00
Aliaksandr Valialkin	9f94c295ab	all: use os.{Read|Write}File instead of ioutil.{Read|Write}File ... The ioutil.{Read|Write}File is deprecated since Go1.16 - see https://tip.golang.org/doc/go1.16#ioutil VictoriaMetrics needs at least Go1.18, so it is safe to remove ioutil usage from source code. This is a follow-up for 02ca2342ab	2022-08-21 23:52:35 +03:00
Aliaksandr Valialkin	4ac79d29ad	app/vmselect: follow-up after 63e0f16062 ... * Explicitly store a pointer to UserReadableError in the error interface. Previously Go automatically converted the value to a pointer before storing in the error interface. * Add Unwrap() method to UserReadableError, so it can be used transparently with the other code, which calls errors.Is() and errors.As(). * Document the change in docs/CHANGELOG.md	2022-08-15 13:50:16 +03:00
Roman Khavronenko	63e0f16062	vmselect: introduce UserReadableError type of error (#2894) ... When read query fails, VM returns rich error message with all the details. While these details might be useful for debugging specific cases, they're usually too verbose for users. Introducing a new error type `UserReadableError` is supposed to allow to return to user only the most important parts of the error trace. This supposed to improve error readability in web interfaces such as VMUI or Grafana. The full error trace is still logged with the full context and can be found in vmselect logs. Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-08-15 13:38:47 +03:00
Dmytro Kozlov	a927814e7b	vmselect/promql: add tests for vmrangeBucketsToLE (#2907) ... * vmselect/promql: add tests for vmrangeBucketsToLE * vmselect/promql: cleanup * vmselect/promql: cleanup * vmselect/promql: fix panic tests want result * vmselect/promql: cleanup * vmselect/promql: update test name * vmselect/promql: fix linter error * vmselect/promql: refactor testcases * vmselect/promql: cleanup * vmselect/promql: remove unused reassign to workers, fix typo * wip * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-26 20:42:41 +03:00
Aliaksandr Valialkin	c888e6b9be	app/vmselect/promql: reduce the diff for f148cffc8a ... This is a follow-up for c826f06366	2022-07-26 19:20:48 +03:00
Alan Liang	c826f06366	vmselect: fix vmrangeBucketsToLE func may panic when ts value equal zero (#2902) ... Co-authored-by: alanwzliang <alanwzliang@tencent.com>	2022-07-25 10:55:13 +03:00
Roman Khavronenko	9ccf695d57	vmselect: return correct error for second part of expression (#2893) ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-20 16:44:28 +02:00
Aliaksandr Valialkin	cc7d499bbd	app/vmselect/promql: execute q1 and q2 from q1 op q2 in parallel if labels pushdown cannot be applied ... This should improve query performance if VictoriaMetrics has enough resources for processing `q1` and `q2` in parallel. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2886	2022-07-19 14:27:48 +03:00
Aliaksandr Valialkin	b5da47bfaf	app/vmselect/promql: properly return q1 series from q1 ifnot q2 when q2 returns nothing	2022-07-18 14:24:54 +03:00
Aliaksandr Valialkin	0792c4ca90	app/vmselect/promql/transform.go: reuse evalNumber() function for constructing timezone_offset() results	2022-07-18 14:24:53 +03:00
Aliaksandr Valialkin	29e53b9f55	app/vmselect/promql: consistency update after 93fbd0c54b	2022-07-13 12:33:14 +03:00
Roman Khavronenko	93fbd0c54b	promql: return step as scrapeInterval when it can't be calculated (#2865) ... The change allows to specify default value for `getScrapeInterval` function when actual interval can't be calculated. Before the change, function were returning `maxSilenceInterval` (5m) in such cases, which may be not correct for instant queries processing. The specific scenario where using `maxSilenceInterval` caused issues is the following: 1. Series becomes stale; 2. Client (in this case vmalert) continues to request series every 15s; 3. Database returns empty results as expected; 4. But at some specific moment of time database returns datapoints from `now()-5m`, because lookback window was extended to `maxSilenceInterval`. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-13 12:27:38 +03:00
Aliaksandr Valialkin	c2197ad139	app/vmselect/promql: validate function name before evaluating its arguments ... This avoids unneeded evaluation of args for unknown functions	2022-07-12 19:48:26 +03:00
Roman Khavronenko	e1a41cfab5	metricsql: properly evaluate timezone_offset over time interval (#2842) ... * metricsql: properly evaluate `timezone_offset` over time interval https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771 Signed-off-by: hagen1778 <roman@victoriametrics.com> * Update docs/CHANGELOG.md Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-08 14:03:56 +03:00
Aliaksandr Valialkin	ae80cf76e0	app/vmselect: make fmt after f3ece83e67	2022-07-05 14:35:24 +03:00
Aliaksandr Valialkin	f3ece83e67	app/vmselect/promql: properly calculate histogram_quantile over unexpected le buckets ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819	2022-07-05 13:19:24 +03:00
Aliaksandr Valialkin	84e373e5c7	app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions	2022-06-30 22:39:38 +03:00
Aliaksandr Valialkin	3e2dd85f7d	all: readability improvements for query traces ... - show dates in human-readable format, e.g. 2022-05-07, instead of a numeric value - limit the maximum length of queries and filters shown in trace messages	2022-06-30 18:20:33 +03:00
Aliaksandr Valialkin	a14188dd8e	app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload ... This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792	2022-06-28 20:18:13 +03:00
Aliaksandr Valialkin	a43f2d0bc5	app/vmselect/promql: show the number of scanned samples in the query trace	2022-06-28 19:26:17 +03:00
Aliaksandr Valialkin	e578549b8a	app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day	2022-06-28 13:02:47 +03:00
Aliaksandr Valialkin	a963b2a0aa	all: show timeRange in traces in human-readable format instead of timestamps in milliseconds	2022-06-27 13:45:51 +03:00
Aliaksandr Valialkin	e6ed92529b	all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package ... This package already has the same name, so there is no need in explicit name	2022-06-21 20:23:32 +03:00
Aliaksandr Valialkin	12ac255dae	lib/querytracer: make it easier to use by passing trace context message to New and NewChild ... The context message can be extended by calling Donef. If there is no need to extend the message, then just call Done.	2022-06-08 21:06:52 +03:00
Aliaksandr Valialkin	41958ed5dd	all: add initial support for query tracing ... See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403	2022-06-01 02:29:23 +03:00
Aliaksandr Valialkin	a7f18f8cb2	app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps	2022-05-09 17:57:39 +03:00
Aliaksandr Valialkin	cce1b6d7f9	app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d	2022-04-27 10:59:03 +03:00
Aliaksandr Valialkin	25fe83577d	app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries ... Previously `vector` time series could be unexpectedly returned from such queries	2022-04-21 15:34:36 +03:00
Aliaksandr Valialkin	d1a9fac894	app/vmselect/promql: fix comparison to nan ... The comparison to nan has been broken in d335cc886c Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150	2022-04-21 14:55:37 +03:00
Aliaksandr Valialkin	de892239a9	app/vmselect/promql: add drop_common_labels() function	2022-04-21 14:20:20 +03:00
Aliaksandr Valialkin	7a44ba1234	app/vmselect/promql: fix q default b where b may have empty time series	2022-04-21 11:42:42 +03:00
Aliaksandr Valialkin	d335cc886c	app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values ... This should prevent from `duplicate time series` errors when executing the following query: kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1) where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series	2022-04-20 22:18:44 +03:00
Aliaksandr Valialkin	ed97908ca9	app/vmselect/promql: rename removeNaNs() to more clear removeEmptySeries()	2022-04-20 19:53:46 +03:00
Aliaksandr Valialkin	61c7f6beae	app/vmselect/promql: allow calling InitRollupResultCache+StopRollupResultCache multiple times during tests ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2406	2022-04-11 12:34:43 +03:00
Aliaksandr Valialkin	6e364e19ef	app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs	2022-03-26 11:29:49 +02:00
Aliaksandr Valialkin	0d47c23a03	app/vmselect/promql: reduce the maximum number of label values, which can be propagated from one side of the binary operation to another side of the binary operation from 10K to 1K ... There are user reports that 10K unique values in a single label filter may lead to performance and memory usage issues	2022-02-24 04:05:18 +02:00
Aliaksandr Valialkin	62b46007c5	lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes ... This should reduce memory usage under high time series churn rate	2022-02-23 13:41:45 +02:00
Aliaksandr Valialkin	b1f94f7f0e	app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130	2022-02-12 15:45:09 +02:00
Aliaksandr Valialkin	96b7de6736	app/vmselect/promql: clarify comments on why the right side of if and and operators are executed at first	2022-02-03 00:26:14 +02:00
Aliaksandr Valialkin	4b850c2a59	app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values ... Such filters may slow down time series search, so just skip them. This is a follow-up for e7f1ceeb84 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827	2022-02-02 23:40:02 +02:00
Aliaksandr Valialkin	2016a2c899	app/vmselect/promql: properly handle foo or bar queries ... Such queries may miss `bar` results after the commit e7f1ceeb84 because common label filters from `foo` could be mistakenly applied to `bar`.	2022-02-01 17:40:51 +02:00
Aliaksandr Valialkin	4bdd10ab90	lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations ... Follow-up for f4989edd96 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007	2022-02-01 00:18:42 +02:00
Aliaksandr Valialkin	a8d22e1223	app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first ... PromQL and MetricsQL operators are case-insensitive	2022-01-31 22:06:15 +02:00
Aliaksandr Valialkin	e7f1ceeb84	app/vmselect/promql: optimize queries, which join on _info metrics. ... Automatically add common filters from one side of binary operation to the other side before sending the query to storage subsystem. See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/ and https://www.robustperception.io/exposing-the-software-version-to-prometheus	2022-01-31 19:32:36 +02:00
Aliaksandr Valialkin	f4989edd96	lib/bytesutil: split Resize() into ResizeNoCopy() and ResizeWithCopy() functions ... Previously bytesutil.Resize() was copying the original byte slice contents to a newly allocated slice. This wasted CPU cycles and memory bandwidth in some places, where the original slice contents wasn't needed after slize resizing. Switch such places to bytesutil.ResizeNoCopy(). Rename the original bytesutil.Resize() function to bytesutil.ResizeWithCopy() for the sake of improved readability. Additionally, allocate new slice with `make()` instead of `append()`. This guarantees that the capacity of the allocated slice exactly matches the requested size. The `append()` could return a slice with bigger capacity as an optimization for further `append()` calls. This could result in excess memory usage when the returned byte slice was cached (for instance, in lib/blockcache). Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007	2022-01-25 15:24:44 +02:00
Aliaksandr Valialkin	dc7b63a793	app/vmselect/promql: properly keep metric names when optimized path is used for aggregate function calculations ... For example, `sum(rate(...) keep_metric_names) by (__name__)` didn't leave the original metric name because of this issue. This is a follup-up for 1bdc71d917 Udates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949	2022-01-17 15:30:30 +02:00
Aliaksandr Valialkin	1bdc71d917	app/vmselect/promql: implement keep_metric_names modifier for transform and rollup functions ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949	2022-01-14 04:14:59 +02:00
Aliaksandr Valialkin	f41846d002	app/vmselect/promql: add stale_samples_over_time() function	2022-01-14 01:48:04 +02:00