468 commits

Author	SHA1	Message	Date
Dmytro Kozlov	d32a6359b0	vmselect/promql: enable search.maxPointsSubqueryPerTimeseries for sub-queries (#2963) ... * vmselect/promql: enable search.maxPointsPerTimeSeriesSubquery for sub-queries * vmselect/promql: cleanup * vmselect/promql: rename config flag * vmselect/promql: add tests * vmselect/promql: use test object instead of log * vmselect/promql: fix posible panic is subquery has more points. add description * vmselect/promql: update tests descriptions * vmselect/promql: update doInternal validation * vmselect/promql: fix linter * vmselect/promql: fix linter * vmselect/promql: update documentation and release notes * wip - Properly apply -search.maxPointsSubqueryPerTimeseries limit to subqueries. Previously the -search.maxPointsPerTimeseries limit was unexpectedly applied to subqueries if it was smaller than the -search.maxPointsSubqueryPerTimeseries . - Clarify docs for -search.maxPointsSubqueryPerTimeseries command-line flag . - Document -search.maxPointsPerTimeseries and -search.maxPointsSubqueryPerTimeseries flags at https://docs.victoriametrics.com/#resource-usage-limits . - Update docs/CHANGELOG.md . Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2922 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-08-24 15:27:41 +03:00
Aliaksandr Valialkin	06f6de6d47	all: use os.{Read|Write}File instead of ioutil.{Read|Write}File ... The ioutil.{Read|Write}File is deprecated since Go1.16 - see https://tip.golang.org/doc/go1.16#ioutil VictoriaMetrics needs at least Go1.18, so it is safe to remove ioutil usage from source code. This is a follow-up for 02ca2342ab	2022-08-21 23:55:20 +03:00
Aliaksandr Valialkin	7d7cf2b6fd	app/vmselect: follow-up after 63e0f16062 ... * Explicitly store a pointer to UserReadableError in the error interface. Previously Go automatically converted the value to a pointer before storing in the error interface. * Add Unwrap() method to UserReadableError, so it can be used transparently with the other code, which calls errors.Is() and errors.As(). * Document the change in docs/CHANGELOG.md	2022-08-15 13:53:19 +03:00
Roman Khavronenko	8a26ec435d	vmselect: introduce UserReadableError type of error (#2894) ... When read query fails, VM returns rich error message with all the details. While these details might be useful for debugging specific cases, they're usually too verbose for users. Introducing a new error type `UserReadableError` is supposed to allow to return to user only the most important parts of the error trace. This supposed to improve error readability in web interfaces such as VMUI or Grafana. The full error trace is still logged with the full context and can be found in vmselect logs. Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-08-15 13:53:18 +03:00
Aliaksandr Valialkin	21f588e9cf	app/vmselect/promql: fix TestVmrangeBucketsToLE test after 8516670582 ... Remove 'AccountID=0, ProjectID=0, ' prefix for cluster version	2022-08-07 00:15:10 +03:00
Dmytro Kozlov	ab645b5fab	vmselect/promql: add tests for vmrangeBucketsToLE (#2907) ... * vmselect/promql: add tests for vmrangeBucketsToLE * vmselect/promql: cleanup * vmselect/promql: cleanup * vmselect/promql: fix panic tests want result * vmselect/promql: cleanup * vmselect/promql: update test name * vmselect/promql: fix linter error * vmselect/promql: refactor testcases * vmselect/promql: cleanup * vmselect/promql: remove unused reassign to workers, fix typo * wip * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-26 20:43:43 +03:00
Aliaksandr Valialkin	8a83c59956	app/vmselect/promql: reduce the diff for f148cffc8a ... This is a follow-up for c826f06366	2022-07-26 19:21:16 +03:00
Alan Liang	f97de473f4	vmselect: fix vmrangeBucketsToLE func may panic when ts value equal zero (#2902) ... Co-authored-by: alanwzliang <alanwzliang@tencent.com>	2022-07-25 10:49:48 +03:00
Roman Khavronenko	04c4f8bafd	vmselect: return correct error for second part of expression (#2893) ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-21 20:40:47 +03:00
Aliaksandr Valialkin	f992f96a88	app/vmselect/promql: execute q1 and q2 from q1 op q2 in parallel if labels pushdown cannot be applied ... This should improve query performance if VictoriaMetrics has enough resources for processing `q1` and `q2` in parallel. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2886	2022-07-19 14:29:41 +03:00
Aliaksandr Valialkin	d920b0afec	app/vmselect/promql: properly return q1 series from q1 ifnot q2 when q2 returns nothing	2022-07-18 14:24:23 +03:00
Aliaksandr Valialkin	f933d5f142	app/vmselect/promql/transform.go: reuse evalNumber() function for constructing timezone_offset() results	2022-07-18 14:24:22 +03:00
Aliaksandr Valialkin	919939ee9f	app/vmselect/promql: consistency update after 93fbd0c54b	2022-07-13 12:34:28 +03:00
Roman Khavronenko	214e01499c	promql: return step as scrapeInterval when it can't be calculated (#2865) ... The change allows to specify default value for `getScrapeInterval` function when actual interval can't be calculated. Before the change, function were returning `maxSilenceInterval` (5m) in such cases, which may be not correct for instant queries processing. The specific scenario where using `maxSilenceInterval` caused issues is the following: 1. Series becomes stale; 2. Client (in this case vmalert) continues to request series every 15s; 3. Database returns empty results as expected; 4. But at some specific moment of time database returns datapoints from `now()-5m`, because lookback window was extended to `maxSilenceInterval`. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-13 12:34:28 +03:00
Aliaksandr Valialkin	5867708b3d	app/vmselect/promql: validate function name before evaluating its arguments ... This avoids unneeded evaluation of args for unknown functions	2022-07-12 19:49:54 +03:00
Roman Khavronenko	78da074ad3	metricsql: properly evaluate timezone_offset over time interval (#2842) ... * metricsql: properly evaluate `timezone_offset` over time interval https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771 Signed-off-by: hagen1778 <roman@victoriametrics.com> * Update docs/CHANGELOG.md Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-08 14:04:40 +03:00
Aliaksandr Valialkin	f4df43f7cc	app/vmselect/netstorage: remove unused auth.Token arg	2022-07-06 12:46:21 +03:00
Aliaksandr Valialkin	10c63f5853	app/vmselect: make fmt after f3ece83e67	2022-07-05 14:56:53 +03:00
Aliaksandr Valialkin	8516670582	app/vmselect/promql: properly calculate histogram_quantile over unexpected le buckets ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819	2022-07-05 13:20:41 +03:00
Aliaksandr Valialkin	fa08220d27	app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2787	2022-06-30 22:40:26 +03:00
Aliaksandr Valialkin	4fb0f15322	all: readability improvements for query traces ... - show dates in human-readable format, e.g. 2022-05-07, instead of a numeric value - limit the maximum length of queries and filters shown in trace messages	2022-06-30 18:19:43 +03:00
Aliaksandr Valialkin	daefb64f38	app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload ... This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792	2022-06-28 20:18:31 +03:00
Aliaksandr Valialkin	9adff0b686	app/vmselect/promql: show the number of scanned samples in the query trace	2022-06-28 19:32:03 +03:00
Aliaksandr Valialkin	399d4c36ae	app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day	2022-06-28 12:55:20 +03:00
Aliaksandr Valialkin	6386f117c8	all: show timeRange in traces in human-readable format instead of timestamps in milliseconds	2022-06-27 13:42:57 +03:00
Aliaksandr Valialkin	fe2269b999	all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package ... This package already has the same name, so there is no need in explicit name	2022-06-21 20:24:28 +03:00
Aliaksandr Valialkin	a9ea3fee38	lib/querytracer: make it easier to use by passing trace context message to New and NewChild ... The context message can be extended by calling Donef. If there is no need to extend the message, then just call Done.	2022-06-08 21:16:12 +03:00
Aliaksandr Valialkin	afced37c0b	all: add initial support for query tracing ... See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403	2022-06-01 02:31:44 +03:00
Aliaksandr Valialkin	68bfa05437	app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps	2022-05-09 17:58:39 +03:00
Aliaksandr Valialkin	6eb1580158	app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d	2022-04-27 10:58:40 +03:00
Aliaksandr Valialkin	dac24aa342	app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries ... Previously `vector` time series could be unexpectedly returned from such queries	2022-04-21 15:34:14 +03:00
Aliaksandr Valialkin	bc54ae9608	app/vmselect/promql: fix comparison to nan ... The comparison to nan has been broken in d335cc886c Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150	2022-04-21 14:56:41 +03:00
Aliaksandr Valialkin	a5cfe5d13e	app/vmselect/promql: add drop_common_labels() function	2022-04-21 14:20:36 +03:00
Aliaksandr Valialkin	7ca08945fd	app/vmselect/promql: fix q default b where b may have empty time series	2022-04-21 11:43:09 +03:00
Aliaksandr Valialkin	25b841c6ed	app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values ... This should prevent from `duplicate time series` errors when executing the following query: kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1) where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series	2022-04-20 22:21:20 +03:00
Aliaksandr Valialkin	0ef7a05fc0	app/vmselect/promql: rename removeNaNs() to more clear removeEmptySeries()	2022-04-20 19:53:24 +03:00
Aliaksandr Valialkin	3605e1743b	app/vmselect/promql: allow calling InitRollupResultCache+StopRollupResultCache multiple times during tests	2022-04-11 12:32:23 +03:00
Aliaksandr Valialkin	b843f0e229	app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs	2022-03-26 11:28:14 +02:00
Aliaksandr Valialkin	99ee1c13c0	app/vmselect/promql: reduce the maximum number of label values, which can be propagated from one side of the binary operation to another side of the binary operation from 10K to 1K ... There are user reports that 10K unique values in a single label filter may lead to performance and memory usage issues	2022-02-24 04:03:28 +02:00
Aliaksandr Valialkin	244c23ea2c	lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes ... This should reduce memory usage under high time series churn rate	2022-02-23 13:42:27 +02:00
Aliaksandr Valialkin	989668beba	app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130	2022-02-12 15:46:43 +02:00
Aliaksandr Valialkin	1173964d8d	app/vmselect/promql: clarify comments on why the right side of if and and operators are executed at first	2022-02-03 00:26:42 +02:00
Aliaksandr Valialkin	10367d7e4c	app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values ... Such filters may slow down time series search, so just skip them. This is a follow-up for e7f1ceeb84 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827	2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin	13c2692ca1	app/vmselect/promql: properly handle foo or bar queries ... Such queries may miss `bar` results after the commit e7f1ceeb84 because common label filters from `foo` could be mistakenly applied to `bar`.	2022-02-01 17:43:10 +02:00
Aliaksandr Valialkin	02b2bfcff3	lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations ... Follow-up for f4989edd96 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007	2022-02-01 11:20:20 +02:00
Aliaksandr Valialkin	784cdcb080	app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first ... PromQL and MetricsQL operators are case-insensitive	2022-01-31 22:07:35 +02:00
Aliaksandr Valialkin	04d6596298	app/vmselect/promql: optimize queries, which join on _info metrics. ... Automatically add common filters from one side of binary operation to the other side before sending the query to storage subsystem. See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/ and https://www.robustperception.io/exposing-the-software-version-to-prometheus	2022-01-31 20:25:15 +02:00
Aliaksandr Valialkin	6232eaa938	lib/bytesutil: split Resize() into ResizeNoCopy() and ResizeWithCopy() functions ... Previously bytesutil.Resize() was copying the original byte slice contents to a newly allocated slice. This wasted CPU cycles and memory bandwidth in some places, where the original slice contents wasn't needed after slize resizing. Switch such places to bytesutil.ResizeNoCopy(). Rename the original bytesutil.Resize() function to bytesutil.ResizeWithCopy() for the sake of improved readability. Additionally, allocate new slice with `make()` instead of `append()`. This guarantees that the capacity of the allocated slice exactly matches the requested size. The `append()` could return a slice with bigger capacity as an optimization for further `append()` calls. This could result in excess memory usage when the returned byte slice was cached (for instance, in lib/blockcache). Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007	2022-01-25 15:28:42 +02:00
Aliaksandr Valialkin	cc76174387	app/vmselect/promql: properly keep metric names when optimized path is used for aggregate function calculations ... For example, `sum(rate(...) keep_metric_names) by (__name__)` didn't leave the original metric name because of this issue. This is a follup-up for 1bdc71d917 Udates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949	2022-01-17 15:33:39 +02:00
Aliaksandr Valialkin	5a1e4a140f	app/vmselect/promql: implement keep_metric_names modifier for transform and rollup functions ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949	2022-01-14 04:15:35 +02:00
Aliaksandr Valialkin	53c9a81f2b	app/vmselect/promql: add stale_samples_over_time() function	2022-01-14 01:49:24 +02:00
Aliaksandr Valialkin	0580a58feb	app/vmselect/promql: add support for @ modifier ... Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier Extend the support with the following features: * Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()` * Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)` returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348	2022-01-13 22:15:19 +02:00
Aliaksandr Valialkin	7d635a2311	app/vmselect/promql: fix limit_offset() test ... The test has been broken in addae7fc6a	2022-01-13 16:58:47 +02:00
Aliaksandr Valialkin	3396f08874	app/vmselect/promql: follow-up after 177e345d8a ... * Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions. * Simplify their implementation * Mention these functions in docs/CHANGELOG.md	2021-12-20 13:15:06 +02:00
匠心零度	cc4725843b	add Prometheus semantics function :changes_prometheus、delta_prometheus、increase_prometheus (#1972) ... Co-authored-by: lirenzuo <lirenzuo@shein.com>	2021-12-20 13:15:05 +02:00
Aliaksandr Valialkin	c170841951	app/vmselect/promql: add timestamp_with_name(m[d]) function ... This function works the same as `timestamp()`, but doesn't remove source time series names. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388	2021-12-15 23:38:09 +02:00
Aliaksandr Valialkin	addae7fc6a	app/vmselect/promql: preserve the order of time series passed to limit_offset() function ... Previously time series passed to `limit_offset()` were shuffled according to hash for their labels. This was unexpected behaviour for most users. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1920 and https://github.com/VictoriaMetrics/VictoriaMetrics/issues/951	2021-12-12 18:07:15 +02:00
Aliaksandr Valialkin	54d4a1c959	app/vmselect: accept optional extra_filters[] query args for all the supported Prometheus querying APIs ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863	2021-12-06 17:33:49 +02:00
Michael Fuller	8a94dd36d1	vmselect: in promql evaluation, return bytes requested when rollup memory limiter is unable to satisfy the request (#1838) ... Co-authored-by: Michael Fuller <mfuller@digitalocean.com>	2021-11-29 12:14:55 +02:00
Aliaksandr Valialkin	84decc4254	app/vmselect/promql: consistently return zero from deriv(const)	2021-11-17 18:01:58 +02:00
Aliaksandr Valialkin	aa6d11a33f	app/vmselect/promql: add now() function, which returns the current timestamp as a floating-point value in seconds	2021-11-17 16:42:11 +02:00
Aliaksandr Valialkin	1eb756fd11	app/vmselect/promql: arrange function names in the code in alphabetical order ... This should simplify code maintenance in the future	2021-11-14 13:54:43 +02:00
Aliaksandr Valialkin	82cf9f26db	app/vmselect/promql: prevent from incorrect calculations for deriv() over multiple samples with identical timestamps	2021-11-12 13:53:30 +02:00
Aliaksandr Valialkin	31486f2244	app/vmselect/promql: properly return durations smaller than one second from duration_over_time() function ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780	2021-11-09 11:42:43 +02:00
Aliaksandr Valialkin	3a1ac218ba	app/vmselect/promql: add duration_over_time(m[d], max_interval) function ... This function calculates the actual lifetime of the time series on the given lookbehdind window `d` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780	2021-11-08 13:14:57 +02:00
Aliaksandr Valialkin	a102fca4ac	app/vmselect/promql: add limit_offset(limit, offset, q) function, which can be used for paging over big number of time series ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1778	2021-11-03 16:03:10 +02:00
Aliaksandr Valialkin	43bf0333eb	app/vmselect/promql: add label_graphite_group() function for extracting groups from Graphite metric names	2021-11-03 13:20:34 +02:00
Aliaksandr Valialkin	99784b21c1	all: fix build issues and tests for Apple M1 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653	2021-10-27 15:07:19 +03:00
Aliaksandr Valialkin	d8ce86d685	app/vmselect/promql: reduce the precision from 15 significant digits to 13 significant digits when comparing float64 results in tests ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1738	2021-10-24 13:32:25 +03:00
Aliaksandr Valialkin	ed994873fd	app/vmselect/promql: randomize the static selection of time series returned from limitk() ... Sort series by a hash calculated from the series labels. This should guarantee "random" selection of the returned time series. Previously the selection could be biased, since time series were sorted alphabetically by label names and label values.	2021-10-16 21:16:34 +03:00
Aliaksandr Valialkin	da0adf446d	app/vmselect/promql: typo fix in comment: didsn't -> didn't	2021-10-16 13:00:10 +03:00
Aliaksandr Valialkin	48c3668b3d	app/vmselect/promql: add atan2 binary operator, which is going to be added in Prometheus 2.31 ... See https://github.com/prometheus/prometheus/pull/9248	2021-10-11 21:17:23 +03:00
Aliaksandr Valialkin	b3d28f3872	app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31 ... See https://github.com/prometheus/prometheus/issues/9233	2021-10-11 21:17:21 +03:00
Aliaksandr Valialkin	f1e1d20ac4	app/vmselect/promql: consistently return the same set of time series from limitk() function ... This is the expected behaviour by most users.	2021-10-08 19:55:29 +03:00
Aliaksandr Valialkin	cbf01f7384	app/vmselect/promql: substitute rollupFuncsCannotAdjustWindow with rollupFuncsCanAdjustWindow ... The list of functions, which can adjust lookbehind window is more limited than the rest of functions, so it is better from maintainability and readability PoV using the allowlist instead of blocklist.	2021-10-07 13:19:22 +03:00
Aliaksandr Valialkin	7e6ec0bbe9	app/vmselect/promql: return back the behaviour for deriv() function when the lookbehind window doesnt contain enough points ... It is expected that the `deriv(m[d])` returns non-empty value if the lookbehind window `d` contains less than 2 samples in the same way as `rate()` does. This is a follow-up after 3e084be06b .	2021-10-07 12:55:58 +03:00
Roman Khavronenko	0a8804d6aa	app/vmselect: make predict_linear and deriv compatible with Prometheus (#1681) ... Previously, `predict_linear` returned slightly different results comparing to Prometheus. The change makes linear regression algorithm compatible with Prometheus. `deriv` was excluded from the list of functions which can adjust the time window for the same reasons.	2021-10-07 12:55:56 +03:00
Aliaksandr Valialkin	ec6eb03d65	app/vmselect/promql: add topk_last and bottomk_last functions	2021-09-30 13:23:27 +03:00
Roman Khavronenko	95ddfda894	app/vmselect: fix binary comparison func (#1667) ... The fix makes the binary comparison func to check for NaNs before executing the actual comparison. This prevents VM to return values for non-existing samples for expressions which contain bool comparisons. Please see added test for example.	2021-09-30 13:04:10 +03:00
Roman Khavronenko	57eb5ef194	app/vmselect: fix testRowsEqual func NaN checks (#1666) ... It appeared, that `testRowsEqual` NaN comparison was incorrect. The fix caused some tests to fail. Please see the change and tests updated. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-30 13:04:08 +03:00
Roman Khavronenko	444a8882f4	app/vmselect: disable time-window adjustment for min/max_over_time funcs (#1658) ... Adjustment results into discrepancy between Prometheus and VM on time windows smaller than scrape interval. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:43:54 +03:00
Roman Khavronenko	f46f847531	app/vmselect: always return zero for stddev func if there is only one value (#1659) ... The fix will always return zero if received set of items consists of one element only, which also means no deviation. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:42:19 +03:00
Roman Khavronenko	72468c9d75	app/vmselect: return NaN instead of 0 for empty value sets (#1660) ... The change affects `count/stddev/stdvar_over_time` funcs and makes them to return NaN instead of zero when there is no datapoints in a time window. This is needed for improving compatibility with Prometheus. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:38:24 +03:00
Roman Khavronenko	c8ac8f44b2	app/vmselect: rm quantile_over_time fast-path optimisations (#1662) ... The removed fast path optimisations weren't consistent with `quantile` function behavior and results into discrepancy. Specifically, results didn't match in cases when: * 0 < phi > 1; * values contain only one element. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:35:59 +03:00
Aliaksandr Valialkin	4b3951fd86	app/vmselect/promql: add rollup_scrape_interval(m[d]) function ... It calculates the min, max and avg scrape intervals for m over the given lookbehind window d	2021-09-27 19:21:55 +03:00
Aliaksandr Valialkin	eff31c10ec	app/vmselect/promql: follow-up after 526dd93b32 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612	2021-09-27 18:57:03 +03:00
Roman Khavronenko	6061464d80	app/vmselect: quantile func compatiblity with Prometheus (#1646) ... * app/vmselect: `quantile` func compatiblity with Prometheus The `quantile` func was previously calculated by https://github.com/valyala/histogram package. The result of such calculation was always the closest real value to requested quantile. While in Prometheus implementation interpolation is used. Such difference may result into discrepancy in output between Prometheus and VictoriaMetrics. This commit adds a Prometheus-like `quantile` function. It also used by other functions which depend on it, such as `quantiles`, `quantile_over_time`, `median` etc. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625 Signed-off-by: hagen1778 <roman@victoriametrics.com> * app/vmselect: `quantile` review fixes * quantile functions were split into multiple to provide different API for already sorted data; * float64sPool is used for reducing allocations. Items in pool may have different sizes, but defining a new pool was complicates due to name collisions; Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-27 18:57:02 +03:00
Aliaksandr Valialkin	a746fe8778	app/vmselect/promql: follow-up after 57b3320478	2021-09-24 01:25:04 +03:00
Roman Khavronenko	7ac0559d4f	app/vmselect: make sorting for query result similar to Prometheus (#1647) ... * app/vmselect: make sorting for query result similar to Prometheus Updated sorting allows to get the order of series in result similar or equal to what Prometheus returns. The change is needed for compatibility reasons. * Update app/vmselect/promql/exec_test.go Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-24 01:25:03 +03:00
Aliaksandr Valialkin	1115c2f235	app/vmselect/promql: align the behavior of or, and and unless operators with on (labels) modifier to Prometheus ... Previously VictoriaMetrics could return unexpected result of the right-hand side operand had multiple time series with the given set of labels mentioned in `on(labels)`. See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1643	2021-09-24 00:56:35 +03:00
Aliaksandr Valialkin	76c650bbda	app/vmselect/promql: add quantiles_over_time("phiLabel", phi1, ..., phiN, m[d]) function for calculating multiple quantiles at once	2021-09-17 23:35:04 +03:00
Aliaksandr Valialkin	81d24045c2	app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function ... This function calculates multiple quantiles over the given buckets at once Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573	2021-09-17 13:39:06 +03:00
Aliaksandr Valialkin	4ed7548ced	app/vmselect/promql: optimize quantiles() calculation ... Calculate quantiles in one go instead of calculating each quantile individually	2021-09-17 12:38:21 +03:00
Aliaksandr Valialkin	3f8de4f82f	app/vmselect/promql: add mad(q) and outliers_mad(tolerance, q) functions to MetricsQL	2021-09-16 13:35:38 +03:00
Aliaksandr Valialkin	19516aedb5	app/vmselect/promql: use Prometheus-compatible label value formatting for count_values function	2021-09-13 13:48:18 +03:00
Aliaksandr Valialkin	9b07cb2988	app/vmselect/promql: keep metric name in rollup_candlestick results, since they don't change the original series meaning ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1600	2021-09-09 19:24:10 +03:00
Aliaksandr Valialkin	39bb6bdd79	app/vmselect/promql: add quantile("phiLabel", phi1, ..., phiN, q) aggregate function to MetricsQL ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573	2021-08-27 18:40:25 +03:00
Aliaksandr Valialkin	335de30083	app/vmselect/promql: make fmt after 0078486ea7	2021-08-23 23:05:34 +03:00
Aliaksandr Valialkin	40b06e84f8	app/vmselect/promql: rename sign() function to sgn() in order to be consistent with Prometheus ... See https://github.com/prometheus/prometheus/pull/8457 for details.	2021-08-23 11:46:29 +03:00
Aliaksandr Valialkin	91534057a3	app/vmselect/prometheus: do not extend [d] to the detected interval between samples for first_over_time(m[d]) ... This is for the sake of consistency with similar change for the last_over_time(m[d]) at a724229b5d	2021-08-21 19:56:56 +03:00