86 commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	5f2905d120	app/vmselect: add sum_eq_over_time, sum_gt_over_time and sum_le_over_time functions to MetricsQL ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4641	2024-02-13 23:40:30 +02:00
Hui Wang	e086ef16da	app/vmselect/promql: properly handle possible negative results caused… (#5608) ... * app/vmselect/promql: properly handle possible negative results caused by float operations precision error in rollup functions like rate() or increase() * fix test	2024-01-22 01:04:50 +02:00
Noah Labrecque	fbb572a180	fix: apply correct bounds to sf and tf (#5274)	2023-11-14 01:19:47 +01:00
Aliaksandr Valialkin	744f8c3fe7	app/vmselect/promql: add outliers_iqr(q) and outlier_iqr_over_time(m[d]) functions ... These functions allow detecting anomalies in series and samples using Interquartile range method. See Outliers section at https://en.wikipedia.org/wiki/Interquartile_range for more details.	2023-10-31 22:14:14 +01:00
Aliaksandr Valialkin	d8afd7fe98	Makefile: update golangci-lint from v1.51.2 to v1.54.2 ... See https://github.com/golangci/golangci-lint/releases/tag/v1.54.2	2023-09-01 10:25:49 +02:00
Damon07	4c509c0b89	{app/vmselect,docs}: support share_eq_over_time#4441 (#4725) ... https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4441 Co-authored-by: wangm <wangmm@tuya.com>	2023-07-31 07:51:09 -07:00
Aliaksandr Valialkin	db7f237da9	app/vmselect/promql: add mad_over_time(m[d]) function ... See https://github.com/prometheus/prometheus/issues/5514	2023-02-11 01:06:39 -08:00
Aliaksandr Valialkin	0cc59b9f95	app/vmselect/promql: add range_stdvar() and range_stddev() functions for calculating variance and deviation over time series on the selected time range	2022-11-17 01:03:02 +02:00
Aliaksandr Valialkin	facf7efa3a	app/vmselect/promql: add range_linear_regression(q) function for calculating simple linear regression for the selected time series on the selected time range	2022-11-17 00:39:17 +02:00
Aliaksandr Valialkin	b05ff477c8	app/vmselect/promql: properly calculate vm_rows_scanned_per_query histogram for rollup functions, which take into account only a few samples on the provided lookbehind window	2022-10-06 23:23:17 +03:00
Roman Khavronenko	4acbf6e7d4	vmselect/rollup: rm workaround for slow-changing counters (#3163) ... The workaround was introduced to fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962. However, it didn't prove itself useful. Instead, it is recommended using `increase_pure` function. Removing the workaround makes VM to produce accurate results when calculating `delta` or `increase` functions over slow-changing counters with vary intervals between data points. Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-09-26 15:39:49 +03:00
Aliaksandr Valialkin	c6b74148cf	app/vmselect/promql: consistently calculate rate_over_sum(m[d]) as sum_over_time(m[d])/d ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3045	2022-09-02 23:19:05 +03:00
Dmytro Kozlov	d32a6359b0	vmselect/promql: enable search.maxPointsSubqueryPerTimeseries for sub-queries (#2963) ... * vmselect/promql: enable search.maxPointsPerTimeSeriesSubquery for sub-queries * vmselect/promql: cleanup * vmselect/promql: rename config flag * vmselect/promql: add tests * vmselect/promql: use test object instead of log * vmselect/promql: fix posible panic is subquery has more points. add description * vmselect/promql: update tests descriptions * vmselect/promql: update doInternal validation * vmselect/promql: fix linter * vmselect/promql: fix linter * vmselect/promql: update documentation and release notes * wip - Properly apply -search.maxPointsSubqueryPerTimeseries limit to subqueries. Previously the -search.maxPointsPerTimeseries limit was unexpectedly applied to subqueries if it was smaller than the -search.maxPointsSubqueryPerTimeseries . - Clarify docs for -search.maxPointsSubqueryPerTimeseries command-line flag . - Document -search.maxPointsPerTimeseries and -search.maxPointsSubqueryPerTimeseries flags at https://docs.victoriametrics.com/#resource-usage-limits . - Update docs/CHANGELOG.md . Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2922 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-08-24 15:27:41 +03:00
Aliaksandr Valialkin	fa08220d27	app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2787	2022-06-30 22:40:26 +03:00
Aliaksandr Valialkin	9adff0b686	app/vmselect/promql: show the number of scanned samples in the query trace	2022-06-28 19:32:03 +03:00
Aliaksandr Valialkin	6eb1580158	app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d	2022-04-27 10:58:40 +03:00
Aliaksandr Valialkin	53c9a81f2b	app/vmselect/promql: add stale_samples_over_time() function	2022-01-14 01:49:24 +02:00
Aliaksandr Valialkin	3396f08874	app/vmselect/promql: follow-up after 177e345d8a ... * Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions. * Simplify their implementation * Mention these functions in docs/CHANGELOG.md	2021-12-20 13:15:06 +02:00
Aliaksandr Valialkin	c170841951	app/vmselect/promql: add timestamp_with_name(m[d]) function ... This function works the same as `timestamp()`, but doesn't remove source time series names. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388	2021-12-15 23:38:09 +02:00
Aliaksandr Valialkin	31486f2244	app/vmselect/promql: properly return durations smaller than one second from duration_over_time() function ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780	2021-11-09 11:42:43 +02:00
Aliaksandr Valialkin	99784b21c1	all: fix build issues and tests for Apple M1 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653	2021-10-27 15:07:19 +03:00
Aliaksandr Valialkin	d8ce86d685	app/vmselect/promql: reduce the precision from 15 significant digits to 13 significant digits when comparing float64 results in tests ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1738	2021-10-24 13:32:25 +03:00
Aliaksandr Valialkin	7e6ec0bbe9	app/vmselect/promql: return back the behaviour for deriv() function when the lookbehind window doesnt contain enough points ... It is expected that the `deriv(m[d])` returns non-empty value if the lookbehind window `d` contains less than 2 samples in the same way as `rate()` does. This is a follow-up after 3e084be06b .	2021-10-07 12:55:58 +03:00
Roman Khavronenko	0a8804d6aa	app/vmselect: make predict_linear and deriv compatible with Prometheus (#1681) ... Previously, `predict_linear` returned slightly different results comparing to Prometheus. The change makes linear regression algorithm compatible with Prometheus. `deriv` was excluded from the list of functions which can adjust the time window for the same reasons.	2021-10-07 12:55:56 +03:00
Roman Khavronenko	57eb5ef194	app/vmselect: fix testRowsEqual func NaN checks (#1666) ... It appeared, that `testRowsEqual` NaN comparison was incorrect. The fix caused some tests to fail. Please see the change and tests updated. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-30 13:04:08 +03:00
Roman Khavronenko	6061464d80	app/vmselect: quantile func compatiblity with Prometheus (#1646) ... * app/vmselect: `quantile` func compatiblity with Prometheus The `quantile` func was previously calculated by https://github.com/valyala/histogram package. The result of such calculation was always the closest real value to requested quantile. While in Prometheus implementation interpolation is used. Such difference may result into discrepancy in output between Prometheus and VictoriaMetrics. This commit adds a Prometheus-like `quantile` function. It also used by other functions which depend on it, such as `quantiles`, `quantile_over_time`, `median` etc. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625 Signed-off-by: hagen1778 <roman@victoriametrics.com> * app/vmselect: `quantile` review fixes * quantile functions were split into multiple to provide different API for already sorted data; * float64sPool is used for reducing allocations. Items in pool may have different sizes, but defining a new pool was complicates due to name collisions; Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-27 18:57:02 +03:00
Aliaksandr Valialkin	76c650bbda	app/vmselect/promql: add quantiles_over_time("phiLabel", phi1, ..., phiN, m[d]) function for calculating multiple quantiles at once	2021-09-17 23:35:04 +03:00
Aliaksandr Valialkin	5dbe88a1c6	app/vmselect/promql: add increase_pure() function to MetricsQL ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962	2021-02-22 19:15:31 +02:00
Aliaksandr Valialkin	df6e399f73	app/vmselect/promql: add tfirst_over_time(m[d]) and tlast_over_time(m[d]) MetricsQL functions for returning timestamps for the first and the last samples in m over d	2021-01-12 16:12:47 +02:00
Aliaksandr Valialkin	8d1031c29a	app/vmselect/promql: return expected increase() result for the first point on the graph with value not exceeding 100	2020-12-15 14:10:50 +02:00
Aliaksandr Valialkin	bd8eef2528	app/vmselect/promql: do not reduce lookbehind window for any_rollup_func(m) to -search.maxStalenessInterval. It should equal to step value passed to /api/v1/query_range as most users expect	2020-12-08 15:17:05 +02:00
Aliaksandr Valialkin	d242c2f2bd	app/vmselect/promql: add count_eq_over_time(m[d], N) and count_ne_over_time(m[d], N) for calculating the number of samples in m over d that are equal / not equal to N	2020-12-05 12:31:01 +02:00
Aliaksandr Valialkin	8f42e59e05	app/vmselect/promql: remove spikes from increase() and delta() results on time series with spare irregular data points ... Do not take into account spare data point value if the next point will is located too far from the current point. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894	2020-11-13 15:23:37 +02:00
Aliaksandr Valialkin	da6d82a8dd	app/vmselect/promql: assume that time series value doesnt change during gaps when calculating increase() and delta() ... This should remove unexpected spikes at the end of gaps. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894	2020-11-13 14:59:32 +02:00
Aliaksandr Valialkin	217c192c88	app/vmselect/promql: improve time series staleness detection ... This should prevent from double counting for time series at the time when it changes label. The most common case is in K8S, which changes pod uid label with each new deployment. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748	2020-10-13 12:20:08 +03:00
Aliaksandr Valialkin	f877e703c8	app/vmselect/promql: fix mode_over_time calculations ... Previously `mode_over_time` could return garbage due to improper shuffling of input data points.	2020-10-13 11:58:30 +03:00
Aliaksandr Valialkin	b8bce348c5	app/vmselect/promql: properly limit implicitly set rollup window to -search.maxStalenessInterval ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/784	2020-09-23 23:24:09 +03:00
Aliaksandr Valialkin	4f06eed1c1	app/vmselect/promql: adjust integrate() calculations to be more similar to calculations from InfluxDB: attempt #2 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/701	2020-09-08 14:36:23 +03:00
Aliaksandr Valialkin	0d0b606455	app/vmselect/promql: adjust integrate() calculations to be more similar to calculations from InfluxDB ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/701	2020-09-08 14:24:02 +03:00
Aliaksandr Valialkin	db91045348	app/vmselect/promql: increase floating point calculations accuracy by dividing by 1e3 instead of multiplying by 1e-3	2020-09-08 14:01:02 +03:00
Aliaksandr Valialkin	478d8f8393	app/vmselect/promql: add count_le_over_time(m[d], le) and count_gt_over_time(m[d], gt) functions ... These functions returns the number of raw samples that don't exceed `le` or are bigger than `gt`. These functions are complement to already existing `share_le_over_time(m[d], le)` and `share_gt_over_time(m[d], gt)`.	2020-09-03 15:28:58 +03:00
Aliaksandr Valialkin	bdb881c43b	app/vmselect/promql: add zscore-related functions: zscore_over_time(m[d]) and zscore(q) by (...)	2020-08-03 21:52:15 +03:00
Aliaksandr Valialkin	338ee47d60	app/vmselect/promql: return non-empty value from rate_over_sum(m[d]) even if a single data point is located in the given [d] window ... Just divide the data point value by the window duration in this case.	2020-07-29 12:37:34 +03:00
Aliaksandr Valialkin	717c554fb0	app/vmselect/promql: remove rollupFuncArg.realPrevValue handling, since the corner case in increase() is handled in another way now ... See e00cfc854d for the approach used now.	2020-07-29 12:37:34 +03:00
Aliaksandr Valialkin	45334f61de	app/vmselect: fix tests for rate_over_sum	2020-07-24 02:35:09 +03:00
Aliaksandr Valialkin	8d1721d128	app/vmselect/promql: add rate_over_sum(m[d]) function to MetricsQL, which returns rate over sum of m values over d duration ... Something like `sum_over_time(m[d]) / d`, but more accurate.	2020-07-24 01:17:15 +03:00
Aliaksandr Valialkin	0755cb3b50	app/vmselect/promql: skip the first value in time series passed to increase() if it exceeds by more than 10x the delta between the next value and the first value ... This should prvent from inflated `increase()` results for time series that start from big initial values. Such cases may occur when a label value changes in a metric without counter reset.	2020-07-21 17:24:28 +03:00
Aliaksandr Valialkin	427fa43ce2	app/vmselect/promql: add mode_over_time(m[d]) function ... See https://en.wikipedia.org/wiki/Mode_(statistics) and https://stackoverflow.com/questions/61134078/promql-query-to-return-the-value-from-a-range-vector-which-occurs-maximum-no-of	2020-07-17 18:29:10 +03:00
Aliaksandr Valialkin	d83c68ca03	app/vmselect/promql: add ascent_over_time(m[d]) and descent_over_time(m[d]) functions ... These functions could be useful in GPS tracking apps for calculating the summary for height gain/loss over the given duration `d`.	2020-05-21 12:06:34 +03:00
Aliaksandr Valialkin	9ed4951ec8	lib/metricsql: move it to a separate repository - github.com/VictoriaMetrics/metrics	2020-04-28 15:30:06 +03:00