github-mirrors/VictoriaMetrics

mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-01 14:47:38 +00:00

Author	SHA1	Message	Date
Haleygo	f5a25ba980	vmselect: fix result in Prometheus query when time is small (#4578 ) vmselect: fix result in Prometheus query when time is small Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>	2023-08-12 13:32:46 -07:00
Aliaksandr Valialkin	1c18b37604	app/vmselect/promql: follow-up for `79e1c6a6fc` - Document the fix at docs/CHANGELOG.md - Add tests with multiple adjancent zero buckets - Simplify the fix a bit Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/296 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4021	2023-03-27 18:06:10 -07:00
Ze'ev Klapow	8f33561797	fix le buckets when adjacent vmrange is empty (#4021 ) There is a bug here where if you have a single bucket like: foo{vmrange="4.084e+02...4.642e+02"} 2 123 The expected output is three le encoded buckets like: foo{le="4.084e+02"} 0 123 foo{le="4.642e+02"} 2 123 foo{le="+Inf"} 2 123 This correctly encodes the start and end of the vmrange. If however, the input contains the previous bucket, and that bucket is empty then you only get the end le and +Inf out currently, i.e: foo{vmrange="7.743e+05...8.799e+05"} 5 123 foo{vmrange="6.813e+05...7.743e+05"} 0 123 results in: foo{le="8.799e+05"} 5 123 foo{le="+Inf"} 5 123 This causes issues when you go to compute a quantile because this means that the assumed lower bound of the buckets is 0 and this we interpolate between 0->end rather than the vmrange start->end as expected.	2023-03-27 18:06:08 -07:00
Aliaksandr Valialkin	511db8c453	app/vmselect/promql: typo fix after `e7f46a0aab`	2023-03-25 01:25:49 -07:00
Aliaksandr Valialkin	cee0fb8071	app/vmselect/promql: follow-up for `7205c79c5a` - Allocate and initialize seriesByWorkerID slice in a single go instead of initializing every item in the list separately. This should reduce CPU usage a bit. - Properly set anti-false sharing padding at timeseriesWithPadding structure - Document the change at docs/CHANGELOG.md Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3966	2023-03-25 01:25:08 -07:00
Zakhar Bessarab	71044221f6	app/vmselect/promql: use lock-less approach to gather results of parallel processing for `evalRollup` funcs (#4004 ) vmselect/promql: refactor `evalRollupNoIncrementalAggregate` to use lock-less approach for parallel workers computation Locking there is causing issues when running on highly multi-core system as it introduces lock contention during results merge. New implementation uses lock less approach to store results per workerID and merges final result in the end, this is expected to significantly reduce lock contention and CPU usage for systems with high number of cores. Related: #3966 Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> * vmselect/promql: add pooling for `timeseriesWithPadding` to reduce allocations Related: #3966 Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> * vmselect/promql: refactor `evalRollupFuncWithSubquery` to avoid using locks Uses same approach as `evalRollupNoIncrementalAggregate` to remove locking between workers and reduce lock contention. Related: #3966 Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> --------- Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>	2023-03-25 01:24:20 -07:00
Aliaksandr Valialkin	d8e218d99e	app/vmselect/promql: pass workerID to the callback inside doParallel() This opens the possibility to remove tssLock from evalRollupFuncWithSubquery() in the follow-up commit from @zekker6 in order to speed up the code for systems with many CPU cores. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3966	2023-03-25 01:23:06 -07:00
Aliaksandr Valialkin	9b5245a836	app/vmselect/promql: fix TestIncrementalAggr test on systems less than 3 CPU cores This is a follow-up for `4856a4cf5a`	2023-03-25 01:21:43 -07:00
Aliaksandr Valialkin	c780c6a280	app/vmselect: optimize incremental aggregates a bit Substitute sync.Map with an ordinary slice indexed by workerID. This should reduce the overhead when updating the incremental aggregate state	2023-03-24 23:50:31 -07:00
oliverpool	a3bc64e7ed	app/vmselect/promql: add test to ensure 8-byte alignment (#3948 ) See `0af9e2b693`	2023-03-24 23:49:04 -07:00
Aliaksandr Valialkin	1050d6bc0f	app/vmselect/promql: prevent from `cannot unmarshal timeseries from rollupResultCache` panic after the upgrade to v1.89.0 The issue has been introduced in `0af9e2b693`	2023-03-12 19:10:48 -07:00
Aliaksandr Valialkin	5a9300be13	app/vmselect: remove data race on updating EvalConfig.IsPartialResponse from concurrently running goroutines This properly returns `is_partial: true` for partial responses.	2023-03-12 16:57:28 -07:00
Aliaksandr Valialkin	c8f8aa85a2	app/vmselect/promql: prevent from SIGBUS crash on architecures, which deny unaligned access to 8-byte words (e.g. ARM) Thanks to @oliverpool for nailing down the root cause of the issue and for the initial attempt to fix it at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3927	2023-03-12 16:56:37 -07:00
Alexander Marshalov	808a06bfd4	fix interpolate function for filling only intermediate gaps (#3816 ) (#3857 ) * fix interpolate function for filling only intermediate gaps (#3816) * Update docs/CHANGELOG.md --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-23 19:30:23 -08:00
Aliaksandr Valialkin	6ff15ca135	app/vmselect: use consistent randomizer in tests Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3683	2023-01-23 19:27:40 -08:00
Aliaksandr Valialkin	e01f52d517	app/vmselect/promql: updates tests for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3664	2023-01-17 23:26:06 -08:00
Aliaksandr Valialkin	ce47faf102	app/vmselect/promql: reduce memory allocations when searching for time series pairs with identical labelsets in `q1 op q2` queries Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3641	2023-01-15 13:00:28 -08:00
Aliaksandr Valialkin	fe8802bbc8	app/vmselect/promql: reduce the number of memory allocations inside getCommonLabelFilters() This should improve performance a bit for `q1 op q2` queries	2023-01-15 12:56:21 -08:00
Aliaksandr Valialkin	d33a65e401	app/vmselect/promql: reduce memory allocations at getCommonLabelFilters() function Intern tag keys and values there	2023-01-12 01:27:34 -08:00
Aliaksandr Valialkin	8a35377cf3	app/vmselect/promql: move the `eval function args in parallel` query trace outside the loop	2023-01-10 22:23:43 -08:00
Aliaksandr Valialkin	c8bd3534cb	app/vmselect/promql: eliminate memory allocation when sorting values inside float64s	2023-01-09 23:06:57 -08:00
Aliaksandr Valialkin	7956b0d974	app/vmselect/promql: pre-allocate memory for values to be merged in mergeTimeseries() This should reduce the number of memory re-allocations	2023-01-09 22:52:19 -08:00
Aliaksandr Valialkin	895d5d9d22	app/vmselect/promql: consistently intern series names obtained from marshalMetricNameSorted This reduces memory allocations when the returned series names are used as map keys later	2023-01-09 22:46:30 -08:00
Aliaksandr Valialkin	12e2bcdf81	app/vmselect/promql: avoid memory allocations and copying from source timeseries to the returned result at timeseriesToResult()	2023-01-09 22:39:15 -08:00
Aliaksandr Valialkin	dd92e2050f	app/vmselect/promql: remove memory allocations from sortMetricTags()	2023-01-09 22:22:58 -08:00
Aliaksandr Valialkin	8050f5a18c	app/vmselect/promql: intern output series names inside timeseriesToResult() This reduces the number of memory allocations for repeated queries, which return (almost) the same set of time series.	2023-01-09 22:20:34 -08:00
Aliaksandr Valialkin	efb3c630fe	app/vmselect/promql: intern output series names during normal aggregation	2023-01-09 22:15:31 -08:00
Aliaksandr Valialkin	c0de651558	app/vmselect/promql: intern output series names during incremental aggregation This should reduce the number of memory allocations for repeated queries	2023-01-09 22:12:05 -08:00
Aliaksandr Valialkin	78114e85d6	vendor: update github.com/VictoriaMetrics/metricsql from v0.50.0 to v0.51.0 Updates https://github.com/VictoriaMetrics/metricsql/pull/7 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3589	2023-01-05 01:50:26 -08:00
Aliaksandr Valialkin	507e719c3f	app/vmselect/promql: add tests for `d3de110070`	2022-12-21 20:25:37 -08:00
Aliaksandr Valialkin	a77bb09e69	app/vmselect/promql: make sure that label_replace() doesn't create an empty dst_label if the src_label doesn't match regex	2022-12-21 20:25:37 -08:00
Aliaksandr Valialkin	7fb314bb59	app/vmselect/promql: do not extend too short lookbehind window for rate() function if it is set explicitly Previously too short lookbehind window d for rate(m[d]) could be automatically extended if it didn't cover at least two raw samples. This was needed in order to guarantee non-empty results from rate(m[d]) on short time ranges. Now the lookbehind window isn't extended if it is set explicitly, since it is expected that the user knows what he is doing. The lookbehind window continues to be extended when needed if it isn't set explicitly. For example, in the case of rate(m). Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3483	2022-12-20 00:18:32 -08:00
Aliaksandr Valialkin	808ce815e4	app/vmselect/promql: allow passing `inf` arg into functions, which accept numeric limit on the number of output time series Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3461	2022-12-10 22:48:01 -08:00
Aliaksandr Valialkin	c902500efc	app/vmselect/promql: add range_trim_spikes(phi, q) function for trimming phi percent of largest spikes per each time series returned by q	2022-12-05 21:55:24 -08:00
Aliaksandr Valialkin	ad548abd6a	app/vmselect/promql: add `range_normalize(q1, ..., qN)` function for normalizing query results into [0..1] value range This may be useful for analyzing correlation between time series with different value ranges	2022-11-21 23:25:35 +02:00
Aliaksandr Valialkin	2d361a834d	app/vmselect/promql: properly return an empty result from limit_offset() if offset exceeds the number of inner time series Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3312	2022-11-21 16:48:10 +02:00
Aliaksandr Valialkin	b0fefe562a	app/vmselect/promql: optimize `e1 op e2` when `e1` returns an empty result Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3349	2022-11-21 16:09:49 +02:00
Aliaksandr Valialkin	0cc59b9f95	app/vmselect/promql: add `range_stdvar()` and `range_stddev()` functions for calculating variance and deviation over time series on the selected time range	2022-11-17 01:03:02 +02:00
Aliaksandr Valialkin	facf7efa3a	app/vmselect/promql: add `range_linear_regression(q)` function for calculating simple linear regression for the selected time series on the selected time range	2022-11-17 00:39:17 +02:00
Aliaksandr Valialkin	ecb71a7221	lib/fs: add canOverwrite arg to WriteFileAtomically when it is allowed to overwrite the file atomically if it already exists	2022-10-26 01:08:35 +03:00
Aliaksandr Valialkin	c92aef39b5	app/vmselect/promql: expose missing metric `vm_cache_size_max_bytes{type="promql/rollupResult"}`	2022-10-23 12:14:07 +03:00
Aliaksandr Valialkin	9bf8d64605	app/vmselect/promql: follow-up for `930f1ee153` Document the change at docs/CHANGELOG.md Apply it to histogram_quantile() in the same way as to histogram_share() Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3225	2022-10-13 12:06:24 +03:00
Siqi Liu	c0fc716426	BUGFIX: properly calculate histogram_quantile with the same value and different string le (#3225 ) Co-authored-by: 647(siki.liu) <siki.liu@huolala.cn>	2022-10-13 12:06:24 +03:00
Aliaksandr Valialkin	a0f3247b14	app/vmselect/promql: properly handle zero and negative values for `-search.maxMemoryPerQuery` This is a follow-up for `04a05f161c` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203	2022-10-12 09:33:16 +03:00
Aliaksandr Valialkin	098c9bda27	app/vmselect: return back the logic for limits the amounts of memory occupied by concurrently executed queries if -search.maxMemoryPerQuery isn't set This is needed for preserving backwards compatibility with the previous releases of VictoriaMetrics. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203	2022-10-10 21:54:18 +03:00
Aliaksandr Valialkin	938ff7bba6	app/vmselect: allow limiting per-query memory usage via -search.maxMemoryPerQuery command-line flag Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203	2022-10-08 01:14:01 +03:00
Aliaksandr Valialkin	b05ff477c8	app/vmselect/promql: properly calculate `vm_rows_scanned_per_query` histogram for rollup functions, which take into account only a few samples on the provided lookbehind window	2022-10-06 23:23:17 +03:00
Aliaksandr Valialkin	baa11a778d	app/vmselect/promql: properly calculate `quantiles_over_time()` over a single raw sample	2022-10-06 22:37:38 +03:00
Aliaksandr Valialkin	5497997b72	app/vmselect/promql: increase scalability of incremental aggregate calculations on systems with many CPU cores Use sync.Map instead of a global mutex there. This should lift scalability limits on systems with many CPU cores.	2022-10-01 20:14:14 +03:00
Aliaksandr Valialkin	98175493d7	app/vmselect/promql: remove empty series before applying aggregate function Previously empty series (e.g. series with all NaN samples) were passed to aggregate functions. Such series must be ingored by all the aggregate functions. So it is better from consistency PoV filtering out empty series before applying aggregate functions.	2022-09-30 08:44:06 +03:00

1 2 3 4 5 ...

476 commits