github-mirrors/VictoriaMetrics

mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00

Author	SHA1	Message	Date
Aliaksandr Valialkin	04a05f161c	app/vmselect: return back the logic for limits the amounts of memory occupied by concurrently executed queries if -search.maxMemoryPerQuery isn't set This is needed for preserving backwards compatibility with the previous releases of VictoriaMetrics. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203	2022-10-10 21:45:13 +03:00
Aliaksandr Valialkin	8e1ccecd97	docs: mention -search.maxMemoryPerQuery in the description to -search.maxConcurrentQueries command-line flag This is a follow-up for `5138eaeea0` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203	2022-10-09 13:56:59 +03:00
Aliaksandr Valialkin	5138eaeea0	app/vmselect: allow limiting per-query memory usage via -search.maxMemoryPerQuery command-line flag Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203	2022-10-08 01:08:05 +03:00
Yury Molodov	27ed4b853e	vmui: auto-update chart after query field removed (#3210 ) * feat: run query after query field removed * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-10-07 23:02:41 +03:00
Aliaksandr Valialkin	8322760647	app/vmselect/vmui: `make vmui-update` after `a54987f671`	2022-10-07 03:31:17 +03:00
Aliaksandr Valialkin	f452c84579	app/vmselect/promql: properly calculate `vm_rows_scanned_per_query` histogram for rollup functions, which take into account only a few samples on the provided lookbehind window	2022-10-06 23:22:24 +03:00
Aliaksandr Valialkin	40e899fd67	app/vmselect/promql: properly calculate `quantiles_over_time()` over a single raw sample	2022-10-06 22:37:21 +03:00
Aliaksandr Valialkin	d9282027e6	app: follow-up after `ec04fcac93` * Optimize fast path for /api/v1/import when importing numeric values * Move the docs about the change from features to bugfixes at docs/CHANGELOG.md * Update tests at lib/protoparser/vmimport Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3161	2022-10-06 14:52:02 +03:00
Dmytro Kozlov	ec04fcac93	Properly parse json when export import metric (#3180 ) * app/vmselect: properly work when export import json from `api/v1/{export, import}` API * app/vmselect: update convert function * app/vmselect: export null if `math.IsNaN(v)` * app/vmselect: get float from json * lib/protoparser: add test * docs: add change log * lib/protoparser: make export import api compatible	2022-10-06 13:54:20 +03:00
Aliaksandr Valialkin	c53b7e66ef	app/vmselect: improve performance scalability on multi-CPU systems for `/api/v1/export/...` endpoints	2022-10-01 22:05:43 +03:00
Aliaksandr Valialkin	49311ae977	app/vmselect/prometheus: improve scalability of `/federate` endpoint on systems with many CPU cores Minimize usage of global lock inside bufferedwriter.Write() when processing `/federate` data on systems with many CPU cores	2022-10-01 20:13:24 +03:00
Aliaksandr Valialkin	fb1cc3cc94	app/vmselect/promql: increase scalability of incremental aggregate calculations on systems with many CPU cores Use sync.Map instead of a global mutex there. This should lift scalability limits on systems with many CPU cores.	2022-10-01 20:00:03 +03:00
Aliaksandr Valialkin	fcc7ab71b3	app/vmselect: do not export NaN values for stale metrics at /federate endpoint Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3185	2022-10-01 19:47:37 +03:00
Aliaksandr Valialkin	725dfb0ed6	lib/httpserver: use 302 redirects instead of 301 redirects Incorrect 301 redirects can be cached by user agents such as web browsers. This can complicate recovery procedure after the incorrect redirect is fixed, e.g. web browser cache must be reset. The related issue - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1752	2022-10-01 16:53:35 +03:00
Aliaksandr Valialkin	b70f815dc4	app/vmselect/promql: remove empty series before applying aggregate function Previously empty series (e.g. series with all NaN samples) were passed to aggregate functions. Such series must be ingored by all the aggregate functions. So it is better from consistency PoV filtering out empty series before applying aggregate functions.	2022-09-30 08:39:54 +03:00
Roman Khavronenko	b64b9b9fec	app/vmselect: ignore empty series for `limit_offset` (#3178 ) * app/vmselect: ignore empty series for `limit_offset` VictoriaMetrics doesn't return empty series (with all NaN values) to the user. But such series are filtered after transform functions. It means `limit_offset` will account for empty series as well. For example, let's consider following data set: ``` time series: foo{label="1"} NaN, NaN, NaN, NaN // empty series foo{label="2"} 1, 2, 3, 4 foo{label="3"} 4, 3, 2, 1 ``` When user requests all series for metric `foo` the empty series will be filtered out: ``` /query=foo: foo{label="v2"} 1, 2, 3, 4 foo{label="v3"} 4, 3, 2, 1 ``` But `limit_offset(1, 1, foo)` is applied to original series, not filtered yet. So it will return `foo{label="v2"}` (skips the first in list) ``` /query=limit_offset(1, 1, foo): foo{label="v2"} 1, 2, 3, 4 ``` Expected result would be to apply `limit_offset` to already filtered list, so in result we receive `foo{label="v3"}`: ``` /query=limit_offset(1, 1, foo): foo{label="v3"} 4, 3, 2, 1 ``` The change does exactly that - filters empty series before applying `limit_offset`. Signed-off-by: hagen1778 <roman@victoriametrics.com> * app/vmselect: ignore empty series for `limit_offset` Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-09-30 08:20:34 +03:00
Roman Khavronenko	166d444159	vmselect/rollup: rm workaround for slow-changing counters (#3163 ) The workaround was introduced to fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962. However, it didn't prove itself useful. Instead, it is recommended using `increase_pure` function. Removing the workaround makes VM to produce accurate results when calculating `delta` or `increase` functions over slow-changing counters with vary intervals between data points. Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-09-26 15:33:25 +03:00
Dmytro Kozlov	b75f1854c5	vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric) (#2982 ) * vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric) * vmselect/promql: fix tests, add documentation * vmselect/promql: update test * vmselect/promql: update for alphanumeric sorting, fix tests * vmselect/promql: remove comments * vmselect/promql: cleanup * vmselect/promql: avoid memory allocations, update functions descriptions * vmselect/promql: make linter happy (remove ineffectual assigment) * vmselect/promql: add test case, fix behavior when strings are equal * vendor: update github.com/VictoriaMetrics/metricsql from v0.44.1 to v0.45.0 this adds support for sort_by_label_numeric and sort_by_label_numeric_desc functions * wip * lib/promscrape: read response body into memory in stream parsing mode before parsing it This reduces scrape duration for targets returning big responses. The response body was already read into memory in stream parsing mode before this change, so this commit shouldn't increase memory usage. * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-09-14 17:41:09 +03:00
Aliaksandr Valialkin	99bc18774c	app/vmselect/vmui: `make vmui-update` after `1304824201`	2022-09-14 13:33:06 +03:00
Yury Molodov	1b41169415	vmui: fix data processing (#3092 ) * fix: change data processing * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-09-12 10:55:11 +03:00
Yury Molodov	9541ef2e9e	vmui: add lists of top queries (#3065 ) * feat: add lists of top queries * fix: change the field label * refactor: add handlers for readability * app/vmselect: `make vmui-update` * docs: document `top queries` tab Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-09-08 21:43:37 +03:00
Dmytro Kozlov	4415c71a2b	vmselect/{promql, prometheus}: show flag names which user can update in error message (#3049 ) * vmselect/{promql, prometheus}: show flag names which user can update in error message * vmselect/{promql, prometheus}: fix typo	2022-09-06 13:25:59 +03:00
Aliaksandr Valialkin	ae31b2363f	app/vmselect/prometheus: follow-up after `50e2524bc2` - Add getCommonParamsWithDefaultDuration function and use it at /api/v1/series, /api/v1/labels and /api/v1/label/.../values - Document the default behaviour for setting 5 minutes time range if start arg isn't passed to /api/v1/series, /api/v1/labels and /api/v1/label/.../values - Document the change at docs/CHANGELOG.md Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3052	2022-09-05 11:55:48 +03:00
匠心零度	50e2524bc2	api prometheus/api/v1/label/../values time not specified, (#3052 ) modify default start values	2022-09-05 11:52:19 +03:00
Aliaksandr Valialkin	7dc632719d	app/vmselect/promql: consistently calculate `rate_over_sum(m[d])` as `sum_over_time(m[d])/d` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3045	2022-09-02 23:18:04 +03:00
Aliaksandr Valialkin	4076277cf0	app/vmselect/promql: evaluate `union()` args in parallel in order to increase query performance Note that the parallel execution of `union()` args may take more memory and CPU time than the sequential execution if args contain heavy queries, which may load all the available CPU, disk and memory resources and vmselect and vmstorage levels.	2022-09-02 19:46:27 +03:00
匠心零度	6d81584d2a	reduce unnecessary vmstorage query (#3031 ) * reduce unnecessary vmstorage query * reduce unnecessary vmstorage query * rollback limit logic /api/v1/label/*	2022-08-30 12:36:54 +03:00
Aliaksandr Valialkin	ad11b8d83d	app/vmselect/promql: follow-up after `2d71b4859c` - Use getScalar() function for obtaining the expected scalar from phi arg - Reduce the error message returned to the user when incorrect phi is passed to histogram_quantiles - Improve the description of this bugfix in the docs/CHANGELOG.md Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3026	2022-08-27 01:35:49 +03:00
Dmytro Kozlov	2d71b4859c	vmselect/promql: fix panic in histogram_quantiles function (#3029 ) * vmselect/promql: fix panic in histogram_quantiles function * Update docs/MetricsQL.md Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com> Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>	2022-08-27 01:33:56 +03:00
Dmytro Kozlov	463ea6897b	vmselect/promql: enable search.maxPointsSubqueryPerTimeseries for sub-queries (#2963 ) * vmselect/promql: enable search.maxPointsPerTimeSeriesSubquery for sub-queries * vmselect/promql: cleanup * vmselect/promql: rename config flag * vmselect/promql: add tests * vmselect/promql: use test object instead of log * vmselect/promql: fix posible panic is subquery has more points. add description * vmselect/promql: update tests descriptions * vmselect/promql: update doInternal validation * vmselect/promql: fix linter * vmselect/promql: fix linter * vmselect/promql: update documentation and release notes * wip - Properly apply -search.maxPointsSubqueryPerTimeseries limit to subqueries. Previously the -search.maxPointsPerTimeseries limit was unexpectedly applied to subqueries if it was smaller than the -search.maxPointsSubqueryPerTimeseries . - Clarify docs for -search.maxPointsSubqueryPerTimeseries command-line flag . - Document -search.maxPointsPerTimeseries and -search.maxPointsSubqueryPerTimeseries flags at https://docs.victoriametrics.com/#resource-usage-limits . - Update docs/CHANGELOG.md . Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2922 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-08-24 15:25:18 +03:00
Aliaksandr Valialkin	343241680b	all: remove the remaining bits of io/ioutil The io/ioutil package is deprecated since Go1.16 - see https://tip.golang.org/doc/go1.16#ioutil VictoriaMetrics requires at least Go1.18, so it is time to remove the io/ioutil from source code This is a follow-up for `02ca2342ab`	2022-08-22 00:20:58 +03:00
Aliaksandr Valialkin	9f94c295ab	all: use os.{Read\|Write}File instead of ioutil.{Read\|Write}File The ioutil.{Read\|Write}File is deprecated since Go1.16 - see https://tip.golang.org/doc/go1.16#ioutil VictoriaMetrics needs at least Go1.18, so it is safe to remove ioutil usage from source code. This is a follow-up for `02ca2342ab`	2022-08-21 23:52:35 +03:00
Aliaksandr Valialkin	4ac79d29ad	app/vmselect: follow-up after `63e0f16062` * Explicitly store a pointer to UserReadableError in the error interface. Previously Go automatically converted the value to a pointer before storing in the error interface. * Add Unwrap() method to UserReadableError, so it can be used transparently with the other code, which calls errors.Is() and errors.As(). * Document the change in docs/CHANGELOG.md	2022-08-15 13:50:16 +03:00
Roman Khavronenko	63e0f16062	vmselect: introduce UserReadableError type of error (#2894 ) When read query fails, VM returns rich error message with all the details. While these details might be useful for debugging specific cases, they're usually too verbose for users. Introducing a new error type `UserReadableError` is supposed to allow to return to user only the most important parts of the error trace. This supposed to improve error readability in web interfaces such as VMUI or Grafana. The full error trace is still logged with the full context and can be found in vmselect logs. Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-08-15 13:38:47 +03:00
Yury Molodov	1513866d51	vmui: shortcut keys legend (#2971 ) * feat: add shortcut modal * feat: add shortcut descriptions * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-08-15 11:54:47 +03:00
Yury Molodov	3cb013aeb8	vmui: graph action on moush hold and move (#2915 ) * fix: change event for graph panning * fix: change detect key * feat: add zoom in with mouse selection * - document the change - run `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-08-07 23:55:09 +03:00
Aliaksandr Valialkin	7478d423c5	app/vmselect/netstorage: cleanup after `92630c1ab4` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2896	2022-08-04 18:28:11 +03:00
Aliaksandr Valialkin	6b0550c023	app/{vmselect,vmalert}: properly generate http redirects if `-http.pathPrefix` command-line flag is set Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2918	2022-08-02 12:59:07 +03:00
Aliaksandr Valialkin	c2bd75926b	app/vmselect/netstorage: initialize tsw.rowsProcessed before calling tsw.f, since tsw.f can modify r.Timestamps and r.Values lengths	2022-07-30 00:39:36 +03:00
Aliaksandr Valialkin	19a0b4679a	app/vmselect/netstorage: re-use random generator used for series shuffle in Result.RunParallel This should reduce CPU usage needed for rand.Rand initialization	2022-07-30 00:30:37 +03:00
Dmytro Kozlov	a927814e7b	vmselect/promql: add tests for vmrangeBucketsToLE (#2907 ) * vmselect/promql: add tests for vmrangeBucketsToLE * vmselect/promql: cleanup * vmselect/promql: cleanup * vmselect/promql: fix panic tests want result * vmselect/promql: cleanup * vmselect/promql: update test name * vmselect/promql: fix linter error * vmselect/promql: refactor testcases * vmselect/promql: cleanup * vmselect/promql: remove unused reassign to workers, fix typo * wip * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-26 20:42:41 +03:00
Aliaksandr Valialkin	c888e6b9be	app/vmselect/promql: reduce the diff for `f148cffc8a` This is a follow-up for `c826f06366`	2022-07-26 19:20:48 +03:00
Alan Liang	c826f06366	vmselect: fix vmrangeBucketsToLE func may panic when ts value equal zero (#2902 ) Co-authored-by: alanwzliang <alanwzliang@tencent.com>	2022-07-25 10:55:13 +03:00
Aliaksandr Valialkin	92630c1ab4	app/vmselect/netstorage: improve the speed of queries over big number of time series on multi-CPU system Reduce inter-CPU communications when processing the query over big number of time series. This should improve performance for queries over big number of time series on systems with many CPU cores. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2896 Based on `b596ac3745` Thanks to @zqyzyq for the idea.	2022-07-25 09:18:44 +03:00
Aliaksandr Valialkin	79d967d35a	app/vmselect/vmui: `make vmui-update` after `edecd2493c`	2022-07-21 20:59:52 +03:00
Roman Khavronenko	9ccf695d57	vmselect: return correct error for second part of expression (#2893 ) Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-20 16:44:28 +02:00
Aliaksandr Valialkin	cc7d499bbd	app/vmselect/promql: execute `q1` and `q2` from `q1 op q2` in parallel if labels pushdown cannot be applied This should improve query performance if VictoriaMetrics has enough resources for processing `q1` and `q2` in parallel. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2886	2022-07-19 14:27:48 +03:00
Aliaksandr Valialkin	b5da47bfaf	app/vmselect/promql: properly return `q1` series from `q1 ifnot q2` when `q2` returns nothing	2022-07-18 14:24:54 +03:00
Aliaksandr Valialkin	0792c4ca90	app/vmselect/promql/transform.go: reuse evalNumber() function for constructing timezone_offset() results	2022-07-18 14:24:53 +03:00
Yury Molodov	17c33132df	vmui: optimize table view (#2867 ) * feat: optimize table view * fix: add column display setting * app/vmselect: `make vmui-update` Also document the change at docs/CHANGELOG.md Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-14 00:15:43 +03:00
Aliaksandr Valialkin	29e53b9f55	app/vmselect/promql: consistency update after `93fbd0c54b`	2022-07-13 12:33:14 +03:00
Roman Khavronenko	93fbd0c54b	promql: return `step` as scrapeInterval when it can't be calculated (#2865 ) The change allows to specify default value for `getScrapeInterval` function when actual interval can't be calculated. Before the change, function were returning `maxSilenceInterval` (5m) in such cases, which may be not correct for instant queries processing. The specific scenario where using `maxSilenceInterval` caused issues is the following: 1. Series becomes stale; 2. Client (in this case vmalert) continues to request series every 15s; 3. Database returns empty results as expected; 4. But at some specific moment of time database returns datapoints from `now()-5m`, because lookback window was extended to `maxSilenceInterval`. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-13 12:27:38 +03:00
Aliaksandr Valialkin	c2197ad139	app/vmselect/promql: validate function name before evaluating its arguments This avoids unneeded evaluation of args for unknown functions	2022-07-12 19:48:26 +03:00
Aliaksandr Valialkin	159c2e15e3	app/vmselect/netstorage: optimize mergeSortBlocks() for the worst case when blocks contain interleaved samples	2022-07-12 12:31:38 +03:00
Aliaksandr Valialkin	8429d4af5a	app/vmselect/netstorage: add mergeSortBlocks benchmark for the worstcase	2022-07-12 12:31:36 +03:00
Aliaksandr Valialkin	cad471037a	app/vmselect/prometheus: follow-up after `3efe33b917`	2022-07-11 20:35:28 +03:00
Dmytro Kozlov	3efe33b917	vmselect/prometeus: Add limit param to api/v1/series api endpoint (#2851 ) * issue-2841: Add limit param to api/v1/series api endpoint * issue-2841: add change log * issue-2841: update logic * issue-2841: simplify logic * issue-2841: simplify logic, add information to documentation	2022-07-11 20:18:30 +03:00
Aliaksandr Valialkin	ce68e76d62	app/vmselect: follow-up after `8667307d73`	2022-07-11 20:14:34 +03:00
Roman Khavronenko	8667307d73	vmselect: cover special cases for vmalert's routing in single-node version (#2845 ) * vmselect: cover special cases for vmalert's routing in single-node version * remove trailing `/` from requests * redirect to vmalert's home page when `/vmalert` is requested. Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: fix review comments Signed-off-by: hagen1778 <roman@victoriametrics.com> * Update app/vmselect/main.go Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-11 19:52:22 +03:00
Aliaksandr Valialkin	5c8eee26bf	all: `make fmt` via the upcoming Go1.19	2022-07-11 19:22:15 +03:00
Aliaksandr Valialkin	cd09f583fe	app/vmselect/netstorage: add benchmarks for mergeSortBlocks This is a follow-up for `743ff84863`	2022-07-11 12:54:48 +03:00
Aliaksandr Valialkin	743ff84863	app/vmselect/netstorage: optimize mergeSortBlocks function - Use binary search instead of linear scan when locating the run of smallest timestamps in blocks with intersected time ranges. This should improve performance when merging blocks with big number of samples - Skip samples with duplicate timestamps. This should increase query performance in cluster version of VictoriaMetrics with the enabled replication.	2022-07-09 00:34:42 +03:00
Roman Khavronenko	e1a41cfab5	metricsql: properly evaluate `timezone_offset` over time interval (#2842 ) * metricsql: properly evaluate `timezone_offset` over time interval https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771 Signed-off-by: hagen1778 <roman@victoriametrics.com> * Update docs/CHANGELOG.md Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-08 14:03:56 +03:00
Aliaksandr Valialkin	2e9ae40d56	app/vmselect/vmui: follow-up after `0bf6841140` * Document the bugfix in docs/CHANGELOG.md * Run `make vmui-update` for updating static js files for vmui, which are included into vmselect	2022-07-08 13:14:17 +03:00
Aliaksandr Valialkin	a6bd442ef9	app/vmui: tune visual presentation of trace view	2022-07-06 14:07:12 +03:00
Aliaksandr Valialkin	bd5b20445e	app/vmselect: add ability to query `vmselect` from another `vmselect`	2022-07-06 13:30:12 +03:00
Aliaksandr Valialkin	17de8a41c2	all: follow-up after `ed89106274`	2022-07-06 12:44:46 +03:00
Aliaksandr Valialkin	77cbbacfdb	lib/vmselectapi: pass storage.SearchQuery to API calls instead of []*storage.TagFilters + storage.TimeRange + maxMetrics This reduces the number of args to vmselectapi calls	2022-07-06 12:37:54 +03:00
Aliaksandr Valialkin	f435924ab3	lib/vmselectapi: pass maxSuffixes arg to tagValueSuffixes RPC call	2022-07-06 12:37:54 +03:00
Aliaksandr Valialkin	e1b8059086	lib/vmselectapi: rename deleteMetrics to more correct deleteSeries	2022-07-06 12:37:54 +03:00
Aliaksandr Valialkin	a60e03b3a7	lib/vmselectapi: use string type for tagKey and tagValuePrefix args at TagValueSuffixes() This improves the API consistency	2022-07-06 12:37:53 +03:00
Roman Khavronenko	ed89106274	vmselect: allow proxying requests to vmalert from single-node (#2834 ) The change allows to proxy requests with prefix `/vmalert` to the vmalert component if `-vmalert.proxyURL` is set. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825 and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2831 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-06 10:47:26 +02:00
Aliaksandr Valialkin	ae80cf76e0	app/vmselect: `make fmt` after `f3ece83e67`	2022-07-05 14:35:24 +03:00
Aliaksandr Valialkin	f3ece83e67	app/vmselect/promql: properly calculate histogram_quantile over unexpected `le` buckets See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819	2022-07-05 13:19:24 +03:00
Aliaksandr Valialkin	84e373e5c7	app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions	2022-06-30 22:39:38 +03:00
Aliaksandr Valialkin	3e2dd85f7d	all: readability improvements for query traces - show dates in human-readable format, e.g. 2022-05-07, instead of a numeric value - limit the maximum length of queries and filters shown in trace messages	2022-06-30 18:20:33 +03:00
Dmytro Kozlov	4d9715f5a8	vmui: update render logic for nested component (#2795 ) * vmui: update render logic for nested component, avoid rerender, remove local storage usage for tracing flag * docs/url-examples.md: fix various documentation issues there * docs: add Troubleshooting doc This doc contains troubleshooting guides for typical problems with VictoriaMetrics. * docs/Troubleshooting.md: add troubleshooting guide for cluster instability * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-30 15:47:12 +03:00
Aliaksandr Valialkin	a14188dd8e	app/vmselect: expose additional histograms at `/metrics` page, which may help get more insights for the query workload This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792	2022-06-28 20:18:13 +03:00
Aliaksandr Valialkin	a43f2d0bc5	app/vmselect/promql: show the number of scanned samples in the query trace	2022-06-28 19:26:17 +03:00
Aliaksandr Valialkin	a5181703b1	app/vmselect/prometheus: reduce the default value for -search.maxSeries from 100k to 30k Production experience shows that 100k is too big for /api/v1/series . It leads to increased CPU usage when Grafana queries /api/v1/series over VictoriaMetrics with big number of time series during auto-completion and when modifying template variables.	2022-06-28 18:22:30 +03:00
Aliaksandr Valialkin	a350d1e81c	lib/storage: return marshaled metric names from SearchMetricNames Previously SearchMetricNames was returning unmarshaled metric names. This wasn't great for vmstorage, which should spend additional CPU time for marshaling the metric names before sending them to vmselect. While at it, remove possible duplicate metric names, which could occur when multiple samples for new time series are ingested via concurrent requests. Also sort the metric names before returning them to the client. This simplifies debugging of the returned metric names across repeated requests to /api/v1/series	2022-06-28 18:17:15 +03:00
Aliaksandr Valialkin	e578549b8a	app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day	2022-06-28 13:02:47 +03:00
Aliaksandr Valialkin	a963b2a0aa	all: show timeRange in traces in human-readable format instead of timestamps in milliseconds	2022-06-27 13:45:51 +03:00
Aliaksandr Valialkin	ba514284f1	lib/storage: add querytracer to more contexts querytracer has been added to the following storage.Storage methods: - RegisterMetricNames - DeleteMetrics - SearchTagValueSuffixes - SearchGraphitePaths	2022-06-27 13:45:51 +03:00
Aliaksandr Valialkin	9a314106ca	app/vmselect/netstorage: remove Get prefix from netstorage functions This makes these function names more consistent with the server side	2022-06-27 00:45:05 +03:00
Dmytro Kozlov	bb7f31541f	vmui: added query tracing (#2748 ) * vmui: added query tracing * vmui: updated ui * vmui: update tracing logic, fix bugs, disable tracing by default * vmui: use empty message as props * vmui: fixed ui, added delete for each tacing data, show query in header * vmui: added timelines * vmui: speedup render * vmui: use memo for sorting * vmui: use Trace model, remove unused functions, simplify part of code * vmui: update recursive logic * vmui: fix set query to header * vmui: code cleanup, remove unused code * vmui: remove unused type, rename component * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-23 22:59:20 +03:00
Aliaksandr Valialkin	51362f9333	app/vmselect: add `-search.setLookbackToStep` command-line flag for making the gap filling algorithm similar to InfluxDB data model This option should override `-search.maxStalenessInterval` for most cases when users migrate from InfluxDB to VictoriaMetrics	2022-06-22 14:19:30 +03:00
Aliaksandr Valialkin	7bf75c7e61	app/vmselect: typo fix in the exported metric name: vm_http_request_total -> vm_http_requests_total	2022-06-22 13:15:31 +03:00
Aliaksandr Valialkin	e6ed92529b	all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package This package already has the same name, so there is no need in explicit name	2022-06-21 20:23:32 +03:00
Dmytro Kozlov	10454d1735	vmui: added focusLabel, enable cardinality app configuratior (#2736 ) * vmui: added focusLabel, enable app configuratior * vmui: set focusLabel if {labelName!=""} * wip * docs/CHANGELOG.md: mention about focusLabel feature in cardinality explorer Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2730 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-17 13:03:02 +03:00
Roman Khavronenko	723d90536c	vmselect: limit `end` param max value by 2d in future (#2729 ) * vmselect: limit `end` param max value by 2d in future The change is applied only to service handlers like `/labels` or `/series` and limits the `end` param by max value <= now() + 2 days. The same limit is applied for the ingested data, so no reason to allow to request data in future far than that. The change is also needed for corner cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2669 where too high `end` value triggers inefficient global index search. Signed-off-by: hagen1778 <roman@victoriametrics.com> * docs/CHANGELOG.md: document the bugfix Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-16 20:46:31 +03:00
Aliaksandr Valialkin	ec7963208d	app/vmselect: accept `focusLabel` query arg at /api/v1/status/tsdb This allows filling the seriesCountByFocusLabelValue list in the /api/v1/status/tsdb response with label values for the specified focusLabel, which contain the highest number of time series. TODO: add this to Cardinality explorer at VMUI - https://docs.victoriametrics.com/#cardinality-explorer	2022-06-14 18:36:54 +03:00
Aliaksandr Valialkin	b6c1ca12b7	lib/storage: show top labels with the highest number of series in cardinality explorer	2022-06-14 16:32:38 +03:00
Dmytro Kozlov	af3dc91a51	vmui: refactor Cardinality panel (#2726 ) * vmui: refactor Cardinality panel * vmui: change width of the search panel * vmui: code cleanup * vmui: code cleanup * vmui: fixed vulnerability (npm audit fix) * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-14 14:39:47 +03:00
Yury Molodov	879670418f	vmui: enhancements (#2638 ) (#2717 ) * feat: make datepicker to be set to last 30 min by default * fix: correct spinner while loading data * feat: change legend style * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-13 09:43:37 +03:00
Aliaksandr Valialkin	374beb350e	app/vmselect: optimize `/api/v1/labels` and `/api/v1/label/.../values` handlers when `match[]` query arg is passed to them	2022-06-12 04:32:13 +03:00
Aliaksandr Valialkin	89b778902b	app/vmselect: add optional `limit` query arg to `/api/v1/labels` and `/api/v1/label_values` endpoints This arg allows limiting the number of sample values returned from these APIs	2022-06-10 09:50:33 +03:00
Aliaksandr Valialkin	483b402bb2	app/vmselect/prometheus: extract common code for obtaining common query args into getCommonParams() function	2022-06-09 20:34:18 +03:00
Aliaksandr Valialkin	2bcb960f17	all: improve query tracing coverage for indexdb search Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403	2022-06-09 20:07:07 +03:00
Aliaksandr Valialkin	a30333a79e	app/vmselect/graphite: remove additional redundant Request.ParseForm() calls after `38c785b851`	2022-06-09 13:28:57 +03:00
Aliaksandr Valialkin	38c785b851	app/vmselect: remove redundant calls to Request.ParseForm() Request.ParseForm() is implicitly called by the first call to Request.FormValue()	2022-06-09 13:11:26 +03:00
Aliaksandr Valialkin	12ac255dae	lib/querytracer: make it easier to use by passing trace context message to New and NewChild The context message can be extended by calling Donef. If there is no need to extend the message, then just call Done.	2022-06-08 21:06:52 +03:00
Dmytro Kozlov	018d2303c4	Cardinality explorer (#2625 ) * Cardinality explorer * vmui, vmselect: updated field name, added description to spinner * make vmui-update * updated const name, make vmui-update * lib/storage: changes calculation for totalSeries values * added static files * wip * wip * wip * wip * docs/CHANGELOG.md: document cardinality explorer feature See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233 Co-authored-by: f41gh7 <nik@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-08 18:43:05 +03:00
Aliaksandr Valialkin	8608dd093c	all: follow-up after `8edb390e21` - Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb - Add /service-discovery page for API compatibility with Prometheus - Properly load bootstrap.min.css from /prometheus/targets - Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there	2022-06-07 00:57:09 +03:00
Dmytro Kozlov	8edb390e21	lib/promscrape: adds service discovery visualization for /targets page(#2675 ) * lib/promscrape: updated template * lib/promscrape: fixed click on unhealthy and all btns * app/vmselect: jquery scripts into static folder Co-authored-by: f41gh7 <nik@victoriametrics.com>	2022-06-03 15:38:45 +02:00
Aliaksandr Valialkin	41958ed5dd	all: add initial support for query tracing See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403	2022-06-01 02:29:23 +03:00
Dmytro Kozlov	11f91532c5	issue-2594: use embedded for static files (#2650 ) embed static js and css files from CDN into vmalert, vmagent and vmsingle binaries. Co-authored-by: f41gh7 <nik@victoriametrics.com> https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594	2022-05-31 01:55:28 +02:00
Aliaksandr Valialkin	69b9cf7161	app/vmselect/vmui: `make vmui-update` after `492a615a88`	2022-05-26 09:40:57 +03:00
Yury Molodov	492a615a88	vmui: import dashboards (#2642 ) * fix: switch dashboards import to fetch * make vmui-update	2022-05-26 09:39:10 +03:00
Aliaksandr Valialkin	c6d543e2f9	app/vmselect/vmui: `make vmui-update`	2022-05-20 14:53:38 +03:00
Yury Molodov	ff74472621	vmui: setup predefined dashboards without build (#2541 ) vmui: support predefined dashboards in json format See https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmselect/vmui/dashboards	2022-05-16 09:42:37 +02:00
Aliaksandr Valialkin	a7f18f8cb2	app/vmselect/promql: do not return values from `label_value()` if the original time series has no values at the selected timestamps	2022-05-09 17:57:39 +03:00
Aliaksandr Valialkin	ca4ca4630b	app/vmselect/vmui: `make vmui-update` after `450d879eaa`	2022-05-05 21:26:01 +03:00
Aliaksandr Valialkin	fc7c7237e3	app/vmselect: follow-up after `8639e79d38`	2022-05-04 23:35:57 +03:00
Nikolay	8639e79d38	app/vmselect: adds proxy for rules API (#2516 ) * app/vmselect: adds proxy for rules and alerts API It allows to visualization for rules at grafana https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1739 * Update app/vmselect/main.go Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com> Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>	2022-05-03 20:55:15 +03:00
Dima Lazerka	1e26dd1f82	Code cleanup (#343 ) * Small code cleanup: remove Request from params * Extract common params to all export handlers * Renamed ExportParams -> exportParams * wip Co-authored-by: Dzmitry Lazerka <dlazerka@gmail.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-03 16:04:16 +03:00
Aliaksandr Valialkin	bca4737fcf	app/vmui: fix up/down arrow keys on multi-line query after `a186434b50`	2022-05-03 10:50:16 +03:00
Aliaksandr Valialkin	a186434b50	app/vmui: execute query by pressing `enter` in the same way as Prometheus does Multi-line query can be entered via `shift-enter` in the query input field	2022-05-02 19:49:29 +03:00
Yury Molodov	87693754d5	vmui: support node v.18 (#2529 ) * fix: add support vmui node18 * fix: remove @mui/styles (legacy styling solution) * Update app/vmui/Dockerfile-build * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-02 18:41:53 +03:00
Aliaksandr Valialkin	7aa5167996	app/vmselect/main.go: move the code /api/v1/status/buildinfo handler to the same location as in the cluster branch	2022-04-29 13:02:14 +03:00
Aliaksandr Valialkin	464325a24b	app/vmselect/vmui: `make vmui-update` after `da04e9d1de`	2022-04-29 12:54:10 +03:00
Dima Lazerka	ed8e88af11	Export "null" in jsonl instead of NaN (#2518 ) * Export "null" in jsonl instead of NaN The NaN appeared because of staleness markers that were added for compatibility. I think it's better to use json `null`, implemented here. Also maybe it also makes sense to add a flag like `?skip-staleness-markers=true` to `/export`, to skip nulls at all? * Update app/vmselect/prometheus/export.qtpl * app/vmselect/prometheus/export.qtpl.go: `make quicktemplate-gen` * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-29 12:49:47 +03:00
Nikolay	c7aad8d441	app/vmselect: adds API /api/v1/status/buildinfo (#2515 ) * app/vmselect: adds API /api/v1/status/buildinfo it should fix an compability error with grafana 8.5 prometheus datasource https://github.com/grafana/grafana/pull/46771 * Update main.go	2022-04-29 11:36:28 +03:00
Aliaksandr Valialkin	cce1b6d7f9	app/vmselect/promql: add `tlast_change_over_time(m[d])` function, which returns the timestamp for the last change of `m` on the given lookbehind window `d`	2022-04-27 10:59:03 +03:00
Yury Molodov	c7693e8bc1	vmui: expression alias (#2495 ) * feat: add alias for queries * docs: update docs for predefined dashboards * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-26 15:59:37 +03:00
Yury Molodov	9b4bff67e0	vmui: add support relative time (#2504 ) * feat: add support relative time * app/vmselect: `make vmui-update` * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-26 15:46:06 +03:00
Aliaksandr Valialkin	25fe83577d	app/vmselect/promql: properly handle `scalar default vector`, `scalar if vector` and `scalar ifnot vector` queries Previously `vector` time series could be unexpectedly returned from such queries	2022-04-21 15:34:36 +03:00
Aliaksandr Valialkin	d1a9fac894	app/vmselect/promql: fix comparison to nan The comparison to nan has been broken in `d335cc886c` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150	2022-04-21 14:55:37 +03:00
Aliaksandr Valialkin	de892239a9	app/vmselect/promql: add `drop_common_labels()` function	2022-04-21 14:20:20 +03:00
Aliaksandr Valialkin	7a44ba1234	app/vmselect/promql: fix `q default b` where `b` may have empty time series	2022-04-21 11:42:42 +03:00
Aliaksandr Valialkin	d335cc886c	app/vmselect/promql: fix `duplicate time series` error on joins against time series filtered by values This should prevent from `duplicate time series` errors when executing the following query: kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending\|Running"}==1) where `kube_pod_status_phase{phase=~"Pending\|Running"}==1` filters out diplicate time series	2022-04-20 22:18:44 +03:00
Aliaksandr Valialkin	ed97908ca9	app/vmselect/promql: rename removeNaNs() to more clear removeEmptySeries()	2022-04-20 19:53:46 +03:00
Yury Molodov	514e3660e2	fix: prevent graph hiding without data (#2456 ) * fix: prevent graph hiding without data * fix: add yaxis labels default * app/vmselect: `make vmui-update` * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-16 17:15:31 +03:00
Aliaksandr Valialkin	ef66b048c9	app/vmui: further improvements for number display on graphs This is a follow-up for `c4d2cd8336` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2409	2022-04-12 16:01:27 +03:00
Aliaksandr Valialkin	7f83dc06c4	app/vmselect: `make vmui-update`	2022-04-12 14:35:19 +03:00
Aliaksandr Valialkin	61c7f6beae	app/vmselect/promql: allow calling InitRollupResultCache+StopRollupResultCache multiple times during tests Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2406	2022-04-11 12:34:43 +03:00
Aliaksandr Valialkin	19ecc4b2c3	app/vmselect: `make vmui-update`	2022-04-01 12:55:21 +03:00
Yury Molodov	f166f80f15	vmui: grid support for predefined panels (#2386 ) * update packages * feat: add setting width for predefined panels * docs: update doc by predefined dashboards * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-01 12:48:17 +03:00
Yury Molodov	c8d29ed78e	vmui: predefined panels (#2243 ) * feat: add basic components for predefined dashboards * fix: change display alert * feat: add autosize and unit for axes * feat: add component for CircularProgress * feat: change layout for predefined dashboards * feat: add override step for predefined panels * feat: add override step for predefined panels * feat: change yaxis limits for predefined panels * fix: rename flag for hide legend * feat: add formatted panel description * feat: add README.md for dashboard setup * feat: validate dashboard settings * feat: add unit for y-ticks * fix: correct display error for dashboards * fix: disable auto refresh after route change * update package-lock.json * fix: add basename for BrowserRouter * fix: add dynamic basename for routing * update packages * feat: add a pre-defined dashboard "per-job resource usage" * feat: display unit in the hover-tooltip * fix: change routing and home layout * fix: change axis width calc * updated packages * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-03-26 13:03:11 +02:00
Aliaksandr Valialkin	6e364e19ef	app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs	2022-03-26 11:29:49 +02:00
Aliaksandr Valialkin	e78f3ac8ac	app/vmselect/bufferedwriter: suppress trivial network errors, which can be generated by remote side These errors include `broken pipe` and `reset by peer`. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2334	2022-03-18 19:28:02 +02:00
Aliaksandr Valialkin	5d9b9b88b9	app/vmselect/vmui: `make vmui-update`	2022-03-18 13:18:26 +02:00
Yury Molodov	227d5182af	vmui: update packages (#2264 ) * update package.json * update package-lock.json * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-03-03 18:49:13 +02:00
Aliaksandr Valialkin	0d47c23a03	app/vmselect/promql: reduce the maximum number of label values, which can be propagated from one side of the binary operation to another side of the binary operation from 10K to 1K There are user reports that 10K unique values in a single label filter may lead to performance and memory usage issues	2022-02-24 04:05:18 +02:00
Aliaksandr Valialkin	62b46007c5	lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes This should reduce memory usage under high time series churn rate	2022-02-23 13:41:45 +02:00
Aliaksandr Valialkin	205d34eae6	app/vmselect: `make vmui-update`	2022-02-22 21:02:37 +02:00
Yury Molodov	5808774e06	vmui: change logic autocomplete (#2196 ) * fix: change autocomplete display logic * fix: change z-index for labels of input fields * fix: change autocomplete display logic * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-21 15:43:55 +02:00
Aliaksandr Valialkin	1d7c877b7b	app/vmui: small fixes * Remove unneeded dependency on `numeral` package * Properly parse numbers obtained from /api/v1/query_range according to https://prometheus.io/docs/prometheus/latest/querying/api/#expression-query-result-formats * Optimize updating processing the received data from /api/v1/query_range * Make smoother zoom on `ctrl+scroll` * Reduce the number of points received from /api/v1/query_range by 2x in order to reduce load on backend	2022-02-14 16:25:46 +02:00
Aliaksandr Valialkin	b1f94f7f0e	app/vmselect/promql: return at most one time series from `absent_over_time()` in the same way as Prometheus does See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130	2022-02-12 15:45:09 +02:00
Roman Khavronenko	e3adcbec6e	lib/promscrape: support prometheus-like duration in scrape configs (#2169 ) * lib/promscrape: support prometheus-like duration in scrape configs The change allows to specify duration values like `1d`, `1w` for fields `scrape_interval`, `scrape_timeout`, etc. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817#issuecomment-1033384766 Signed-off-by: hagen1778 <roman@victoriametrics.com> * lib/blockcache: make linter happy Signed-off-by: hagen1778 <roman@victoriametrics.com> * lib/promscrape: support prometheus-like duration in scrape configs * add support for extra fields `scrape_align_interval` and `scrape_offset`; * support Prometheus duration parsing for `__scrape_interval__` and `__scrape_duration__` labels; Signed-off-by: hagen1778 <roman@victoriametrics.com> * wip * wip * docs/CHANGELOG.md: document the feature Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-11 16:17:00 +02:00
Aliaksandr Valialkin	af77f449da	app/vmselect/vmui: `make vmui-update`	2022-02-08 17:26:50 +02:00
Aliaksandr Valialkin	96b7de6736	app/vmselect/promql: clarify comments on why the right side of `if` and `and` operators are executed at first	2022-02-03 00:26:14 +02:00
Aliaksandr Valialkin	4b850c2a59	app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values Such filters may slow down time series search, so just skip them. This is a follow-up for `e7f1ceeb84` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827	2022-02-02 23:40:02 +02:00
Aliaksandr Valialkin	2016a2c899	app/vmselect/promql: properly handle `foo or bar` queries Such queries may miss `bar` results after the commit `e7f1ceeb84` because common label filters from `foo` could be mistakenly applied to `bar`.	2022-02-01 17:40:51 +02:00
Aliaksandr Valialkin	4bdd10ab90	lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations Follow-up for `f4989edd96` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007	2022-02-01 00:18:42 +02:00
Aliaksandr Valialkin	a8d22e1223	app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first PromQL and MetricsQL operators are case-insensitive	2022-01-31 22:06:15 +02:00
Aliaksandr Valialkin	e7f1ceeb84	app/vmselect/promql: optimize queries, which join on `_info` metrics. Automatically add common filters from one side of binary operation to the other side before sending the query to storage subsystem. See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/ and https://www.robustperception.io/exposing-the-software-version-to-prometheus	2022-01-31 19:32:36 +02:00
Aliaksandr Valialkin	7edf8be3bc	app/vmselect/vmui: `make vmui-update`	2022-01-31 12:36:59 +02:00
Aliaksandr Valialkin	74a2297dcc	app/vmselect/vmui: `make vmui-update` after `e3995572bb`	2022-01-25 17:51:46 +02:00
Aliaksandr Valialkin	f4989edd96	lib/bytesutil: split Resize() into ResizeNoCopy() and ResizeWithCopy() functions Previously bytesutil.Resize() was copying the original byte slice contents to a newly allocated slice. This wasted CPU cycles and memory bandwidth in some places, where the original slice contents wasn't needed after slize resizing. Switch such places to bytesutil.ResizeNoCopy(). Rename the original bytesutil.Resize() function to bytesutil.ResizeWithCopy() for the sake of improved readability. Additionally, allocate new slice with `make()` instead of `append()`. This guarantees that the capacity of the allocated slice exactly matches the requested size. The `append()` could return a slice with bigger capacity as an optimization for further `append()` calls. This could result in excess memory usage when the returned byte slice was cached (for instance, in lib/blockcache). Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007	2022-01-25 15:24:44 +02:00
Aliaksandr Valialkin	285bb2bbec	app/vmselect/vmui: `make vmui-update`	2022-01-24 12:19:49 +02:00
Yury Molodov	ad5059f2d3	vmui: fixed display type switching (#2088 ) * fix: correct switch display type * docs/CHANGELOG.md: document the bugfix Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-21 16:56:22 +02:00
Yury Molodov	adbb821eac	vmui: fix time range selector (#2085 ) * fix: add date validate for time range * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document the bugfix Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-21 12:02:38 +02:00
Aliaksandr Valialkin	d79a915583	app/vmui: fix `last 6 months` time range picker Previously it was incorrectly selecting 6 minutes instead of 6 months. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1960	2022-01-18 23:28:10 +02:00
Aliaksandr Valialkin	672fcba223	app/vmui: properly calculate graph range for y axis Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2037	2022-01-18 22:22:24 +02:00
Aliaksandr Valialkin	5a77c86e97	app/vmui: reduce the refresh interval during graph scrolling/zooming from 1 second to 300 milliseconds One second feels too laggy, so let's reduce the refresh interval to 300 milliseconds. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064	2022-01-18 21:48:12 +02:00
Yury Molodov	8bdc45ba00	fix: remove buffer period (#2078 ) * fix: remove buffer period * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document the implemented feature Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-18 21:42:56 +02:00
Yury Molodov	70737ea4ac	vmui: correct url encoding (#2067 ) * fix: correct encode multi-line queries * fix: change autocomplete for correct arrows work * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document the bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2039 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-18 21:31:46 +02:00
Yury Molodov	8e3f9c1fbb	vmui: correct calc axes limits (#2058 ) * fix: correct calc axes limits * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-18 15:43:41 +02:00
Yury Molodov	fcd33fc409	vmui: change layout (#2054 ) * fix: change query reset * feat: replace @codemirror to text field * feat: switch to Preact from React * fix: optimize mui imports * feat: move time selector to Header * checkout * fix: remove unused vars * update package-lock.json * fix: correct styles * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-18 12:44:22 +02:00
Aliaksandr Valialkin	dc7b63a793	app/vmselect/promql: properly keep metric names when optimized path is used for aggregate function calculations For example, `sum(rate(...) keep_metric_names) by (__name__)` didn't leave the original metric name because of this issue. This is a follup-up for `1bdc71d917` Udates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949	2022-01-17 15:30:30 +02:00
Aliaksandr Valialkin	1bdc71d917	app/vmselect/promql: implement `keep_metric_names` modifier for transform and rollup functions Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949	2022-01-14 04:14:59 +02:00
Aliaksandr Valialkin	f41846d002	app/vmselect/promql: add `stale_samples_over_time()` function	2022-01-14 01:48:04 +02:00
Aliaksandr Valialkin	c883c15878	app/vmselect/promql: add support for `@` modifier Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier Extend the support with the following features: * Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()` * Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)` returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348	2022-01-13 22:12:06 +02:00
Aliaksandr Valialkin	9469696e46	app/vmselect/promql: fix limit_offset() test The test has been broken in `addae7fc6a`	2022-01-13 16:59:28 +02:00
Yury Molodov	7d5ed49d23	vmui: switching to Preact (#2053 ) * feat: replace @codemirror to text field * feat: switch to Preact from React * fix: optimize mui imports * fix: remove unused vars * update package-lock.json * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-11 10:32:17 +02:00
Aliaksandr Valialkin	ceade70d4e	app/vmselect/vmui: run `make vmui-update` after `89ff7b2465`	2022-01-03 21:03:37 +02:00
Yury Molodov	681a800086	vmui: legend fixes (#1995 ) * feat: add a reset query by clicking the logo * feat: add sequence number for query fields * feat: invert behavior on the graph's legend * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-23 12:14:16 +02:00
Roman Khavronenko	543bd0ea0c	vmselect: update /query_exemplars placeholder (#2000 ) Grafana expects `data` in response to be a slice and logs an err if it is not: ``` err="[]v1.ExemplarQueryResult: decode slice: expect [ or n, but found , error found in #0 byte of ...\|\|..., bigger context ...\|\|..." ``` https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1999 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-12-23 11:53:50 +02:00
Yury Molodov	4b40acd964	vmui: add custom start range (#1989 ) * feat: add custom start range * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-21 20:19:33 +02:00
Yury Molodov	718c352946	vmui: graph fixes (#1982 ) * fix: remove disabling custom step when zooming * feat: add a dynamic calc of the width of the graph * fix: add validate y-axis limits * fix: correct axis limits for value 0 * fix: change logic create time series * fix: change types for tooltip * fix: correct points on the line * fix: change the logic for set graph width * fix: stop checking the period when auto-refresh is enabled * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-20 17:37:02 +02:00
Aliaksandr Valialkin	6d77cc9b08	app/vmselect/vmui: `make vmui-update`	2021-12-20 13:51:02 +02:00
Aliaksandr Valialkin	974d9c0eee	app/vmselect/promql: follow-up after `177e345d8a` * Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions. * Simplify their implementation * Mention these functions in docs/CHANGELOG.md	2021-12-20 13:19:44 +02:00
匠心零度	177e345d8a	add Prometheus semantics function :changes_prometheus、delta_prometheus、increase_prometheus (#1972 ) Co-authored-by: lirenzuo <lirenzuo@shein.com>	2021-12-20 12:32:43 +02:00
Aliaksandr Valialkin	e6d4641bf0	app/vmselect/vmui: `make vmui-update`	2021-12-17 11:01:09 +02:00
Aliaksandr Valialkin	193331d522	app/vmselect: de-duplicate data exported via `/api/v1/export/csv` by default Previously the exported data wasn't de-duplicated. Now it is possible to export the raw data without deduplication by passing reduce_mem_usage=1 query arg to /api/v1/export/csv See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837	2021-12-17 10:57:39 +02:00
Aliaksandr Valialkin	5efe377a26	app/vmselect/promql: add `timestamp_with_name(m[d])` function This function works the same as `timestamp()`, but doesn't remove source time series names. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388	2021-12-15 23:37:07 +02:00
Yury Molodov	9baad51004	vmui: introduce application mode (#1949 ) * feat: add a label for the Query field * fix: change zoom position * fix: add description and error code to alerts * fix: correct logic query history * fix: correct update query history * feat: add custom step * update package-lock.json * feat: introduce application mode * build vmui * Revert "build vmui" This reverts commit `c0e2415550`. * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-15 21:33:25 +02:00
Aliaksandr Valialkin	92070cbb67	lib/storage: return dedup interval in milliseconds from GetDedupInterval() This removes duplicate .Milliseconds() calls after GetDedupInterval() calls.	2021-12-15 13:26:38 +02:00
Aliaksandr Valialkin	1d20a19c7d	lib/storage: explicitly pass dedupInterval to DeduplicateSamples() and deduplicateSamplesDuringMerge() This improves the code readability and debuggability, since the output of these functions stops depending on global state.	2021-12-14 20:49:12 +02:00
Yury Molodov	49e6a921df	vmui: custom step (#1942 ) * feat: add a label for the Query field * fix: change zoom position * fix: add description and error code to alerts * fix: correct logic query history * fix: correct update query history * feat: add custom step * update package-lock.json * docs: document that VMUI now supports overriding of `step` query arg, which is passed to `/api/v1/query_range` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-14 14:51:45 +02:00
Yury Molodov	b5b701d590	vmui: minor fixes (#1936 ) * feat: add a label for the Query field * fix: change zoom position * fix: add description and error code to alerts * fix: correct logic query history * fix: correct update query history * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document bugfixes Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-13 13:42:37 +02:00
Aliaksandr Valialkin	ce80a0ce5e	app/vmselect/vmui: `make vmui-update`	2021-12-13 13:32:54 +02:00
Yury Molodov	7c3b6365f0	vmui: add a label for the Query field (#1923 ) * feat: add a label for the Query field * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-12 19:06:39 +02:00
Aliaksandr Valialkin	d1f8915ed1	app/vmselect/promql: preserve the order of time series passed to `limit_offset()` function Previously time series passed to `limit_offset()` were shuffled according to hash for their labels. This was unexpected behaviour for most users. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1920 and https://github.com/VictoriaMetrics/VictoriaMetrics/issues/951	2021-12-12 18:04:58 +02:00
Yury Molodov	c1fd93e8a0	vmui: multiple queries (#1916 ) * feat: change duration by "enter" * fix: optimize data processing for chart * feat: set minimum step to 1ms * update dependencies * feat: remove save the last query to local storage * fix: handle an error in a table with subqueries * feat: store display type in URL * Revert "feat: store display type in URL" This reverts commit `ccc242c69a`. * feat: store display type in URL * refactor: move the time setting to a folder * refactor: move the query configurator to a folder * refactor: move the auth settings to a folder * feat: improve styles * feat: add multi query * update package-lock * feat: add display multiple queries * feat: add limits for multiple queries * update dependencies * feat: add history for multiple queries * feat: add line type to legend * feat: change style for switch * feat: change the logic for axes limits for multiple queries * update package-lock.json * update dependencies * feat: add the filter to legend * wip * lib/httpserver: add missing 127.0.0.1 hostname to the logged address for http and pprof server if the address starts with ':' This allows copy-pasting the url to http server from logs. * lib/httpserver: add missing 127.0.0.1 hostname to the logged address for http and pprof server if the address starts with ':' This allows copy-pasting the url to http server from logs. Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-08 16:40:15 +02:00
Aliaksandr Valialkin	ff15a752c1	app/vmselect: accept optional `extra_filters[]` query args for all the supported Prometheus querying APIs See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863	2021-12-06 17:07:09 +02:00
Aliaksandr Valialkin	732a0cd3e1	app/vmselect/vmui: `make vmui-update`	2021-12-06 10:19:09 +02:00
Aliaksandr Valialkin	f67427ae61	app/vmselect/vmui: `make vmui-update`	2021-11-30 01:38:24 +02:00
Yury Molodov	1b58d126c0	vmui: optimize render (#1852 ) * feat: change duration by "enter" * fix: optimize data processing for chart * feat: set minimum step to 1ms * update dependencies * update package-lock * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-11-30 00:56:48 +02:00
Aliaksandr Valialkin	91243ad5cd	app/vmselect/vmui: `make vmui-update`	2021-11-29 21:55:08 +02:00
Aliaksandr Valialkin	ce31e837eb	app/vmselect/vmui: `make vmui-update`	2021-11-29 12:22:59 +02:00
Michael Fuller	cf8c171f85	vmselect: in promql evaluation, return bytes requested when rollup memory limiter is unable to satisfy the request (#1838 ) Co-authored-by: Michael Fuller <mfuller@digitalocean.com>	2021-11-22 13:20:42 +03:00
Aliaksandr Valialkin	9bee043ff2	app/vmselect/promql: consistently return zero from `deriv(const)`	2021-11-17 18:02:05 +02:00
Aliaksandr Valialkin	b900560b83	app/vmselect/promql: add `now()` function, which returns the current timestamp as a floating-point value in seconds	2021-11-17 16:35:30 +02:00
Aliaksandr Valialkin	f43586c63c	app/vmselect/promql: arrange function names in the code in alphabetical order This should simplify code maintenance in the future	2021-11-14 13:55:06 +02:00
Aliaksandr Valialkin	9fa098d8e3	app/vmselect/promql: prevent from incorrect calculations for `deriv()` over multiple samples with identical timestamps	2021-11-12 13:50:43 +02:00
Aliaksandr Valialkin	e5ac9d8e57	all: consistently return `application/json` content-type without `charset=utf-8` The `application/json` content-type has utf-8 encoding by default. See https://stackoverflow.com/questions/9254891/what-does-content-type-application-json-charset-utf-8-really-mean Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897	2021-11-09 18:04:44 +02:00
Aliaksandr Valialkin	840ac283ef	app/vmselect/promql: properly return durations smaller than one second from duration_over_time() function Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780	2021-11-09 11:41:56 +02:00
Yury Molodov	b78ab88a1c	vmui: migration MUI Core v4 to v5 (#1795 ) * migration MUI Core v4 to v5 * app/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-11-08 15:45:59 +02:00
Aliaksandr Valialkin	3419ac1d36	app/vmselect/promql: add `duration_over_time(m[d], max_interval)` function This function calculates the actual lifetime of the time series on the given lookbehdind window `d` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780	2021-11-08 13:14:09 +02:00
Yury Molodov	6d1d558c4f	vmui: fix graph reset (#1788 ) * feat: add query history * fix: change detect keyUp for nav query history * feat: set default query history * feat: change graph legend * update dependencies * update codemirror version * fix: correct update period time after zoom/pan * fix: optimize data processing for the graph * fix: eliminate memory leaks related to mouse events * fix: correct display of straight line * Merge branch 'master' into vmui-fix-reset-graph * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-11-05 20:56:57 +02:00
Aliaksandr Valialkin	2178335618	app/vmselect: `make vmui-update`	2021-11-04 12:13:12 +02:00
Aliaksandr Valialkin	27044b84d2	app/vmselect/promql: add `limit_offset(limit, offset, q)` function, which can be used for paging over big number of time series Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1778	2021-11-03 16:02:27 +02:00
Aliaksandr Valialkin	43a58bd618	app/vmselect/promql: add `label_graphite_group()` function for extracting groups from Graphite metric names	2021-11-03 13:19:08 +02:00
Yury Molodov	47ee3744f2	vmui: correct migration material-ui (#1758 ) * migration material-ui * fix: rollback popover * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-27 20:41:26 +03:00
Aliaksandr Valialkin	1952ab99aa	all: fix build issues and tests for Apple M1 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653	2021-10-27 15:06:34 +03:00
Yury Molodov	956cf83e7b	vmui: update dependencies (#1754 ) * update dependencies * update codemirror version * app/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-26 19:31:20 +03:00
Aliaksandr Valialkin	df8f967040	app/vmselect/promql: reduce the precision from 15 significant digits to 13 significant digits when comparing float64 results in tests See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1738	2021-10-24 13:31:14 +03:00
Yury Molodov	2b266cb87e	vmui: query history (#1732 ) * feat: add query history * fix: change detect keyUp for nav query history * feat: set default query history * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-22 12:21:22 +03:00
Yury Molodov	a3e09a57c2	vmui: features (#1711 ) * feat: initial uPlot graph * feat: add zoom/pan for graph * fix: add zoom by ctrl/mac * fix: remove unused code * feat: add toggle cache for fetch * feat: add fix y-axis limits * fix: stop point events while panning * fix: change getting cursor position when scaling * feat: add cursor tooltip to graph * fix: uninstall chart.js * fix: change link for create an issue * fix: set default cache value to true * app/vmalert: follow-up after `0e2486df56` * docs/CHANGELOG.md: document `5416e18007` * app/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-18 15:16:57 +03:00
Aliaksandr Valialkin	da97e58979	app/vmselect/promql: randomize the static selection of time series returned from `limitk()` Sort series by a hash calculated from the series labels. This should guarantee "random" selection of the returned time series. Previously the selection could be biased, since time series were sorted alphabetically by label names and label values.	2021-10-16 21:16:49 +03:00
Aliaksandr Valialkin	cae174b11c	app/vmselect/promql: typo fix in comment: didsn't -> didn't	2021-10-16 13:00:34 +03:00
Aliaksandr Valialkin	a5001b9c20	app/vmselect/promql: add `atan2` binary operator, which is going to be added in Prometheus 2.31 See https://github.com/prometheus/prometheus/pull/9248	2021-10-11 21:15:53 +03:00
Aliaksandr Valialkin	81c6720392	app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31 See https://github.com/prometheus/prometheus/issues/9233	2021-10-11 21:01:33 +03:00
Aliaksandr Valialkin	92b92d4d2c	app/vmselect/promql: consistently return the same set of time series from `limitk()` function This is the expected behaviour by most users.	2021-10-08 19:53:52 +03:00
Aliaksandr Valialkin	0ff8fcac6a	app/vmui: follow-up after `7bfb44113e` * Run `vmui-update` * Document the changes in README.md and CHANGELOG.md	2021-10-08 15:09:29 +03:00
Aliaksandr Valialkin	2748255c8b	app/vmselect/promql: substitute rollupFuncsCannotAdjustWindow with rollupFuncsCanAdjustWindow The list of functions, which can adjust lookbehind window is more limited than the rest of functions, so it is better from maintainability and readability PoV using the allowlist instead of blocklist.	2021-10-07 13:18:42 +03:00
Aliaksandr Valialkin	c45210a6f9	app/vmselect/promql: return back the behaviour for `deriv()` function when the lookbehind window doesnt contain enough points It is expected that the `deriv(m[d])` returns non-empty value if the lookbehind window `d` contains less than 2 samples in the same way as `rate()` does. This is a follow-up after `3e084be06b` .	2021-10-07 12:52:27 +03:00
Roman Khavronenko	3e084be06b	app/vmselect: make `predict_linear` and `deriv` compatible with Prometheus (#1681 ) Previously, `predict_linear` returned slightly different results comparing to Prometheus. The change makes linear regression algorithm compatible with Prometheus. `deriv` was excluded from the list of functions which can adjust the time window for the same reasons.	2021-10-07 12:50:49 +03:00
Aliaksandr Valialkin	0e3de5a0cc	app/vmselect/promql: add `topk_last` and `bottomk_last` functions	2021-09-30 13:22:52 +03:00
Roman Khavronenko	a31407006c	app/vmselect: fix binary comparison func (#1667 ) The fix makes the binary comparison func to check for NaNs before executing the actual comparison. This prevents VM to return values for non-existing samples for expressions which contain bool comparisons. Please see added test for example.	2021-09-30 12:24:17 +03:00
Roman Khavronenko	344490d89b	app/vmselect: fix `testRowsEqual` func NaN checks (#1666 ) It appeared, that `testRowsEqual` NaN comparison was incorrect. The fix caused some tests to fail. Please see the change and tests updated. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-30 12:08:47 +03:00
Yury Molodov	893af0a92c	vmui: fixed bug with time range (time zone) (#1661 ) * fix: set date in query string in utc format * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-29 01:00:44 +03:00
Roman Khavronenko	5dc84bf210	app/vmselect: disable time-window adjustment for `min/max_over_time` funcs (#1658 ) Adjustment results into discrepancy between Prometheus and VM on time windows smaller than scrape interval. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:43:21 +03:00
Roman Khavronenko	de810031bf	app/vmselect: always return zero for `stddev` func if there is only one value (#1659 ) The fix will always return zero if received set of items consists of one element only, which also means no deviation. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:38:55 +03:00
Roman Khavronenko	dd536b475c	app/vmselect: return NaN instead of 0 for empty value sets (#1660 ) The change affects `count/stddev/stdvar_over_time` funcs and makes them to return NaN instead of zero when there is no datapoints in a time window. This is needed for improving compatibility with Prometheus. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:37:04 +03:00
Roman Khavronenko	03cd93bf1a	app/vmselect: rm `quantile_over_time` fast-path optimisations (#1662 ) The removed fast path optimisations weren't consistent with `quantile` function behavior and results into discrepancy. Specifically, results didn't match in cases when: * 0 < phi > 1; * values contain only one element. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:35:14 +03:00
Yury Molodov	a64155d91e	vmui: use Chart.js as default engine for graph (#1634 ) * feat: add Plotly as default engine for graph * fix: remove unused components * feat: use Chart.js as default engine graph * fix: correct styles for loader * feat: add zoom/pan for chart * feat: add height for chart * fix: remove unused code * fix: remove empty units from duration * fix: change debounce for pan to 500ms * fix: add utility for plugins register globally * fix: optimize render graph * feat: add buffer data for zoom * fix: add limits for zoom in/out * fix: change update data while zooming * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-27 22:26:14 +03:00
Aliaksandr Valialkin	2efe0acfc9	app/vmselect/promql: add `rollup_scrape_interval(m[d])` function It calculates the min, max and avg scrape intervals for m over the given lookbehind window d	2021-09-27 19:21:24 +03:00
Aliaksandr Valialkin	c4c77aa2dd	app/vmselect/promql: follow-up after `526dd93b32` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612	2021-09-27 18:55:39 +03:00
Roman Khavronenko	526dd93b32	app/vmselect: `quantile` func compatiblity with Prometheus (#1646 ) * app/vmselect: `quantile` func compatiblity with Prometheus The `quantile` func was previously calculated by https://github.com/valyala/histogram package. The result of such calculation was always the closest real value to requested quantile. While in Prometheus implementation interpolation is used. Such difference may result into discrepancy in output between Prometheus and VictoriaMetrics. This commit adds a Prometheus-like `quantile` function. It also used by other functions which depend on it, such as `quantiles`, `quantile_over_time`, `median` etc. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625 Signed-off-by: hagen1778 <roman@victoriametrics.com> * app/vmselect: `quantile` review fixes * quantile functions were split into multiple to provide different API for already sorted data; * float64sPool is used for reducing allocations. Items in pool may have different sizes, but defining a new pool was complicates due to name collisions; Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-27 18:02:41 +03:00
Aliaksandr Valialkin	8ed95e82c6	app/vmselect/promql: follow-up after `57b3320478`	2021-09-24 01:24:18 +03:00
Roman Khavronenko	57b3320478	app/vmselect: make sorting for query result similar to Prometheus (#1647 ) * app/vmselect: make sorting for query result similar to Prometheus Updated sorting allows to get the order of series in result similar or equal to what Prometheus returns. The change is needed for compatibility reasons. * Update app/vmselect/promql/exec_test.go Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-24 01:03:12 +03:00
Aliaksandr Valialkin	e564411a62	app/vmselect/promql: align the behavior of `or`, `and` and `unless` operators with `on (labels)` modifier to Prometheus Previously VictoriaMetrics could return unexpected result of the right-hand side operand had multiple time series with the given set of labels mentioned in `on(labels)`. See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1643	2021-09-24 00:46:25 +03:00
Aliaksandr Valialkin	a22aa0608b	app/vmselect: fix accessing `/graphite/*` endpoints	2021-09-21 13:56:35 +03:00
Aliaksandr Valialkin	9a3d0c43b5	app/vmselect/promql: add `quantiles_over_time("phiLabel", phi1, ..., phiN, m[d])` function for calculating multiple quantiles at once	2021-09-17 23:35:10 +03:00
Aliaksandr Valialkin	2951dd0a57	app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function This function calculates multiple quantiles over the given buckets at once Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573	2021-09-17 13:32:39 +03:00
Aliaksandr Valialkin	5a44be0e52	app/vmselect/promql: optimize quantiles() calculation Calculate quantiles in one go instead of calculating each quantile individually	2021-09-17 12:33:42 +03:00
f41gh7	9375b60c5f	adds stub for functions api	2021-09-16 13:49:52 +03:00
Aliaksandr Valialkin	e60dfc96ff	app/vmselect/promql: add `mad(q)` and `outliers_mad(tolerance, q)` functions to MetricsQL	2021-09-16 13:33:53 +03:00
Aliaksandr Valialkin	eca75cc650	app/vmselect/prometheus: make more clear log messages for errors during sending data to remote clients	2021-09-16 12:56:58 +03:00
Aliaksandr Valialkin	2f86d4cf38	app/vmui: follow-up after `777ff75874` The commit contains the following changes: - Show vmui when requesting /graph page in order to be compatible with Prometheus datasource in Grafana. - Properly encode query args at vmui url. - Set the number of points on the graph to the number of horizontal pixels divided by 2. Previously it was hardcoded to 30. - Do not save server url to persistent storage at browser, since it should be always obtained from the url. - Run `make vmui-update` for updating vmui embedded into VictoriaMetrics.	2021-09-15 17:40:48 +03:00
Aliaksandr Valialkin	d1a16e0891	app/vmselect/promql: use Prometheus-compatible label value formatting for `count_values` function	2021-09-13 13:48:06 +03:00
Aliaksandr Valialkin	f4dead529f	lib/storage: properly search series by multiple tag filters matching empty labels such as foo{bar=~"baz\|",x=~"y\|"} Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1601 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/395	2021-09-09 21:09:21 +03:00
Aliaksandr Valialkin	ea943911bc	app/vmselect/promql: keep metric name in `rollup_candlestick` results, since they don't change the original series meaning Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1600	2021-09-09 19:21:18 +03:00
Aliaksandr Valialkin	5ea689d61b	app/vmselect/promql: add `quantile("phiLabel", phi1, ..., phiN, q)` aggregate function to MetricsQL See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573	2021-08-27 18:37:20 +03:00
Aliaksandr Valialkin	bec18e4fe9	app/vmselect: add `-search.disableAutoCacheReset` command-line option for disabling automatic cache reset when a sample with old timestamp outside -search.cacheTimestampOffset is inserted See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1570	2021-08-27 17:15:31 +03:00
Aliaksandr Valialkin	6c5760db9c	app/vmselect/promql: `make fmt` after `0078486ea7`	2021-08-23 23:06:00 +03:00
Aliaksandr Valialkin	0078486ea7	app/vmselect/promql: rename `sign()` function to `sgn()` in order to be consistent with Prometheus See https://github.com/prometheus/prometheus/pull/8457 for details.	2021-08-23 11:45:51 +03:00
Aliaksandr Valialkin	4f3a5742eb	app/vmselect/prometheus: do not extend `[d]` to the detected interval between samples for `first_over_time(m[d])` This is for the sake of consistency with similar change for the last_over_time(m[d]) at `a724229b5d`	2021-08-21 19:56:14 +03:00
Roman Khavronenko	b5d6a0e499	vmselect: update `vm_request_duration_seconds` value when request fails (#1537 ) Before, metric `vm_request_duration_seconds` was update only on successful attempts which could be misleading. For example, timeout errors on netstorage request may be not accounted in the metric and won't be visible on dashboards. Using `defer` statement to update the metric after query arguments validation may improve the situation.	2021-08-19 13:58:54 +03:00
Aliaksandr Valialkin	a724229b5d	app/vmselect/promql: do not override `[d]` at `last_over_time(m[d])` if `[d]` is smaller than `scrape_interval` Since most users do not expect the overriding of explicitly set `[d]`.	2021-08-19 10:31:48 +03:00
Aliaksandr Valialkin	cdc372bb98	app/vmselect: add `-search.noStaleMarkers` command-line flag for disabling stale markers handling in queries This option allows reducing CPU usage a bit when VictoriaMetrics is used for collecting and processing non-Prometheus data. For example, InfluxDB line protocol, Graphite, OpenTSDB, CSV, etc.	2021-08-18 13:59:02 +03:00
Aliaksandr Valialkin	db1e62495b	app/vmselect/promql: add bitmap_and(), bitmap_or() and bitmap_xor() functions to MetricsQL Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1541	2021-08-17 13:21:21 +03:00
Aliaksandr Valialkin	bd14b0887e	app/vmselect/promql: move common condition to dropStaleNaNs in order to improve code maintainability	2021-08-17 11:01:16 +03:00
Aliaksandr Valialkin	113f0a8a07	app/vmselect/promql: drop staleness marks before calling rollupConfig.Do This allows dropping staleness marks only once and then calculate multiple rollup functions on the result. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526	2021-08-15 13:21:10 +03:00
Aliaksandr Valialkin	25997a70f1	Revert "app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions" This reverts commit 94dfcb6747a3b29a11d14e71bea21a2312bb6346. It is better to remove staleness marks (decimal.StaleNaN) before calling rollupConfig.Do, e.g. in preFunc	2021-08-15 13:19:16 +03:00
Aliaksandr Valialkin	73d7b568da	app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions Prometheus stalenss marks shouldn't be changed in removeCounterResets. Otherwise they will be converted to an ordinary NaN values, which couldn't be removed in dropStaleNaNs() function later. This may result in incorrect calculations for rollup functions. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526	2021-08-14 12:45:57 +03:00
Aliaksandr Valialkin	4401464c22	all: add support for Prometheus staleness markers Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845	2021-08-13 12:10:17 +03:00
Aliaksandr Valialkin	95dd5a48bb	app/vmselect: `make vmui-update` after the commit 4ae14df864a7e327955f44941295a286175423b3	2021-08-11 13:41:41 +03:00
Aliaksandr Valialkin	e92fde7945	app/vmselect/promql: add `present_over_time(m[d])` function, which will be available starting from Prometheus 2.29.0 See https://github.com/prometheus/prometheus/releases/tag/v2.29.0-rc.0 and https://github.com/prometheus/prometheus/pull/9097	2021-08-03 16:11:49 +03:00
Nikolay	7bbff7fb86	adds /rules and /alerts api for grafana (#1504 ) Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-08-02 17:28:09 +03:00
Aliaksandr Valialkin	a1911e1330	app/vmselect/netstorage: unpack time series data in mostly local big chunks This should improve performance on multi-CPU systems for queries selecting time series with big number of raw samples	2021-07-30 12:03:17 +03:00
Aliaksandr Valialkin	74ffaa45d9	app/vmselect/netstorage: do not query Go maps with unsafe string keys, since this breaks in Go 1.17	2021-07-30 09:57:53 +03:00
Aliaksandr Valialkin	192dfbfd90	app/vmselect: follow-up for `ed95bc9531` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1493	2021-07-29 09:53:28 +03:00
arnoldyahad	00af4ff5a4	Add case prometheus/rules for grafana 8 (#1502 )	2021-07-29 09:53:27 +03:00
Aliaksandr Valialkin	8ee8660ac4	app/vmselect: follow-up for `626073bca8` * Rename -search.maxMetricsPointSearch to -search.maxSamplesPerQuery, so it is more consistent with the existing -search.maxSamplesPerSeries * Move the -search.maxSamplesPerQuery from vmstorage to vmselect, so it could effectively limit the number of raw samples obtained from all the vmstorage nodes * Document the -search.maxSamplesPerQuery in docs/CHANGELOG.md	2021-07-28 18:00:23 +03:00
Aliaksandr Valialkin	8b7917cd81	all: add `go:build` lines for Go1.17 See https://tip.golang.org/doc/go1.17#gofmt for more details	2021-07-26 15:48:21 +03:00
Aliaksandr Valialkin	95aff47330	app/vmselect: prevent from possible deadlock when f callback blocks inside RunParallel	2021-07-26 15:47:30 +03:00
Aliaksandr Valialkin	fcaf152480	app/vmselect: `make vmui-update` after `a91d41f12a`	2021-07-26 10:31:11 +03:00
Aliaksandr Valialkin	05672ffc32	app/vmselect/promql: properly handle `(a op b) default N` if `(a op b)` returns NaN series The result should be a series with `N` values and `a op b` labels. Previously such series has been removed from the result.	2021-07-16 01:44:58 +03:00
Aliaksandr Valialkin	ed10141ff8	app/vmselect/netstorage: use more scalable algorithm for ditributing the work among among multiple channels on systems with big number of CPU cores	2021-07-16 00:35:23 +03:00
Aliaksandr Valialkin	ca75432e66	app/vmselect: do not track queries with less than 1ms execution time at /api/v1/status/top_queries This should improve the readability and usefullness of the /api/v1/status/top_queries when debugging slow queries or queries that take too much cpu time.	2021-07-15 16:44:28 +03:00
Aliaksandr Valialkin	f4e81aef7e	app/vmselect/netstorage: add `-search.maxSamplesPerSeries` command-line option for limiting the number of samples a query can process per each series This should prevent from out of memory crashes like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1067	2021-07-15 16:03:28 +03:00
Aliaksandr Valialkin	e6ef97a5ee	app/vmselect/netstorage: improve scalability of series unpacking on multi-CPU systems	2021-07-15 15:41:58 +03:00
Aliaksandr Valialkin	a925d5a3e1	app/vmselect/promql: duration handling improvements in MetricsQL queries - Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600 - Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])	2021-07-12 17:16:41 +03:00
Aliaksandr Valialkin	8c764e88f0	app/vmui: move source code from https://github.com/VictoriaMetrics/vmui to app/vmui Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413	2021-07-09 17:15:23 +03:00
Aliaksandr Valialkin	c5f0b454f0	app/vmselect: follow-up after `aa11ef6d3b` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413	2021-07-07 17:43:35 +03:00
tony	e9e35a7d6a	add vmui for vmselect component (#1431 ) Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-07-07 17:33:02 +03:00
Aliaksandr Valialkin	ed944313b0	app/{vminsert,vmselect}: export vminsert_request_duration_seconds and vmselect_request_duration_seconds histograms	2021-07-07 13:25:21 +03:00
Aliaksandr Valialkin	766edbc421	lib/httpserver: print full requestURI in httpserver.Errorf This should simplify debugging.	2021-07-07 13:09:40 +03:00
Aliaksandr Valialkin	cdfae0117a	app/vmselect/promql: return the last timestamp for the max / min value from `tmax_over_time()` and `tmin_over_time()` function as most users expect	2021-06-23 14:19:00 +03:00
Aliaksandr Valialkin	83a4db813e	app/vmselect: log slow requests to all the `/api/v1/*` handlers if their execution time exceeds `-search.logSlowQueryDuration`	2021-06-18 19:04:42 +03:00
Roman Khavronenko	fb4f758715	promql: fix `increase_pure` calculation for cases with stale series (#1381 ) Due to staleness handling, increase_pure were using incorrect previous value during calculation in cases where series disappears for period longer than staleness period and then returns back. The fix suppose to account for a real datapoint value before staleness takes place. The fix should remove unexpected spikes while using `increase_pure` for staled series.	2021-06-15 17:37:19 +03:00
John Belmonte	67b17cdd68	spelling fix: synonym (#1363 )	2021-06-10 08:32:52 +03:00
Aliaksandr Valialkin	d0dca62026	app/vmselect/promql: typo fix in the comment	2021-06-09 18:34:31 +03:00
Aliaksandr Valialkin	0842bb9294	app/vmselect/promql: add `timezone_offset(tz)` function Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1306	2021-05-20 11:53:09 +03:00
Aliaksandr Valialkin	832651c6c2	app/vmselect: follow up after `8a0678678b` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1168	2021-05-12 17:18:30 +03:00
Nikolay	8a0678678b	Adds tsdb match filters (#1282 ) * init work on filters * init propose for status filters * fixes tsdb status adds test * fix bug * removes checks from test	2021-05-12 15:18:45 +03:00
Aliaksandr Valialkin	6bc52fe41a	all: rename https://victoriametrics.github.io to https://docs.victoriametrics.com	2021-04-20 20:16:17 +03:00
Aliaksandr Valialkin	544821b719	app/vmselect/promql: fix tests after `d3fa0ccabd`	2021-04-08 00:18:01 +03:00
Aliaksandr Valialkin	d3fa0ccabd	app/vmselect/promql: properly detect aggregate `topk` and `bottomk` aggregate functions in order to disable duplicate sorting Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189	2021-04-08 00:09:40 +03:00
Aliaksandr Valialkin	cb12a8f0a8	app/vmselect: return `data:null` instead of `data:[]` from `/api/v1/query_exemplars`, since Grafana throws an error otherwise Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1186	2021-04-07 23:34:06 +03:00
Aliaksandr Valialkin	1177dca3da	app/vmselect: do not sort series returned from `topk` and `bottomk` functions, since these series are already sorted in user-expected order Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189	2021-04-07 14:16:08 +03:00
Aliaksandr Valialkin	7d23598b33	app/vmselect: return dumb response on `/api/v1/query_exemplars` request Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1186	2021-04-05 23:25:08 +03:00
Aliaksandr Valialkin	7a0b964e8d	app/vmselect/promql: do not delete `dst_label` if `src_label` is empty in `label_copy(q, src_label, dst_label)` and `label_move(q, src_label, dst_label)`	2021-04-03 22:05:06 +03:00
Aliaksandr Valialkin	3055ab0115	app/vmselect/promql: add ability to set label value additionally to label name for the remaining sum of time series returned from `topk_` and `bottomk_` functions in the form: `topk_min(N, m, "label=value")`	2021-04-02 23:55:54 +03:00
Aliaksandr Valialkin	c79e4a2f90	app/vmselect/promql: remove the limit on the number of time series that can be sorted, since it may confuse users Always sort time series returned from `/api/v1/query` and `/api/v1/query_range` unless `sort_*` function is used at top level of the query.	2021-04-02 15:02:08 +03:00
Aliaksandr Valialkin	e7fdea5953	app/vmselect: add `-search.maxStatusRequestDuration` command-line flag for limiting the duration of requests to `/api/v1/status/*` and `/api/v1/series/count`	2021-03-30 21:41:35 +03:00
Aliaksandr Valialkin	7962cf1af8	app/vmselect: prevent from possible incomplete query results after timed out query Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711	2021-03-30 13:35:45 +03:00
Aliaksandr Valialkin	aa81039b42	app/vmselect: log the metric which trigger rollup result cache reset This should help finding the source of stale metrics	2021-03-25 21:31:39 +02:00
Aliaksandr Valialkin	0b2a66db30	app/vmselect/promql: do not merge time series during requests to `/api/v1/query` See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1141	2021-03-25 13:56:07 +02:00
Aliaksandr Valialkin	331a6a2015	app/vmselect/graphite: accept and enforce `extra_label` in all the Graphite APIs	2021-03-23 15:29:16 +02:00
Aliaksandr Valialkin	b521d1d4f2	app/vmselect: move getEnforcedTagFiltersFromRequest to searchtuils, since it will be used in Graphite functions soon	2021-03-23 14:16:29 +02:00
Aliaksandr Valialkin	71b72304ae	app/vmselect: improve description for `-search.maxPointsPerTimeseries` command-line flag	2021-03-22 16:45:34 +02:00
Aliaksandr Valialkin	cef010d5f7	app/vmselect/promql: increment key prefix for faster reset for rollup result cache	2021-03-22 11:59:07 +02:00
Aliaksandr Valialkin	b0c956a178	app/vmselect/graphite: follow-up after `529d7be26b`	2021-03-18 16:30:20 +02:00
Nikolay	529d7be26b	changes metricsFind api (#1137 ) it should be able mitigate crash if label value contains *,[ or { symbols	2021-03-18 16:12:02 +02:00
Aliaksandr Valialkin	b1713e3fcd	app/vmselect/promql: typo fix after `9666834045`	2021-03-17 15:12:11 +02:00
Aliaksandr Valialkin	9666834045	app/vmselect/promql: merge adjancent buckets with the smallest summary number of hits in `buckets_limit()` function This should improve accuracy for the returned buckets	2021-03-17 14:31:41 +02:00
Aliaksandr Valialkin	7bc728bf53	app/vmselect: add `vm_index_search_duration_seconds` histogram for monitoring the performance of index search	2021-03-17 01:17:41 +02:00
Aliaksandr Valialkin	0ce557951f	app/vmselect/netstorage: reduce mutex contention when unpacking data on a system with high number of CPU cores	2021-03-16 21:51:31 +02:00
Aliaksandr Valialkin	f0a4157f89	app/vmselect/promql: do not crash if `histogram_over_time()` function name contains uppercase letters such as `Histogram_over_time()`	2021-03-16 12:24:21 +02:00
Aliaksandr Valialkin	85a95bf60c	all: various fixes in command-line flag descriptions	2021-03-15 21:59:25 +02:00
Aliaksandr Valialkin	923cdb0552	app/vmselect/promql: reduce overhead on scrape interval estimation It should be enough to use the first 20 datapoints instead of 100 datapoints for scrape interval estimation.	2021-03-15 20:31:33 +02:00
Aliaksandr Valialkin	82aab87446	app/vmselect/promql: fix tests after `2dae0a2c47`	2021-03-15 20:18:59 +02:00
Aliaksandr Valialkin	2dae0a2c47	app/vmselect: add `round_digits` query arg to `/api/v1/query` and `/api/v1/query_range` handlers for limiting the number of decimal digits after the point	2021-03-15 12:36:33 +02:00
Aliaksandr Valialkin	2096c6e464	app/vmselect/prometheus: typo fix after `7c002023d7`	2021-03-12 12:19:36 +02:00
Aliaksandr Valialkin	7c002023d7	app/vmselect/prometheus: do not include datapoints with timestamps matching `t-d` when returning results from `/api/v1/query?query=m[d]&time=t` as Prometheus does	2021-03-12 12:16:50 +02:00
Aliaksandr Valialkin	fe8b12fbad	app/vmselect/promql: follow up for `433fff0006`	2021-03-09 12:48:44 +02:00
Nikolay	433fff0006	duplicate timeseries fix for prometheus_buckets function (#1119 ) * try fix for prometheus_buckets * merge possible end of the bucket collision	2021-03-09 12:26:23 +02:00
Aliaksandr Valialkin	a14053ffa0	app/vmselect/promql: add `histogram_avg()`, `histogram_stddev()` and `histogram_stdvar()` functions to MetricsQL	2021-03-04 14:12:07 +02:00
Aliaksandr Valialkin	2b53add6b2	app/vmselect/querystats: show the number of matching queries in the top by average duration and in the top by summary duration This should help debugging slow queries.	2021-02-28 19:40:19 +02:00
Nikolay	186c078fac	adds enforced tag filters into cache key (#1095 )	2021-02-27 00:15:53 +02:00
Aliaksandr Valialkin	d86e9b49c4	app/vmselect/promql: increase accuracy for `buckets_limit()` function for small limits by skipping the first and the last buckets during merge The first and the last buckets are usually `[0 ... leMin]` and `(leMax ... +Inf)`. If they are merged with adjancent buckets, then the resulting accuracy can suffer.	2021-02-26 22:56:36 +02:00
Aliaksandr Valialkin	98854e5f2b	app/vmselect: add `sign(q)` and `clamp(q, min, max)` functions, which will be added in the upcoming Prometheus release See https://twitter.com/roidelapluie/status/1363428376162295811 The `last_over_time(m[d])` function already exists in MetricsQL.	2021-02-24 17:24:56 +02:00
Aliaksandr Valialkin	f4135b0d14	app/vmselect/promql: properly calculate `histogram_quantile() over zero buckets and only a single non-zero` le="+Inf"` bucket like Prometheus does	2021-02-24 00:42:22 +02:00
Aliaksandr Valialkin	fa03e0d210	app/vmselect/promql: add `increase_pure()` function to MetricsQL Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962	2021-02-22 19:14:15 +02:00
Aliaksandr Valialkin	9c70c1f21f	app/vmselect/promql: reduce the probability of `duplicate time series` errors when querying Kubernetes metrics	2021-02-18 22:07:29 +02:00
Aliaksandr Valialkin	c4756f94da	app/vmselect/netstorage: reuse timeseriesWork objects in order to reduce memory allocations	2021-02-16 16:08:53 +02:00
Aliaksandr Valialkin	5a401225c7	app/vmselect/netstorage: use unsafe string as a key for a map when the map already contains the given key This should prevent from a memory allocation and a string copy.	2021-02-16 15:43:10 +02:00
Aliaksandr Valialkin	4e39bf148c	vendor: update github.com/VictoriaMetrics/metrics from v1.13.1 to v1.14.0 The new version switches from log-linear histograms to log-based histograms, which provide up to 3.6 times better accuracy.	2021-02-15 15:12:29 +02:00
Aliaksandr Valialkin	acdb401585	app/vmselect/prometheus: treat `match` query arg in the same way as `match[]` query arg	2021-02-11 15:02:21 +02:00
Aliaksandr Valialkin	2d33230793	app/vmselect/promql: properly make copies of EvalConfig	2021-02-11 12:41:15 +02:00
Aliaksandr Valialkin	7a3a9421f3	app/vmselect/promql: make a copy of EvalConfig when executing `q1` and `q2` in parallel for `q1 binary_op q2` This should prevent from data races if the underlying functions modify EvalConfig contents.	2021-02-10 23:05:55 +02:00
Aliaksandr Valialkin	04faea8b45	app/vmselect: parallelize `q1 <binary_op> q2` queries by running `q1` and `q2` in parallel This should reduce query execution times.	2021-02-10 22:59:39 +02:00
Aliaksandr Valialkin	553016ea99	lib/storage: disable composite index usage when querying old data	2021-02-10 14:57:50 +02:00
Aliaksandr Valialkin	c7dccebaef	lib/storage: optimize search by label filters matching big number of time series	2021-02-10 00:44:54 +02:00
Aliaksandr Valialkin	803a00102a	app/{vminsert,vmselect}: accept requests to paths with `/graphite` and `/prometheus` prefixes This should improve compatibility with path prefixes from VictoriaMetrics cluster. See https://victoriametrics.github.io/Cluster-VictoriaMetrics.html#url-format	2021-02-04 20:01:18 +02:00
Aliaksandr Valialkin	4d43ab0875	app/vmselect: typo fix when stripping url path prefixes	2021-02-04 19:29:25 +02:00
Aliaksandr Valialkin	8629fd8a72	app/vmselect: deprecate `-search.treatDotsAsIsInRegexps` in favor to `{__graphite__="foo.*.bar"}` syntax	2021-02-03 20:36:01 +02:00
Aliaksandr Valialkin	d16f22f3a1	app/vmselect,lib/storage: properly parse Graphite selectors with inner wildcards Example: foo{bar{x,yz},a[b-c],*de}	2021-02-03 20:14:22 +02:00
Aliaksandr Valialkin	157c02622b	app/vmselect: add ability to set Graphite-compatible filter via `{__graphite__="foo.*.bar"}` syntax	2021-02-03 01:21:54 +02:00
Aliaksandr Valialkin	b43b498fd8	app/vmselect: add ability to pass `extra_label=<label>=<value>` query arg to Prometheus Querying API This enforced `{label="value"}` label filter to the query. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1021	2021-02-01 18:04:17 +02:00
Nikolay	195341a7cf	Graphite vmalert wip (#112 ) * init implementation for graphite alerts * adds graphite support for vmalert * small fix * changes vmalert graphite api with type * updates tests * small fix * fixes graphite parse * Fixes graphite from time	2021-02-01 15:05:32 +02:00
Aliaksandr Valialkin	c164a8d231	app/vmselect/promql: improve documentation for `-search.maxPointsPertimeseries` command-line flag This should reduce incorrect usage and assumptions for this flag. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1020	2021-01-22 13:00:10 +02:00
Aliaksandr Valialkin	8749c2dd92	app/vmselect: add `-search.maxStepForPointsAdjustment` command-line flag, which can be used for disabling adjustment for points returned from `/api/v1/query_range` handler if they have timestamps closer than `-search.latencyOffset` to the current time	2021-01-19 22:56:32 +02:00
Aliaksandr Valialkin	011c5da785	app/vmselect/graphite: extract getCanonicalPath() function from loop body inside getCanonicalPaths()	2021-01-18 17:30:26 +02:00
Aliaksandr Valialkin	1051d8aa2d	app/vmselect/promql: add ability to pass multiple labels to `sort_by_label` and `sort_by_label_desc` functions See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992	2021-01-13 12:44:51 +02:00
Aliaksandr Valialkin	acf1a2c72b	app/vmselect/promql: properly parse escaped multibyte utf8 code sequences in metric names and labels names See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/990	2021-01-13 10:59:42 +02:00
Aliaksandr Valialkin	1837f2f7d3	app/vmselect/promql: add `tfirst_over_time(m[d])` and `tlast_over_time(m[d])` MetricsQL functions for returning timestamps for the first and the last samples in `m` over `d`	2021-01-12 16:12:12 +02:00
Aliaksandr Valialkin	c86286ec1d	app/vmselect/promql: do not ajdust `offset` value provided in the query Previously it could be modified in order to improve response cache hit ratio. This is unneeded, since cache hit ratio should remain good because the query time range should be already aligned to multiple of `step` values. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976	2020-12-27 14:09:25 +02:00
Aliaksandr Valialkin	4b7105a65b	app/vmselect: sync query stats handling with cluster version	2020-12-27 13:00:29 +02:00
Aliaksandr Valialkin	df0309eae0	app/vmselect/promql: simplify defer call for querystats.RegisterQuery	2020-12-27 12:06:04 +02:00
Aliaksandr Valialkin	ad4e6a9283	app/vmselect/querystats: reduce the default number of last queries to track from 100K to 20K This should reduce memory usage in constrained environments	2020-12-25 17:40:47 +02:00
Aliaksandr Valialkin	59183f66d0	app/vmselect: refactor `/api/v1/stats/top_queries` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907	2020-12-25 16:44:29 +02:00
Nikolay	86630350bf	Adds query stats handler (#945 ) * Adds query stat handler, for query and query_range api, victoriametrics tracks query execution time, stats are expored at /api/v1/status/queries endpoint with topN param https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907 * fixed query stats bugs * improves queryStats tracker * improves query stat * small fix * fix tests * added more tests * fixes 386 tests * naming fixes * adds drop for outdated records	2020-12-25 16:42:05 +02:00
Aliaksandr Valialkin	88ac4dfc07	app/vmselect: properly parse negative combined offsets such as `-1h2m3s` Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.	2020-12-19 01:23:46 +02:00
Aliaksandr Valialkin	11fa458e39	app/vmselect/promql: return expected `increase()` result for the first point on the graph with value not exceeding 100	2020-12-15 13:40:46 +02:00
Aliaksandr Valialkin	5ebfc275e6	app/victoria-metrics: automatically reset response cache when samples with too timestamps older than `now - search.cacheTimestampOffset` are ingested	2020-12-14 13:08:28 +02:00
Aliaksandr Valialkin	ba8ac08739	app/vmselect/graphite: properly handle wildcards and charsets inside curly braces For example, `foo{bar,[a-f]ab}` should match `foobar`, `foobar123`, `foofab`, etc. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/952	2020-12-11 17:24:12 +02:00
Aliaksandr Valialkin	4146fc4668	all: properly handle CPU limits set on the host system/container This can reduce memory usage on systems with enabled CPU limits. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946	2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin	1906f841c9	app/vmselect/promql: do not reduce lookbehind window for `any_rollup_func(m)` to `-search.maxStalenessInterval`. It should equal to `step` value passed to `/api/v1/query_range` as most users expect	2020-12-08 15:16:17 +02:00
Aliaksandr Valialkin	007dbf273d	app/vmselect/graphite: remove duplicate `name` tag from `/tags/autoComplete/tags` handler Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/942	2020-12-07 01:08:52 +02:00
Aliaksandr Valialkin	e2e8ef86d9	app/vmselect/promql: add `count_eq_over_time(m[d], N)` and `count_ne_over_time(m[d], N)` for calculating the number of samples in `m` over `d` that are equal / not equal to `N`	2020-12-05 12:30:46 +02:00
Aliaksandr Valialkin	66379cc69f	app/vmselect/promql: add `label_uppercase(q, label1, ... labelN)` and `label_lowercase(q, label1, ... labelN)` functions Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/936	2020-12-03 21:47:36 +02:00
Aliaksandr Valialkin	44a34a0f5f	app/vmselect/promql: `make fmt`	2020-12-02 21:33:35 +02:00
Aliaksandr Valialkin	1982505c2b	app/vmselect/promql: return `nan` from `minute(m)` when `m` equals to `nan` This aligns VictoriaMetrics behaviour with Prometheus behaviour. The issue has been spotted in https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/	2020-12-02 20:16:58 +02:00
Aliaksandr Valialkin	9d87496b50	app/vmselect/promql: do not return `0` value from `sum_over_time(m[d])` when there are no samples on the given `d` window. This aligns the behaviour of `sum_over_time()` with other `_over_time()` functions and with Prometheus behavior.	2020-12-02 13:12:50 +02:00
Aliaksandr Valialkin	91a4c279cc	app/vmselect: return `metric` values from `time() cmp_op metric` query when `cmp_op` comparison is true This aligns MetricsQL behavior to Prometheus' one. The issue has been identified at https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/	2020-12-02 12:09:34 +02:00
Aliaksandr Valialkin	700bda8e2e	app/vmselect/promql: return `nan` from `a >bool b` if `a` is `nan` in the same way as Prometheus does	2020-12-02 00:28:26 +02:00
Aliaksandr Valialkin	efdc3c71af	app/vmselect/searchutils: return elapsed time in Deadline.String() output This should improve debuggability for error messages containing Deadline.String() output	2020-12-01 00:15:18 +02:00
Aliaksandr Valialkin	a724dde90a	app/vmselect: protect `/tags/delSeries` with `-deleteAuthKey` in the same way as `/api/v1/admin/tsdb/delete_series`	2020-11-23 15:35:59 +02:00
Aliaksandr Valialkin	f0c207fae2	app/vmselect: add `/tags/delSeries` handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#removing-series-from-the-tagdb	2020-11-23 15:27:21 +02:00
Aliaksandr Valialkin	d3794eb994	app/{vminsert,vmselect}: move `/tags/tagSeries` and `/tags/tagMultiSeries` api from vminsert to vmselect This is needed for consistency, since all the `/tags*` api handlers are located in vmselect.	2020-11-23 12:33:19 +02:00
Aliaksandr Valialkin	f765985947	lib/fs: replace fs.OpenReaderAt with fs.MustOpenReaderAt All the callers for fs.OpenReaderAt expect that the file will be opened. So it is better to log fatal error inside fs.MustOpenReaderAt instead of leaving this to the caller.	2020-11-23 09:57:21 +02:00
Aliaksandr Valialkin	2859a452d4	app/vmselect: add remoteAddr to slow query log in order to improve debuggability This will simplify identifying the client that sends slow queries to VictoriaMetrics.	2020-11-18 20:38:32 +02:00
Aliaksandr Valialkin	266788be14	app/vmselect: use storage.NewSearchQuery() instead of constructing storage.SearchQuery in-place This should prevent from bugs when AccountID and ProjectID aren't set in storage.SearchQuery.	2020-11-16 18:24:00 +02:00
Aliaksandr Valialkin	cf18df367d	app/vmselect/netstorage: apply Graphite filter after substituting `__name__` with `name`	2020-11-16 15:52:16 +02:00
Aliaksandr Valialkin	2c67232565	app/vmselect/graphite: add `/tags/autoComplete/values` handler from Graphite Tags API	2020-11-16 15:29:35 +02:00
Aliaksandr Valialkin	86f99c6b55	app/vmselect/graphite: add `/tags/autoComplete/tags` handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#auto-complete-support	2020-11-16 14:50:05 +02:00
Aliaksandr Valialkin	3c1434118e	app/vmselect/prometheus: return `__name__` label if `match[]` query to `/api/v1/labels` matches at least a single time series	2020-11-16 13:54:34 +02:00
Aliaksandr Valialkin	27a417bcd3	app/vmselect/prometheus: improve performance for `/api/v1/labels` and `/api/v1/label/<labelName>/values` on time ranges exceeding one day when `match[]` query arg is set	2020-11-16 13:51:59 +02:00
Aliaksandr Valialkin	6fa806f1ca	app/vmselect/prometheus: fix deadlock in `/api/v1/series` on a time range exceeding one day	2020-11-16 13:30:47 +02:00
Aliaksandr Valialkin	465923b181	app/vmselect/graphite: add /tags/findSeries handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags	2020-11-16 12:53:13 +02:00
Aliaksandr Valialkin	414cd39659	app/vmselect/graphite: apply filter then limit	2020-11-16 04:09:14 +02:00
Aliaksandr Valialkin	d100341394	app/vmselect/graphite: add `/tags/<tag_name>` handler for Graphite Tags API	2020-11-16 03:42:25 +02:00
Aliaksandr Valialkin	6251762787	app/vmselect/graphite: add `/tags` handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags	2020-11-16 03:00:01 +02:00
Aliaksandr Valialkin	d9d01f976b	app/vmselect/promql: remove spikes from `increase()` and `delta()` results on time series with spare irregular data points Do not take into account spare data point value if the next point will is located too far from the current point. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894	2020-11-13 15:23:44 +02:00

... 6 7 8 9 10 ...

1111 commits