2555 commits

Author	SHA1	Message	Date
Roman Khavronenko	a645a95bd6	docs: improve troubleshooting docs for vmalert (#3812) ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-02-13 17:29:30 +01:00
Oleksandr Redko	9fff48c3e3	app,lib: fix typos in comments (#3804)	2023-02-13 13:27:13 +01:00
Aliaksandr Valialkin	b026ebe91e	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-02-11 20:54:02 -08:00
Aliaksandr Valialkin	438b2e11bd	app/vmauth: allow specifying max_concurrent_requests value on a per-user basis bigger than the -maxConcurrentPerUserRequests value	2023-02-11 20:53:08 -08:00
Aliaksandr Valialkin	e4a61581e1	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-02-11 12:53:25 -08:00
Aliaksandr Valialkin	25a9017a72	app/vmui: show median instead of avg on graph tooltip and line legend ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3706	2023-02-11 12:51:12 -08:00
Aliaksandr Valialkin	7b41c9ac72	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-02-11 01:07:13 -08:00
Aliaksandr Valialkin	b11bdc46be	app/vmselect/promql: add mad_over_time(m[d]) function ... See https://github.com/prometheus/prometheus/issues/5514	2023-02-11 01:06:20 -08:00
Aliaksandr Valialkin	ed4492ddd5	all: update alpine base docker image from 1.17.1 to 1.17.2 ... See https://alpinelinux.org/posts/Alpine-3.17.2-released.html	2023-02-11 00:37:17 -08:00
Aliaksandr Valialkin	c1d42f3288	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-02-11 00:33:44 -08:00
Aliaksandr Valialkin	776391917f	app/vmauth: improve load balancing by sending incoming requests to backends with the lowest number of concurrent requests ... While at it, stop sending requests to unavailable backend for 3 seconds before the next attempt. This should reduce the amounts of useless work and the number of useless network packets when the backend is temporarily unavailable.	2023-02-11 00:30:31 -08:00
Aliaksandr Valialkin	f3625e4f3f	app/vmauth: add -maxConcurrentPerUserRequests command-line option for limiting the number of concurrent requests on a per-user basis ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3346	2023-02-10 21:58:21 -08:00
Aliaksandr Valialkin	70f8911ca7	app/vmauth: automatically retry failing GET requests on the remaining backends	2023-02-09 21:05:55 -08:00
Dmytro Kozlov	f582f9e8ab	app/vmauth: add concurrent requests limit per auth record (#3749) ... * app/vmauth: add concurent requests limit per auth record * app/vmauth: added clarification comment * app/vmauth: remove unused code * app/vmauth: move read from limiter * app/vmauth: fix text * app/vmauth: fix comments * - Clarify the docs for the max_concurrent_requests option at docs/vmauth.md - Clarify the description of the change at docs/CHANGELOG.md - Make sure that the -maxConcurrentRequests takes precedence over per-user max_concurrent_requests - Update tests for verifying that the max_concurrent_requests option is parsed properly Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3346 --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-09 20:03:01 -08:00
Aliaksandr Valialkin	4167344edb	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-02-09 19:13:40 -08:00
Aliaksandr Valialkin	73358571ee	app/vmalert: follow-up after d3c64aae8768d58781ee7e358bd7f3d8e0eb836d ... - Document the change at docs/CHANGELOG.md - Add `Reading rules from object storage` section to docs/vmalert.md - Add `s3` prefix to command-line flags related to the configuration of s3 and gcs clients - Explicitly mention that reading rules from object storage is supported only in enterprise version	2023-02-09 18:52:00 -08:00
Roman Khavronenko	14c20c1843	vmalert: support object storage for rules (#519) ... * vmalert: support object storage for rules Support loading of alerting and recording rules from object storages `gcs://`, `gs://`, `s3://`. * review fixes	2023-02-09 18:50:48 -08:00
Aliaksandr Valialkin	37f48cdaa5	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-02-09 14:07:32 -08:00
Aliaksandr Valialkin	0e0095d350	all: run apk update && apk upgrade in base Alpine Docker image in order to get all the recent security fixes	2023-02-09 14:01:32 -08:00
Aliaksandr Valialkin	a39140baef	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-02-09 13:08:34 -08:00
Aliaksandr Valialkin	513707a8c7	app/vmui: UX enhancements for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3706 ... - Display `min` value additionally to `avg`, `max` and `last` - Allow copy-n-pasting metric name with its labels from both legend and tooltup	2023-02-09 11:04:51 -08:00
Aliaksandr Valialkin	f40661e7b7	docs/vmagent.md: clarify that automatically generated metrics contain all the target-specific labels, including instance and job	2023-02-09 11:04:51 -08:00
Yury Molodov	bff18cb5dd	vmui: lazy loading predefined panels (#3795) ... * fix: change logic lazy loading predefined panels * app/vmselect/vmui: `make vmui-update` --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-09 00:11:55 -08:00
Yury Molodov	e1063ce3c1	vmui: improve tenant selector (#3794) ... * fix: change styles tenant selector (#3792) * docs/CHANGELOG.md: document the change Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3792 --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-09 00:08:59 -08:00
Yury Molodov	8afc0aef8d	vmui: add last/max/avg values (#3789) ... * feat: add last/max/avg values (#3706) * fix: change filter exclude values * app/vmui: wip - improve the visualization for avg/max/last values - make getAvgFromArray() function resilient against inf/undefined/nil - export getLastFromArray() function, which is resilient against inf/undefined/nil - run `make vmui-update` --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-08 22:41:20 -08:00
Yury Molodov	75bcf86a31	fix: turn off the local dashboards(#3740) (#3793)	2023-02-08 11:13:15 -08:00
Roman Khavronenko	e83f14210d	Vmalert fixes (#3788) ... * vmalert: use group's ID in UI to avoid collisions Identical group names are allowed. So we should used IDs for various groupings and aggregations in UI. Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: prevent disabling state updates tracking The minimum number of update states to track is now set to 1. Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: properly update `debug` and `update_entries_limit` params on hot-reload Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: display `debug` field for rule in UI Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: exclude `updates` field from json marhsaling This field isn't correctly marshaled right now. And implementing the correct marshaling for it doesn't seem right, since json representation is mostly used by systems like Grafana. And Grafana doesn't expect this field to be present. Signed-off-by: hagen1778 <roman@victoriametrics.com> * fix test for disabled state Signed-off-by: hagen1778 <roman@victoriametrics.com> * fix test for disabled state Signed-off-by: hagen1778 <roman@victoriametrics.com> --------- Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-02-08 14:34:03 +01:00
Roman Khavronenko	c32d8ea29e	vmalert: update docs (#3770) ... vmalert: update flags description Signed-off-by: hagen1778 <roman@victoriametrics.com> --------- Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-02-06 09:51:30 +01:00
Roman Khavronenko	6fd10e8871	vmalert: speed up state restore procedure on start (#3758) ... * vmalert: speed up state restore procedure on start Alerts state restore procedure has been changed to become asynchronous. It doesn't block groups start anymore which significantly improves vmalert's startup time. Instead, state restore is called by each group in their goroutines after the first rules evaluation. While previously state restore attempt was made for all loaded alerting rules, now it is called only for alerts which became active after the first evaluation. This reduces the amount of API calls to the configured remote read URL. This also means that `remoteRead.ignoreRestoreErrors` command-line flag becomes deprecated now and will have no effect if configured. See relevant issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2608 Signed-off-by: hagen1778 <roman@victoriametrics.com> * make lint happy Signed-off-by: hagen1778 <roman@victoriametrics.com> * Apply suggestions from code review --------- Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-03 19:46:13 -08:00
Aliaksandr Valialkin	0a824d9490	app/vmselect/vmui: make vmui-update after e4c04b6dbe	2023-02-03 19:34:01 -08:00
Yury Molodov	e4c04b6dbe	vmui: set light theme for app mode (#3748) ... * fix: set light theme for app mode * fix: check inputTenantID flag * fix: rename inputTenantID to useTenantID	2023-02-03 19:31:37 -08:00
Yury Molodov	f63f487787	vmui: mobile view (#3742) ... * feat: add detect the system theme * fix: change logic fetch tenants * feat: add docs and info to cardinality page * feat: add mobile view #3707	2023-02-03 19:27:57 -08:00
Aliaksandr Valialkin	32e46ea35f	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-02-01 13:02:21 -08:00
Aliaksandr Valialkin	607b542222	vendor: make vendor-update	2023-02-01 12:23:23 -08:00
Zakhar Bessarab	4cf37c5e70	app/vmbackup: fix deleting snapshot after backup completion (#3735) (#3736) ... Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>	2023-02-01 11:23:58 +01:00
Aliaksandr Valialkin	3d331e4c5d	app/vmselect/vmui: make vmui-update after dcc5616126	2023-01-31 13:24:43 -08:00
Yury Molodov	dcc5616126	vmui: improvement the theme (#3731) ... * feat: add detect the system theme * fix: change logic fetch tenants * feat: add docs and info to cardinality page --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-31 12:54:59 -08:00
Roman Khavronenko	1cbdcd391c	docs: mention -vmalert.proxyURL in vmalert docs (#3730) ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-01-30 16:28:33 +01:00
Yury Molodov	ac14d50c18	vmui: add select of Tenant ID (#3673) ... * feat: add select of tenantID * feat: replace tenantID to default url * fix: move the tenantID selector to the top header * fix: hide tenantID selector by condition * fix: correct z-index * app/vmselect/vmui: `make vmui-update` --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-27 15:53:14 -08:00
Aliaksandr Valialkin	06f6b76521	app/vmagent: properly return 200 response code when importing data via Prometheus PushGateway protocol ... This is the same fix as has been already applied to app/vminsert at cdb6d651e9 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1415	2023-01-27 14:40:02 -08:00
Aliaksandr Valialkin	a0c8b86eab	docs/vmauth.md: update docs after ff39a91147 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3346	2023-01-27 14:10:19 -08:00
Aliaksandr Valialkin	ff39a91147	app/vmauth: limit the number of concurrent requests served by vmauth with the -maxConcurrentRequests command-line flag ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3346 This commit is based on the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3486	2023-01-27 14:07:30 -08:00
Aliaksandr Valialkin	372b1688d7	app/vmauth: do not use net/http/httputil.ReverseProxy ... This allows better controlling requests to backends and providing better error logging. For example, if the backend was unavailable, then the ReverseProxy was logging the error message without client ip and the initial request uri. This could harden debugging. This is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3486	2023-01-27 13:40:05 -08:00
Aliaksandr Valialkin	1b81d8f542	lib/netutil: move IsTrivialNetworkError() function there, since it is used in multiple places across the code	2023-01-27 13:24:30 -08:00
Aliaksandr Valialkin	7e355080ce	app/vmauth: pass the target url to reverse proxy via context.Value instead of request header ... This is less hacky way, since it doesn't clash with request headers	2023-01-27 12:15:52 -08:00
Aliaksandr Valialkin	6faaefef7b	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-27 11:38:43 -08:00
Aliaksandr Valialkin	36941d6d75	app/vmauth: consistency renaming: UserInfo.URLMap -> UserInfo.URLMaps ... This is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3486	2023-01-27 00:19:02 -08:00
Aliaksandr Valialkin	5cd89aaaa1	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-27 00:05:34 -08:00
Aliaksandr Valialkin	0890adde67	docs: update command-line descriptions after 73256fe438	2023-01-27 00:00:37 -08:00
Aliaksandr Valialkin	3a21fde0f3	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-26 23:54:43 -08:00
Aliaksandr Valialkin	cb374677a9	app/vmagent/prometheusimport: delete the temporary directory created by vmagent after the test is complete ... This is a follow-up for 1cfa183c2b	2023-01-26 23:21:24 -08:00
Nikolay	73256fe438	lib/netutil: init implimentation of proxy protocol (#3687) ... * lib/netutil: init implimentation of proxy protocol https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3335 * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-26 23:08:35 -08:00
Roman Khavronenko	95ee86b600	docs: specify the time window for series_limit (#3708) ... Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-01-25 09:30:20 -08:00
Aliaksandr Valialkin	274627943e	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-25 09:23:08 -08:00
Aliaksandr Valialkin	28f66f0079	docs: update the list of command-line flags according to the latest changes	2023-01-25 09:20:24 -08:00
Aliaksandr Valialkin	d655d6b047	lib/streamaggr: add ability to de-duplicate input samples before aggregation	2023-01-25 09:14:49 -08:00
Aliaksandr Valialkin	21140318cc	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-24 09:33:54 -08:00
Yury Molodov	99d49e3ceb	vmui: include fonts in its bundle (#3705) ... * feat: include fonts in the build * fix: reduce size fonts * wip - Document the change at docs/CHANGELOG.md - Run `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-24 09:30:56 -08:00
Yury Molodov	20ad848c5d	vmui: improvements to the UI styles (#3704) ... * feat: add dark theme * update packages * feat: add multilevel menu (#3678) * fix: correct styles * fix: update link to cardinality-explorer * fix: remove unused scss variables * docs/CHANGELOG.md: document the changes Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-24 09:20:31 -08:00
Aliaksandr Valialkin	c496f06ca3	app/vmagent/{promremotewrite,vmimport}: remove unused functions InsertHandlerForReader() ... Thanks to 1cfa183c2b , where the first such function has been removed	2023-01-23 22:31:29 -08:00
Aliaksandr Valialkin	f7acdb13db	app/{vmagent,vminsert}: follow-up for 1cfa183c2b ... - Call httpserver.GetQuotedRemoteAddr() and httpserver.GetRequestURI() only when the error occurs. This saves CPU time on fast path when there are no parsing errors. - Create a helper function - httpserver.LogError() - for logging the error with the request uri and remote addr context.	2023-01-23 22:26:53 -08:00
Artem Navoiev	1cfa183c2b	add error handler for parsing prometheus text format to vmagent and v… (#3693) ... * add error handler for parsing prometheus text format to vmagent and vminsert Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * fix typo Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * typo Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * fix variables naming and error message Signed-off-by: Artem Navoiev <tenmozes@gmail.com> Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2023-01-23 22:14:34 -08:00
Yury Molodov	3536bef36e	vmui: add open graph and twitter card tags (#3697) ... * feat: add open graph and twitter card tags * app/vmui: spelling fixes Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-23 22:04:46 -08:00
Aliaksandr Valialkin	30e968df6d	app/vmselect: use consistent randomizer in tests ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3683	2023-01-23 19:27:25 -08:00
Aliaksandr Valialkin	f2b40dbe9a	app/vmalert: use consistent randomizer in tests ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3683	2023-01-23 19:25:10 -08:00
Artem Navoiev	7c1daade15	tests: use DebugFlush instead of vmstorage stop. This simplifies the logic and allows to remove test-only methodds (#3694) ... Signed-off-by: Artem Navoiev <tenmozes@gmail.com> Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2023-01-23 14:45:59 +01:00
Aliaksandr Valialkin	74df30456b	app/vmselect: make vmui-update after df7b81b44d	2023-01-20 12:07:13 -08:00
Yury Molodov	df7b81b44d	vmui: add support for time zone selection for older versions of browsers (#3680) ... * fix: add check for support of getting time zones * vmui: add support for time zone selection for older versions of browsers	2023-01-20 11:47:53 -08:00
Aliaksandr Valialkin	a5975c31c2	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-18 12:02:26 -08:00
Tobias Jungel	777038fe44	app/vmbackup: prevent password leaks (#3672) ... This prevents vmbackup from leaking passwords into logs like shown below. 2023-01-11T15:00:01.050Z info VictoriaMetrics/lib/logger/flag.go:12 build version: vmbackup-20221214-211706-tags-v1.85.1-0-g09a70d3e9 2023-01-11T15:00:01.050Z info VictoriaMetrics/lib/logger/flag.go:13 command-line flags 2023-01-11T15:00:01.050Z info VictoriaMetrics/lib/logger/flag.go:20 -dst="fs:///vm-backups/latest" 2023-01-11T15:00:01.050Z info VictoriaMetrics/lib/logger/flag.go:20 -snapshot.createURL="http://user:super_sercret123@victoriametricspshot/create" 2023-01-11T15:00:01.050Z info VictoriaMetrics/lib/logger/flag.go:20 -storageDataPath="/storage" 2023-01-11T15:00:01.050Z info VictoriaMetrics/app/vmbackup/main.go:53 Snapshot create url http://user:super_sercret123@victoriametrics:8428/snapshot/create 2023-01-11T15:00:01.050Z info VictoriaMetrics/app/vmbackup/main.go:60 Snapshot delete url http://user:super_sercret123@victoriametrics:8428/snapshot/delete	2023-01-18 11:35:21 -08:00
Aliaksandr Valialkin	fad61eafc1	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-18 01:42:05 -08:00
Aliaksandr Valialkin	e867df5ef5	app/vmui: increase perceived performance by 2.5x by reducing the delay before the query execution from 0.8s to 0.3s ... The delay cannot be removed, since it is used for limiting the rate of queries sent to VictoriaMetrics during graph scrolling.	2023-01-18 01:33:48 -08:00
Aliaksandr Valialkin	7737321133	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-18 00:01:57 -08:00
Aliaksandr Valialkin	0c625185cb	app/vmselect/promql: updates tests for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3664	2023-01-17 23:25:45 -08:00
Aliaksandr Valialkin	a2ab1f0ec9	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-17 21:49:03 -08:00
Yury Molodov	fcef2ff6b2	vmui: correctly display range results in Table view (#3657) ... * fix: properly display range results * fix: set range values to empty array * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-17 21:03:28 -08:00
Yury Molodov	e58921aa8f	vmui: give more visually different colors to graph lines (#3656) ... * feat: make more different colors of graph lines * docs/CHANGELOG.md: give more visually different colors to graph lines	2023-01-17 20:25:37 -08:00
Aliaksandr Valialkin	c94020f7dc	app/vmui: do not round the number in formatPrettyNumber() if the range isn't set	2023-01-17 19:50:44 -08:00
Aliaksandr Valialkin	09d7fa2737	lib/{mergeset,storage}: do not slow down concurrently executed queries during assisted merges ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3647 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3641 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/291	2023-01-16 14:31:52 -08:00
Yury Molodov	04c408e986	vmui: make the step input field global across all the tabs and views (#3644) ... * feat: make the step input field global * fix: correct get step from url * fix: set minimumSignificantDigits to 1 * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-15 13:47:08 -08:00
Aliaksandr Valialkin	cdb6d651e9	app/vminsert: return 200 OK status code when importing data in pushgateway format ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1415 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636	2023-01-15 13:29:53 -08:00
Aliaksandr Valialkin	207a62a3c2	app/vmselect/promql: reduce memory allocations when searching for time series pairs with identical labelsets in q1 op q2 queries ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3641	2023-01-15 13:03:23 -08:00
Aliaksandr Valialkin	27afe7bc38	app/vmselect/promql: reduce the number of memory allocations inside getCommonLabelFilters() ... This should improve performance a bit for `q1 op q2` queries	2023-01-15 13:03:23 -08:00
Yury Molodov	7fd82c0d3a	feat: make nav menu as links (#3646)	2023-01-13 09:45:07 +01:00
Aliaksandr Valialkin	be8fba9b6a	app/vmselect/netstorage: tune the number of blocks per series which should be unpacked by a single goroutine instead of spinning up multiple goroutines ... This reduces overhead on time series data unpacking for typical cases, this reducing CPU usage at vmselect	2023-01-12 09:31:44 -08:00
Aliaksandr Valialkin	7067e8206c	app/vmselect/promql: reduce memory allocations at getCommonLabelFilters() function ... Intern tag keys and values there	2023-01-12 01:27:41 -08:00
Aliaksandr Valialkin	a092df3f84	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-12 01:13:39 -08:00
Aliaksandr Valialkin	9f5b5708ff	app/vmselect: handle the /custom-dashboards request from /graph/ page in the same way as from the /vmui/ page ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3322	2023-01-11 23:41:04 -08:00
Aliaksandr Valialkin	a194982117	app/vmselect: follow-up after 820312a2b1 ... - Move the feature description at the correct place at docs/CHANGELOG.md - Run `make vmui-update` - Various cosmetic fixes Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3322	2023-01-11 23:28:00 -08:00
Dmytro Kozlov	820312a2b1	app/vmui: define custom path for dashboards json file (#3545) ... * app/vmui: define custom path for dashboards json file * app/vmui: remove unneeded code * app/vmui: move handler to own file, fix show dashboards, * app/vmui: move flag to handler, add flag description * app/vmauth: fix part of the comments * feat: add store for dashboards * fix: prevent fetch dashboards for app mode * app/vmauth: use simple cache for predefined dashboards * app/vmauth: update dashboards doc * app/vmauth: fix ci * app/vmui: decrease timeout * app/vmselect: removed cache, fix comments * app/vmselect: remove unused const * app/vmselect: fix error log, use slice byte instead of struct Co-authored-by: Yury Moladau <yurymolodov@gmail.com>	2023-01-11 23:06:07 -08:00
Dmytro Kozlov	20af29294e	app/vmctl: add remote read protocol integration tests (#3626)	2023-01-11 23:00:10 -08:00
Yury Molodov	5c8bb029b5	vmui: small changes on explore metrics page (#3634) ... * fix: change issue link * fix: remove legend toggle * fix: move select graph size * feat: save url params on explore metrics page * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-11 22:16:10 -08:00
Aliaksandr Valialkin	c3f178aa53	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-11 01:34:37 -08:00
Aliaksandr Valialkin	dff47c73b7	app/vmselect: improve logging when the incoming query cannot be executed because of timeout in the wait queue	2023-01-11 01:06:05 -08:00
Aliaksandr Valialkin	65b9dcfcca	app/vmselect/promql: typo fix after 0771d57860	2023-01-11 01:05:31 -08:00
Aliaksandr Valialkin	0771d57860	app/vmselect/promql: make a copy of per-series timestamps before their modification ... The per-series timestamps are usually shared among series, so it is unsafe modifying them. The issue has been appeared after the optimization at 2f3ddd4884	2023-01-11 00:59:13 -08:00
Aliaksandr Valialkin	31fc29599f	app/vmselect/promql: move the eval function args in parallel query trace outside the loop	2023-01-10 22:23:30 -08:00
Aliaksandr Valialkin	c449714c0a	deployment/docker: update Go builder from v1.19.4 to v1.19.5 ... See https://github.com/golang/go/issues?q=milestone%3AGo1.19.5+label%3ACherryPickApproved	2023-01-10 18:43:04 -08:00
Aliaksandr Valialkin	393e7636be	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-10 16:23:24 -08:00
Aliaksandr Valialkin	2d294cca59	deployment/docker: update Alpine base image from v3.17.0 to v3.17.1 ... See https://alpinelinux.org/posts/Alpine-3.17.1-released.html	2023-01-10 16:14:40 -08:00
Aliaksandr Valialkin	ebc200846c	Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files	2023-01-10 16:11:27 -08:00
Aliaksandr Valialkin	95ce1ba6ce	lib/httpserver: directly pass flag value to CheckAuthFlag() ... There is no sense in passing a pointer to flag value there. This is a follow-up for 4225a0bd75	2023-01-10 15:52:23 -08:00
Zakhar Bessarab	4225a0bd75	Use httpAuth.* flags as a fallback for endpoints protected by *AuthKey flags (#3582) ... * {lib/server, app/}: use `httpAuth.*` flag as fallback for `*AuthKey` if it is not set * lib/ingestserver/opentsdbhttp: fix opentdb HTTP handler not respecting `httpAuth.*` flags * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-10 15:46:13 -08:00
Dmytro Kozlov	0811000bb0	app/vmctl: Add insecure skip verify flag for remote read protocol (#3611) ... * app/vmctl: Add insecure skip verify flag for remote read protocol	2023-01-10 23:18:49 +01:00
Artem Navoiev	37c52ccaf4	vmagent: add minimal scrape file exampe for vmagent quick start (#3627) ... * vmagent: add minimal scrape file exampe for vmagent quick start Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * replace example with link to your prometheus.yml in docker Signed-off-by: Artem Navoiev <tenmozes@gmail.com> Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2023-01-10 23:16:10 +01:00
Aliaksandr Valialkin	53d871d0b1	app/vmselect/netstorage: reduce tail latency during query processing ... Previously the selected time series were split evenly among available CPU cores for further processing - e.g unpacking the data and applying the given rollup function to the unpacked data. Some time series could be processed slower than others. This could result in uneven work distribution among available CPU cores, e.g. some CPU cores could complete their work sooner than others. This could slow down query execution. The new algorithm allows stealing time series to process from other CPU cores when all the local work is done. This should reduce the maximum time needed for query execution (aka tail latency). The new algorithm should also scale better on systems with many CPU cores, since every CPU processes locally assigned time series without inter-CPU communications. The inter-CPU communications are used only when all the local work is finished and the pending work from other CPUs needs to be stealed.	2023-01-10 13:43:14 -08:00
Aliaksandr Valialkin	e640ff72f1	app/vmselect/netstorage: reduce memory allocations when unpacking time series ... Unpack time series with less than 400K samples in the currently running goroutine. Previously a new goroutine was being started for unpacking the samples. This was requiring additional memory allocations.	2023-01-09 23:18:17 -08:00
Aliaksandr Valialkin	9a563a6aef	app/vmselect/promql: eliminate memory allocation when sorting values inside float64s	2023-01-09 23:06:46 -08:00
Aliaksandr Valialkin	30ed33fae0	app/vmselect/promql: pre-allocate memory for values to be merged in mergeTimeseries() ... This should reduce the number of memory re-allocations	2023-01-09 22:51:17 -08:00
Aliaksandr Valialkin	645c24dc5f	app/vmselect/promql: consistently intern series names obtained from marshalMetricNameSorted ... This reduces memory allocations when the returned series names are used as map keys later	2023-01-09 22:45:40 -08:00
Aliaksandr Valialkin	2f3ddd4884	app/vmselect/promql: avoid memory allocations and copying from source timeseries to the returned result at timeseriesToResult()	2023-01-09 22:38:59 -08:00
Aliaksandr Valialkin	26cf680468	app/vmselect/promql: remove memory allocations from sortMetricTags()	2023-01-09 22:22:15 -08:00
Aliaksandr Valialkin	4f0c11ee93	app/vmselect/promql: intern output series names inside timeseriesToResult() ... This reduces the number of memory allocations for repeated queries, which return (almost) the same set of time series.	2023-01-09 22:19:56 -08:00
Aliaksandr Valialkin	562d6bca08	app/vmselect/promql: intern output series names during normal aggregation	2023-01-09 22:15:24 -08:00
Aliaksandr Valialkin	21ee9a1fab	app/vmselect/promql: intern output series names during incremental aggregation ... This should reduce the number of memory allocations for repeated queries	2023-01-09 22:11:36 -08:00
Aliaksandr Valialkin	df2a494a7c	app/vmselect/netstorage: pre-allocate 4 block references per each time series during querying ... Usually the number of blocks returned per each time series during queries is around 4. So it is a good idea to pre-allocate 4 block references per time series in order to reduce the number of memory allocations.	2023-01-09 22:03:23 -08:00
Aliaksandr Valialkin	c5e0f527bc	app/vmselect/netstorage: cache canonical MetricName for time series returned from the storage ... This reduces memory allocations for repeated queries, which return (almost) the same set of time series.	2023-01-09 21:53:10 -08:00
Aliaksandr Valialkin	7afcca0c51	all: use metricsql.CompileRegexp instead of regexp.Compile for compiling regexps used in graphite queries ... This should speed up repeated queries, since metricsql.CompileRegexp returns regexps from the cache on subsequent calls for the same input regexp.	2023-01-09 21:43:08 -08:00
Aliaksandr Valialkin	c38a10e143	app/vmselect/netstorage: eliminate memory allocation for sortBlocksHeap arg when calling mergeSortBlocks()	2023-01-09 21:08:51 -08:00
Aliaksandr Valialkin	1f9d605988	app/vmselect/netstorage: consistently select the sample with the biggest value out of samples with identical timestamps ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3333 This fix is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3620 , but doesn't slow down the common case with merging replicated data blocks so significantly. Benchmark results: Before the change: BenchmarkMergeSortBlocks/replicationFactor-1-4 13968 85643 ns/op 956.53 MB/s 1700 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-2-4 10806 109171 ns/op 1500.77 MB/s 2191 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-3-4 8887 130623 ns/op 1881.45 MB/s 2660 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-4-4 7440 157348 ns/op 2082.52 MB/s 3174 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-5-4 6534 184473 ns/op 2220.38 MB/s 3612 B/op 1 allocs/op BenchmarkMergeSortBlocks/overlapped-blocks-bestcase-4 13419 85205 ns/op 961.44 MB/s 2213 B/op 1 allocs/op BenchmarkMergeSortBlocks/overlapped-blocks-worstcase-4 579 1894900 ns/op 43.23 MB/s 46760 B/op 1 allocs/op After the change: BenchmarkMergeSortBlocks/replicationFactor-1-4 13832 85298 ns/op 960.40 MB/s 1716 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-2-4 8833 134222 ns/op 1220.66 MB/s 2675 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-3-4 6487 184830 ns/op 1329.65 MB/s 3636 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-4-4 4977 236318 ns/op 1386.61 MB/s 4733 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-5-4 4088 296734 ns/op 1380.36 MB/s 5761 B/op 1 allocs/op BenchmarkMergeSortBlocks/overlapped-blocks-bestcase-4 14083 84067 ns/op 974.47 MB/s 2110 B/op 1 allocs/op BenchmarkMergeSortBlocks/overlapped-blocks-worstcase-4 536 2043534 ns/op 40.09 MB/s 50511 B/op 1 allocs/op	2023-01-09 13:01:48 -08:00
Aliaksandr Valialkin	af263fe881	all: small improvements in error messages and command-line flag descriptions related to concurrency limiters	2023-01-07 00:11:44 -08:00
Aliaksandr Valialkin	293e4dc77b	app/{vminsert,vmstorage}: add comments on why storage.AddRows() is called without limiting the number of concurrent calls	2023-01-06 22:40:07 -08:00
Aliaksandr Valialkin	c63755c316	lib/writeconcurrencylimiter: improve the logic behind -maxConcurrentInserts limit ... Previously the -maxConcurrentInserts was limiting the number of established client connections, which write data to VictoriaMetrics. Some of these connections could be idle. Such connections do not consume big amounts of CPU and RAM, so there is a little sense in limiting the number of such connections. So now the -maxConcurrentInserts command-line option limits the number of concurrently executed insert requests, not including idle connections. It is recommended removing -maxConcurrentInserts command-line option, since the default value for this option should work good for most cases.	2023-01-06 22:20:19 -08:00
Aliaksandr Valialkin	e7637885a6	app/vmselect: improve error message when the request cannot be started because too many concurrent requests are already executed	2023-01-06 22:10:42 -08:00
Karan Sharma	48f371a46c	lib/promscrape: add Prometheus-compatible service discovery for Nomad (#3549) ... Add nomad_sd_config support for service discovery	2023-01-05 23:03:58 +01:00
Zakhar Bessarab	01bc0c94ab	doc: add vmbackupmanager monitoring section (#3605) ... * doc: add vmbackupmanager monitoring section Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>	2023-01-05 16:03:06 +01:00
Aliaksandr Valialkin	0dea3b71da	lib/promscrape: pre-fetch metric_relabel_configs rules when debugging metric relabeling for a particular target ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407	2023-01-05 03:26:49 -08:00
Yury Molodov	2460e0f51e	vmui: improve Explore metrics (#3598) ... * feat: add multiple select * feat: improve explore interface * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-05 02:23:04 -08:00
Aliaksandr Valialkin	04dff34de4	vendor: update github.com/VictoriaMetrics/metricsql from v0.50.0 to v0.51.0 ... Updates https://github.com/VictoriaMetrics/metricsql/pull/7 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3589	2023-01-05 01:50:10 -08:00
Aliaksandr Valialkin	fd175ad80b	docs: update -help outputs for vm* tools	2023-01-03 23:27:06 -08:00
Aliaksandr Valialkin	fa13bbc48a	app/{vmagent,vminsert}: add support for streaming aggregation ... See https://docs.victoriametrics.com/stream-aggregation.html Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3460	2023-01-03 22:19:21 -08:00
Aliaksandr Valialkin	f33e687723	app/vmagent/remotewrite: improve descriptions for -remoteWrite.relabelConfig and -remoteWrite.urlRelabelConfig ... - Cross-reference these command-line flags. - Add a link to https://docs.victoriametrics.com/vmagent.html#relabeling	2023-01-03 22:04:49 -08:00
Aliaksandr Valialkin	6db5c3801e	app/vmbackup: remove superflouos whitespace after 8fe21ec707	2023-01-03 21:52:40 -08:00
yanggang	8fe21ec707	Fix flag help message for the backups types. (#3577) ... Signed-off-by: yanggang <gang.yang@daocloud.io>	2023-01-03 10:56:42 +01:00
yanggang	fa842d6534	fix typo for json values. (#3576) ... Signed-off-by: yanggang <gang.yang@daocloud.io>	2023-01-03 10:55:38 +01:00
yanggang	93e935bcaa	Fix vmctl command hint for vm-native-step-interval (#3575) ... Signed-off-by: yanggang <gang.yang@daocloud.io>	2023-01-03 10:54:53 +01:00
Roman Khavronenko	4b3d8eb573	vmalert: mention specifics of Alertmanager HA mode (#3573) ... Stress the importance of specifying of all Alertmanager URLs in vmalert's `-notifier.url` or `notifier.config` if it runs in cluster mode. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3547 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-12-30 09:54:03 +01:00
Aliaksandr Valialkin	05fa11b296	app/vmui: reuse the series color in line tooltip	2022-12-29 15:32:41 -08:00
Aliaksandr Valialkin	1794f3d46e	app/vmui: small usability improvements ... - Show in the line tooltip the number of the query which generates the given line. This simplifies comparison of lines generated by multiple queries. - Show metric name as __name__ label in the line tooltip in the same way as other labels are shown there. This makes the label information in the tooltip more consistent. - Properly quote label values with JSON.stringify(). This prevents from improper formatting when label values contain doublequote chars. - Remove double curly braces artifact at graph legend for lines without names and labels. - Properly use modifier for regular expressions across the code.	2022-12-29 14:52:51 -08:00
Aliaksandr Valialkin	b8bc62431a	app/vmselect/vmui: make vmui-update after 1720bddb4f	2022-12-29 12:20:06 -08:00
Yury Molodov	1720bddb4f	fix: display correct graph tooltip title (#3562)	2022-12-29 12:06:48 -08:00
Roman Khavronenko	83870aeb8d	tests: attempt to fix flaky graphite test (#3567) ... Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-12-29 11:48:47 -08:00
ChenyuanHu	0a9e1d64fe	app/vmselect/prometheus: no need manually call queryDuration.UpdateDuration (#3564) ... There is no need to manually call `queryDuration.UpdateDuration(startTime)`, because `defer queryDuration.UpdateDuration(startTime)` is executed at the beginning of the function(L660).	2022-12-29 14:18:00 +01:00
Roman Khavronenko	6588fcbfca	vmalert: allow configuring the default number of stored rule's update states (#3556) ... Allow configuring the default number of stored rule's update states in memory via global `-rule.updateEntriesLimit` command-line flag or per-rule via rule's `update_entries_limit` configuration param. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-12-29 12:36:44 +01:00
Aliaksandr Valialkin	3dc684634e	app/vmselect/searchutils: accept partial RFC3339 values at time, start and end query args ... This simplifies manual usage of the APIs. For example, the following query would return the results over the 2022 year. /api/v1/query_range?start=2022&end=2023&step=1d&query=... This is equivalent to: /api/v1/query_range?start=2022-01-01T00:00:00Z&end=2023-01-01T00:00:00Z&step=1d&query=...	2022-12-28 19:41:54 -08:00
Yury Molodov	d2f89b55b7	vmui: fix step field (#3561) ... * feat: use a unit next to the step value * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-12-28 16:00:51 -08:00
Artem Navoiev	7d9c4bebc0	update links to grafana dashboards (#3534) ... docs: update links to grafana dashboards Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2022-12-25 17:36:20 +01:00
Aliaksandr Valialkin	b11c806d1c	app/vmui: show min, max and avg lines at Explore metrics graphs when instance is selected in the same way as when only the job is selected ... This improves consistency of the graphs. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3386	2022-12-23 23:21:11 -08:00
Aliaksandr Valialkin	2a7e392bb3	app/vmselect/vmui: make vmui-update after 0dca224ec3	2022-12-23 22:25:04 -08:00
Aliaksandr Valialkin	0dca224ec3	app/vmui: small improvements for header panel ... - Rename `Custom panel` tab to more clear `Query` tab - Rename `Cardinality` tab to `Explore cardinality`, so it becomes consistent with `Explore metrics` tab - Move `Dashboards` tab to the end, since it isn't used too much	2022-12-23 22:24:31 -08:00