github-mirrors/VictoriaMetrics

mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-11 14:53:49 +00:00

Author	SHA1	Message	Date
Alexander Marshalov	808a06bfd4	fix interpolate function for filling only intermediate gaps (#3816 ) (#3857 ) * fix interpolate function for filling only intermediate gaps (#3816) * Update docs/CHANGELOG.md --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-23 19:30:23 -08:00
Aliaksandr Valialkin	cc2a82a7dc	all: update Go builder from Go1.20.0 to Go1.20.1 See https://github.com/golang/go/issues?q=milestone%3AGo1.20.1+label%3ACherryPickApproved	2023-02-14 23:13:32 -08:00
Aliaksandr Valialkin	49f69a7068	all: update alpine base docker image from 1.17.1 to 1.17.2 See https://alpinelinux.org/posts/Alpine-3.17.2-released.html	2023-02-11 12:16:14 -08:00
Aliaksandr Valialkin	34379d4cf1	all: run `apk update && apk upgrade` in base Alpine Docker image in order to get all the recent security fixes	2023-02-09 14:03:02 -08:00
Aliaksandr Valialkin	82b5fa2fd0	app/vmui: UX enhancements for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3706 - Display `min` value additionally to `avg`, `max` and `last` - Allow copy-n-pasting metric name with its labels from both legend and tooltup	2023-02-09 11:05:57 -08:00
Aliaksandr Valialkin	dc7b92bc34	docs/vmagent.md: clarify that automatically generated metrics contain all the target-specific labels, including `instance` and `job`	2023-02-09 11:05:57 -08:00
Yury Molodov	13f55fc21e	vmui: lazy loading predefined panels (#3795 ) * fix: change logic lazy loading predefined panels * app/vmselect/vmui: `make vmui-update` --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-09 00:12:29 -08:00
Yury Molodov	5a7c63a324	vmui: improve tenant selector (#3794 ) * fix: change styles tenant selector (#3792) * docs/CHANGELOG.md: document the change Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3792 --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-09 00:09:36 -08:00
Yury Molodov	54bfd22ec5	vmui: add last/max/avg values (#3789 ) * feat: add last/max/avg values (#3706) * fix: change filter exclude values * app/vmui: wip - improve the visualization for avg/max/last values - make getAvgFromArray() function resilient against inf/undefined/nil - export getLastFromArray() function, which is resilient against inf/undefined/nil - run `make vmui-update` --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-08 22:42:05 -08:00
Yury Molodov	0454f9489c	fix: turn off the local dashboards(#3740 ) (#3793 )	2023-02-08 11:24:28 -08:00
Roman Khavronenko	4e922eb93b	Vmalert fixes (#3788 ) * vmalert: use group's ID in UI to avoid collisions Identical group names are allowed. So we should used IDs for various groupings and aggregations in UI. Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: prevent disabling state updates tracking The minimum number of update states to track is now set to 1. Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: properly update `debug` and `update_entries_limit` params on hot-reload Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: display `debug` field for rule in UI Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: exclude `updates` field from json marhsaling This field isn't correctly marshaled right now. And implementing the correct marshaling for it doesn't seem right, since json representation is mostly used by systems like Grafana. And Grafana doesn't expect this field to be present. Signed-off-by: hagen1778 <roman@victoriametrics.com> * fix test for disabled state Signed-off-by: hagen1778 <roman@victoriametrics.com> * fix test for disabled state Signed-off-by: hagen1778 <roman@victoriametrics.com> --------- Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-02-08 08:45:25 -08:00
Roman Khavronenko	80bf0bcf8c	vmalert: update docs (#3770 ) vmalert: update flags description Signed-off-by: hagen1778 <roman@victoriametrics.com> --------- Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-02-07 09:28:59 -08:00
Roman Khavronenko	96db7ac52c	vmalert: speed up state restore procedure on start (#3758 ) * vmalert: speed up state restore procedure on start Alerts state restore procedure has been changed to become asynchronous. It doesn't block groups start anymore which significantly improves vmalert's startup time. Instead, state restore is called by each group in their goroutines after the first rules evaluation. While previously state restore attempt was made for all loaded alerting rules, now it is called only for alerts which became active after the first evaluation. This reduces the amount of API calls to the configured remote read URL. This also means that `remoteRead.ignoreRestoreErrors` command-line flag becomes deprecated now and will have no effect if configured. See relevant issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2608 Signed-off-by: hagen1778 <roman@victoriametrics.com> * make lint happy Signed-off-by: hagen1778 <roman@victoriametrics.com> * Apply suggestions from code review --------- Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-02-03 19:46:41 -08:00
Aliaksandr Valialkin	c9a32ebaf7	app/vmselect/vmui: `make vmui-update` after `e4c04b6dbe`	2023-02-03 19:34:20 -08:00
Yury Molodov	ccdf388094	vmui: set light theme for app mode (#3748 ) * fix: set light theme for app mode * fix: check inputTenantID flag * fix: rename inputTenantID to useTenantID	2023-02-03 19:34:18 -08:00
Yury Molodov	05873dee89	vmui: mobile view (#3742 ) * feat: add detect the system theme * fix: change logic fetch tenants * feat: add docs and info to cardinality page * feat: add mobile view #3707	2023-02-03 19:30:38 -08:00
Aliaksandr Valialkin	2d1ea90bcb	vendor: `make vendor-update`	2023-02-01 12:48:10 -08:00
Dmytro Kozlov	b7cc0bfe3c	app/vmctl: Add insecure skip verify flag for remote read protocol (#3611 ) * app/vmctl: Add insecure skip verify flag for remote read protocol	2023-02-01 12:45:11 -08:00
Zakhar Bessarab	326a702559	app/vmbackup: fix deleting snapshot after backup completion (#3735 ) (#3736 ) Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>	2023-02-01 08:58:11 -08:00
Zakhar Bessarab	626bd22157	fix: vmselect multi-level setup panic (#3738 ) * app/vmselect/netstorage: fix panic for multi-level cluster setup when `replicationFactor` was set and request contained `trace` parameter (#3734) Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> * app/vmselect/netstorage: use correct context for retry Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> --------- Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>	2023-02-01 08:56:36 -08:00
Aliaksandr Valialkin	2049114e1f	app/vmselect/vmui: `make vmui-update` after `dcc5616126`	2023-01-31 13:24:54 -08:00
Yury Molodov	89c4982658	vmui: improvement the theme (#3731 ) * feat: add detect the system theme * fix: change logic fetch tenants * feat: add docs and info to cardinality page --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-31 12:55:44 -08:00
Roman Khavronenko	d93ac2b1ea	docs: mention `-vmalert.proxyURL` in vmalert docs (#3730 ) Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-01-31 10:49:49 -08:00
Yury Molodov	730025d1dc	vmui: add select of Tenant ID (#3673 ) * feat: add select of tenantID * feat: replace tenantID to default url * fix: move the tenantID selector to the top header * fix: hide tenantID selector by condition * fix: correct z-index * app/vmselect/vmui: `make vmui-update` --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-27 15:54:09 -08:00
Aliaksandr Valialkin	cd25945ff6	app/vmagent: properly return 200 response code when importing data via Prometheus PushGateway protocol This is the same fix as has been already applied to app/vminsert at `cdb6d651e9` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1415	2023-01-27 14:40:26 -08:00
Aliaksandr Valialkin	8838c07360	docs/vmauth.md: update docs after `ff39a91147` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3346	2023-01-27 14:10:29 -08:00
Aliaksandr Valialkin	18bf18b8b9	app/vmauth: limit the number of concurrent requests served by `vmauth` with the `-maxConcurrentRequests` command-line flag See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3346 This commit is based on the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3486	2023-01-27 14:07:53 -08:00
Aliaksandr Valialkin	27ec56ffb1	app/vmauth: do not use net/http/httputil.ReverseProxy This allows better controlling requests to backends and providing better error logging. For example, if the backend was unavailable, then the ReverseProxy was logging the error message without client ip and the initial request uri. This could harden debugging. This is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3486	2023-01-27 13:40:53 -08:00
Aliaksandr Valialkin	bccbe07c33	lib/netutil: move IsTrivialNetworkError() function there, since it is used in multiple places across the code	2023-01-27 13:24:44 -08:00
Aliaksandr Valialkin	341026902e	app/vmauth: pass the target url to reverse proxy via context.Value instead of request header This is less hacky way, since it doesn't clash with request headers	2023-01-27 12:16:55 -08:00
Aliaksandr Valialkin	7dc38fba72	app/vmauth: consistency renaming: UserInfo.URLMap -> UserInfo.URLMaps This is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3486	2023-01-27 00:19:46 -08:00
Aliaksandr Valialkin	4cf4c307ea	docs: update command-line descriptions after `73256fe438`	2023-01-27 00:01:14 -08:00
Aliaksandr Valialkin	1a1f3e7149	app/vmagent/prometheusimport: delete the temporary directory created by vmagent after the test is complete This is a follow-up for `1cfa183c2b`	2023-01-26 23:25:34 -08:00
Nikolay	ebebaecd94	lib/netutil: init implimentation of proxy protocol (#3687 ) * lib/netutil: init implimentation of proxy protocol https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3335 * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-26 23:25:22 -08:00
Roman Khavronenko	29ea196bf0	docs: specify the time window for `series_limit` (#3708 ) Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-01-25 09:31:09 -08:00
Aliaksandr Valialkin	bd809db4d9	docs: update the list of command-line flags according to the latest changes	2023-01-25 09:22:23 -08:00
Aliaksandr Valialkin	5defa99a2e	lib/streamaggr: add ability to de-duplicate input samples before aggregation	2023-01-25 09:22:03 -08:00
Yury Molodov	29fd95d426	vmui: include fonts in its bundle (#3705 ) * feat: include fonts in the build * fix: reduce size fonts * wip - Document the change at docs/CHANGELOG.md - Run `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-24 09:31:37 -08:00
Yury Molodov	97aa9b03de	vmui: improvements to the UI styles (#3704 ) * feat: add dark theme * update packages * feat: add multilevel menu (#3678) * fix: correct styles * fix: update link to cardinality-explorer * fix: remove unused scss variables * docs/CHANGELOG.md: document the changes Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-24 09:21:20 -08:00
Aliaksandr Valialkin	417b90b8ad	app/vmagent/{promremotewrite,vmimport}: remove unused functions InsertHandlerForReader() Thanks to `1cfa183c2b` , where the first such function has been removed	2023-01-23 22:42:46 -08:00
Aliaksandr Valialkin	4b3a207705	app/{vmagent,vminsert}: follow-up for `1cfa183c2b` - Call httpserver.GetQuotedRemoteAddr() and httpserver.GetRequestURI() only when the error occurs. This saves CPU time on fast path when there are no parsing errors. - Create a helper function - httpserver.LogError() - for logging the error with the request uri and remote addr context.	2023-01-23 22:41:08 -08:00
Artem Navoiev	0ac0cfdc69	add error handler for parsing prometheus text format to vmagent and v… (#3693 ) * add error handler for parsing prometheus text format to vmagent and vminsert Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * fix typo Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * typo Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * fix variables naming and error message Signed-off-by: Artem Navoiev <tenmozes@gmail.com> Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2023-01-23 22:36:23 -08:00
Yury Molodov	bad7213fcb	vmui: add open graph and twitter card tags (#3697 ) * feat: add open graph and twitter card tags * app/vmui: spelling fixes Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-23 22:33:42 -08:00
Aliaksandr Valialkin	6ff15ca135	app/vmselect: use consistent randomizer in tests Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3683	2023-01-23 19:27:40 -08:00
Aliaksandr Valialkin	ef7683f2e0	app/vmalert: use consistent randomizer in tests Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3683	2023-01-23 19:25:32 -08:00
Aliaksandr Valialkin	b046af8a4d	app/vmselect: `make vmui-update` after `df7b81b44d`	2023-01-20 12:07:29 -08:00
Yury Molodov	57f7ffca5b	vmui: add support for time zone selection for older versions of browsers (#3680 ) * fix: add check for support of getting time zones * vmui: add support for time zone selection for older versions of browsers	2023-01-20 12:07:28 -08:00
Tobias Jungel	8bdc63aab9	app/vmbackup: prevent password leaks (#3672 ) This prevents vmbackup from leaking passwords into logs like shown below. 2023-01-11T15:00:01.050Z info VictoriaMetrics/lib/logger/flag.go:12 build version: vmbackup-20221214-211706-tags-v1.85.1-0-g09a70d3e9 2023-01-11T15:00:01.050Z info VictoriaMetrics/lib/logger/flag.go:13 command-line flags 2023-01-11T15:00:01.050Z info VictoriaMetrics/lib/logger/flag.go:20 -dst="fs:///vm-backups/latest" 2023-01-11T15:00:01.050Z info VictoriaMetrics/lib/logger/flag.go:20 -snapshot.createURL="http://user:super_sercret123@victoriametricspshot/create" 2023-01-11T15:00:01.050Z info VictoriaMetrics/lib/logger/flag.go:20 -storageDataPath="/storage" 2023-01-11T15:00:01.050Z info VictoriaMetrics/app/vmbackup/main.go:53 Snapshot create url http://user:super_sercret123@victoriametrics:8428/snapshot/create 2023-01-11T15:00:01.050Z info VictoriaMetrics/app/vmbackup/main.go:60 Snapshot delete url http://user:super_sercret123@victoriametrics:8428/snapshot/delete	2023-01-18 11:40:52 -08:00
Aliaksandr Valialkin	46645f5d94	app/vmui: increase perceived performance by 2.5x by reducing the delay before the query execution from 0.8s to 0.3s The delay cannot be removed, since it is used for limiting the rate of queries sent to VictoriaMetrics during graph scrolling.	2023-01-18 01:33:51 -08:00
Aliaksandr Valialkin	e01f52d517	app/vmselect/promql: updates tests for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3664	2023-01-17 23:26:06 -08:00
Yury Molodov	0561ba3557	vmui: correctly display range results in Table view (#3657 ) * fix: properly display range results * fix: set range values to empty array * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-17 21:04:50 -08:00
Yury Molodov	6e033b93b0	vmui: give more visually different colors to graph lines (#3656 ) * feat: make more different colors of graph lines * docs/CHANGELOG.md: give more visually different colors to graph lines	2023-01-17 20:26:23 -08:00
Aliaksandr Valialkin	0498377837	app/vmui: do not round the number in formatPrettyNumber() if the range isn't set	2023-01-17 19:51:43 -08:00
Aliaksandr Valialkin	103dfd0525	lib/{mergeset,storage}: do not slow down concurrently executed queries during assisted merges Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3647 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3641 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/291	2023-01-16 14:45:40 -08:00
Yury Molodov	060780af69	vmui: make the step input field global across all the tabs and views (#3644 ) * feat: make the step input field global * fix: correct get step from url * fix: set minimumSignificantDigits to 1 * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-15 13:47:48 -08:00
Aliaksandr Valialkin	556484a52f	app/vminsert: return 200 OK status code when importing data in pushgateway format Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1415 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636	2023-01-15 13:26:09 -08:00
Yury Molodov	e37bac07a0	feat: make nav menu as links (#3646 )	2023-01-15 13:01:56 -08:00
Aliaksandr Valialkin	ce47faf102	app/vmselect/promql: reduce memory allocations when searching for time series pairs with identical labelsets in `q1 op q2` queries Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3641	2023-01-15 13:00:28 -08:00
Aliaksandr Valialkin	fe8802bbc8	app/vmselect/promql: reduce the number of memory allocations inside getCommonLabelFilters() This should improve performance a bit for `q1 op q2` queries	2023-01-15 12:56:21 -08:00
Aliaksandr Valialkin	26f6cfd3b2	app/vmselect/netstorage: tune the number of blocks per series which should be unpacked by a single goroutine instead of spinning up multiple goroutines This reduces overhead on time series data unpacking for typical cases, this reducing CPU usage at vmselect Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3641	2023-01-12 09:35:15 -08:00
Aliaksandr Valialkin	41b0b951f3	app/vmselect/netstorage: unpack series blocks in the current goroutine if their count doesnt exceed 100 This should improve performance a bit for common case	2023-01-12 01:31:38 -08:00
Aliaksandr Valialkin	d33a65e401	app/vmselect/promql: reduce memory allocations at getCommonLabelFilters() function Intern tag keys and values there	2023-01-12 01:27:34 -08:00
Aliaksandr Valialkin	820357f434	app/vmselect: follow-up after `820312a2b1` - Move the feature description at the correct place at docs/CHANGELOG.md - Run `make vmui-update` - Various cosmetic fixes Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3322	2023-01-11 23:37:51 -08:00
Dmytro Kozlov	f2862f088b	app/vmui: define custom path for dashboards json file (#3545 ) * app/vmui: define custom path for dashboards json file * app/vmui: remove unneeded code * app/vmui: move handler to own file, fix show dashboards, * app/vmui: move flag to handler, add flag description * app/vmauth: fix part of the comments * feat: add store for dashboards * fix: prevent fetch dashboards for app mode * app/vmauth: use simple cache for predefined dashboards * app/vmauth: update dashboards doc * app/vmauth: fix ci * app/vmui: decrease timeout * app/vmselect: removed cache, fix comments * app/vmselect: remove unused const * app/vmselect: fix error log, use slice byte instead of struct Co-authored-by: Yury Moladau <yurymolodov@gmail.com>	2023-01-11 23:33:30 -08:00
Dmytro Kozlov	99fbb2948b	app/vmctl: add remote read protocol integration tests (#3626 )	2023-01-11 23:03:17 -08:00
Yury Molodov	4295ca2ce2	vmui: small changes on explore metrics page (#3634 ) * fix: change issue link * fix: remove legend toggle * fix: move select graph size * feat: save url params on explore metrics page * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-11 22:17:06 -08:00
Artem Navoiev	e91814e332	vmagent: add minimal scrape file exampe for vmagent quick start (#3627 ) * vmagent: add minimal scrape file exampe for vmagent quick start Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * replace example with link to your prometheus.yml in docker Signed-off-by: Artem Navoiev <tenmozes@gmail.com> Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2023-01-11 01:23:07 -08:00
Aliaksandr Valialkin	f7130d571d	app/vmselect: improve logging when the incoming query cannot be executed because of timeout in the wait queue	2023-01-11 01:12:25 -08:00
Aliaksandr Valialkin	675e0fa0ee	app/vmselect/promql: typo fix after `0771d57860`	2023-01-11 01:06:18 -08:00
Aliaksandr Valialkin	3d22532bb8	app/vmselect/promql: make a copy of per-series timestamps before their modification The per-series timestamps are usually shared among series, so it is unsafe modifying them. The issue has been appeared after the optimization at `2f3ddd4884`	2023-01-11 00:59:02 -08:00
Aliaksandr Valialkin	8a35377cf3	app/vmselect/promql: move the `eval function args in parallel` query trace outside the loop	2023-01-10 22:23:43 -08:00
Aliaksandr Valialkin	6d1bd761f8	deployment/docker: update Go builder from v1.19.4 to v1.19.5 See https://github.com/golang/go/issues?q=milestone%3AGo1.19.5+label%3ACherryPickApproved	2023-01-10 18:42:34 -08:00
Aliaksandr Valialkin	ee73c9a5e6	deployment/docker: update Alpine base image from v3.17.0 to v3.17.1 See https://alpinelinux.org/posts/Alpine-3.17.1-released.html	2023-01-10 16:15:06 -08:00
Aliaksandr Valialkin	aa027529eb	lib/httpserver: directly pass flag value to CheckAuthFlag() There is no sense in passing a pointer to flag value there. This is a follow-up for `4225a0bd75`	2023-01-10 15:59:55 -08:00
Zakhar Bessarab	10f314cdbd	Use `httpAuth.` flags as a fallback for endpoints protected by `AuthKey` flags (#3582 ) * {lib/server, app/}: use `httpAuth.` flag as fallback for `AuthKey` if it is not set * lib/ingestserver/opentsdbhttp: fix opentdb HTTP handler not respecting `httpAuth.` flags Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-10 15:57:55 -08:00
Aliaksandr Valialkin	98931449c1	app/vmselect/netstorage: reduce tail latency during query processing Previously the selected time series were split evenly among available CPU cores for further processing - e.g unpacking the data and applying the given rollup function to the unpacked data. Some time series could be processed slower than others. This could result in uneven work distribution among available CPU cores, e.g. some CPU cores could complete their work sooner than others. This could slow down query execution. The new algorithm allows stealing time series to process from other CPU cores when all the local work is done. This should reduce the maximum time needed for query execution (aka tail latency). The new algorithm should also scale better on systems with many CPU cores, since every CPU processes locally assigned time series without inter-CPU communications. The inter-CPU communications are used only when all the local work is finished and the pending work from other CPUs needs to be stealed.	2023-01-10 13:42:26 -08:00
Aliaksandr Valialkin	158a280822	app/vmselect/netstorage: reduce memory allocations when unpacking time series Unpack time series with less than 4M samples in the currently running goroutine. Previously a new goroutine was being started for unpacking the samples. This was requiring additional memory allocations.	2023-01-09 23:17:34 -08:00
Aliaksandr Valialkin	c8bd3534cb	app/vmselect/promql: eliminate memory allocation when sorting values inside float64s	2023-01-09 23:06:57 -08:00
Aliaksandr Valialkin	7956b0d974	app/vmselect/promql: pre-allocate memory for values to be merged in mergeTimeseries() This should reduce the number of memory re-allocations	2023-01-09 22:52:19 -08:00
Aliaksandr Valialkin	895d5d9d22	app/vmselect/promql: consistently intern series names obtained from marshalMetricNameSorted This reduces memory allocations when the returned series names are used as map keys later	2023-01-09 22:46:30 -08:00
Aliaksandr Valialkin	12e2bcdf81	app/vmselect/promql: avoid memory allocations and copying from source timeseries to the returned result at timeseriesToResult()	2023-01-09 22:39:15 -08:00
Aliaksandr Valialkin	dd92e2050f	app/vmselect/promql: remove memory allocations from sortMetricTags()	2023-01-09 22:22:58 -08:00
Aliaksandr Valialkin	8050f5a18c	app/vmselect/promql: intern output series names inside timeseriesToResult() This reduces the number of memory allocations for repeated queries, which return (almost) the same set of time series.	2023-01-09 22:20:34 -08:00
Aliaksandr Valialkin	efb3c630fe	app/vmselect/promql: intern output series names during normal aggregation	2023-01-09 22:15:31 -08:00
Aliaksandr Valialkin	c0de651558	app/vmselect/promql: intern output series names during incremental aggregation This should reduce the number of memory allocations for repeated queries	2023-01-09 22:12:05 -08:00
Aliaksandr Valialkin	abbac2c27c	app/vmselect/netstorage: pre-allocate 4 block references per each time series during querying Usually the number of blocks returned per each time series during queries is around 4. So it is a good idea to pre-allocate 4 block references per time series in order to reduce the number of memory allocations.	2023-01-09 22:08:30 -08:00
Aliaksandr Valialkin	2483c67579	app/vmselect/netstorage: cache canonical MetricName for time series returned from the storage This reduces memory allocations for repeated queries, which return (almost) the same set of time series.	2023-01-09 21:56:27 -08:00
Aliaksandr Valialkin	b7a4650ab0	all: use metricsql.CompileRegexp instead of regexp.Compile for compiling regexps used in graphite queries This should speed up repeated queries, since metricsql.CompileRegexp returns regexps from the cache on subsequent calls for the same input regexp.	2023-01-09 21:45:34 -08:00
Aliaksandr Valialkin	9f02f5a05a	app/vmselect/netstorage: eliminate memory allocation for sortBlocksHeap arg when calling mergeSortBlocks()	2023-01-09 21:29:01 -08:00
Aliaksandr Valialkin	96f04c9863	app/vmselect/netstorage: consistently select the sample with the biggest value out of samples with identical timestamps Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3333 This fix is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3620 , but doesn't slow down the common case with merging replicated data blocks so significantly. Benchmark results: Before the change: BenchmarkMergeSortBlocks/replicationFactor-1-4 13968 85643 ns/op 956.53 MB/s 1700 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-2-4 10806 109171 ns/op 1500.77 MB/s 2191 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-3-4 8887 130623 ns/op 1881.45 MB/s 2660 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-4-4 7440 157348 ns/op 2082.52 MB/s 3174 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-5-4 6534 184473 ns/op 2220.38 MB/s 3612 B/op 1 allocs/op BenchmarkMergeSortBlocks/overlapped-blocks-bestcase-4 13419 85205 ns/op 961.44 MB/s 2213 B/op 1 allocs/op BenchmarkMergeSortBlocks/overlapped-blocks-worstcase-4 579 1894900 ns/op 43.23 MB/s 46760 B/op 1 allocs/op After the change: BenchmarkMergeSortBlocks/replicationFactor-1-4 13832 85298 ns/op 960.40 MB/s 1716 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-2-4 8833 134222 ns/op 1220.66 MB/s 2675 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-3-4 6487 184830 ns/op 1329.65 MB/s 3636 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-4-4 4977 236318 ns/op 1386.61 MB/s 4733 B/op 1 allocs/op BenchmarkMergeSortBlocks/replicationFactor-5-4 4088 296734 ns/op 1380.36 MB/s 5761 B/op 1 allocs/op BenchmarkMergeSortBlocks/overlapped-blocks-bestcase-4 14083 84067 ns/op 974.47 MB/s 2110 B/op 1 allocs/op BenchmarkMergeSortBlocks/overlapped-blocks-worstcase-4 536 2043534 ns/op 40.09 MB/s 50511 B/op 1 allocs/op	2023-01-09 12:58:18 -08:00
Aliaksandr Valialkin	5876821a16	all: small improvements in error messages and command-line flag descriptions related to concurrency limiters	2023-01-07 00:12:24 -08:00
Aliaksandr Valialkin	b275983403	lib/writeconcurrencylimiter: improve the logic behind -maxConcurrentInserts limit Previously the -maxConcurrentInserts was limiting the number of established client connections, which write data to VictoriaMetrics. Some of these connections could be idle. Such connections do not consume big amounts of CPU and RAM, so there is a little sense in limiting the number of such connections. So now the -maxConcurrentInserts command-line option limits the number of concurrently executed insert requests, not including idle connections. It is recommended removing -maxConcurrentInserts command-line option, since the default value for this option should work good for most cases.	2023-01-06 22:07:16 -08:00
Aliaksandr Valialkin	20e9598254	lib/vmselectapi: limit the number of concurrently executed requests This should prevent from out of memory errors when big number of vmselect nodes send many concurrent requests to vmstorage The limit can be controlled at vmstorage via the following command-line flags: - search.maxConcurrentRequests - search.maxQueueDuration See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#resource-usage-limits	2023-01-06 18:39:46 -08:00
Aliaksandr Valialkin	cd705b0f69	app/vmselect: improve error message when the request cannot be started because too many concurrent requests are already executed	2023-01-06 18:19:05 -08:00
Karan Sharma	8f42c5a024	lib/promscrape: add Prometheus-compatible service discovery for Nomad (#3549 ) Add nomad_sd_config support for service discovery	2023-01-05 18:07:02 -08:00
Zakhar Bessarab	eee7e75278	doc: add vmbackupmanager monitoring section (#3605 ) * doc: add vmbackupmanager monitoring section Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>	2023-01-05 16:49:59 -08:00
Aliaksandr Valialkin	1d16cc9349	lib/promscrape: pre-fetch metric_relabel_configs rules when debugging metric relabeling for a particular target Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407	2023-01-05 03:28:14 -08:00
Yury Molodov	802febab74	vmui: improve `Explore metrics` (#3598 ) * feat: add multiple select * feat: improve explore interface * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-05 02:24:05 -08:00
Aliaksandr Valialkin	78114e85d6	vendor: update github.com/VictoriaMetrics/metricsql from v0.50.0 to v0.51.0 Updates https://github.com/VictoriaMetrics/metricsql/pull/7 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3589	2023-01-05 01:50:26 -08:00
Aliaksandr Valialkin	ac890b3081	docs: update `-help` outputs for vm* tools	2023-01-03 23:27:31 -08:00

1 2 3 4 5 ...

2372 commits