5125 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Aliaksandr Valialkin	f4e9393076	vendor: make vendor-update	2022-10-28 00:08:50 +03:00
Denys Holius	6e5308ecb2	.github/workflows/codeql-analysis.yml: specifically setting the Go version (#3277) ... see https://github.com/github/codeql-action/issues/1059	2022-10-28 00:08:50 +03:00
Dmytro Kozlov	3123059407	vmalert: escape query params if external alert source defined (#3267) ... vmalert: escape query args if external alert source defined	2022-10-28 00:08:50 +03:00
Aliaksandr Valialkin	450a32970a	lib/envtemplate: allow referring env vars from other env vars via %{ENV_VAR} syntax ... This is a follow-up for 02096e06d0	2022-10-26 14:51:02 +03:00
Aliaksandr Valialkin	46450995ae	app/vmselect/vmui: make vmui-update after eae6063450	2022-10-26 02:51:09 +03:00
Yury Molodov	1847a207f8	fix: change step setting field (#3270) ... Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-10-26 02:47:41 +03:00
Yury Molodov	346d49ae61	vmui: limit the number of decimal places to 10 characters (#3258) ... * fix: limit the number of decimal places to 10 characters * fix: add float numbers stabilizer	2022-10-26 02:43:43 +03:00
Yury Molodov	ab880afa50	vmui: add responsive styles for small screens (#3256) ... * fix: add responsive styles for small screens * fix: correct additional settings margins * docs/CHANGELOG.md: add responsive styles	2022-10-26 02:40:41 +03:00
Aliaksandr Valialkin	c92d1fdf8e	docs/Cluster-VictoriaMetrics.md: update docs about env vars usage in command-line flags ... This is a follow-up for 02096e06d0	2022-10-26 01:55:25 +03:00
Aliaksandr Valialkin	36b92f07f7	lib/envflag: allow referring environment variables in command-line flags	2022-10-26 01:55:23 +03:00
Aliaksandr Valialkin	75990de795	docs: run make docs-sync after 2ed3d49c26	2022-10-26 01:14:54 +03:00
Aliaksandr Valialkin	ecb71a7221	lib/fs: add canOverwrite arg to WriteFileAtomically when it is allowed to overwrite the file atomically if it already exists	2022-10-26 01:08:35 +03:00
Aliaksandr Valialkin	9d0652fc6e	docs/Makefile: fix docs-up Makefile command after 9ccd22c1f6	2022-10-25 17:54:14 +03:00
Denys Holius	8c0f93aa55	Docs: add guide "How to delete and replace metrics" (#2829) ... docs: add guide how to delete and replace metrics	2022-10-25 17:54:13 +03:00
Aliaksandr Valialkin	4f53147ed4	app/{vminsert,vmselect}/netstorage: allow calling Init()+MustStop() in a loop ... Previously netstorage.MustStop() call didn't free up all the resources, so the subsequent call to nestorage.Init() would panic. This allows writing tests, which call nestorage.Init() + nestorage.MustStop() in a loop.	2022-10-25 14:43:05 +03:00
Aliaksandr Valialkin	596251eb87	app/vmselect/vmui: make vmui-update after 274e235bf7	2022-10-24 21:29:57 +03:00
Aliaksandr Valialkin	608c977978	docs/guides/migrate-from-influx.md: properly display images at https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/guides/migrate-from-influx.md ... This is a follow-up for 42375679db	2022-10-24 21:29:55 +03:00
Nguyen Van Duc	f7a1001b55	Fix images not display on key concepts document (#3266)	2022-10-24 21:23:38 +03:00
Aliaksandr Valialkin	46a37fbeac	docs/CHANGELOG.md: document 274e235bf7 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3240	2022-10-24 21:03:03 +03:00
Yury Molodov	96a579c13b	vmui: optimize memory (#3255) ... * fix: change series limit logic * fix: remove spread operators	2022-10-24 21:03:02 +03:00
Yury Molodov	5de090ab43	vmui: extend options for app mode (#3252) ... * feat: add vmui customization for dbaas * feat: extends vmui customization for dbaas * fix: move input tenandId after query * fix: change serverURL when changing tenantID * fix: remove options * docs: add options description	2022-10-24 20:49:46 +03:00
Aliaksandr Valialkin	8ea84432ef	docs/enterprise.md: describe all the enteprise features in a short doc at https://docs.victoriametrics.com/enterprise.html	2022-10-24 18:03:22 +03:00
Aliaksandr Valialkin	754208b124	docs/CHANGELOG.md: typo fixes	2022-10-24 17:00:18 +03:00
Aliaksandr Valialkin	a6d4711ac6	lib/storage: add support for retention filters (aka multiple retentions for distinct sets of time series) ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/143 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/289	2022-10-24 16:41:59 +03:00
Aliaksandr Valialkin	22a5405cfa	docs/CHANGELOG.md: document 69b27275d2b2bf1bdae0d8b887b44bde2a787649	2022-10-24 16:08:35 +03:00
Aliaksandr Valialkin	8946371466	vendor: make vendor-update	2022-10-24 15:50:57 +03:00
Aliaksandr Valialkin	51f2e473f5	lib/storage: skip blocks outside the configured retention during search ... Blocks outside the configured retention are eventually deleted during background merge. But such blocks may reside in the storage for long time until background merge. Previously VictoriaMetrics could spend additional CPU time on processing such blocks during search queries. Now these blocks are skipped.	2022-10-24 02:56:13 +03:00
Aliaksandr Valialkin	2fc82b846e	lib/storage: do not pass retentionMsecs and isReadOnly args explicitly - access them via Storage arg ... This makes code easier to read. This is a follow-up after d2d30581a0	2022-10-24 01:32:56 +03:00
Aliaksandr Valialkin	d51f9b9284	lib/storage: small code cleanups	2022-10-24 01:17:58 +03:00
Aliaksandr Valialkin	5ace1587e6	lib/storage: re-use newTestStorage() instead of manually initializing Storage mock ... This is a follow-up for d2d30581a0	2022-10-23 16:24:42 +03:00
Aliaksandr Valialkin	57ea7a3ee8	lib/storage: pass Storage to table and partition instead of getDeletedMetricIDs callback ... This improves code readability a bit.	2022-10-23 16:11:02 +03:00
Aliaksandr Valialkin	63419d8e7c	lib/storage: small refactoring: move retentionDeadline to blockStreamMerger ... This allows defining per-block retention in the future by updating the getRetentionDeadline function	2022-10-23 16:11:01 +03:00
Aliaksandr Valialkin	31071347ca	lib/storage: use a single reference to the currently merged block - bsm.Block during the block merge loop	2022-10-23 14:09:14 +03:00
Aliaksandr Valialkin	5d0a91afd5	lib/storage: properly pass uint64 constant to fmt.Errorf on 32-bit platforms	2022-10-23 12:48:43 +03:00
Aliaksandr Valialkin	2dd93449d8	lib/storage: subsitute searchTSIDs functions with more lightweight searchMetricIDs function ... The searchTSIDs function was searching for metricIDs matching the the given tag filters and then was locating the corresponding TSID entries for the found metricIDs. The TSID entries aren't needed when searching for time series names (aka MetricName), so this commit removes the uneeded TSID search from the implementation of /api/v1/series API. This improves perfromance of /api/v1/series calls. This commit also improves performance a bit for /api/v1/query and /api/v1/query_range calls, since now these calls cache small metricIDs instead of big TSID entries in the indexdb/tagFilters cache (now this cache is named indexdb/tagFiltersToMetricIDs) without the need to compress the saved entries in order to save cache space. This commit also removes concurrency limiter during searching for matching time series, which was introduced in 8f16388428, since the concurrency for all the read queries is already limited with -search.maxConcurrentRequests command-line flag. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648	2022-10-23 12:43:44 +03:00
Aliaksandr Valialkin	c92aef39b5	app/vmselect/promql: expose missing metric vm_cache_size_max_bytes{type="promql/rollupResult"}	2022-10-23 12:14:07 +03:00
Aliaksandr Valialkin	fe5611d6e1	lib/storage: free up memory occupied by Storage.pendingHourEntries after a temporary spike in its memory usage ... This reduces vmstorage memory usage by up to 20% in production workload	2022-10-21 14:59:14 +03:00
Aliaksandr Valialkin	32b6ce691b	lib/storage: move common code to newRawRowsBlock() function	2022-10-21 14:46:06 +03:00
Aliaksandr Valialkin	2f8861ed9c	lib/storage: simplify code a bit after 3f5959c053	2022-10-21 14:39:44 +03:00
Aliaksandr Valialkin	1fb2be0cae	lib/{mergeset,storage}: simplify the code a bit after ae55ad8749	2022-10-21 14:33:15 +03:00
Aliaksandr Valialkin	af648279ce	lib/storage: validate timestamps in the block only if they use encoding, which needs validation ... This reduces CPU usage when there is no sense in validating timestamps. This is a follow-up for 5fa9525498 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2998 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3011	2022-10-21 00:54:37 +03:00
Aliaksandr Valialkin	edf3b7be47	lib/storage: try generating initial parts from inmemory rows with identical sizes under high ingestion rate ... This should improve background merge rate under high load a bit	2022-10-20 23:27:44 +03:00
Aliaksandr Valialkin	4d71023eb9	lib/workingsetcache: increase default cache expiration from 10 minutes to 20 minutes ... This increases the maximum time for cache population with new entries from 20 minutes to 40 minutes. This This change shouldn't increase memory usage for caches, since the prev cache cleaner should free up memory by deleting unused prev cache as soon as possible. See 08ca45d238 for details on prev cache cleaner.	2022-10-20 21:59:08 +03:00
Aliaksandr Valialkin	9a52b56b89	lib/workingsetcache: move the cleaner for the prev cache into a separate goroutine ... This makes the code more clear after d906d8573e	2022-10-20 21:59:02 +03:00
Aliaksandr Valialkin	324e119172	lib/procutil: stop immediately after receiving the second SIGINT or SIGTERM signal ... Previously VictoriaMetrics apps could stop responding to SIGINT and SIGTERM signals if they hang for some reason in graceful shutdown procedure.	2022-10-20 21:58:49 +03:00
Aliaksandr Valialkin	6855de311c	lib/{mergeset,storage}: avoid unaligned 64-bit atomic operation panic on 32-bit platforms ... The panic has been introduced in 68f3a02589 While at it, add padding to shard structs in order to avoid false sharing on mordern CPUs This should improve scalability on systems with many CPU cores	2022-10-20 16:24:46 +03:00
Aliaksandr Valialkin	526bc8a8b0	lib/workingsetcache: drop the previous cache whenever it recieves less than 5% of requests comparing to the current cache ... This means that the majority of requests are successfully served from the current cache, so the previous cache can be reset in order to free up memory.	2022-10-20 10:48:46 +03:00
Aliaksandr Valialkin	42cda38dbc	lib/workingsetcache: use per-bucket stats counters instead of global stats counters for cache hits/misses ... This should improve cache scalability on systems with many CPU cores.	2022-10-20 10:48:46 +03:00
Aliaksandr Valialkin	f22bea242f	lib/workingsetcache: randomize interval for swapping curr and prev caches ... This should make CPU usage smoother over time, since different caches will be swapped at different times.	2022-10-20 10:48:46 +03:00
Aliaksandr Valialkin	0126ea31e7	docs/CHANGELOG.md: move the BUGFIX line for 1059c4d84a into correct place	2022-10-18 20:40:37 +03:00