Aliaksandr Valialkin
02096e06d0
lib/envflag: allow referring environment variables in command-line flags
2022-10-26 01:52:05 +03:00
Aliaksandr Valialkin
5d82e7d64a
docs: run make docs-sync after 2ed3d49c26
2022-10-26 01:14:31 +03:00
omahs
2ed3d49c26
Fix: typos ( #3269 )
...
Fix: typos
2022-10-26 01:12:54 +03:00
Aliaksandr Valialkin
c4265322f4
lib/fs: add canOverwrite arg to WriteFileAtomically when it is allowed to overwrite the file atomically if it already exists
2022-10-26 01:07:34 +03:00
Aliaksandr Valialkin
db8abd000e
docs/Makefile: fix docs-up Makefile command after 9ccd22c1f6
2022-10-25 17:53:24 +03:00
Aliaksandr Valialkin
d9bbf24183
app/{vminsert,vmselect}/netstorage: allow calling Init()+MustStop() in a loop
...
Previously netstorage.MustStop() call didn't free up all the resources,
so the subsequent call to nestorage.Init() would panic.
This allows writing tests, which call nestorage.Init() + nestorage.MustStop() in a loop.
2022-10-25 17:47:17 +03:00
Denys Holius
9ccd22c1f6
Docs: add guide "How to delete and replace metrics" ( #2829 )
...
docs: add guide how to delete and replace metrics
2022-10-25 09:02:14 -04:00
Aliaksandr Valialkin
b7882dc9af
app/vmselect/vmui: make vmui-update after 274e235bf7
2022-10-24 21:29:13 +03:00
Aliaksandr Valialkin
15849cb571
docs/guides/migrate-from-influx.md: properly display images at https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/guides/migrate-from-influx.md
...
This is a follow-up for 42375679db
2022-10-24 21:28:31 +03:00
Nguyen Van Duc
42375679db
Fix images not display on key concepts document ( #3266 )
2022-10-24 21:22:41 +03:00
Aliaksandr Valialkin
b1324631b1
docs/CHANGELOG.md: document 274e235bf7
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3240
2022-10-24 21:02:17 +03:00
Yury Molodov
274e235bf7
vmui: optimize memory ( #3255 )
...
* fix: change series limit logic
* fix: remove spread operators
2022-10-24 20:51:24 +03:00
Yury Molodov
59199a98dd
vmui: extend options for app mode ( #3252 )
...
* feat: add vmui customization for dbaas
* feat: extends vmui customization for dbaas
* fix: move input tenandId after query
* fix: change serverURL when changing tenantID
* fix: remove options
* docs: add options description
2022-10-24 20:45:41 +03:00
Aliaksandr Valialkin
c52c23c272
docs/enterprise.md: describe all the enteprise features in a short doc at https://docs.victoriametrics.com/enterprise.html
2022-10-24 18:02:03 +03:00
Aliaksandr Valialkin
cac28ae0ae
docs/CHANGELOG.md: typo fixes
2022-10-24 16:59:55 +03:00
Aliaksandr Valialkin
8e998aa1a1
lib/storage: add support for retention filters (aka multiple retentions for distinct sets of time series)
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/143
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/289
2022-10-24 16:40:20 +03:00
Aliaksandr Valialkin
d7e657e5f9
docs/CHANGELOG.md: document 69b27275d2b2bf1bdae0d8b887b44bde2a787649
2022-10-24 16:08:15 +03:00
Aliaksandr Valialkin
692af2d17c
vendor: make vendor-update
2022-10-24 15:47:01 +03:00
Aliaksandr Valialkin
dba218a8ce
lib/storage: skip blocks outside the configured retention during search
...
Blocks outside the configured retention are eventually deleted during background merge.
But such blocks may reside in the storage for long time until background merge.
Previously VictoriaMetrics could spend additional CPU time on processing such blocks
during search queries. Now these blocks are skipped.
2022-10-24 02:52:44 +03:00
Aliaksandr Valialkin
e2f0b76ebf
lib/storage: do not pass retentionMsecs and isReadOnly args explicitly - access them via Storage arg
...
This makes code easier to read.
This is a follow-up after d2d30581a0
2022-10-24 01:31:04 +03:00
Aliaksandr Valialkin
89a1108b1a
lib/storage: small code cleanups
2022-10-24 01:17:47 +03:00
Aliaksandr Valialkin
05512fdd74
lib/storage: re-use newTestStorage() instead of manually initializing Storage mock
...
This is a follow-up for d2d30581a0
2022-10-23 16:24:00 +03:00
Aliaksandr Valialkin
d2d30581a0
lib/storage: pass Storage to table and partition instead of getDeletedMetricIDs callback
...
This improves code readability a bit.
2022-10-23 16:10:04 +03:00
Aliaksandr Valialkin
54f35c175c
lib/storage: small refactoring: move retentionDeadline to blockStreamMerger
...
This allows defining per-block retention in the future by updating the getRetentionDeadline function
2022-10-23 16:10:02 +03:00
Aliaksandr Valialkin
187e294a53
lib/storage: use a single reference to the currently merged block - bsm.Block during the block merge loop
2022-10-23 14:08:57 +03:00
Aliaksandr Valialkin
d0a9ca1bc2
lib/storage: properly pass uint64 constant to fmt.Errorf on 32-bit platforms
2022-10-23 12:48:00 +03:00
Aliaksandr Valialkin
5e4dfe50c6
lib/storage: subsitute searchTSIDs functions with more lightweight searchMetricIDs function
...
The searchTSIDs function was searching for metricIDs matching the the given tag filters
and then was locating the corresponding TSID entries for the found metricIDs.
The TSID entries aren't needed when searching for time series names (aka MetricName),
so this commit removes the uneeded TSID search from the implementation of /api/v1/series API.
This improves perfromance of /api/v1/series calls.
This commit also improves performance a bit for /api/v1/query and /api/v1/query_range calls,
since now these calls cache small metricIDs instead of big TSID entries
in the indexdb/tagFilters cache (now this cache is named indexdb/tagFiltersToMetricIDs)
without the need to compress the saved entries in order to save cache space.
This commit also removes concurrency limiter during searching for matching time series,
which was introduced in 8f16388428https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648
2022-10-23 12:23:47 +03:00
Aliaksandr Valialkin
a10647e0bf
app/vmselect/promql: expose missing metric vm_cache_size_max_bytes{type="promql/rollupResult"}
2022-10-23 12:13:47 +03:00
Aliaksandr Valialkin
4128ad71e2
lib/storage: move common code to newRawRowsBlock() function
2022-10-21 14:46:55 +03:00
Aliaksandr Valialkin
b5674164c6
lib/storage: simplify code a bit after 3f5959c053
2022-10-21 14:39:27 +03:00
Aliaksandr Valialkin
fd7c86ae25
lib/{mergeset,storage}: simplify the code a bit after ae55ad8749
2022-10-21 14:33:03 +03:00
Aliaksandr Valialkin
99d67ac8ad
lib/storage: validate timestamps in the block only if they use encoding, which needs validation
...
This reduces CPU usage when there is no sense in validating timestamps.
This is a follow-up for 5fa9525498https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2998
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3011
2022-10-21 00:52:32 +03:00
Aliaksandr Valialkin
3f5959c053
lib/storage: try generating initial parts from inmemory rows with identical sizes under high ingestion rate
...
This should improve background merge rate under high load a bit
2022-10-20 23:28:24 +03:00
Aliaksandr Valialkin
891ff6af2a
lib/workingsetcache: increase default cache expiration from 10 minutes to 20 minutes
...
This increases the maximum time for cache population with new entries from 20 minutes to 40 minutes.
This
This change shouldn't increase memory usage for caches, since the prev cache cleaner
should free up memory by deleting unused prev cache as soon as possible.
See 08ca45d238
2022-10-20 21:48:25 +03:00
Aliaksandr Valialkin
08ca45d238
lib/workingsetcache: move the cleaner for the prev cache into a separate goroutine
...
This makes the code more clear after d906d8573e
2022-10-20 21:45:29 +03:00
Aliaksandr Valialkin
4cd173bbaa
lib/procutil: stop immediately after receiving the second SIGINT or SIGTERM signal
...
Previously VictoriaMetrics apps could stop responding to SIGINT and SIGTERM signals
if they hang for some reason in graceful shutdown procedure.
2022-10-20 21:40:20 +03:00
Aliaksandr Valialkin
150e99d403
lib/{mergeset,storage}: avoid unaligned 64-bit atomic operation panic on 32-bit platforms
...
The panic has been introduced in 68f3a02589
2022-10-20 16:25:43 +03:00
Aliaksandr Valialkin
d906d8573e
lib/workingsetcache: drop the previous cache whenever it recieves less than 5% of requests comparing to the current cache
...
This means that the majority of requests are successfully served from the current cache,
so the previous cache can be reset in order to free up memory.
2022-10-20 10:47:58 +03:00
Aliaksandr Valialkin
817aeafd69
lib/workingsetcache: use per-bucket stats counters instead of global stats counters for cache hits/misses
...
This should improve cache scalability on systems with many CPU cores.
2022-10-20 09:12:17 +03:00
Aliaksandr Valialkin
9c02c39487
lib/workingsetcache: randomize interval for swapping curr and prev caches
...
This should make CPU usage smoother over time, since different caches
will be swapped at different times.
2022-10-20 08:42:43 +03:00
Aliaksandr Valialkin
cba9696a14
docs/CHANGELOG.md: move the BUGFIX line for 1059c4d84a into correct place
2022-10-18 20:38:57 +03:00
Nikolay
1059c4d84a
lib/promscrape/discovery/kubernetes: correctly wrap error ( #3250 )
...
* lib/promscrape/discovery/kubernetes: correctly wrap error
follow-up after 1304824201
2022-10-18 20:37:42 +03:00
Roman Khavronenko
4e0ea95f26
vmalert: lower severity level for RW retries ( #3237 )
...
The message about dropped data still remains at `error` level.
The change supposed to make log message more clear about how
serious it is.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-10-18 14:27:20 +02:00
Aliaksandr Valialkin
b4f7243110
vendor: make vendor-update
2022-10-18 10:56:38 +03:00
Aliaksandr Valialkin
3bb3893b2d
app/vmselect/vmui: make vmui-update after 080562030d
2022-10-18 10:50:35 +03:00
Yury Molodov
080562030d
fix: remove rounding of axis limits ( #3238 )
2022-10-18 10:47:58 +03:00
Aliaksandr Valialkin
069401a304
all: log error when environment variables referred from -promscrape.config are missing
...
This should prevent from using incorrect config files
2022-10-18 10:47:16 +03:00
Aliaksandr Valialkin
fb50730ba7
lib/storage: double the number of rawRows shards on multi-core systems
...
This should increase data ingestion scalability on multi-core systems at the cost of slightly higher memory usage
2022-10-17 18:19:51 +03:00
Aliaksandr Valialkin
ae55ad8749
lib/{storage,mergeset}: do not hold per-shard lock in fast path when adding per-shard items to the flush list
2022-10-17 18:01:26 +03:00
Aliaksandr Valialkin
b6e8c1403a
lib/promrelabel: add relabeling tests when the source label is missing
2022-10-17 14:47:52 +03:00
