VictoriaMetrics/lib
Aliaksandr Valialkin 84fb59b0ba lib/storage: move deletedMetricIDs set from indexDB to Storage
This makes consitent the list of deleted metricIDs when it is used from both the current indexDB and the previous indexDB (aka extDB).
This should fix the issue, which could lead to storing new samples under deleted metricIDs after indexDB rotation.
See more details at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347#issuecomment-861232136 .

Thanks to @tangqipengleoo for the initial analysis and the pull request - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .

This commit resolves the issue in more generic way compared to https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .

The downside of the commit is the deletedMetricIDs set isn't cleaned from the metricIDs outside the retention. It needs app restart.
This should be OK in most cases.
2021-06-15 15:04:30 +03:00
..
backup lib/backup: increase backup chunk size from 128MB to 1GB 2021-01-13 12:16:35 +02:00
bloomfilter lib/bloomfilter: fix TestLimiterConcurrent 2021-05-24 05:17:36 +03:00
buildinfo all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
bytesutil Perform conversion from string to []byte according to rule #6 at https://golang.org/pkg/unsafe/#Pointer 2020-08-05 11:55:58 +03:00
cgroup lib/cgroup: document the ability to detect cgroup v2 memory and cpu limits. This is follow-up for b50024812e 2021-05-13 09:26:20 +03:00
decimal lib/decimal: prevent exponent overflow when processing values close to zero 2021-03-05 18:52:47 +02:00
encoding lib: dump compressed block contents on error during decompression 2020-08-15 14:44:33 +03:00
envflag lib/envflag: add -envflag.prefix for setting optional prefix for environment vars 2020-03-30 15:51:19 +03:00
envtemplate all: support %{ENV_VAR} placeholders in yaml configs in all the vm* components 2020-08-13 17:15:25 +03:00
fastnum Makefile: add build and test rules with enabled race detector. These rules have -race suffix 2020-03-05 12:03:38 +02:00
fasttime lib: extract common code for returning fast unix timestamp into lib/fasttime 2020-05-14 23:02:07 +03:00
filestream fixes solaris build (#1345) 2021-05-31 09:21:23 +03:00
flagutil improve docs for cli flags (#1202) 2021-04-12 12:28:04 +03:00
fs fixes solaris build (#1345) 2021-05-31 09:21:23 +03:00
httpserver app/vmauth: properly handle http.ErrAbortHandler panic 2021-06-11 12:50:25 +03:00
influxutils lib/influxutils: return response compatible with InfluxDB 1.8.4 2021-03-15 22:19:59 +02:00
ingestserver lib/ingestserver: properly close incoming connections during graceful shutdown 2021-05-08 19:52:58 +03:00
leveledbytebufferpool lib/promscrape: an attempt to reduce memory usage when vmagent scrapes targets with varying number of metrics 2021-03-15 11:45:39 +02:00
logger improve docs for cli flags (#1202) 2021-04-12 12:28:04 +03:00
memory fixes solaris build (#1345) 2021-05-31 09:21:23 +03:00
mergeset Revert "lib/mergeset: remove a pool for inmemoryBlock structs" 2021-05-28 01:09:32 +03:00
metricsql lib/metricsql: move it to a separate repository - github.com/VictoriaMetrics/metrics 2020-04-28 15:28:22 +03:00
netutil lib/netutil: enable IPv6 UDP listening if -enableTCP6 command-line flag is passed to VictoriaMetrics 2021-03-17 00:16:17 +02:00
pacelimiter lib/pacelimiter: increase scalability for multi-CPU system 2020-08-06 18:32:59 +03:00
persistentqueue lib/persistentqueue: eliminate possible data race when obtaining vm_persistentqueue_bytes_pending metric value 2021-04-27 00:25:52 +03:00
procutil Adds windows build (#1040) 2021-02-27 00:37:07 +02:00
promauth lib/promauth: follow-up after 5b8176c68e 2021-05-22 18:01:11 +03:00
prompb app/vminsert: moved -maxInsertRequestSize command-line flag out of lib/prompb in order to prevent its inclusion in vmselect and vmstorage apps 2020-01-28 23:02:08 +02:00
prompbmarshal all: use %w instead of %s for wrapping errors in fmt.Errorf 2020-06-30 23:05:11 +03:00
promrelabel lib/promrelabel: add tests for labelsToString() function 2021-06-04 20:42:46 +03:00
promscrape lib/promscrape: show the number of samples collected during the last scrape at /targets and /api/v1/targets pages 2021-06-14 14:04:00 +03:00
protoparser lib/protoparser: stop reading the input stream as soon as the callback provided by the caller returns error 2021-06-14 15:18:49 +03:00
proxy lib/promscrape: reload auth tokens from files every second 2021-05-14 20:00:08 +03:00
storage lib/storage: move deletedMetricIDs set from indexDB to Storage 2021-06-15 15:04:30 +03:00
storagepacelimiter lib/storage: remove prioritizing of merging small parts over merging big parts, since it doesn't work as expected 2020-07-30 19:57:27 +03:00
syncwg all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
timerpool lib/timerpool: use timer pool in concurrency limiters 2019-05-28 17:20:10 +03:00
uint64set lib/uint64set: store pointers to bucket16 instead of bucket16 objects in bucket32 2021-05-25 14:20:52 +03:00
workingsetcache all: properly handle CPU limits set on the host system/container 2020-12-08 21:07:29 +02:00
writeconcurrencylimiter all: properly handle CPU limits set on the host system/container 2020-12-08 21:07:29 +02:00