VictoriaMetrics/lib/persistentqueue
Aliaksandr Valialkin eee6f1e56d lib/persistentqueue: make the persistent queue more durable against unclean shutdown (kill -9, OOM, hard reset)
The strategy is:

- Periodical flushing of inmemory blocks to files, so they aren't lost on unclean shutdown.
- Periodical syncing of metadata for persisted queues, so the metadata remains in sync with the persisted data.
- Automatic adjusting of too big chunk size when opening the queue. The chunk size may be bigger than the writer offset after unclean shutdown.
- Skipping of broken chunk file if it cannot be read.
- Fsyncing finalized chunk files.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/687
2020-09-16 18:13:24 +03:00
..
fastqueue.go lib/persistentqueue: make the persistent queue more durable against unclean shutdown (kill -9, OOM, hard reset) 2020-09-16 18:13:24 +03:00
fastqueue_test.go all: fix golangci-lint issues 2020-03-10 19:40:03 +02:00
fastqueue_timing_test.go app/vmagent: add -remoteWrite.maxDiskUsagePerURL for limiting the maximum disk usage for each -remoteWrite.url buffer 2020-03-03 19:49:20 +02:00
persistentqueue.go lib/persistentqueue: make the persistent queue more durable against unclean shutdown (kill -9, OOM, hard reset) 2020-09-16 18:13:24 +03:00
persistentqueue_test.go all: use %w instead of %s for wrapping errors in fmt.Errorf 2020-06-30 23:33:46 +03:00
persistentqueue_timing_test.go app/vmagent: add -remoteWrite.maxDiskUsagePerURL for limiting the maximum disk usage for each -remoteWrite.url buffer 2020-03-03 19:49:20 +02:00