From 3544e66a95a9a5132f3f8e197d48d6bd7a54a5f3 Mon Sep 17 00:00:00 2001 From: Roman Khavronenko <roman@victoriametrics.com> Date: Tue, 6 Jun 2023 10:54:39 +0200 Subject: [PATCH] docs: mention checksums and data corruption (#4404) Signed-off-by: hagen1778 <roman@victoriametrics.com> (cherry picked from commit 96b40b044c7e9237cd3ad2ec582621449775797c) Signed-off-by: hagen1778 <roman@victoriametrics.com> --- README.md | 2 +- docs/README.md | 8 +++++++- docs/Single-server-VictoriaMetrics.md | 8 +++++++- 3 files changed, 15 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 9ad46b048d..26c0f457bf 100644 --- a/README.md +++ b/README.md @@ -1147,7 +1147,7 @@ Below is the output for `/path/to/vmselect -help`: -search.maxGraphiteTagKeys int The maximum number of tag keys returned from Graphite /tags, /tags/autoComplete/*, /tags/findSeries API (default 100000) -search.maxGraphiteTagValues int - The maximum number of tag values returned Graphite /tags/<tag_name> API (default 100000) + The maximum number of tag values returned Graphite /tags/<tag_name> API (default 100000) -search.maxLookback duration Synonym to -search.lookback-delta from Prometheus. The value is dynamically detected from interval between time series datapoints if not set. It can be overridden on per-query basis via max_lookback arg. See also '-search.maxStalenessInterval' flag, which has the same meaining due to historical reasons -search.maxMemoryPerQuery size diff --git a/docs/README.md b/docs/README.md index 601bcbbe60..c05e616b07 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1547,7 +1547,13 @@ occurs in the middle of writing the `part` to disk - such incompletely written ` are automatically deleted on the next VictoriaMetrics start. The same applies to merge process — `parts` are either fully merged into a new `part` or fail to merge, -leaving the source `parts` untouched. +leaving the source `parts` untouched. However, due to hardware issues data on disk may be corrupted regardless of +VictoriaMetrics process. VictoriaMetrics can detect corruption during decompressing, decoding or sanity checking +of the data blocks. But **it cannot fix the corrupted data**. Data parts that fail to load on startup need to be deleted +or restored from backups. This is why it is recommended performing +[regular backups](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#backups). + +VictoriaMetrics doesn't use checksums for stored data blocks. See why [here](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3011). VictoriaMetrics doesn't merge parts if their summary size exceeds free disk space. This prevents from potential out of disk space errors during merge. diff --git a/docs/Single-server-VictoriaMetrics.md b/docs/Single-server-VictoriaMetrics.md index bd0615035a..4e5d487741 100644 --- a/docs/Single-server-VictoriaMetrics.md +++ b/docs/Single-server-VictoriaMetrics.md @@ -1555,7 +1555,13 @@ occurs in the middle of writing the `part` to disk - such incompletely written ` are automatically deleted on the next VictoriaMetrics start. The same applies to merge process — `parts` are either fully merged into a new `part` or fail to merge, -leaving the source `parts` untouched. +leaving the source `parts` untouched. However, due to hardware issues data on disk may be corrupted regardless of +VictoriaMetrics process. VictoriaMetrics can detect corruption during decompressing, decoding or sanity checking +of the data blocks. But **it cannot fix the corrupted data**. Data parts that fail to load on startup need to be deleted +or restored from backups. This is why it is recommended performing +[regular backups](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#backups). + +VictoriaMetrics doesn't use checksums for stored data blocks. See why [here](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3011). VictoriaMetrics doesn't merge parts if their summary size exceeds free disk space. This prevents from potential out of disk space errors during merge.