github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
VictoriaMetrics/lib/storage
History
Zakhar Bessarab bea3431ed1
lib/storage/partition: add check to ensure parts exist on disk (#5017) 
* lib/storage/partition: add check to ensure parts exist on disk

If part exists in parts.json but is missing on disk there will be a misleading error similar to "unexpected number of substrings in the part name".

This change forces verification of part existence and throws a correct error in case it is missing on disk.

Such issue can be result of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005 or disk corruption.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/partition: use filepath.Join instead of string concatenation

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/partition: add action points for error message

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* all: add a check for missing part in lib/mergeset and lib/logstorage

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-09-19 11:17:41 +02:00
..
block.go lib/storage: validate timestamps in the block only if they use encoding, which needs validation 2022-10-21 00:52:32 +03:00
block_header.go lib/storage: verify that timestamps in block are in the range specified by blockHeader.{Min,Max}Timestamp when upacking the block 2022-09-06 13:08:09 +03:00
block_header_test.go lib/storage: typo fix: umarshal -> unmarshal 2021-03-02 20:47:59 +02:00
block_stream_merger.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
block_stream_reader.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:46:12 -07:00
block_stream_reader_test.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
block_stream_reader_timing_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:46:12 -07:00
block_stream_writer.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:46:12 -07:00
block_stream_writer_timing_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:46:12 -07:00
block_test.go lib/storage: use deterministic random generator in tests 2023-01-23 20:10:32 -08:00
dedup.go app,lib: fix typos in comments (#3804) 2023-02-13 13:27:13 +01:00
dedup_test.go lib/storage: follow-up after 7c0ae3a86a 2022-12-08 18:16:57 -08:00
dedup_timing_test.go lib/storage: follow-up after 7c0ae3a86a 2022-12-08 18:16:57 -08:00
filenames.go lib/storage: consistently use OS-independent separator in file paths 2023-03-25 14:33:58 -07:00
index_db.go lib/storage: handle fatal errors inside indexSearch.getTSIDByMetricID() instead of returning them to the caller 2023-09-15 11:55:42 +02:00
index_db_test.go flagutil: Make .Msecs private (#4906) 2023-09-03 10:33:37 +02:00
index_db_timing_test.go Add flagutil.Duration to avoid conversion bugs (#4835) 2023-09-01 09:27:51 +02:00
inmemory_part.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:46:12 -07:00
inmemory_part_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:46:12 -07:00
inmemory_part_timing_test.go lib/storage: use deterministic random generator in tests 2023-01-23 20:10:32 -08:00
merge.go lib/storage: optimization: do not scan block for rows outside retention if it is covered by the retention 2022-12-03 22:14:12 -08:00
merge_test.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
merge_timing_test.go lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage() 2023-07-13 17:13:24 -07:00
metaindex_row.go all: subsitute ioutil.ReadAll with io.ReadAll 2022-08-22 00:16:37 +03:00
metaindex_row_test.go lib/storage: correctly use maxBlockSize in various checks 2020-09-24 18:12:56 +03:00
metric_name.go app/vmselect/promql: add the ability to copy all the labels from one side of group_left()/group_right() operation 2023-07-17 19:07:39 -07:00
metric_name_test.go app/vminsert: add support for data ingestion via other vminsert nodes 2021-05-08 19:52:57 +03:00
part.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:46:12 -07:00
part_header.go lib/storage: fix a bug, which prevents from reading pre-v1.90.0 parts 2023-04-14 22:33:08 -07:00
part_search.go lib/storage: optimize partSearch.searchBHS() for common case when the TSID for the current block header is bigger or equal to the current tsid 2022-12-19 10:28:03 -08:00
part_search_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:46:12 -07:00
part_search_timing_test.go lib/storage: skip missing tsids in the current block header by using binary search 2022-12-14 22:06:51 -08:00
partition.go lib/storage/partition: add check to ensure parts exist on disk (#5017) 2023-09-19 11:17:41 +02:00
partition_search.go lib/storage: fixes finalDedup for backfilled data (#3737) 2023-02-01 09:54:21 -08:00
partition_search_test.go lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage() 2023-07-13 17:13:24 -07:00
partition_test.go lib/storage: remove unused code (#4050) 2023-04-02 21:24:42 -07:00
raw_block.go all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
raw_row.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:46:12 -07:00
search.go app,lib: fix typos in comments (#3804) 2023-02-13 13:27:13 +01:00
search_test.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
storage.go lib/storage: handle fatal errors inside indexSearch.getTSIDByMetricID() instead of returning them to the caller 2023-09-15 11:55:42 +02:00
storage_test.go flagutil: Make .Msecs private (#4906) 2023-09-03 10:33:37 +02:00
storage_timing_test.go lib/storage: replace OpenStorage() with MustOpenStorage() 2023-04-14 23:02:40 -07:00
table.go lib/storage: do not create flock.lock files at partition directories, since it is created at the Storage level 2023-06-19 22:48:37 -07:00
table_search.go lib/storage: do not pass retentionMsecs and isReadOnly args explicitly - access them via Storage arg 2022-10-24 01:31:04 +03:00
table_search_test.go lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage() 2023-07-13 17:13:24 -07:00
table_search_timing_test.go lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage() 2023-07-13 17:13:24 -07:00
table_test.go flagutil: Make .Msecs private (#4906) 2023-09-03 10:33:37 +02:00
table_timing_test.go lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage() 2023-07-13 17:13:24 -07:00
tag_filters.go lib/storage: optimize matching speed for non-trivial regexp filters 2022-10-01 12:06:06 +03:00
tag_filters_test.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:12 +03:00
tag_filters_timing_test.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:12 +03:00
time.go app,lib: fix typos in comments (#3804) 2023-02-13 13:27:13 +01:00
time_test.go all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
tsid.go all: remove the remaining mentions of cluster version 2019-11-21 23:18:22 +02:00
tsid_test.go all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00