VictoriaMetrics/lib/storage
Aliaksandr Valialkin 5e4dfe50c6
lib/storage: subsitute searchTSIDs functions with more lightweight searchMetricIDs function
The searchTSIDs function was searching for metricIDs matching the the given tag filters
and then was locating the corresponding TSID entries for the found metricIDs.

The TSID entries aren't needed when searching for time series names (aka MetricName),
so this commit removes the uneeded TSID search from the implementation of /api/v1/series API.
This improves perfromance of /api/v1/series calls.

This commit also improves performance a bit for /api/v1/query and /api/v1/query_range calls,
since now these calls cache small metricIDs instead of big TSID entries
in the indexdb/tagFilters cache (now this cache is named indexdb/tagFiltersToMetricIDs)
without the need to compress the saved entries in order to save cache space.

This commit also removes concurrency limiter during searching for matching time series,
which was introduced in 8f16388428, since the concurrency
for all the read queries is already limited with -search.maxConcurrentRequests command-line flag.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648
2022-10-23 12:23:47 +03:00
..
block.go lib/storage: validate timestamps in the block only if they use encoding, which needs validation 2022-10-21 00:52:32 +03:00
block_header.go lib/storage: verify that timestamps in block are in the range specified by blockHeader.{Min,Max}Timestamp when upacking the block 2022-09-06 13:08:09 +03:00
block_header_test.go lib/storage: typo fix: umarshal -> unmarshal 2021-03-02 20:47:59 +02:00
block_stream_merger.go lib/storage: set bsm.Block to nil on error, so the previous block couldn't be used. 2022-01-20 20:13:14 +02:00
block_stream_reader.go lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations 2022-02-01 00:18:42 +02:00
block_stream_reader_test.go all: use %w instead of %s for wrapping errors in fmt.Errorf 2020-06-30 23:05:11 +03:00
block_stream_reader_timing_test.go all: use %w instead of %s for wrapping errors in fmt.Errorf 2020-06-30 23:05:11 +03:00
block_stream_writer.go lib/storage: substitute remaining calls to fs.MustRemoveAll with fs.MustRemoveDirAtomic 2022-09-13 16:17:38 +03:00
block_stream_writer_timing_test.go lib/storage: deduplicate samples more thoroughly 2021-12-15 15:59:58 +02:00
block_test.go lib/storage: verify that timestamps in block are in the range specified by blockHeader.{Min,Max}Timestamp when upacking the block 2022-09-06 13:08:09 +03:00
dedup.go lib/storage: leave the last sample per each discrete interval during the deduplicaton 2022-05-02 21:50:45 +03:00
dedup_test.go lib/storage: leave the last sample per each discrete interval during the deduplicaton 2022-05-02 21:50:45 +03:00
dedup_timing_test.go lib/storage: explicitly pass dedupInterval to DeduplicateSamples() and deduplicateSamplesDuringMerge() 2021-12-14 20:49:12 +02:00
index_db.go lib/storage: subsitute searchTSIDs functions with more lightweight searchMetricIDs function 2022-10-23 12:23:47 +03:00
index_db_test.go lib/storage: subsitute searchTSIDs functions with more lightweight searchMetricIDs function 2022-10-23 12:23:47 +03:00
index_db_timing_test.go lib/storage: properly take into account already registered series when -storage.maxHourlySeries or -storage.maxDailySeries limits are enabled 2022-06-20 13:47:47 +03:00
inmemory_part.go lib/{mergeset,storage}: switch from sync.Pool to chan-based pool for inmemoryPart objects 2021-07-06 16:28:41 +03:00
inmemory_part_test.go all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
inmemory_part_timing_test.go all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
merge.go lib/storage: deduplicate samples more thoroughly 2021-12-15 15:59:58 +02:00
merge_test.go app/vmstorage: support for -retentionPeriod smaller than one month 2020-10-20 14:31:44 +03:00
merge_timing_test.go app/vmstorage: support for -retentionPeriod smaller than one month 2020-10-20 14:31:44 +03:00
metaindex_row.go all: subsitute ioutil.ReadAll with io.ReadAll 2022-08-22 00:16:37 +03:00
metaindex_row_test.go lib/storage: correctly use maxBlockSize in various checks 2020-09-24 18:12:56 +03:00
metric_name.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:12 +03:00
metric_name_test.go app/vminsert: add support for data ingestion via other vminsert nodes 2021-05-08 19:52:57 +03:00
part.go lib/mergeset: tune caches size limits for indexdb/dataBlocks and indexdb/indexBlocks 2022-01-21 12:45:43 +02:00
part_header.go all: use os.{Read|Write}File instead of ioutil.{Read|Write}File 2022-08-21 23:52:35 +03:00
part_header_test.go all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
part_search.go lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations 2022-02-01 00:18:42 +02:00
part_search_test.go app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day 2022-06-28 13:02:47 +03:00
partition.go lib/storage: move common code to newRawRowsBlock() function 2022-10-21 14:46:55 +03:00
partition_search.go all: make fmt via the upcoming Go1.19 2022-07-11 19:22:15 +03:00
partition_search_test.go app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day 2022-06-28 13:02:47 +03:00
partition_test.go lib/{mergeset,storage}: improve the detection of the needed free space for background merge 2021-08-25 09:35:44 +03:00
raw_block.go all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
raw_row.go lib/storage: deduplicate samples more thoroughly 2021-12-15 15:59:58 +02:00
search.go lib/storage: subsitute searchTSIDs functions with more lightweight searchMetricIDs function 2022-10-23 12:23:47 +03:00
search_test.go app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day 2022-06-28 13:02:47 +03:00
storage.go lib/storage: subsitute searchTSIDs functions with more lightweight searchMetricIDs function 2022-10-23 12:23:47 +03:00
storage_test.go lib/vmselectapi: rename deleteMetrics to more correct deleteSeries 2022-07-06 12:37:54 +03:00
storage_timing_test.go app/vmstorage: add ability to limit series cardinality via -storage.maxHourlySeries and -storage.maxDailySeries command-line flags 2021-05-20 14:15:19 +03:00
table.go lib/storage: atomically remove partitions, which went outside the configured retention 2022-09-13 16:17:37 +03:00
table_search.go optimized code (#2103) 2022-01-28 14:15:41 +02:00
table_search_test.go app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day 2022-06-28 13:02:47 +03:00
table_search_timing_test.go app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day 2022-06-28 13:02:47 +03:00
table_test.go lib/storage: stop background merge when storage enters read-only mode 2022-06-01 14:36:45 +03:00
table_timing_test.go lib/storage: stop background merge when storage enters read-only mode 2022-06-01 14:36:45 +03:00
tag_filters.go lib/storage: optimize matching speed for non-trivial regexp filters 2022-10-01 12:06:06 +03:00
tag_filters_test.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:12 +03:00
tag_filters_timing_test.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:12 +03:00
time.go all: readability improvements for query traces 2022-06-30 18:20:33 +03:00
time_test.go all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
tsid.go all: remove the remaining mentions of cluster version 2019-11-21 23:18:22 +02:00
tsid_test.go all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00