mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-12-01 14:47:38 +00:00
fe5f16b662
This allows reducing the amounts of data, which must be read during queries over logs with big number of fields (aka "wide events").
This, in turn, improves query performance when the data, which needs to be scanned during the query, doesn't fit OS page cache.
(cherry picked from commit 7a62eefa34)
71 lines
3.1 KiB
Go
71 lines
3.1 KiB
Go
package logstorage
|
|
|
|
// partFormatLatestVersion is the latest format version for parts.
|
|
//
|
|
// See partHeader.FormatVersion for details.
|
|
const partFormatLatestVersion = 2
|
|
|
|
// bloomValuesMaxShardsCount is the number of shards for bloomFilename and valuesFilename files.
|
|
//
|
|
// The partHeader.FormatVersion and partFormatLatestVersion must be updated when this number changes.
|
|
const bloomValuesMaxShardsCount = 128
|
|
|
|
// maxUncompressedIndexBlockSize contains the maximum length of uncompressed block with blockHeader entries aka index block.
|
|
//
|
|
// The real block length can exceed this value by a small percentage because of the block write details.
|
|
const maxUncompressedIndexBlockSize = 128 * 1024
|
|
|
|
// maxUncompressedBlockSize is the maximum size of uncompressed block in bytes.
|
|
//
|
|
// The real uncompressed block can exceed this value by up to 2 times because of block merge details.
|
|
const maxUncompressedBlockSize = 2 * 1024 * 1024
|
|
|
|
// maxRowsPerBlock is the maximum number of log entries a single block can contain.
|
|
const maxRowsPerBlock = 8 * 1024 * 1024
|
|
|
|
// maxColumnsPerBlock is the maximum number of columns per block.
|
|
//
|
|
// It isn't recommended setting this value to too big value, because this may result
|
|
// in excess memory usage during data ingestion and significant slowdown during query execution,
|
|
// since every column header is unpacked in every matching block during query execution.
|
|
const maxColumnsPerBlock = 1_000
|
|
|
|
// MaxFieldNameSize is the maximum size in bytes for field name.
|
|
//
|
|
// Longer field names are truncated during data ingestion to MaxFieldNameSize length.
|
|
const MaxFieldNameSize = 128
|
|
|
|
// maxConstColumnValueSize is the maximum size in bytes for const column value.
|
|
//
|
|
// Const column values are stored in columnsHeader, which is read every time the corresponding block is scanned during search queries.
|
|
// So it is better to store bigger values in regular columns in order to speed up search speed.
|
|
const maxConstColumnValueSize = 256
|
|
|
|
// maxIndexBlockSize is the maximum size of the block with blockHeader entries (aka indexBlock)
|
|
const maxIndexBlockSize = 8 * 1024 * 1024
|
|
|
|
// maxTimestampsBlockSize is the maximum size of timestamps block
|
|
const maxTimestampsBlockSize = 8 * 1024 * 1024
|
|
|
|
// maxValuesBlockSize is the maximum size of values block
|
|
const maxValuesBlockSize = 8 * 1024 * 1024
|
|
|
|
// maxBloomFilterBlockSize is the maximum size of bloom filter block
|
|
const maxBloomFilterBlockSize = 8 * 1024 * 1024
|
|
|
|
// maxColumnsHeaderSize is the maximum size of columnsHeader block
|
|
const maxColumnsHeaderSize = 8 * 1024 * 1024
|
|
|
|
// maxColumnsHeaderIndexSize is the maximum size of columnsHeaderIndex block
|
|
const maxColumnsHeaderIndexSize = 8 * 1024 * 1024
|
|
|
|
// maxDictSizeBytes is the maximum length of all the keys in the valuesDict.
|
|
//
|
|
// Dict is stored in columnsHeader, which is read every time the corresponding block is scanned during search qieries.
|
|
// So it is better to store bigger values in regular columns in order to speed up search speed.
|
|
const maxDictSizeBytes = 256
|
|
|
|
// maxDictLen is the maximum number of entries in the valuesDict.
|
|
//
|
|
// it shouldn't exceed 255, since the dict len is marshaled into a single byte.
|
|
const maxDictLen = 8
|