github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
VictoriaMetrics/lib/logstorage
History
Aliaksandr Valialkin 202eb429a7
lib/logstorage: refactor storage format to be more efficient for querying wide events 
It has been appeared that VictoriaLogs is frequently used for collecting logs with tens of fields.
For example, standard Kuberntes setup on top of Filebeat generates more than 20 fields per each log.
Such logs are also known as "wide events".

The previous storage format was optimized for logs with a few fields. When at least a single field
was referenced in the query, then the all the meta-information about all the log fields was unpacked
and parsed per each scanned block during the query. This could require a lot of additional disk IO
and CPU time when logs contain many fields. Resolve this issue by providing an (field -> metainfo_offset)
index per each field in every data block. This index allows reading and extracting only the needed
metainfo for fields used in the query. This index is stored in columnsHeaderIndexFilename ( columns_header_index.bin ).
This allows increasing performance for queries over wide events by 10x and more.

Another issue was that the data for bloom filters and field values across all the log fields except of _msg
was intermixed in two files - fieldBloomFilename ( field_bloom.bin ) and fieldValuesFilename ( field_values.bin ).
This could result in huge disk read IO overhead when some small field was referred in the query,
since the Operating System usually reads more data than requested. It reads the data from disk
in at least 4KiB blocks (usually the block size is much bigger in the range 64KiB - 512KiB).
So, if 512-byte bloom filter or values' block is read from the file, then the Operating System
reads up to 512KiB of data from disk, which results in 1000x disk read IO overhead. This overhead isn't visible
for recently accessed data, since this data is usually stored in RAM (aka Operating System page cache),
but this overhead may become very annoying when performing the query over large volumes of data
which isn't present in OS page cache.

The solution for this issue is to split bloom filters and field values across multiple shards.
This reduces the worst-case disk read IO overhead by at least Nx where N is the number of shards,
while the disk read IO overhead is completely removed in best case when the number of columns doesn't exceed N.
Currently the number of shards is 8 - see bloomValuesShardsCount . This solution increases
performance for queries over large volumes of newly ingested data by up to 1000x.

The new storage format is versioned as v1, while the old storage format is version as v0.
It is stored in the partHeader.FormatVersion.

Parts with the old storage format are converted into parts with the new storage format during background merge.
It is possible to force merge by querying /internal/force_merge HTTP endpoint - see https://docs.victoriametrics.com/victorialogs/#forced-merge .
2024-10-16 17:35:07 +02:00
..
arena.go lib/logstorage: work-in-progress 2024-05-20 04:08:30 +02:00
arena_test.go lib/logstorage: work-in-progress 2024-05-20 04:08:30 +02:00
bitmap.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
bitmap_test.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
bitmap_timing_test.go lib/logstorage: work-in-progress 2024-06-10 18:42:19 +02:00
block.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
block_data.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
block_data_test.go lib/logstorage: initial implementation of pipes in LogsQL 2024-05-12 16:33:31 +02:00
block_header.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
block_header_test.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
block_result.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
block_search.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
block_stream_merger.go all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:20:37 +02:00
block_stream_reader.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
block_stream_writer.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
block_test.go lib/logstorage: initial implementation of pipes in LogsQL 2024-05-12 16:33:31 +02:00
block_timing_test.go app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
bloomfilter.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
bloomfilter_test.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
column_names.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
column_names_test.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
consts.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
datadb.go app/vlstorage: add support for forced merge via /internal/force_merge HTTP endpoint 2024-10-13 22:20:31 +02:00
datadb_test.go app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
encoding.go lib/encoding: optimize UnmarshalVarUint64, UnmarshalVarInt64 and UnmarshalBytes a bit 2024-05-14 01:23:54 +02:00
encoding_test.go lib/logstorage: fix make test-pure tests 2023-07-04 13:14:30 -07:00
encoding_timing_test.go app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
fields_set.go lib/logstorage: work-in-progress 2024-05-30 16:19:23 +02:00
fields_set_test.go lib/logstorage: work-in-progress 2024-05-20 04:08:30 +02:00
filenames.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
filter.go lib/logstorage: work-in-progress 2024-05-20 04:08:30 +02:00
filter_and.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_and_test.go lib/logstorage: preserve the order of tokens to check against bloom filters in AND filters 2024-09-08 12:27:30 +02:00
filter_any_case_phrase.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_any_case_phrase_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_any_case_prefix.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_any_case_prefix_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_day_range.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
filter_day_range_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_exact.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_exact_prefix.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_exact_prefix_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_exact_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_in.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_in_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_ipv4_range.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_ipv4_range_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_len_range.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_len_range_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_noop.go lib/logstorage: work-in-progress 2024-06-04 01:49:02 +02:00
filter_not.go lib/logstorage: allow using - instead of ! as a shorthand for NOT operator in LogsQL 2024-09-27 13:14:47 +02:00
filter_not_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_or.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_or_test.go lib/logstorage: preserve the order of tokens to check against bloom filters in AND filters 2024-09-08 12:27:30 +02:00
filter_phrase.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_phrase_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_prefix.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_prefix_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_range.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_range_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_regexp.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_regexp_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_sequence.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_sequence_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_stream.go lib/logstorage: support skipping _stream: prefix for stream filters 2024-09-27 13:14:46 +02:00
filter_stream_id.go lib/logstorage: work-in-progress 2024-06-24 23:27:12 +02:00
filter_stream_id_test.go lib/logstorage: work-in-progress 2024-06-24 23:27:12 +02:00
filter_string_range.go lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch 2024-10-13 14:29:02 +02:00
filter_string_range_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_test.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
filter_time.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
filter_time_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
filter_timing_test.go lib/logstorage: initial implementation of pipes in LogsQL 2024-05-12 16:33:31 +02:00
filter_week_range.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
filter_week_range_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
hash128.go app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
hash128_test.go app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
hash128_timing_test.go lib/logstorage: consistently use atomic.* types instead of atomic.* functions on regular types 2024-02-23 23:46:13 +02:00
hash_tokenizer.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
hash_tokenizer_test.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
hash_tokenizer_timing_test.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
if_filter.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
index_block_header.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
index_block_header_test.go app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
indexdb.go lib/logstorage: work-in-progress 2024-05-20 04:08:30 +02:00
indexdb_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
inmemory_part.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
inmemory_part_test.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
inmemory_part_timing_test.go app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
json_parser.go lib/logstorage: work-in-progress 2024-06-17 12:13:18 +02:00
json_parser_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
log_rows.go lib/logstorage: work-in-progress 2024-06-17 12:13:18 +02:00
log_rows_timing_test.go app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
logfmt_parser.go lib/logstorage: work-in-progress 2024-06-04 01:49:02 +02:00
logfmt_parser_test.go app/vlselect: do not show empty fields in query results 2024-10-14 23:43:58 +02:00
parser.go lib/logstorage: disallow using pipe names as the first unquoted words in filter pipe 2024-10-09 16:10:13 +02:00
parser_test.go lib/logstorage: disallow using pipe names as the first unquoted words in filter pipe 2024-10-09 16:10:13 +02:00
part.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
part_header.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
part_header_test.go app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
partition.go app/vlstorage: add support for forced merge via /internal/force_merge HTTP endpoint 2024-10-13 22:20:31 +02:00
partition_test.go lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
pattern.go lib/logstorage: work-in-progress 2024-05-24 03:06:55 +02:00
pattern_test.go lib/logstorage: work-in-progress 2024-05-24 03:06:55 +02:00
pattern_timing_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
pipe.go lib/logstorage: disallow using by as the first word in log filters, since it frequently clashes with stats by(...) pipe where stats word is omitted 2024-10-09 15:53:15 +02:00
pipe_blocks_count.go lib/logstorage: add blocks_count pipe 2024-09-25 19:17:48 +02:00
pipe_blocks_count_test.go lib/logstorage: add blocks_count pipe 2024-09-25 19:17:48 +02:00
pipe_copy.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_copy_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
pipe_delete.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_delete_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
pipe_drop_empty_fields.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_drop_empty_fields_test.go lib/logstorage: work-in-progress 2024-06-04 01:49:02 +02:00
pipe_extract.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_extract_regexp.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_extract_regexp_test.go lib/logstorage: work-in-progress 2024-05-28 19:29:41 +02:00
pipe_extract_test.go lib/logstorage: work-in-progress 2024-05-28 19:29:41 +02:00
pipe_field_names.go lib/logstorage: add blocks_count pipe 2024-09-25 19:17:48 +02:00
pipe_field_names_test.go lib/logstorage: work-in-progress 2024-05-24 03:06:55 +02:00
pipe_field_values.go lib/logstorage: add len pipe for calculating byte length of log field values 2024-10-03 18:21:10 +02:00
pipe_field_values_test.go lib/logstorage: add len pipe for calculating byte length of log field values 2024-10-03 18:21:10 +02:00
pipe_fields.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_fields_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
pipe_filter.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_filter_test.go lib/logstorage: work-in-progress 2024-06-10 18:42:19 +02:00
pipe_format.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_format_test.go lib/logstorage: work-in-progress 2024-06-05 03:18:12 +02:00
pipe_len.go lib/logstorage: add len pipe for calculating byte length of log field values 2024-10-03 18:21:10 +02:00
pipe_len_test.go lib/logstorage: add len pipe for calculating byte length of log field values 2024-10-03 18:21:10 +02:00
pipe_limit.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_limit_test.go lib/logstorage: work-in-progress 2024-05-28 19:29:41 +02:00
pipe_math.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_math_test.go lib/logstorage: substitute | operator with or operator at math pipe 2024-09-06 22:44:14 +02:00
pipe_offset.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_offset_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
pipe_pack.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_pack_json.go lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
pipe_pack_json_test.go app/vlselect: do not show empty fields in query results 2024-10-14 23:43:58 +02:00
pipe_pack_logfmt.go lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
pipe_pack_logfmt_test.go lib/logstorage: quote logfmt strings only if they contain special chars, which could break logfmt parsing and/or reading 2024-10-07 14:31:30 +02:00
pipe_rename.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_rename_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
pipe_replace.go lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
pipe_replace_regexp.go lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
pipe_replace_regexp_test.go lib/logstorage: work-in-progress 2024-05-25 21:36:16 +02:00
pipe_replace_test.go lib/logstorage: work-in-progress 2024-05-25 21:36:16 +02:00
pipe_sort.go lib/logstorage: postpone initialization of per-shard stateSizeBudget until the first call to pipeProcessor.writeBlock() 2024-09-29 10:29:13 +02:00
pipe_sort_test.go lib/logstorage: support order alias for sort pipe 2024-09-29 09:51:27 +02:00
pipe_sort_topk.go lib/logstorage: postpone initialization of per-shard stateSizeBudget until the first call to pipeProcessor.writeBlock() 2024-09-29 10:29:13 +02:00
pipe_stats.go lib/logstorage: postpone initialization of per-shard stateSizeBudget until the first call to pipeProcessor.writeBlock() 2024-09-29 10:29:13 +02:00
pipe_stats_test.go lib/logstorage: work-in-progress 2024-05-30 16:19:23 +02:00
pipe_stream_context.go lib/logstorage: postpone initialization of per-shard stateSizeBudget until the first call to pipeProcessor.writeBlock() 2024-09-29 10:29:13 +02:00
pipe_stream_context_test.go lib/logstorage: improve performance for stream_context pipe over streams with big number of log entries 2024-09-26 22:22:23 +02:00
pipe_top.go lib/logstorage: do not count dictionary values which have no matching logs in count_uniq stats function 2024-10-01 13:34:45 +02:00
pipe_top_test.go lib/logstorage: work-in-progress 2024-06-17 12:13:18 +02:00
pipe_uniq.go lib/logstorage: do not count dictionary values which have no matching logs in count_uniq stats function 2024-10-01 13:34:45 +02:00
pipe_uniq_test.go lib/logstorage: return the expected hits results from uniq pipe when the number of unique values reaches the specified limit 2024-09-29 10:51:09 +02:00
pipe_unpack.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_unpack_json.go lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
pipe_unpack_json_test.go lib/logstorage: work-in-progress 2024-05-25 21:36:16 +02:00
pipe_unpack_logfmt.go lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
pipe_unpack_logfmt_test.go lib/logstorage: work-in-progress 2024-06-04 01:49:02 +02:00
pipe_unpack_syslog.go lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
pipe_unpack_syslog_test.go lib/logstorage: work-in-progress 2024-06-17 12:13:18 +02:00
pipe_unroll.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_unroll_test.go lib/logstorage: work-in-progress 2024-05-30 16:19:23 +02:00
pipe_update.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
pipe_utils_test.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
rows.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
rows_test.go lib/logstorage: use quicktemplate.AppendJSONString instead of strconv.AppendQuote for encoding JSON strings 2024-07-05 01:22:23 +02:00
stats_avg.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_avg_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_count.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
stats_count_empty.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
stats_count_empty_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_count_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_count_uniq.go lib/logstorage: do not count dictionary values which have no matching logs in count_uniq stats function 2024-10-01 13:34:45 +02:00
stats_count_uniq_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_max.go lib/logstorage: improve performance for stream_context pipe over streams with big number of log entries 2024-09-26 22:22:23 +02:00
stats_max_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_median.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_median_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_min.go lib/logstorage: improve performance for stream_context pipe over streams with big number of log entries 2024-09-26 22:22:23 +02:00
stats_min_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_quantile.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
stats_quantile_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_row_any.go lib/logstorage: read timestamps column when it is really needed during query execution 2024-09-25 19:17:47 +02:00
stats_row_any_test.go app/vlselect: do not show empty fields in query results 2024-10-14 23:43:58 +02:00
stats_row_max.go lib/logstorage: improve performance for stream_context pipe over streams with big number of log entries 2024-09-26 22:22:23 +02:00
stats_row_max_test.go app/vlselect: do not show empty fields in query results 2024-10-14 23:43:58 +02:00
stats_row_min.go lib/logstorage: improve performance for stream_context pipe over streams with big number of log entries 2024-09-26 22:22:23 +02:00
stats_row_min_test.go app/vlselect: do not show empty fields in query results 2024-10-14 23:43:58 +02:00
stats_sum.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_sum_len.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_sum_len_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_sum_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
stats_uniq_values.go lib/logstorage: return the expected hits results from uniq pipe when the number of unique values reaches the specified limit 2024-09-29 10:51:09 +02:00
stats_uniq_values_test.go lib/logstorage: work-in-progress 2024-05-25 00:30:58 +02:00
stats_values.go lib/logstorage: return the expected hits results from uniq pipe when the number of unique values reaches the specified limit 2024-09-29 10:51:09 +02:00
stats_values_test.go lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
storage.go app/vlstorage: add support for forced merge via /internal/force_merge HTTP endpoint 2024-10-13 22:20:31 +02:00
storage_search.go lib/logstorage: make a copy of s.partitions slice when performing queries over the selected partitions 2024-10-13 22:14:34 +02:00
storage_search_test.go lib/logstorage: return the expected hits results from uniq pipe when the number of unique values reaches the specified limit 2024-09-29 10:51:09 +02:00
storage_test.go lib/logstorage: consistently use nsecsPerDay constant and remove nsecPerDay constant 2024-09-06 16:17:04 +02:00
stream_filter.go lib/logstorage: allow special chars in unquoted _stream tag names and values 2024-10-07 15:10:03 +02:00
stream_filter_test.go lib/logstorage: allow special chars in unquoted _stream tag names and values 2024-10-07 15:10:03 +02:00
stream_id.go lib/logstorage: improve performance for streamID.marshalString() by more than 2x 2024-09-24 18:35:04 +02:00
stream_id_test.go lib/logstorage: add benchmark for streamID.marshalString 2024-09-24 18:31:38 +02:00
stream_id_timing_test.go lib/logstorage: add benchmark for streamID.marshalString 2024-09-24 18:31:38 +02:00
stream_tags.go app/vlinsert/loki: use easyproto instead for parsing Loki protobuf messages 2024-07-10 03:05:17 +02:00
stringbucket.go lib/logstorage: initial implementation of pipes in LogsQL 2024-05-12 16:33:31 +02:00
syslog_parser.go app/vlinsert: support _time field without timezone information during data ingestion 2024-09-26 12:49:35 +02:00
syslog_parser_test.go app/vlselect: do not show empty fields in query results 2024-10-14 23:43:58 +02:00
tenant_id.go lib/logstorage: improve performance for streamID.marshalString() by more than 2x 2024-09-24 18:35:04 +02:00
tenant_id_test.go lib: consistently use f-tests instead of table-driven tests 2024-07-09 22:40:50 +02:00
tokenizer.go lib/logstorage: refactor storage format to be more efficient for querying wide events 2024-10-16 17:35:07 +02:00
tokenizer_test.go lib/logstorage: initial implementation of pipes in LogsQL 2024-05-12 16:33:31 +02:00
tokenizer_timing_test.go app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
u128.go lib/logstorage: improve performance for streamID.marshalString() by more than 2x 2024-09-24 18:35:04 +02:00
u128_test.go app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
values_encoder.go lib/logstorage: avoid redundant copying of column names and column values for dictionary-encoded columns during querying 2024-10-13 13:25:38 +02:00
values_encoder_test.go app/vlinsert: support _time field without timezone information during data ingestion 2024-09-26 12:49:35 +02:00
values_encoder_timing_test.go lib/logstorage: properly parse timezone offset at TryParseTimestampRFC3339Nano() 2024-06-25 14:53:38 +02:00