15 commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	81f3e07e1e	lib/logstorage: do not count dictionary values which have no matching logs in count_uniq stats function ... Create blockResultColumn.forEachDictValue* helper functions for visiting matching dictionary values. These helper functions should prevent from counting dictionary values without matching logs in the future. This is a follow-up for 0c0f013a60 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7152	2024-10-01 13:36:27 +02:00
Aliaksandr Valialkin	dbcf06cd85	lib/logstorage: skip values with zero hits for 'uniq', 'top' and 'field_values' pipes ... See https://github.com/VictoriaMetrics/victorialogs-datasource/issues/72#issuecomment-2352078483	2024-09-30 14:16:21 +02:00
Aliaksandr Valialkin	fce6fc745a	lib/logstorage: return the expected hits results from uniq pipe when the number of unique values reaches the specified limit ... Previously `uniq` pipe could return zero `hits` if the number of found unique values equals the specified limit. This wasn't expected in most cases.	2024-09-29 10:53:44 +02:00
Aliaksandr Valialkin	58d1e517de	lib/logstorage: clear hits slice obtained from encoding.GetUint64s() before updating it with hits for valueTypeDict column ... encoding.GetUint64s() returns uninitialized slice, which may contain arbitrary values. So values in this slice must be reset to zero before using it for counting hits in `uniq` and `top` pipes.	2024-09-29 10:29:50 +02:00
Aliaksandr Valialkin	b5d94f06f5	lib/logstorage: postpone initialization of per-shard stateSizeBudget until the first call to pipeProcessor.writeBlock() ... This simplifies pipeProcessor initialization logic a bit. This also doesn't mangle the original maxStateSize value, which is used in error messages when the state size exceeds maxStateSize.	2024-09-29 10:29:49 +02:00
Aliaksandr Valialkin	246c339e3d	lib/logstorage: read timestamps column when it is really needed during query execution ... Previously timestamps column was read unconditionally on every query. This could significantly slow down queries, which do not need reading this column like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7070 .	2024-09-25 19:18:37 +02:00
Aliaksandr Valialkin	dd62a2b9d6	lib/logstorage: work-in-progress	2024-06-27 14:21:03 +02:00
Aliaksandr Valialkin	ce5e4c842a	lib/logstorage: fix golangci-lint warnings	2024-05-26 02:02:41 +02:00
Aliaksandr Valialkin	03fe4c8963	lib/logstorage: work-in-progress	2024-05-25 21:36:24 +02:00
Aliaksandr Valialkin	7a2a2f173e	lib/logstorage: work-in-progress	2024-05-24 03:07:07 +02:00
Aliaksandr Valialkin	04d0dd2542	lib/logstorage: work-in-progress	2024-05-22 21:01:28 +02:00
Aliaksandr Valialkin	582e7d5439	lib/logstorage: work-in-progress	2024-05-20 04:09:15 +02:00
Aliaksandr Valialkin	28626db066	lib/logstorage: work-in-progress ... (cherry picked from commit 0aa19a2837)	2024-05-16 09:35:55 +02:00
Aliaksandr Valialkin	32193b6059	lib/encoding: optimize UnmarshalVarUint64, UnmarshalVarInt64 and UnmarshalBytes a bit ... Change the return values for these functions - now they return the unmarshaled result plus the size of the unmarshaled result in bytes, so the caller could re-slice the src for further unmarshaling. This improves performance of these functions in hot loops of VictoriaLogs a bit.	2024-05-14 01:30:25 +02:00
Aliaksandr Valialkin	147704aab0	lib/logstorage: initial implementation of pipes in LogsQL ... See https://docs.victoriametrics.com/victorialogs/logsql/#pipes	2024-05-12 16:36:01 +02:00