Commit graph

8 commits

Author SHA1 Message Date
Aliaksandr Valialkin
6155b85a13
lib/logstorage: make sure that chunkedAllocator is isn't used from concurrently running goroutines
This is needed in order to avoid data races
2025-02-17 15:36:37 +01:00
Aliaksandr Valialkin
262ccf7963
lib/logstorage: explicitly pass statsFunc to statsProcessor methods
This allows reducing the state of every statsProcessor by removing pointer to the corresponding statsFunc.
For example, this reduces statsCountProcessor size by 2x.
2025-01-13 07:23:20 +01:00
Aliaksandr Valialkin
a2003de567
lib/logstorage: stats pipe: stop finalizeStats() as soon as the query is canceled
Previoysly finalizeStats() for some functions such as count_uniq() could run for long periods
of time after the query is canceled, since stopCh wan't propagated to finalizeStats().
2025-01-13 07:23:20 +01:00
Aliaksandr Valialkin
6b0da64b30
lib/logstorage: reduce memory allocations at stats and top pipes
Use chunked allocator in order to reduce memory allocations. It allocates objects from slices of up to 64Kb size.
This improves performance for `stats` and `top` pipes by up to 2x when they are applied to big number of `by (...)` groups.

Also parallelize execution of `count_uniq`, `count_uniq_hash` and `uniq_values` stats functions,
so they are executed faster on hosts with many CPU cores when applied to fields with big number
of unique values.
2024-12-23 19:45:02 +01:00
Aliaksandr Valialkin
95dfdc434a
lib/logstorage: eliminate memory allocations when finalizing per-group values calculated by stats pipe
This improves query performance a bit when `stats by (...)` returns millions of individual `by (...)` groups
2024-12-18 22:41:06 +01:00
Aliaksandr Valialkin
04d0dd2542
lib/logstorage: work-in-progress 2024-05-22 21:01:28 +02:00
Aliaksandr Valialkin
582e7d5439
lib/logstorage: work-in-progress 2024-05-20 04:09:15 +02:00
Aliaksandr Valialkin
147704aab0
lib/logstorage: initial implementation of pipes in LogsQL
See https://docs.victoriametrics.com/victorialogs/logsql/#pipes
2024-05-12 16:36:01 +02:00