Commit graph

7 commits

Author SHA1 Message Date
Aliaksandr Valialkin
186aa3bb0e
lib/logstorage: explicitly pass statsFunc to statsProcessor methods
This allows reducing the state of every statsProcessor by removing pointer to the corresponding statsFunc.
For example, this reduces statsCountProcessor size by 2x.
2025-01-13 04:49:39 +01:00
Aliaksandr Valialkin
e368f687a7
lib/logstorage: stats pipe: stop finalizeStats() as soon as the query is canceled
Previoysly finalizeStats() for some functions such as count_uniq() could run for long periods
of time after the query is canceled, since stopCh wan't propagated to finalizeStats().
2025-01-13 03:38:09 +01:00
Aliaksandr Valialkin
60f9f44150
lib/logstorage: reduce memory allocations at stats and top pipes
Use chunked allocator in order to reduce memory allocations. It allocates objects from slices of up to 64Kb size.
This improves performance for `stats` and `top` pipes by up to 2x when they are applied to big number of `by (...)` groups.

Also parallelize execution of `count_uniq`, `count_uniq_hash` and `uniq_values` stats functions,
so they are executed faster on hosts with many CPU cores when applied to fields with big number
of unique values.
2024-12-22 02:13:02 +01:00
Aliaksandr Valialkin
524f0e8d8b
lib/logstorage: eliminate memory allocations when finalizing per-group values calculated by stats pipe
This improves query performance a bit when `stats by (...)` returns millions of individual `by (...)` groups
2024-12-17 15:17:01 +01:00
Aliaksandr Valialkin
22107421eb
lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
Aliaksandr Valialkin
ad505a7a9a
lib/logstorage: work-in-progress 2024-05-20 04:08:30 +02:00
Aliaksandr Valialkin
0aa19a2837
lib/logstorage: work-in-progress 2024-05-15 04:55:44 +02:00