Aliaksandr Valialkin db557b86ee app/vmagent/remotewrite: follow-up for f153f54d11 ... - Move the remaining code responsible for stream aggregation initialization from remotewrite.go to streamaggr.go . This improves code maintainability a bit. - Properly shut down streamaggr.Aggregators initialized inside remotewrite.CheckStreamAggrConfigs(). This prevents from potential resource leaks. - Use separate functions for initializing and reloading of global stream aggregation and per-remoteWrite.url stream aggregation. This makes the code easier to read and maintain. This also fixes INFO and ERROR logs emitted by these functions. - Add an ability to specify `name` option in every stream aggregation config. This option is used as `name` label in metrics exposed by stream aggregation at /metrics page. This simplifies investigation of the exposed metrics. - Add `path` label additionally to `name`, `url` and `position` labels at metrics exposed by streaming aggregation. This label should simplify investigation of the exposed metrics. - Remove `match` and `group` labels from metrics exposed by streaming aggregation, since they have little practical applicability: it is hard to use these labels in query filters and aggregation functions. - Rename the metric `vm_streamaggr_flushed_samples_total` to less misleading `vm_streamaggr_output_samples_total` . This metric shows the number of samples generated by the corresponding streaming aggregation rule. This metric has been added in the commit 861852f262 . See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6462 - Remove the metric `vm_streamaggr_stale_samples_total`, since it is unclear how it can be used in practice. This metric has been added in the commit 861852f262 . See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6462 - Remove Alias and aggrID fields from streamaggr.Options struct, since these fields aren't related to optional params, which could modify the behaviour of the constructed streaming aggregator. Convert the Alias field to regular argument passed to LoadFromFile() function, since this argument is mandatory. - Pass Options arg to LoadFromFile() function by reference, since this structure is quite big. This also allows passing nil instead of Options when default options are enough. - Add `name`, `path`, `url` and `position` labels to `vm_streamaggr_dedup_state_size_bytes` and `vm_streamaggr_dedup_state_items_count` metrics, so they have consistent set of labels comparing to the rest of streaming aggregation metrics. - Convert aggregator.aggrStates field type from `map[string]aggrState` to `[]aggrOutput`, where `aggrOutput` contains the corresponding `aggrState` plus all the related metrics (currently only `vm_streamaggr_output_samples_total` metric is exposed with the corresponding `output` label per each configured output function). This simplifies and speeds up the code responsible for updating per-output metrics. This is a follow-up for the commit 2eb1bc4f81 . See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6604 - Added missing urls to docs ( https://docs.victoriametrics.com/stream-aggregation/ ) in error messages. These urls help users figuring out why VictoriaMetrics or vmagent generates the corresponding error messages. The urls were removed for unknown reason in the commit 2eb1bc4f81 . - Fix incorrect update for `vm_streamaggr_output_samples_total` metric in flushCtx.appendSeriesWithExtraLabel() function. While at it, reduce memory usage by limiting the maximum number of samples per flush to 10K. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5467 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6268		2024-07-15 20:24:01 +02:00
..
avg.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:20:37 +02:00
count_samples.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:20:37 +02:00
count_series.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:20:37 +02:00
dedup.go	lib/streamaggr: prevent from data race inside dedupAggrShard when samplesBuf can be updated in pushSamples() while their values are read in the flush() loop without das.mu lock	2024-06-11 17:31:16 +02:00
dedup_test.go	lib/streamaggr: reduce memory allocations by using dedupAggrSample buffer per each dedupAggrShard	2024-06-10 16:38:42 +02:00
dedup_timing_test.go	lib/streamaggr: reduce the number of duplicates per each sample in BenchmarkDedupAggr from 100 to 2	2024-06-10 16:38:41 +02:00
deduplicator.go	app/vmagent/remotewrite: follow-up for f153f54d11	2024-07-15 20:24:01 +02:00
deduplicator_test.go	app/vmagent/remotewrite,lib/streamaggr: re-use common code in tests after 879771808b	2024-07-03 15:21:36 +02:00
deduplicator_timing_test.go	lib/streamaggr: added stale samples metric, added metrics labels (#6462)	2024-07-01 14:56:17 +02:00
histogram_bucket.go	app/vmagent/remotewrite: follow-up for f153f54d11	2024-07-15 20:24:01 +02:00
last.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:20:37 +02:00
max.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:20:37 +02:00
min.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:20:37 +02:00
quantiles.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:20:37 +02:00
rate.go	app/vmagent/remotewrite: follow-up for f153f54d11	2024-07-15 20:24:01 +02:00
stddev.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:20:37 +02:00
stdvar.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:20:37 +02:00
streamaggr.go	app/vmagent/remotewrite: follow-up for f153f54d11	2024-07-15 20:24:01 +02:00
streamaggr_test.go	app/vmagent/remotewrite: follow-up for f153f54d11	2024-07-15 20:24:01 +02:00
streamaggr_timing_test.go	app/vmagent/remotewrite: follow-up for f153f54d11	2024-07-15 20:24:01 +02:00
sum_samples.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:20:37 +02:00
total.go	app/vmagent/remotewrite: follow-up for f153f54d11	2024-07-15 20:24:01 +02:00
unique_samples.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:20:37 +02:00