VictoriaMetrics/lib/streamaggr
Andrii Chubatiuk 131367fb59
lib/streamaggr: add option to ignore first N aggregation intervals (#6137)
Stream aggregation may yield inaccurate results if it processes incomplete data.
This issue can arise when data is sourced from clients that maintain a queue of unsent data, such as Prometheus or vmagent.
 If the queue isn't fully cleared within the aggregation interval, only a portion of the time series may be included in that period, leading to distorted calculations.
To mitigate this we add an option to ignore first N aggregation intervals. It is expected, that client queues
will be cleared during the time while aggregation ignores first N intervals and all subsequent aggregations
will be correct.

(cherry picked from commit c0e4ccb7b5)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-22 14:34:36 +02:00
..
avg.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00
count_samples.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00
count_series.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00
dedup.go lib/streamaggr: follow-up for 15e33d56f1 2024-03-17 23:23:57 +02:00
dedup_test.go lib/streamaggr: pick sample with bigger timestamp or value on deduplicator (#5939) 2024-03-17 23:06:37 +02:00
dedup_timing_test.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00
deduplicator.go lib/streamaggr: pick sample with bigger timestamp or value on deduplicator (#5939) 2024-03-17 23:06:37 +02:00
deduplicator_test.go lib/streamaggr: follow-up for 15e33d56f1 2024-03-17 23:23:57 +02:00
deduplicator_timing_test.go all: fix golangci-lint(revive) warnings after 0c0ed61ce7 2024-04-03 03:00:45 +03:00
histogram_bucket.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00
last.go lib/streamaggr: ignore out of order samples for last output 2024-03-18 01:03:58 +02:00
max.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00
min.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00
quantiles.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00
stddev.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00
stdvar.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00
streamaggr.go lib/streamaggr: add option to ignore first N aggregation intervals (#6137) 2024-04-22 14:34:36 +02:00
streamaggr_test.go all: fix golangci-lint(revive) warnings after 0c0ed61ce7 2024-04-03 03:00:45 +03:00
streamaggr_timing_test.go all: fix golangci-lint(revive) warnings after 0c0ed61ce7 2024-04-03 03:00:45 +03:00
sum_samples.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00
total.go lib/streamaggr: ignore out of order samples when calculating increase, increase_prometheus, total and total_prometheus outputs 2024-03-17 23:24:14 +02:00
unique_samples.go lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:13:50 +02:00