github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity

docs: mention staleNaN handling during deduplication

Browse source 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5587

Signed-off-by: hagen1778 <roman@victoriametrics.com>
This commit is contained in:
Artem Navoiev 2024-01-08 20:20:09 +01:00 committed by hagen1778
parent 91ccea236f
commit d374595e31
No known key found for this signature in database
GPG key ID: 3BF75F3741CA9640
2 changed files with 5 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
README.md
View file

@ -1767,6 +1767,9 @@ This aligns with the [staleness rules in Prometheus](https://prometheus.io/docs/
If multiple raw samples have **the same timestamp** on the given `-dedup.minScrapeInterval` discrete interval, If multiple raw samples have **the same timestamp** on the given `-dedup.minScrapeInterval` discrete interval,
then the sample with **the biggest value** is kept. then the sample with **the biggest value** is kept.
Prometheus stale markers are respected as any other value. If raw sample with the biggest timestamp on `-dedup.minScrapeInterval`
has a stale marker as a value - it will be kept after the deduplication.
Please note, [labels](https://docs.victoriametrics.com/keyConcepts.html#labels) of raw samples should be identical Please note, [labels](https://docs.victoriametrics.com/keyConcepts.html#labels) of raw samples should be identical
in order to be deduplicated. For example, this is why [HA pair of vmagents](https://docs.victoriametrics.com/vmagent.html#high-availability) in order to be deduplicated. For example, this is why [HA pair of vmagents](https://docs.victoriametrics.com/vmagent.html#high-availability)
needs to be identically configured. needs to be identically configured.

2
lib/storage/dedup.go
View file

@ -25,6 +25,8 @@ func isDedupEnabled() bool {
} }
// DeduplicateSamples removes samples from src* if they are closer to each other than dedupInterval in milliseconds. // DeduplicateSamples removes samples from src* if they are closer to each other than dedupInterval in milliseconds.
// DeduplicateSamples treats StaleNaN (Prometheus stale markers) as values and doesn't skip them on purpose - see
// https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5587
func DeduplicateSamples(srcTimestamps []int64, srcValues []float64, dedupInterval int64) ([]int64, []float64) { func DeduplicateSamples(srcTimestamps []int64, srcValues []float64, dedupInterval int64) ([]int64, []float64) {
if !needsDedup(srcTimestamps, dedupInterval) { if !needsDedup(srcTimestamps, dedupInterval) {
// Fast path - nothing to deduplicate // Fast path - nothing to deduplicate