VictoriaMetrics/app/vmselect
Zakhar Bessarab 401ae72587
app/vmselect/promql: propagate lower bucket values when fixing a histogram (#6547)
### Describe Your Changes

In most cases histograms are exposed in sorted manner with lower buckets
being first. This means that during scraping buckets with lower bounds
have higher chance of being updated earlier than upper ones.

Previously, values were propagated from upper to lower bounds, which
means that in most cases that would produce results higher than expected
once all buckets will become updated.
Propagating from upper bound effectively limits highest value of
histogram to the value of previous scrape. Once the data will become
consistent in the subsequent evaluation this causes spikes in the
result.

Changing propagation to be from lower to higher buckets reduces value
spikes in most cases due to nature of the original inconsistency.

 See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4580

An example histogram with previous(red) and updated(blue) versions:

![1719565540](https://github.com/VictoriaMetrics/VictoriaMetrics/assets/1367798/605c5e60-6abe-45b5-89b2-d470b60127b8)

This also makes logic of filling nan values with lower buckets values: [1 2 3 nan nan nan] => [1 2 3 3 3 3] obsolete.
Since buckets are now fixed from lower ones to upper this happens in the main loop, so there is no need in a second one.

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Andrii Chubatiuk <andrew.chubatiuk@gmail.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6a4bd5049b)
2024-07-10 15:17:08 +02:00
..
clusternative all: fix golangci-lint(revive) warnings after 0c0ed61ce7 2024-04-03 03:00:45 +03:00
deployment Rootless docker images by default (#358) 2020-03-27 21:18:32 +02:00
graphite all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
graphiteql app/vmselect: use strings.EqualFold instead of strings.ToLower where appropriate 2024-05-12 10:21:24 +02:00
multiarch all: run apk update && apk upgrade in base Alpine Docker image in order to get all the recent security fixes 2023-02-09 14:03:02 -08:00
netstorage all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
prometheus all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
promql app/vmselect/promql: propagate lower bucket values when fixing a histogram (#6547) 2024-07-10 15:17:08 +02:00
querystats app/vmselect: do not track queries with less than 1ms execution time at /api/v1/status/top_queries 2021-07-15 16:53:35 +03:00
searchutils lib/logstorage: work-in-progress 2024-06-27 14:21:03 +02:00
static/css app/vmselect: properly serve static css for /select/.../metric-relabel-debug page 2022-12-10 03:04:43 -08:00
vmui app/vmui: run make vmui-update after 65f414acee 2024-06-24 23:21:19 +02:00
main.go lib/logstorage: work-in-progress 2024-06-27 14:21:03 +02:00
Makefile Add build support for loong64 (#6222) 2024-05-10 14:32:05 +02:00
README.md app/vmselect: expose vmui at /vmselect/<accountID>/vmui/ instead of /vmselect/<accountID>/prometheus/vmui/ 2021-07-10 12:32:21 +03:00
vmui.go app/vmselect: add missing whitespace into the description for -vmui.defaultTimezone command-line flag 2024-02-01 14:49:48 +02:00

vmselect performs the following tasks:

  • Splits incoming selects to tasks for vmstorage nodes and issues these tasks to all the vmstorage nodes in the cluster.

  • Merges responses from all the vmstorage nodes and returns a single response.

The vmui directory contains static contents built from app/vmui package with make vmui-update command. The vmui page is available at http://<vmselect>:8481/select/<accountID>/vmui/.