Aliaksandr Valialkin
cfc1193d15
app/vmselect/netstorage: limit the maximum brsPool size to 32Kb at ProcessSearchQuery()
...
This avoids slow path in Go runtime for allocating objects bigger than 32Kb -
see 704401ffa0/src/runtime/malloc.go (L11)5dd37ad836https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5527
2024-01-23 14:12:27 +02:00
Aliaksandr Valialkin
fe4ea30a79
app/vmselect/netstorage: limit the size of metricNamesBuf to 32Kb in order to avoid slow path at Go runtime for allocating a byte slice of bigger size
...
See 704401ffa0/src/runtime/malloc.go (L11)508c608062https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5527
2024-01-23 13:50:59 +02:00
Aliaksandr Valialkin
1880c656c1
app/vmselect/vmui: run make vmui-update in order to sync recent changes in app/vmui
2024-01-23 04:31:57 +02:00
Yury Molodov
1db2b991b7
vmui: query report ( #5497 )
...
* vmui: add query analyzer page
* vmui: fix tabs for query analyzer
* vmui: add help to export query
* vmui: add time params to query analyzer
* docs/vmui: add query analyzer
* vmui: fix validation JSON form
---------
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-23 04:26:04 +02:00
Yury Molodov
3a26e4d6ec
vmui: add flag for default timezone setting ( #5611 )
...
* vmui: add flag for default timezone setting #5375
* vmui: validate timezone before client return
* Update app/vmselect/vmui.go
---------
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-23 04:15:14 +02:00
Yury Molodov
574d69775e
vmui: fix cache autocomplete ( #5591 )
...
* vmui: fix the logic of closing the popper #5470
* vmui: fix the logic of caching autocomplete results #5472
---------
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-23 04:06:39 +02:00
Aliaksandr Valialkin
e20cbfcbc3
app/vmselect/promql: remove superflouos memory allocations at aggrPrepareSeries()
...
While at it, also remove unneeded map lookup
2024-01-23 02:29:14 +02:00
Aliaksandr Valialkin
cd4f641d32
app/{vmstorage,vmselect}: disable vmstorage->vmselect RPC compression by default in order to improve query performance
2024-01-23 02:29:13 +02:00
Aliaksandr Valialkin
b7cc1af3eb
app/vmselect/promql/aggr_incremental.go: eliminate unnecessary memory allocation in incrementalAggrFuncContext.updateTimeseries
2024-01-23 02:29:13 +02:00
Aliaksandr Valialkin
953b96ced2
app/vmselect/netstorage: remove tswPool, since it isnt efficient
2024-01-23 02:29:13 +02:00
Aliaksandr Valialkin
68a59bfabd
app/vmselect/netstorage: avoid metricName->blockRef lookup when processing multiple blocks for the same time series
...
This saves a few CPU cycles for common case
2024-01-23 02:29:12 +02:00
Aliaksandr Valialkin
f8a9ef8cbd
app/vmselect/netstorage: group per-vmstorage fields at tmpBlocksFileWrapperShard
...
This improves code readability a bit
2024-01-23 02:29:12 +02:00
Aliaksandr Valialkin
d52b121222
app/vmselect/netstorage: use []blockRef from blockRefPool in order to reduce memory allocations
2024-01-23 02:29:11 +02:00
Aliaksandr Valialkin
5b05224eb9
app/vmselect/netstorage: substitute pointer to blockRefs by brssPool index at the metricName->blockRefs map
...
This should reduce the pressure on Go GC, since it will see lower number of pointers.
This change has been extracted from https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5527
2024-01-23 02:29:11 +02:00
Aliaksandr Valialkin
b289f15f02
app/vmselect/netstorage: reduce the number of allocations for blockRefs objects in ProcessSearchQuery()
...
This should reduce pressure on Go GC at vmselect
The change has been extracted from https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5527
2024-01-23 02:29:11 +02:00
Aliaksandr Valialkin
2ab9a75cca
app/vmselect/netstorage: reduce the number of memory allocations in ProcessSearchQuery() by storing all the metric names in a single byte slice
...
This reduces the number of memory allocations at the cost of possible memory usage increase,
since now different metric name strings may hold references to the previous byte slice.
This is good tradeoff, since ProcessSearchQuery is called in vmselect, and vmselect isn't usually limited by memory.
This change has been extracted from https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5527
2024-01-23 02:29:10 +02:00
hagen1778
8138499439
app/vmctl/backoff: fix flaky test
...
The change removes artificial delay before returning error, which sometimes
caused less retry events than expected.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-22 18:42:05 +02:00
hagen1778
ede466be56
docs: fix Grafana link example for vmalert
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-22 18:41:38 +02:00
Aliaksandr Valialkin
d52fd73f18
all: add up to 10% random jitter to the interval between periodic tasks performed by various components
...
This should smooth CPU and RAM usage spikes related to these periodic tasks,
by reducing the probability that multiple concurrent periodic tasks are performed at the same time.
2024-01-22 18:39:16 +02:00
Aliaksandr Valialkin
3230525c36
docs: use persistent links to Grafana dashboards
...
These links do not depend on the dashboard name, so they do not break after the renaming of the dashboard.
This is a follow-up for ff33e60a3d
2024-01-22 01:45:42 +02:00
Aliaksandr Valialkin
d4a1a28543
app/vmselect: handle negative time range start in a generic manner inside NewSearchQuery()
...
This is a follow-up for cf03e11d89https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5553
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5630
2024-01-22 01:39:27 +02:00
Aliaksandr Valialkin
885ee160c2
all: allow dynamically reading *AuthKey flag values from files and urls
...
Examples:
1) -metricsAuthKey=file:///abs/path/to/file - reads flag value from the given absolute filepath
2) -metricsAuthKey=file://./relative/path/to/file - reads flag value from the given relative filepath
3) -metricsAuthKey=http://some-host/some/path?query_arg=abc - reads flag value from the given url
The flag value is automatically updated when the file contents changes.
2024-01-22 01:23:23 +02:00
Aliaksandr Valialkin
5f5fcab217
all: call atomic.Load* in front of atomic.CompareAndSwap* at places where the atomic.CompareAndSwap* returns false most of the time
...
This allows avoiding slow inter-CPU synchornization induced by atomic.CompareAndSwap*
2024-01-22 01:13:41 +02:00
Nikolay
73c51072e6
app/vmauth: adds metric_labels and backend_errors counter ( #5585 )
...
* app/vmauth: adds metric_labels and backend_errors counter
it must improve observability for user requests with new metric - per user backend errors counter.
it's needed to calculate requests fail rate to the configured backends.
metric_labels configuration allows to perform additional aggregations on top of multiple users from configuration section.
It could be multiple clients or clients with separate read/write tokens
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5565
* wip
---------
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-22 01:09:51 +02:00
Yury Molodov
0582ec5c8c
vmui: add autofocus to input for desktop version #5479 ( #5592 )
2024-01-22 01:09:27 +02:00
Hui Wang
e086ef16da
app/vmselect/promql: properly handle possible negative results caused… ( #5608 )
...
* app/vmselect/promql: properly handle possible negative results caused by float operations precision error in rollup functions like rate() or increase()
* fix test
2024-01-22 01:04:50 +02:00
Roman Khavronenko
fe4934f0ec
app/vmui: send step param for instant queries ( #5639 )
...
The change reverts https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3896
due to reasons explained in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3896#issuecomment-1896704401
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-22 00:25:31 +02:00
Roman Khavronenko
148e14b3f2
app/vmselect: properly calculate start param for queries with too big look-behind window ( #5630 )
...
Properly determine time range search for instant queries with too big look-behind window like `foo[100y]`.
Previously, such queries could return empty responses even if `foo` is present in database.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5553
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-21 23:47:09 +02:00
Aliaksandr Valialkin
2b67944eb4
app/vmselect/graphite: properly handle -N index for the array of N items
...
This is a follow-up for 70cd09e736https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5581
2024-01-17 00:16:37 +02:00
Aliaksandr Valialkin
b3823bc091
app/vmctl: disallow insecure https connections to vm-native-dst-addr and vm-native-src-addr by default
...
It is better from security PoV to disallow insecure https connections
to vm-native-dst-addr and vm-native-src-addr . This also maintains backwards compatibility
with vmctl before the commit 828aca82e9https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5595
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5606
2024-01-17 00:15:50 +02:00
Yury Molodov
9588b9bd19
vmui/vmanomaly: add support models that produce only anomaly_score ( #5594 )
...
* vmui/vmanomaly: add support models that produce only `anomaly_score`
* vmui/vmanomaly: fix display legend
* vmui/vmanomaly: update comment on anomaly threshold
2024-01-17 00:12:43 +02:00
Aliaksandr Valialkin
b7fcdb1985
deployment/docker: update Go builder from Go1.21.5 to Go1.21.6
2024-01-17 00:05:24 +02:00
Aliaksandr Valialkin
84f11a9e6d
app/vmselect/promql: simplify the code after 388d020b7c
...
Add a test, which verifies the correct sorting of float64 slices with NaNs.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5506
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509
2024-01-16 22:35:51 +02:00
Aliaksandr Valialkin
6ba2fd3312
app/vmselect/promql: follow-up for ce4f26db02
...
- Document the bugfix at docs/CHANGELOG.md
- Filter out NaN values before sorting as suggested at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509#discussion_r1447369218
- Revert unrelated changes in lib/filestream and lib/fs
- Use simpler test at app/vmselect/promql/exec_test.go
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5506
2024-01-16 22:13:13 +02:00
Zongyang
cb37df5723
FIX bottomk doesn't return any data when there are no time range overlap between timeseries ( #5509 )
...
* FIX sort order in bottomk
* Add lessWithNaNsReversed for bottomk
* Add ut for TopK
* Move lt from loop
* FIX lint
* FIX lint
* FIX lint
* Mod log format
---------
Co-authored-by: xiaozongyang <xiaozngyang@kanyun.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-16 22:12:49 +02:00
Aliaksandr Valialkin
015c0a4d1a
app/vmselect/promql: consistently sort results of a or b query
...
Previously the order of results returned from `a or b` query could change with each request
because the sorting for such query has been disabled in order to satisfy
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763 .
This commit executes `a or b` query as `sortByMetricName(a) or sortByMetricName(b)`.
This makes the order of returned time series consistent across requests,
while maintaining the requirement from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763 ,
e.g. `b` results are consistently put after `a` results.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5393
2024-01-16 22:12:15 +02:00
Aliaksandr Valialkin
825cfdb5ef
app/vmstorage: expose proper types for storage metrics when -metrics.exposeMetadata command-line flag is set
...
This is a follow-up for 326a77c697
2024-01-16 22:03:31 +02:00
Aliaksandr Valialkin
9b449dfadf
app/vmstorage: deregister storage metrics before stopping the storage
...
This prevents from possible nil pointer dereference issues when the storage metrics
are read after the storage is stopped.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
2024-01-16 21:26:03 +02:00
Aliaksandr Valialkin
9e5e514faf
lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop()
...
Previously the was a race condition when the background goroutine still could try collecting metrics
from already stopped resources after returning from pushmetrics.Stop().
Now the pushmetrics.Stop() waits until the background goroutine is stopped before returning.
This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5549
and the commit fe2d9f6646https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
2024-01-16 21:18:22 +02:00
rbizos
62db64e71b
Handling negative index in Graphite groupByNode/aliasByNode ( #5581 )
...
Handeling the error case with -1
Signed-off-by: Raphael Bizos <r.bizos@criteo.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-01-16 20:55:27 +02:00
Aliaksandr Valialkin
d566aa7d78
lib/prompbmarshal: switch to github.com/VictoriaMetrics/easyproto
2024-01-16 20:48:30 +02:00
Aliaksandr Valialkin
063ea8c773
app/vmalert/remotewrite: properly calculate vmalert_remotewrite_dropped_rows_total
...
It was calculating the number of dropped time series instead of the number of dropped samples.
While at it, drop vmalert_remotewrite_dropped_bytes_total metric, since it was inconsistently calculated -
at one place it was calculating raw protobuf-encoded sample sizes, while at another place it was calculating
the size of snappy-compressed prompbmarshal.WriteRequest protobuf message.
Additionally, this metric has zero practical sense, so just drop it in order to reduce the level of confusion.
2024-01-16 20:47:13 +02:00
Aliaksandr Valialkin
f7b589e38a
lib/prompb: switch to github.com/VictoriaMetrics/easyproto
2024-01-16 20:43:09 +02:00
Aliaksandr Valialkin
7d40506744
lib/prompb: change type of Label.Name and Label.Value from []byte to string
...
This makes it more consistent with lib/prompbmarshal.Label
2024-01-16 20:41:37 +02:00
Dmytro Kozlov
b95d6f5f5e
app/vmctl: add insecure skip verify flags for source and destination addresses for native protocol ( #5606 )
...
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5595
2024-01-16 20:29:41 +02:00
hagen1778
2a7207f38a
app/all: follow-up after 84d710beab
...
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-09 13:17:09 +01:00
zhdd99
84d710beab
lib/pushmetrics: fix a panic caused by pushing metrics during the graceful shutdown process of vmstorage nodes. ( #5549 )
...
Co-authored-by: zhangdongdong <zhangdongdong@kuaishou.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-01-09 13:01:03 +01:00
Dmytro Kozlov
37a76de800
app/vmui: fix broken link for the statistic inaccuracy explanation ( #5568 )
...
(cherry picked from commit 105c6b2eb7
2024-01-08 20:15:04 +01:00
Hui Wang
c14e229b20
vmalert: automatically add exported_ prefix for original evaluation… ( #5398 )
...
automatically add `exported_` prefix for original evaluation result label if it's conflicted with external or reserved one,
previously it was overridden.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5161
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 1f477aba41
2023-12-22 16:10:33 +01:00
Aliaksandr Valialkin
12de0d39eb
lib/protoparser/datadogv2: take into account source_type_name field, since it contains useful value such as kubernetes, docker, system, etc.
2023-12-21 23:05:52 +02:00
