Commit graph

1133 commits

Author SHA1 Message Date
Alexander Rickardsson
d27dc3721b vmalert: enable configuring explicit path (#1536)
* vmalert: allow to disable automatically added path to remote write address via disablePathAppend flag
* docs: update docs to include remoteWrite.disablePathAppend
2021-08-16 14:58:05 +03:00
Aliaksandr Valialkin
48920bdef8 app/vmagent/remotewrite: expose vmagent_remotewrite_send_duration_seconds_total metric
This metric can be used for determining high saturation of every connection to remote storage with
an alerting query `rate(vmagent_remotewrite_send_duration_seconds_total) > 0.9s`.
This query triggers when a connection is satureated by more than 90%
2021-08-15 13:34:07 +03:00
Aliaksandr Valialkin
5420c3d967 app/vmselect/promql: drop staleness marks before calling rollupConfig.Do
This allows dropping staleness marks only once and then calculate multiple rollup functions on the result.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
2021-08-15 13:22:26 +03:00
Aliaksandr Valialkin
6c4c54eaad Revert "app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions"
This reverts commit 94dfcb6747a3b29a11d14e71bea21a2312bb6346.

It is better to remove staleness marks (decimal.StaleNaN) before calling rollupConfig.Do, e.g. in preFunc
2021-08-15 13:22:24 +03:00
Aliaksandr Valialkin
af4a306d7b app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions
Prometheus stalenss marks shouldn't be changed in removeCounterResets. Otherwise they will be converted to an ordinary NaN values,
which couldn't be removed in dropStaleNaNs() function later. This may result in incorrect calculations for rollup functions.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
2021-08-14 12:45:31 +03:00
Aliaksandr Valialkin
c1f81f08d4 all: add support for Prometheus staleness markers
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
2021-08-13 12:13:15 +03:00
Aliaksandr Valialkin
b35ae791f1 app/vmselect: make vmui-update after the commit 4ae14df864a7e327955f44941295a286175423b3 2021-08-11 13:42:53 +03:00
Aliaksandr Valialkin
f60ff85dbe app/vmui: actualize Dockerfiles 2021-08-11 13:42:53 +03:00
Aliaksandr Valialkin
9eb828b2c2 app/vminsert: add vm_rpc_send_duration_seconds_total metric per each vminsert->vmstorage link
This metric is useful for determining high link saturation with the following alerting rule:

rate(vm_rpc_send_duration_seconds_total) > 0.9s
2021-08-11 11:42:33 +03:00
Aliaksandr Valialkin
90efb5831b lib/envflag: add a link to docs for -envflag.enable 2021-08-11 10:32:40 +03:00
Yury Molodov
aca2cb245e vmui: fix layout and add server url by default (#1519)
* fix: change layout for correctly display big query

* fix: set default server from url

* fix: change get default server url
2021-08-06 12:16:53 +03:00
Roman Khavronenko
d5ba8248cc vmalert: expose new metrics for tracking number of produced samples during last evaluation (#1518)
* vmalert: expose new metrics for tracking number of produced samples during last evaluation

Two new metrics were added to track the number of samples produced during the last evaluation:
* vmalert_recording_rules_last_evaluation_samples
* vmalert_alerting_rules_last_evaluation_samples

The gauge type is used to remain consistent with Prometheus metric
`prometheus_rule_group_last_evaluation_samples` which is on the group level.
However, the counter type was considered as well.

Two metrics instead of one are used to make it easier to separate recording and
alerting rules. It is likely, number of samples produced by recording rules is
more important so people will refer to it more frequently.

The expected usage of the new metric is the following:
```
   - alert: RecordingRuleReturnsEmptyResults
        expr: sum(vmalert_recording_rules_last_evaluation_samples) by(recording) < 1
        annotations:
          summary: Recording rule {{$labels.recording}} returns empty results.
            Please verify expression correctness.
```

Addresses https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1494

* vmalert: rename `vmalert_alerts_error` to `vmalert_alerting_rules_error` to remain consistent with recording rules metrics
2021-08-05 10:02:35 +03:00
Aliaksandr Valialkin
13d438d808 app/vmagent: typo fix in the description for -remoteWrite.queues 2021-08-05 10:00:58 +03:00
Aliaksandr Valialkin
b877538622 app/vmagent: follow-up after fe445f753b
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1491
2021-08-05 09:51:00 +03:00
Omar Ghader
fe445f753b
feature: Add multitenant for vmagent (#1505)
* feature: Add multitenant for vmagent

* Minor fix

* Fix rcs index out of range

* Minor fix

* Fix multi Init

* Fix multi Init

* Fix multi Init

* Add default multi

* Adjust naming

* Add TenantInserted metrics

* Add TenantInserted metrics

* fix: remove unused metrics for vmagent

* fix: remove unused metrics for vmagent

Co-authored-by: mghader <marc.ghader@ubisoft.com>
Co-authored-by: Sebastian YEPES <syepes@gmail.com>
2021-08-05 09:44:29 +03:00
Qifei Wan
095bb90879 app/vmalert: update config state metrics if config parsed failed (#1507) 2021-08-03 16:12:48 +03:00
Aliaksandr Valialkin
60cfa5f100 app/vmselect/promql: add present_over_time(m[d]) function, which will be available starting from Prometheus 2.29.0
See https://github.com/prometheus/prometheus/releases/tag/v2.29.0-rc.0 and https://github.com/prometheus/prometheus/pull/9097
2021-08-03 12:21:53 +03:00
wusphinx
511e5c2e68 Update TimeSelector.tsx (#1515)
delete garbled code
2021-08-03 11:14:56 +03:00
Nikolay
3f3ad13753 adds /rules and /alerts api for grafana (#1504)
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-08-02 17:29:49 +03:00
Aliaksandr Valialkin
99004a6a40 app/vmselect/netstorage: unpack time series data in mostly local big chunks
This should improve performance on multi-CPU systems for queries selecting time series with big number of raw samples
2021-07-30 12:26:33 +03:00
Aliaksandr Valialkin
c473d8ffe1 li/storage: re-use the per-day inverted index search code for searching in global index
This allows removing a big pile of outdated code for global index search.

This may help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1486
2021-07-30 10:28:20 +03:00
Aliaksandr Valialkin
cbb81c2ce9 app/vmselect/netstorage: do not query Go maps with unsafe string keys, since this breaks in Go 1.17 2021-07-30 10:28:19 +03:00
Aliaksandr Valialkin
b709fa387a app/vmselect: follow-up for ed95bc9531
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1493
2021-07-29 09:48:47 +03:00
arnoldyahad
ed95bc9531
Add case prometheus/rules for grafana 8 (#1502) 2021-07-29 06:15:35 +03:00
assassins
6ab0001a1f Performance optimization (#1481)
There are redundant steps
2021-07-28 19:29:22 +03:00
Aliaksandr Valialkin
49bf3abf67 app/vmselect: follow-up for 626073bca8
* Rename -search.maxMetricsPointSearch to -search.maxSamplesPerQuery, so it is more consistent with the existing -search.maxSamplesPerSeries
* Move the -search.maxSamplesPerQuery from vmstorage to vmselect, so it could effectively limit the number of raw samples obtained from all the vmstorage nodes
* Document the -search.maxSamplesPerQuery in docs/CHANGELOG.md
2021-07-28 18:00:04 +03:00
匠心零度
626073bca8
protection vmselect ,avoid metrics point too much let vmselect cup load very, very high (#1478)
* protection vmselect……

* protection vmselect……

* protection vmselect……

* All checks have failed,fix

Co-authored-by: lirenzuo <lirenzuo@shein.com>
2021-07-28 14:39:35 +03:00
Aliaksandr Valialkin
5d255846ac all: add go:build lines for Go1.17
See https://tip.golang.org/doc/go1.17#gofmt for more details
2021-07-26 15:50:46 +03:00
Aliaksandr Valialkin
3921d8afae app/vmselect: prevent from possible deadlock when f callback blocks inside RunParallel 2021-07-26 15:50:45 +03:00
Aliaksandr Valialkin
c3e6ce1db9 app/vmselect: make vmui-update after a91d41f12a 2021-07-26 10:32:01 +03:00
Yury Molodov
401de2dca4 Vmui/query editor (#1472)
* fix: move request button to server input

* feat: add switch for query autocomplete

* refactor: rename state for popover open

* feat: add detect os by userAgent

* fix: change hotkey to run query for mac

* fix: change detect mac os

* fix: change div to span inside Typography

Co-authored-by: yury <yurymolodov@victoriametrics.com>
2021-07-23 21:08:58 +03:00
Aliaksandr Valialkin
b047feeb8b app/vmselect/promql: properly handle (a op b) default N if (a op b) returns NaN series
The result should be a series with `N` values and `a op b` labels. Previously such series has been removed from the result.
2021-07-16 01:44:24 +03:00
Aliaksandr Valialkin
b92702f6d5 app/vmselect/netstorage: use more scalable algorithm for ditributing the work among among multiple channels on systems with big number of CPU cores 2021-07-16 00:35:36 +03:00
Aliaksandr Valialkin
df117f85bd app/vmselect: do not track queries with less than 1ms execution time at /api/v1/status/top_queries
This should improve the readability and usefullness of the /api/v1/status/top_queries when debugging slow queries
or queries that take too much cpu time.
2021-07-15 16:53:35 +03:00
Aliaksandr Valialkin
5830ce2706 app/vmselect/netstorage: add -search.maxSamplesPerSeries command-line option for limiting the number of samples a query can process per each series
This should prevent from out of memory crashes like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1067
2021-07-15 16:53:35 +03:00
Aliaksandr Valialkin
6c42db87a8 app/vmselect/netstorage: improve scalability of series unpacking on multi-CPU systems 2021-07-15 15:40:53 +03:00
Aliaksandr Valialkin
3059e4feec app/vmui/README.md: typo fix: naviate->navigate 2021-07-15 15:02:56 +03:00
Aliaksandr Valialkin
9add9d86a6 app/vmselect/promql: duration handling improvements in MetricsQL queries
- Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600
- Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])
2021-07-12 17:19:32 +03:00
Aliaksandr Valialkin
d98e22fe50 app/vmalert: accept Prometheus-like durations in interval config option inside group section 2021-07-12 12:36:22 +03:00
Aliaksandr Valialkin
f5fa177141 Revert "app/vmselect: expose vmui at /select/<accountID>/prometheus/vmui additionally to /select/<accountID>/vmui"
This reverts commit 885a79def6799f288e14df05b35a12569659ab85.

Reason for revert: Grafana doesn't allows accessing /select/<accountID>/prometheus/vmui :(
2021-07-12 09:08:43 +03:00
Aliaksandr Valialkin
ddaa12050d app/vmselect: expose vmui at /select/<accountID>/prometheus/vmui additionally to /select/<accountID>/vmui
The /select/<accountID>/prometheus/vmui is needed for accessing via server-side Prometheus datasource for Grafana.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-10 12:52:25 +03:00
Aliaksandr Valialkin
0b98f6c7ff app/vmselect: expose vmui at /vmselect/<accountID>/vmui/ instead of /vmselect/<accountID>/prometheus/vmui/
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-10 12:32:21 +03:00
Aliaksandr Valialkin
98e049ba6d app/vmui: move source code from https://github.com/VictoriaMetrics/vmui to app/vmui
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-09 17:13:51 +03:00
Aliaksandr Valialkin
2c5e1cd893 app/vmselect: move web ui from /ui to /select/<accountID>/prometheus/ui
This way the UI is available for every tenant (aka accountID) and the UI can automatically determine the needed per-tenant datasource path from page referer.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-08 13:14:50 +03:00
Aliaksandr Valialkin
acb7a95c64 app/vmselect: follow-up after aa11ef6d3b
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4
2021-07-07 17:45:09 +03:00
tony
aa11ef6d3b
add vmui for vmselect component (#1431)
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-07-07 17:04:23 +03:00
Aliaksandr Valialkin
9c19719ad6 app/{vminsert,vmselect}: export vminsert_request_duration_seconds and vmselect_request_duration_seconds histograms 2021-07-07 13:27:23 +03:00
Aliaksandr Valialkin
ceda2b1df4 lib/httpserver: print full requestURI in httpserver.Errorf
This should simplify debugging.
2021-07-07 13:11:29 +03:00
Aliaksandr Valialkin
22c6e64bbc lib/storage: consistency renaming: tagCache -> tagFiltersCache
This improves code readability
2021-07-06 11:03:30 +03:00
Aliaksandr Valialkin
44855f0c9b app/{vmselect,vmstorage}: clarify the description for -dedup.minScrapeInterval command-line flag
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1426
2021-07-02 15:06:41 +03:00