Commit graph

1208 commits

Author SHA1 Message Date
Roman Khavronenko
6061464d80
app/vmselect: quantile func compatiblity with Prometheus (#1646)
* app/vmselect: `quantile` func compatiblity with Prometheus

The `quantile` func was previously calculated by https://github.com/valyala/histogram
package. The result of such calculation was always the closest real value to
requested quantile. While in Prometheus implementation interpolation is used.
Such difference may result into discrepancy in output between Prometheus and
VictoriaMetrics.

This commit adds a Prometheus-like `quantile` function. It also used by other
functions which depend on it, such as `quantiles`, `quantile_over_time`, `median` etc.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* app/vmselect: `quantile` review fixes

* quantile functions were split into multiple to provide
different API for already sorted data;
* float64sPool is used for reducing allocations. Items in pool may have
different sizes, but defining a new pool was complicates due to name collisions;

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-27 18:57:02 +03:00
Aliaksandr Valialkin
a746fe8778
app/vmselect/promql: follow-up after 57b3320478 2021-09-24 01:25:04 +03:00
Roman Khavronenko
7ac0559d4f
app/vmselect: make sorting for query result similar to Prometheus (#1647)
* app/vmselect: make sorting for query result similar to Prometheus

Updated sorting allows to get the order of series in result similar or equal
to what Prometheus returns.
The change is needed for compatibility reasons.

* Update app/vmselect/promql/exec_test.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-24 01:25:03 +03:00
Aliaksandr Valialkin
1115c2f235
app/vmselect/promql: align the behavior of or, and and unless operators with on (labels) modifier to Prometheus
Previously VictoriaMetrics could return unexpected result of the right-hand side operand
had multiple time series with the given set of labels mentioned in `on(labels)`.

See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1643
2021-09-24 00:56:35 +03:00
Aliaksandr Valialkin
f93146c81e
docs: make docs-sync 2021-09-23 20:55:29 +03:00
Roman Khavronenko
f65ca2ac05
app/vmctl: fix misleading comment about cluster version for native mode (#1648)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1637
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-23 20:47:56 +03:00
Roman Khavronenko
1bdccfc2bf
app/vmalert: remove unnecessary omitempty tag for interval param (#1649)
`omitempty` tag resulted into skipping this param on marshaling,
which was used as a checksum for groups configuration. Since on
config reload checksums are compared before applying changes,
any change to `interval` only didn't trigger config reload.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1641
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-23 20:47:56 +03:00
Roman Khavronenko
aa052097cf
app/vmalert: support http.pathPrefix flag in UI (#1636)
The change makes UI to respect `http.pathPrefix` flag
for API or navigation items links.
2021-09-21 18:15:08 +03:00
Aliaksandr Valialkin
3f1104ea74
app/vmselect: fix accessing /graphite/* endpoints 2021-09-21 14:06:18 +03:00
Aliaksandr Valialkin
e8700b57b9 docs/vmagent.md: typo fix 2021-09-20 16:49:32 +03:00
Aliaksandr Valialkin
233c6a8a82 docs/vmagent.md: typo fixes in Prometheus staleness markers docs 2021-09-20 16:44:42 +03:00
Nikolay
dd53abf36d changes protoparser apis for accepting reading from io.Reader (#1624)
adds InsertHandlerForReader apis to vmagent
2021-09-20 14:54:20 +03:00
n4mine
a5fcc9e356
fix: typo, dddresses -> addresses (#1630) 2021-09-20 14:27:34 +03:00
Aliaksandr Valialkin
76c650bbda app/vmselect/promql: add quantiles_over_time("phiLabel", phi1, ..., phiN, m[d]) function for calculating multiple quantiles at once 2021-09-17 23:35:04 +03:00
Aliaksandr Valialkin
81d24045c2 app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function
This function calculates multiple quantiles over the given buckets at once

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
2021-09-17 13:39:06 +03:00
Aliaksandr Valialkin
4ed7548ced app/vmselect/promql: optimize quantiles() calculation
Calculate quantiles in one go instead of calculating each quantile individually
2021-09-17 12:38:21 +03:00
Roman Khavronenko
a07fa92ef2 vmalert: add new metric vmalert_remotewrite_flush_duration_seconds (#1622) 2021-09-16 14:01:24 +03:00
f41gh7
348f705d95 follow-up after e2f99214f3e518f79c1c96ff918820a69d350ba1 2021-09-16 13:56:28 +03:00
Roman Khavronenko
05bf6cca82 vmselect: add stub for graphite/functions requests
The request `graphite/functions` is sent by Grafana datasource
and results into error logs in vmselect.
2021-09-16 13:51:37 +03:00
Aliaksandr Valialkin
3f8de4f82f app/vmselect/promql: add mad(q) and outliers_mad(tolerance, q) functions to MetricsQL 2021-09-16 13:35:38 +03:00
Aliaksandr Valialkin
a4eaf1a0a1 app/vmselect/prometheus: make more clear log messages for errors during sending data to remote clients 2021-09-16 13:35:37 +03:00
Aliaksandr Valialkin
7ad54041fe app/{vminsert,vmselect}: automatically add missing port in -storageNode lists passed to vminsert and vmselect
This should simplify manual setup of the cluster according to https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#cluster-setup
2021-09-15 18:04:30 +03:00
Aliaksandr Valialkin
794d941125 app/vmui: follow-up after 777ff75874
The commit contains the following changes:

- Show vmui when requesting /graph page in order to be compatible with Prometheus datasource in Grafana.
- Properly encode query args at vmui url.
- Set the number of points on the graph to the number of horizontal pixels divided by 2. Previously it was hardcoded to 30.
- Do not save server url to persistent storage at browser, since it should be always obtained from the url.
- Run `make vmui-update` for updating vmui embedded into VictoriaMetrics.
2021-09-15 17:41:00 +03:00
Yury Molodov
df87362ef6 vmui: change query params compatible with prometheus (#1619)
* feat: change url params for compatible prometheus

* style: add comment for TimeParams

* fix: change get default server for single version

* fix: change function for get query string value
2021-09-15 16:36:39 +03:00
Roman Khavronenko
286b79dc48 vmalert: create basic auth config only if args aren't empty (#1618)
* vmalert: create basic auth config only if args aren't empty

follow-up after 68721f6

* vmalert: make lint happy
2021-09-15 09:35:38 +03:00
Aliaksandr Valialkin
63fb5a5ca5 docs/vmalert.md: follow-up after 68721f6e7d
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1608
2021-09-14 14:48:45 +03:00
Roman Khavronenko
eea6317d40 vmalert: support bearer token for datasource, remotewrite and remoteread (#1614)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1608
2021-09-14 14:48:43 +03:00
Aliaksandr Valialkin
5e5ce27df7 app/vmauth: do not log invalid auth tokens by default for security reasons
The logging can be enabled by passing `-logInvalidAuthTokens` command-line flag to vmauth
2021-09-14 12:20:17 +03:00
Aliaksandr Valialkin
a8f1b614e5 docs/vmagent.md: mention out of order sample errors, which are typically emitted by Thanos, Cortex or Prometheus 2021-09-13 19:36:09 +03:00
Aliaksandr Valialkin
a1b298a842 app/vminsert/netstorage: disable rerouting by default
Production clusters work more stable with the disabled rerouting during rolling restarts and/or
during spikes in time series churn rate. So it would be better disabling the rerouting by default.

The re-routing can be enabled by passing `-disableRerouting=false` command-line flag to `vminsert` nodes.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/791
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1054
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1165
2021-09-13 18:50:47 +03:00
Aliaksandr Valialkin
2932f495f3 docs/CHANGELOG.md: document 5494bc02a6 2021-09-13 17:14:45 +03:00
Roman Khavronenko
7df731ea91 vmalert: add flag to limit the max value for auto-resovle duration for alerts (#1609)
* vmalert: add flag to limit the max value for auto-resovle duration for alerts

The new flag `rule.maxResolveDuration` suppose to limit max value for
alert.End param, which is used by notifiers like Alertmanager for alerts auto resolve.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1586
2021-09-13 17:14:45 +03:00
Aliaksandr Valialkin
184e145570 docs: consistency renaming: Influx -> InfluxDB 2021-09-13 17:14:45 +03:00
Aliaksandr Valialkin
101cdef0a7 app/vmui/Dockerfile-web: update Go builder from 1.16.7 to 1.17.1 and Alpine base image from 3.14.1 to 3.14.2 2021-09-13 17:14:45 +03:00
Aliaksandr Valialkin
b721110902 docs/vmbackup.md: update the outdated link to vmbackupmanager 2021-09-13 14:17:53 +03:00
Roman Khavronenko
9dd121db36 vmalert: display extra filter labels in UI (#1613) 2021-09-13 14:17:51 +03:00
Aliaksandr Valialkin
19516aedb5 app/vmselect/promql: use Prometheus-compatible label value formatting for count_values function 2021-09-13 13:48:18 +03:00
Aliaksandr Valialkin
2b64c1c95e docs/Single-server-VictoriaMetrics.md: refer to relabeling section for vmagent
This removes duplicate docs about additional relabeling actions supported by VictoriaMetrics components
2021-09-12 11:40:23 +03:00
Aliaksandr Valialkin
5aaaa686a4 lib/promscrape: send stale markers for disappeared metrics like Prometheus does 2021-09-11 11:02:56 +03:00
Aliaksandr Valialkin
e1632dc0df docs/vmalert.md: typo fix in Multitenancy chapter 2021-09-10 17:57:43 +03:00
Aliaksandr Valialkin
c2f37f049b lib/storage: properly search series by multiple tag filters matching empty labels such as foo{bar=~"baz|",x=~"y|"}
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1601
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/395
2021-09-09 21:12:53 +03:00
Aliaksandr Valialkin
9b07cb2988 app/vmselect/promql: keep metric name in rollup_candlestick results, since they don't change the original series meaning
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1600
2021-09-09 19:24:10 +03:00
Aliaksandr Valialkin
dca6f0f7de app/vminsert/netstorage: remove the limit on the number of -storageNode addresses
There is no any reasons to limit the number of `-storageNode` addresses to 255.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1599
2021-09-09 19:11:36 +03:00
Aliaksandr Valialkin
d613a018c8 lib/promscrape: add ability to set series_limit and stream_parse options via relabeling
This allows managing these options on a per-target basis.

Typical use case: to manage these options for pods via Kubernetes annotations.
2021-09-09 18:51:23 +03:00
Aliaksandr Valialkin
986807ebd9 docs/vmagent.md: typo fix in Relabeling chapter 2021-09-09 16:39:34 +03:00
Aliaksandr Valialkin
75c3514c5c lib/promrelabel: add keep_metrics and drop_metrics actions to relabeling rules
These actions simlify metrics filtering. For example,

- action: keep_metrics
  regex: 'foo|bar|baz'

would leave only metrics with `foo`, `bar` and `baz` names, while the rest of metrics will be deleted.

The commit also makes possible to split long regexps into multiple lines. For example, the following config is equivalent to the config above:

- action: keep_metrics
  regex:
  - foo
  - bar
  - baz
2021-09-09 16:25:09 +03:00
Aliaksandr Valialkin
d72593cca2 app/vmalert: document GroupAlerts
This makes golint happy
2021-09-07 22:50:48 +03:00
Aliaksandr Valialkin
608ed19655 app/vmalert: follow-up after 21f022e5f0 2021-09-07 22:45:47 +03:00
Roman Khavronenko
bb26aa9b1f vmalert: add initial UI implementation (#1602)
New UI pages:
/ - welcome page with API handlers list;
/groups - list of all rules per group;
/alerts - list of all active alerts;
/groupID/alertID/status - status of the active alert;
2021-09-07 22:45:45 +03:00
Aliaksandr Valialkin
304661c547 docs/vmagent.md: add API path for Prometheus text exposition format 2021-09-07 16:15:28 +03:00