Commit graph

3225 commits

Author SHA1 Message Date
Roman Khavronenko
6061464d80
app/vmselect: quantile func compatiblity with Prometheus (#1646)
* app/vmselect: `quantile` func compatiblity with Prometheus

The `quantile` func was previously calculated by https://github.com/valyala/histogram
package. The result of such calculation was always the closest real value to
requested quantile. While in Prometheus implementation interpolation is used.
Such difference may result into discrepancy in output between Prometheus and
VictoriaMetrics.

This commit adds a Prometheus-like `quantile` function. It also used by other
functions which depend on it, such as `quantiles`, `quantile_over_time`, `median` etc.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* app/vmselect: `quantile` review fixes

* quantile functions were split into multiple to provide
different API for already sorted data;
* float64sPool is used for reducing allocations. Items in pool may have
different sizes, but defining a new pool was complicates due to name collisions;

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-27 18:57:02 +03:00
Aliaksandr Valialkin
469820873f
vendor: make vendor-update 2021-09-27 18:01:46 +03:00
Aliaksandr Valialkin
a957d420f4
docs/BestPractices.md
docs/BestPractices.md: update the doc
2021-09-27 17:51:57 +03:00
Aliaksandr Valialkin
a746fe8778
app/vmselect/promql: follow-up after 57b3320478 2021-09-24 01:25:04 +03:00
Roman Khavronenko
7ac0559d4f
app/vmselect: make sorting for query result similar to Prometheus (#1647)
* app/vmselect: make sorting for query result similar to Prometheus

Updated sorting allows to get the order of series in result similar or equal
to what Prometheus returns.
The change is needed for compatibility reasons.

* Update app/vmselect/promql/exec_test.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-24 01:25:03 +03:00
Aliaksandr Valialkin
1115c2f235
app/vmselect/promql: align the behavior of or, and and unless operators with on (labels) modifier to Prometheus
Previously VictoriaMetrics could return unexpected result of the right-hand side operand
had multiple time series with the given set of labels mentioned in `on(labels)`.

See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1643
2021-09-24 00:56:35 +03:00
Nikolay
8148bfc06d
moves prod images build into alpine container with musl (#1640)
adds gcc and musl-dev to builder container
2021-09-24 00:20:33 +03:00
Aliaksandr Valialkin
6fbfa026ca
docs/CHANGELOG.md: cut v1.66.2 2021-09-23 22:53:27 +03:00
Aliaksandr Valialkin
d15d036a5a
lib/storage: properly handle {__name__=~"prefix(suffix1|suffix2)",other_label="..."} queries
They were broken in the commit 00cbb099b6

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1644
2021-09-23 21:52:31 +03:00
Aliaksandr Valialkin
f93146c81e
docs: make docs-sync 2021-09-23 20:55:29 +03:00
Aliaksandr Valialkin
7edb65a9a3
docs/CHANGELOG.md: document 0e35fc9538
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1641
2021-09-23 20:47:56 +03:00
Roman Khavronenko
f65ca2ac05
app/vmctl: fix misleading comment about cluster version for native mode (#1648)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1637
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-23 20:47:56 +03:00
Roman Khavronenko
1bdccfc2bf
app/vmalert: remove unnecessary omitempty tag for interval param (#1649)
`omitempty` tag resulted into skipping this param on marshaling,
which was used as a checksum for groups configuration. Since on
config reload checksums are compared before applying changes,
any change to `interval` only didn't trigger config reload.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1641
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-23 20:47:56 +03:00
Aaron France
e398d83829
fix: typo in vmagent.md (#1642) 2021-09-23 20:47:56 +03:00
Aliaksandr Valialkin
3b501ca859
vendor: make vendor-update 2021-09-23 15:06:57 +03:00
Aliaksandr Valialkin
d8de26bbfd
lib/promscrape: add vm_promscrape_max_scrape_size_exceeded_errors_total metric for counting of the failed scrapes due to the exceeded response size
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1639
2021-09-23 14:48:16 +03:00
Aliaksandr Valialkin
9b76515216
docs/CaseStudies.md: add a case study for Grammarly 2021-09-23 13:12:20 +03:00
Aliaksandr Valialkin
e128915c6e
docs/Articles.md: add https://cer6erus.medium.com/superset-bi-with-victoria-metrics-a109d3e91bc6 2021-09-23 10:44:14 +03:00
Aliaksandr Valialkin
baeb2fce3a
vendor: update github.com/valyala/gozstd from v1.12.0 to v1.13.0 2021-09-22 20:06:52 +03:00
Aliaksandr Valialkin
5e5d5fa8fb
docs/CaseStudies.md: fix a link to third-party articles 2021-09-22 03:53:08 +03:00
Aliaksandr Valialkin
47422e9918
docs/CHANGELOG.md: cut v1.66.1 2021-09-22 01:47:26 +03:00
Aliaksandr Valialkin
25361d176d
docs/CHANGELOG.md: document 9ca1cbced1
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1635
2021-09-21 23:17:26 +03:00
Aliaksandr Valialkin
86bafe796c
lib/httpserver: add -enterprise and/or -cluster suffixes to short_version label of vm_app_version metric
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1635
2021-09-21 23:12:50 +03:00
Aliaksandr Valialkin
c3e1f87048
lib/promrelabel: fix parsing regex: true in relabeling rules 2021-09-21 23:01:40 +03:00
Roman Khavronenko
aa052097cf
app/vmalert: support http.pathPrefix flag in UI (#1636)
The change makes UI to respect `http.pathPrefix` flag
for API or navigation items links.
2021-09-21 18:15:08 +03:00
Aliaksandr Valialkin
0e7b2b724f
vendor: temporarily stick to v0.93.3 for cloud.google.com/go until the binary size bloat issue is resolved
This returns back VictoriaMetrics binary size from 24Mb to 18Mb.

See https://github.com/googleapis/google-cloud-go/issues/4783
2021-09-21 18:13:33 +03:00
Aliaksandr Valialkin
3f1104ea74
app/vmselect: fix accessing /graphite/* endpoints 2021-09-21 14:06:18 +03:00
Aliaksandr Valialkin
e8700b57b9 docs/vmagent.md: typo fix 2021-09-20 16:49:32 +03:00
Aliaksandr Valialkin
233c6a8a82 docs/vmagent.md: typo fixes in Prometheus staleness markers docs 2021-09-20 16:44:42 +03:00
Aliaksandr Valialkin
d43ebb8314 docs/CHANGELOG.md: cut v1.66.0 2021-09-20 15:20:36 +03:00
Nikolay
dd53abf36d changes protoparser apis for accepting reading from io.Reader (#1624)
adds InsertHandlerForReader apis to vmagent
2021-09-20 14:54:20 +03:00
Aliaksandr Valialkin
3b9fd99735 vendor: make vendor-update 2021-09-20 14:44:49 +03:00
n4mine
a5fcc9e356
fix: typo, dddresses -> addresses (#1630) 2021-09-20 14:27:34 +03:00
Aliaksandr Valialkin
4150772ef8 docs/Articles.md: add a link to Open-source strategy at VictoriaMetrics 2021-09-20 11:46:38 +03:00
Aliaksandr Valialkin
76c650bbda app/vmselect/promql: add quantiles_over_time("phiLabel", phi1, ..., phiN, m[d]) function for calculating multiple quantiles at once 2021-09-17 23:35:04 +03:00
Aliaksandr Valialkin
727521d20a docs/CHANGELOG.md: document 0e09fdb8b0 2021-09-17 18:47:39 +03:00
Nikolay
1ab2f844a2 makes filters optional for ec2 api requests (#1627)
filters can be applied only for DescribeInstances requests, like prometheus does.
related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626
2021-09-17 18:12:25 +03:00
Aliaksandr Valialkin
81d24045c2 app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function
This function calculates multiple quantiles over the given buckets at once

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
2021-09-17 13:39:06 +03:00
Aliaksandr Valialkin
9333c4111e docs/CHANGELOG.md: document the change in enterprise apps, which allows passing -version without -eula flag
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1621
2021-09-17 12:38:22 +03:00
Aliaksandr Valialkin
4ed7548ced app/vmselect/promql: optimize quantiles() calculation
Calculate quantiles in one go instead of calculating each quantile individually
2021-09-17 12:38:21 +03:00
Aliaksandr Valialkin
f9910113b5 docs/FAQ.md: extend VictoriaMetrics vs TimescaleDB section with real user experience
See also https://github.com/timescale/promscale/issues/427 , which is mentioned in the https://abiosgaming.com/press/high-cardinality-aggregations/
2021-09-16 20:47:15 +03:00
Roman Khavronenko
a07fa92ef2 vmalert: add new metric vmalert_remotewrite_flush_duration_seconds (#1622) 2021-09-16 14:01:24 +03:00
Roman Khavronenko
cec67a3129 docs: fix indentation for FAQ document (#1620) 2021-09-16 14:01:22 +03:00
f41gh7
348f705d95 follow-up after e2f99214f3e518f79c1c96ff918820a69d350ba1 2021-09-16 13:56:28 +03:00
Roman Khavronenko
05bf6cca82 vmselect: add stub for graphite/functions requests
The request `graphite/functions` is sent by Grafana datasource
and results into error logs in vmselect.
2021-09-16 13:51:37 +03:00
Aliaksandr Valialkin
3f8de4f82f app/vmselect/promql: add mad(q) and outliers_mad(tolerance, q) functions to MetricsQL 2021-09-16 13:35:38 +03:00
Aliaksandr Valialkin
a4eaf1a0a1 app/vmselect/prometheus: make more clear log messages for errors during sending data to remote clients 2021-09-16 13:35:37 +03:00
Aliaksandr Valialkin
7d910c6a51 vendor: make vendor-update 2021-09-15 18:24:08 +03:00
Aliaksandr Valialkin
7ad54041fe app/{vminsert,vmselect}: automatically add missing port in -storageNode lists passed to vminsert and vmselect
This should simplify manual setup of the cluster according to https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#cluster-setup
2021-09-15 18:04:30 +03:00
Aliaksandr Valialkin
9ebccddfc2 docs/CHANGELOG.md: document 777ff75874 2021-09-15 17:45:18 +03:00