Commit graph

3300 commits

Author SHA1 Message Date
Aliaksandr Valialkin
99011c6b63
lib/promscrape: always initialize http client for stream parsing mode
Stream parsing mode can be automatically enabled when scraping targets with big response bodies
exceeding the -promscrape.minResponseSizeForStreamParse , so it must be always initialized.
2021-10-16 13:19:48 +03:00
Aliaksandr Valialkin
0f4fda1bda
lib/promscrape: store the last scraped response in compressed form if its size exceeds -promscrape.minResponseSizeForStreamParse
This should reduce memory usage when scraping targets with big response bodies.
2021-10-16 13:00:11 +03:00
Aliaksandr Valialkin
da0adf446d
app/vmselect/promql: typo fix in comment: didsn't -> didn't 2021-10-16 13:00:10 +03:00
Aliaksandr Valialkin
0452a8d4e8
lib/promscrape: store the full response in stream parsing mode in scrapeWork.lastScrape byte slice
This allows sending staleness marks and properly calculate scrape_series_added metric in stream parsing mode
at the cost of the increased memory usage, since now the potentially big response is kept
in the lastScrape byte slice per each scrapeWork.

In practice the memory usage increase shouldn't be big, since the response size
is usually much smaller than the parsed metrics from this response after the relabeling,
which usually adds a big pile of target-specific labels per each metric.
2021-10-15 15:26:24 +03:00
Aliaksandr Valialkin
3e9beb0f8d
lib/promscrape/discovery/kubernetes: rename endpointslices.go -> endpointslice.go in order to be consistent with EndpointSlice struct name
This is a follow-up for 31b42b30b6
2021-10-15 12:27:31 +03:00
Aliaksandr Valialkin
72c79d7689
docs/FAQ.md: improve wording on why MetricsQL isnt 100% compatible with PromQL 2021-10-14 16:23:17 +03:00
Aliaksandr Valialkin
ae1ee5ba4a
docs/CHANGELOG.md: document the change at 7fcbd3fa4b 2021-10-14 14:38:14 +03:00
Aliaksandr Valialkin
1beb38d476
docs/FAQ.md: add an entry explaining why MetricsQL isn't 100% compatible with PromQL 2021-10-14 12:50:24 +03:00
Aliaksandr Valialkin
25421fa2ae
lib/promscrape: add -promscrape.minResponseSizeForStreamParse command-line option for automatic switching to stream parsing mode when scraping targets with big responses
This should reduce memory usage when vmagent scrapes targets with non-uniform response sizes.
This is common case in Kubernetes monitoring.
2021-10-14 12:30:55 +03:00
Aliaksandr Valialkin
bee130cc78
lib/promscrape: return error if sample_limit or series_limit options are set when stream parsing mode is enabled 2021-10-14 12:30:54 +03:00
Roman Khavronenko
f393145843
Adjust http.Transport.MaxIdleConns setting for vmauth/vmalert services (#1704)
* vmalert: adjust `http.Transport.MaxIdleConns` value accordingly to `http.Transport.MaxIdleConnsPerHost`

`http.Transport.MaxIdleConnsPerHost` setting is controlled by `datasource.maxIdleConnections` flag,
while `http.Transport.MaxIdleConns` is inherited from DefaultTransport and is equal to `100`.
The fix adjusts `http.Transport.MaxIdleConns` value if it is lower than `http.Transport.MaxIdleConnsPerHost`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmauth: adjust `http.Transport.MaxIdleConns` value accordingly to `http.Transport.MaxIdleConnsPerHost`

`http.Transport.MaxIdleConnsPerHost` setting is controlled by `maxIdleConnsPerBackend` flag,
while `http.Transport.MaxIdleConns` is inherited from DefaultTransport and is equal to `100`.
The fix adjusts `http.Transport.MaxIdleConns` value if it is lower than `http.Transport.MaxIdleConnsPerHost`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-10-13 19:23:52 +03:00
Aliaksandr Valialkin
73dbad3c2a
docs/CHANGELOG.md: document e3c8304deb 2021-10-13 16:44:34 +03:00
Aliaksandr Valialkin
5b7d90d178
lib/promscrape: add ability to show the original labels for discovered targets at /targets page
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1698
2021-10-13 16:44:34 +03:00
Roman Khavronenko
14995eece6
vmalert: add Source link to alerts UI (#1701)
The source link is controlled by `external.url` and `external.alert.source`
flags, in the same way as for alertmanager notifications.
The source link is added to Alerts list view, and specific Alert view.
2021-10-13 15:26:57 +03:00
Aliaksandr Valialkin
9568961054
docs/MetricsQL.md: add missing blank line before the link to github.com/VictoriaMetrics/metricsql package 2021-10-13 15:11:46 +03:00
Roman Khavronenko
5dab25e8ad
lib/promscrape: make errcheck happy (#1703) 2021-10-13 15:11:45 +03:00
Aliaksandr Valialkin
c3a729d458
lib/promscrape: shard targets among cluster nodes after relabeling is applied
This guarantees that targets with the same set of labels go to the same vmagent node.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1687#issuecomment-940629495
2021-10-12 17:06:37 +03:00
Aliaksandr Valialkin
aeedfe2fe2
app/vmagent: expose -promscrape.config contents at /config page as Prometheus does
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695
2021-10-12 16:27:37 +03:00
Aliaksandr Valialkin
1e56d19c09
docs/FAQ.md: add a solution against high churn rate 2021-10-12 10:33:08 +03:00
Roman Khavronenko
49a7ff62df
docs: mention "PromQL compliance" in MetricsQL docs (#1691) 2021-10-11 21:52:28 +03:00
Aliaksandr Valialkin
4878fcedac
vendor: make vendor-update 2021-10-11 21:52:27 +03:00
Aliaksandr Valialkin
48c3668b3d
app/vmselect/promql: add atan2 binary operator, which is going to be added in Prometheus 2.31
See https://github.com/prometheus/prometheus/pull/9248
2021-10-11 21:17:23 +03:00
Aliaksandr Valialkin
b3d28f3872
app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31
See https://github.com/prometheus/prometheus/issues/9233
2021-10-11 21:17:21 +03:00
Aliaksandr Valialkin
d00f8b8800
docs/MetricsQL.md: clarify docs for union() function 2021-10-11 17:41:10 +03:00
Aliaksandr Valialkin
84aa08d93a
lib/promscrape: use Prometheus format for target labels at /targets page
This should simplify copy-pasting the labels to/from PromQL / MetricsQL
2021-10-11 12:42:18 +03:00
Denys Holius
6199bd28a2
Added some fixes (#1690)
* removed not needed description

* added some fixes and fixed typos
2021-10-11 12:42:18 +03:00
Aliaksandr Valialkin
f17c4b74c5
docs/MetricsQL.md: add a link to https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e 2021-10-11 11:02:03 +03:00
Roman Khavronenko
de4fbe9402
docs: add "PromQL compliance" article (#1689)
* docs: add "PromQL compliance" article

* Update docs/Articles.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-11 11:01:59 +03:00
Aliaksandr Valialkin
ab3dcf3f77
deployment/docker/docker-compose.yml: upgrade Grafana from v8.1.2 to v8.2.0 2021-10-08 20:37:32 +03:00
Aliaksandr Valialkin
f1e1d20ac4
app/vmselect/promql: consistently return the same set of time series from limitk() function
This is the expected behaviour by most users.
2021-10-08 19:55:29 +03:00
Aliaksandr Valialkin
a7a1305395
lib/storage: fix unaligned access on 32-bit architectures.
The bug has been introduced at a171916ef5
2021-10-08 19:38:20 +03:00
Denys Holius
d1ecffb0f6
Added fixes and improvements (#1677)
* added guide for VM operator

* Update docs/guides/getting-started-with-vm-operator.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Update docs/guides/getting-started-with-vm-operator.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Fixed different typos and added improvements from proposals

* move remoteWrite.url to other place

* fixed typo

* rephrased vminsert explanation

* remove not needed parameters for default setup

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2021-10-08 18:58:42 +03:00
Aliaksandr Valialkin
39aa1217c6
deployment/docker: update Go builder version from Go1.17.1 to Go1.17.2
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.2+label%3ACherryPickApproved
2021-10-08 17:53:06 +03:00
Aliaksandr Valialkin
20fa4b01cb
vendor: make vendor-update 2021-10-08 16:06:31 +03:00
Aliaksandr Valialkin
aae6f7dfd0
docs/CHANGELOG.md: cut v1.67.0 2021-10-08 16:00:43 +03:00
Aliaksandr Valialkin
a47754b689
lib/protoparser/clusternative: typo fix after 4fddcf4c83 2021-10-08 15:38:47 +03:00
Aliaksandr Valialkin
cf87050d83
app/vmui: follow-up after 7bfb44113e
* Run `vmui-update`
* Document the changes in README.md and CHANGELOG.md
2021-10-08 15:11:09 +03:00
Yury Molodov
41a83a1cc6
vmui: use uPlot as default engine for graph (#1683)
* feat: initial uPlot graph

* feat: add zoom/pan for graph

* fix: add zoom by ctrl/mac

* fix: remove unused code
2021-10-08 15:11:07 +03:00
Aliaksandr Valialkin
4fddcf4c83
app/{vminsert,vmstorage}: follow-up after a171916ef5
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/269
2021-10-08 14:09:51 +03:00
Nikolay
a171916ef5
Adds read-only mode for vmstorage node (#1680)
* adds read-only mode for vmstorage
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/269

* changes order a bit

* moves isFreeDiskLimitReached var to storage struct
renames functions to be consistent
change protoparser api - with optional storage limit check for given openned storage

* renames freeSpaceLimit to ReadOnly
2021-10-08 12:52:56 +03:00
Aliaksandr Valialkin
cbf01f7384
app/vmselect/promql: substitute rollupFuncsCannotAdjustWindow with rollupFuncsCanAdjustWindow
The list of functions, which can adjust lookbehind window is more limited than the rest of functions,
so it is better from maintainability and readability PoV using the allowlist instead of blocklist.
2021-10-07 13:19:22 +03:00
Aliaksandr Valialkin
7e6ec0bbe9
app/vmselect/promql: return back the behaviour for deriv() function when the lookbehind window doesnt contain enough points
It is expected that the `deriv(m[d])` returns non-empty value if the lookbehind window `d`
contains less than 2 samples in the same way as `rate()` does.

This is a follow-up after 3e084be06b .
2021-10-07 12:55:58 +03:00
Roman Khavronenko
0a8804d6aa
app/vmselect: make predict_linear and deriv compatible with Prometheus (#1681)
Previously, `predict_linear` returned slightly different results comparing
to Prometheus. The change makes linear regression algorithm compatible
with Prometheus.

`deriv` was excluded from the list of functions which can adjust the time
window for the same reasons.
2021-10-07 12:55:56 +03:00
Aliaksandr Valialkin
64b6f3f1c8
app/vminsert: fix uneven distribution of time series among storage nodes
Use distinct seed for distribution hash calculations on the second level of vminsert nodes.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1672
2021-10-07 12:22:39 +03:00
Aliaksandr Valialkin
cea79d2013
docs/guides: follow-up after 05a1396247 2021-10-06 14:47:28 +03:00
Thomas Danielsson
8d26b0f3f5
fix: typo metric_relabel_configs (#1674)
metric_ralabel_configs -> metric_relabel_configs
2021-10-06 14:47:26 +03:00
Ziqi Zhao
1db3aeab36
fix some typos (#1678)
Co-authored-by: 柘远 <zzq237937@alibaba-inc.com>
2021-10-06 14:43:56 +03:00
Aliaksandr Valialkin
bb45d747ad
vendor: make vendor-update 2021-10-05 10:31:30 +03:00
Aliaksandr Valialkin
89700433cc
docs/vmagent.md: update docs after 3e9a939a990c8b608414388c96f68eb062364ae7 2021-10-05 10:23:37 +03:00
Aliaksandr Valialkin
f434fcbbd7
vendor: make vendor-update 2021-09-30 17:54:45 +03:00