Commit graph

3667 commits

Author SHA1 Message Date
Aliaksandr Valialkin
92d01db85a
lib/protoparser/prometheus: optimize GetRowsDiff() function
This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1745 ,
since the provided profile shows that the majority of CPU and memory is spent in this function
during `streamParse` when `-promscrape.noStaleMarkers` wasn't set.
2021-10-27 18:55:25 +03:00
Aliaksandr Valialkin
16f1aaf0b5
lib/protoparser/prometheus: add a benchmark for GetRowsDiff 2021-10-27 18:55:23 +03:00
Aliaksandr Valialkin
99784b21c1
all: fix build issues and tests for Apple M1
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653
2021-10-27 15:07:19 +03:00
Aliaksandr Valialkin
6c1701d908
.github/workflows/main.yml: checkout code before installing dependencies
Dependencies depend on Makefile rules from the code, so code checkout must run first
2021-10-26 22:09:38 +03:00
Aliaksandr Valialkin
f86dba1f5c
docs/CHANGELOG.md: mention the issue about missing proxy_url config option at /config page
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755
2021-10-26 22:07:07 +03:00
Aliaksandr Valialkin
d65912e47c
.github/workflows/main.yml: re-use makefile rules for installing goling, errcheck and golangci-lint 2021-10-26 22:05:09 +03:00
Aliaksandr Valialkin
ad445a06cd
lib/promscrape: properly show proxy_url option value at /config page
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755
2021-10-26 21:24:22 +03:00
Aliaksandr Valialkin
aece35158d
vendor: make vendor-update 2021-10-26 19:38:27 +03:00
Yury Molodov
e7591577f0
vmui: update dependencies (#1754)
* update dependencies

* update codemirror version

* app/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-26 19:32:31 +03:00
Aliaksandr Valialkin
eb783b0ef3
app/vmbackup/README.md: sync with docs/vmbackup.md after e706fb5686 2021-10-26 19:21:26 +03:00
Dima Lazerka
464f5d0910
Fix doc: vmbackup splits by 1 GiB not 100 MB (#1756)
This is a follow-up for bdd0a1cdb2
2021-10-26 19:21:25 +03:00
Denys Holius
56970caded
fixed wrong path for npm dependabot checks (#1744) 2021-10-26 19:16:35 +03:00
Aliaksandr Valialkin
9e8697c6c3
docs/CHANGELOG.md: document 3dbdf1632e 2021-10-25 14:46:52 +03:00
Roman Khavronenko
5321127add
vmalert: allow groups with empty rules for compatibility reasons (#1742)
Prometheus allows to have groups with no rules, so we should support
it in vmalert as well for compatibility reasons.
It is also allowed to hot-reload empty groups by adding or removing rules.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-10-25 14:46:51 +03:00
Aliaksandr Valialkin
2d7d67372f
docs/Cluster-VictoriaMetrics.md: add links with the explanation of active time series and series churn rate 2021-10-24 18:39:16 +03:00
Aliaksandr Valialkin
30ade321a6
docs/CaseStudies.md: fix a link to AbisoGaming case study 2021-10-24 18:37:19 +03:00
Aliaksandr Valialkin
d8ce86d685
app/vmselect/promql: reduce the precision from 15 significant digits to 13 significant digits when comparing float64 results in tests
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1738
2021-10-24 13:32:25 +03:00
Aliaksandr Valialkin
7ec253a3da
docs/CHANGELOG.md: typo fix 2021-10-22 21:13:03 +03:00
Aliaksandr Valialkin
9e0a67cf13
go.mod: remove outdated replacement 2021-10-22 19:47:10 +03:00
Aliaksandr Valialkin
1bdfa475f3
docs/CHANGELOG.md: cut v1.68.0 2021-10-22 19:38:01 +03:00
Aliaksandr Valialkin
e18aa6e07e
deployment/docker: update Grafana from v8.2.0 to v8.2.2 2021-10-22 19:30:06 +03:00
Aliaksandr Valialkin
064abfba8c
vendor: make vendor-update 2021-10-22 19:28:27 +03:00
Aliaksandr Valialkin
1780c25e05
docs/vmauth.md: make docs-sync 2021-10-22 19:28:27 +03:00
Yurii Kravets
f7cb850f8d
Updated docker-compose.yml (#1730)
* Updated docker-compose.yml

Added one more Vmstorage to compose and pointed vmselect and vminsert to it.

* Update prometheus.yml

According to requested changes for new vmstorage

* Update docker-compose

Added port mapping

* Updated docker-compose

Set same ports for both storages

* Update prometheus.yml

Changed target for vmstorage1

* Updated docker compose

Changed the flags for vmstorage1

* Update docker compose

Changed flags for vmisert and vmselect to point to vmstorage1

* Update docker compose

Made the vmstorage names uniform

* Update prometheus.ymk

* Update docker-compose.yml

Removed additional flags as they are using default values.

* Update prometheus.yml

Put vmstorage targets under the same job

* Update prometheus.yml

Changed targets format

* Update docker-compose.yml

Set two different volumes for every vmstorage node

* Update deployment/docker/prometheus.yml

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-22 19:26:02 +03:00
Aliaksandr Valialkin
559dd996c4
app/vmauth: add ability to specify http headers to send in requests to backends
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1736
2021-10-22 19:11:35 +03:00
Aliaksandr Valialkin
b08f51f5d3
lib/promscrape: do not populate response body to memory in stream parsing mode if -promscrape.noStaleMarkers is set
The response body isn't used if -promscrape.noStaleMarkers is set after the commit 2876137c92 ,
so there is no sense in pupulating it in memory. This should reduce memory usage when scraping big responses.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1728#issuecomment-949630694
2021-10-22 16:49:21 +03:00
Aliaksandr Valialkin
fcdb0c415e
docs/CHANGELOG.md: document 43a7984cd8 2021-10-22 14:00:50 +03:00
Aliaksandr Valialkin
6bc10f0623
lib/promscrape: do not sort original labels and do not intern label string for the original labels before the sharding code is executed
This should reduce CPU and memory usage in shard mode when service discovery finds big number of scrape targets with many long labels.
See https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets

This is a follow-up after 9882cda8b9

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1728
2021-10-22 13:55:39 +03:00
Aliaksandr Valialkin
a8bcc3c276
lib/promscrape: reduce memory usage if -promscrape.noStaleMarkers command-line flag is passed
Do not store in memory the response from the last scrape per each target if -promscrape.noStaleMarkers option is enabled.
This should reduce memory usage when the scraped targets return large responses.
2021-10-22 13:22:08 +03:00
Roman Khavronenko
e0f21d6000
vmalert: correctly calculate alert ID including extra labels (#1734)
Previously, ID for alert entity was generated without alertname or groupname.
This led to collision, when multiple alerting rules within the same group
producing same labelsets. E.g. expr: `sum(metric1) by (job) > 0` and
expr: `sum(metric2) by (job) > 0` could result into same labelset `job: "job"`.

The issue affects only UI and Web API parts of vmalert, because alert ID is used
only for displaying and finding active alerts. It does not affect state restore
procedure, since this label was added right before pushing to remote storage.

The change now adds all extra labels right after receiving response from the datasource.
And removes adding extra labels before pushing to remote storage.

Additionally, change introduces a new flag `Restored` which will be displayed in UI
for alerts which have been restored from remote storage on restart.
2021-10-22 12:31:35 +03:00
Aliaksandr Valialkin
ad40e70d39
docs/CHANGELOG.md: document a3684fe3de 2021-10-22 12:29:27 +03:00
Nikolay
83e1dfccba
adds tab as second separator for graphite text protocol (#1733)
* adds tab as second separator for graphite text protocol

* changes indexFunc for indexAny

* Update lib/protoparser/graphite/parser_test.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-22 12:29:27 +03:00
Yury Molodov
3ccc37a98b
vmui: query history (#1732)
* feat: add query history

* fix: change detect keyUp for nav query history

* feat: set default query history

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-22 12:29:27 +03:00
Aliaksandr Valialkin
d56e676d71
lib/flagutil: do not expose sensitive info (passwords, keys and urls) at /flags page 2021-10-20 00:51:15 +03:00
Aliaksandr Valialkin
5705f4b6d1
lib/httpserver: expose command-line flags at /flags page
This should simplify debugging.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695
2021-10-20 00:46:54 +03:00
Aliaksandr Valialkin
a105b71116
lib/envflag: use flag.Set for setting the flags from env vars
This should make visible the set flags at flag.Visit(), which is used later for logging
and exporting the `is_set` label for these flags at /metrics page
2021-10-20 00:46:53 +03:00
Aliaksandr Valialkin
93511b4be7
lib/storage: log a warning when the -storageDataPath has less than -storage.minFreeDiskSpaceBytes
This should improve the debuggability of the readonly feature.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1727
2021-10-19 23:58:09 +03:00
Roman Khavronenko
ca49853664
vmalert: make group.ID() thread-safe (#1726)
Commit fixes potential race condition when group update
and generating of ID() happens simultaneously.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-10-19 16:45:52 +03:00
Roman Khavronenko
627224d493
vmalert: properly init SIGHUP listener before starting group manager (#1725)
Regression was introduced during code refactoring. It potentially
could lead to situation when SIGHUP signals were ignored while
vmalert was still busy with initing group manager.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-10-19 16:36:08 +03:00
Aliaksandr Valialkin
0a1982b294
app/vmauth: fix metric name prefixes: vmagent -> vmauth 2021-10-19 15:30:16 +03:00
Aliaksandr Valialkin
5df7ef323f
docs/Single-server-VictoriaMetrics.md: add a link to VMUI at VictoriaMetrics playground 2021-10-19 14:42:35 +03:00
Aliaksandr Valialkin
ea69eef375
lib/promscrape/discovery/kubernetes: log a warning if role: endpoints discovers more than 1000 targets per a single endpoint
In this case `role: endpointslice` must be used instead.

See the following references:

* https://kubernetes.io/docs/reference/labels-annotations-taints/#endpoints-kubernetes-io-over-capacity
* https://github.com/kubernetes/kubernetes/pull/99975
* https://github.com/prometheus/prometheus/issues/7572#issuecomment-934779398
2021-10-19 13:22:28 +03:00
Aliaksandr Valialkin
948e9b8f50
docs/CHANGELOG.md: document 146a5b504c 2021-10-19 11:24:44 +03:00
Aliaksandr Valialkin
a7de3712e6
docs/CHANGELOG.md: document cbcc622786 2021-10-19 09:00:05 +03:00
Aliaksandr Valialkin
e4ebcebc8a
deployment/docker/alerts.yml: formatting fixes after 865a60f13e 2021-10-19 09:00:05 +03:00
Nikolay
e84a063209
changes job source for /target api (#1723)
use jobNameOriginal instead of relabeled as prometheus does

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1707
2021-10-19 09:00:05 +03:00
Roman Khavronenko
d763837130
dashboards: add cardnilaity limiter panels for vmagent (#1720)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-10-19 09:00:05 +03:00
Yurii Kravets
34f52de3a5
Update alerts.yml
Added Series Limit day\hour alerts
2021-10-19 09:00:05 +03:00
Aliaksandr Valialkin
79709e2586
vendor: return back the previous google.golang.org/genproto version, since the latest version leads to compile errors
The following errors:

    vendor/cloud.google.com/go/storage/storage.go:1447:53: o.GetCustomerEncryption().GetKeySha256 undefined (type *"google.golang.org/genproto/googleapis/storage/v2".Object_CustomerEncryption has no field or method GetKeySha256)
    vendor/cloud.google.com/go/storage/writer.go:439:10: q.GetCommittedSize undefined (type *"google.golang.org/genproto/googleapis/storage/v2".QueryWriteStatusResponse has no field or method GetCommittedSize)
2021-10-18 15:37:29 +03:00
Aliaksandr Valialkin
c92746bb01
vendor: make vendor-update 2021-10-18 15:23:46 +03:00