Commit graph

1849 commits

Author SHA1 Message Date
Aliaksandr Valialkin
caeb74f068 app/vmselect: reduce memory usage when query touches big number of time series 2020-11-04 17:04:04 +02:00
Aliaksandr Valialkin
ae91a6883c lib/{storage,mergeset}: clean cached index blocks and inmemory blocks more aggressively
Previously such blocks were cleaned after they weren't accessed during 10 minutes.
Now they are cleaned after one minute of missing access. This should reduce memory usage in general case.
2020-11-04 17:04:04 +02:00
Nikolay
e4182dd896
reduces memory usage for vmagent, (#880)
* reduces memory usage for vmagent,
limits count of droppedTarget, that can be stored for /api/v1/targets page up to 999 items,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878

* Update app/vmagent/README.md

* Update app/vmagent/README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2020-11-04 17:03:43 +02:00
Aliaksandr Valialkin
b9e5172aa2 Revert "lib/promscrape: add -promscrape.dropOriginalLabels command-line flag for reducing memory usage when discovering big number of scrape targets"
This reverts commit 5a9743211f.
2020-11-04 11:45:02 +02:00
Aliaksandr Valialkin
600f225cff Revert "docs/vmagent.md: mention about -promscrape.dropOriginalLabels"
This reverts commit bd81f926a4.
2020-11-04 11:44:57 +02:00
Aliaksandr Valialkin
bd81f926a4 docs/vmagent.md: mention about -promscrape.dropOriginalLabels 2020-11-04 11:16:33 +02:00
Aliaksandr Valialkin
5a9743211f lib/promscrape: add -promscrape.dropOriginalLabels command-line flag for reducing memory usage when discovering big number of scrape targets
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
2020-11-04 11:08:57 +02:00
Aliaksandr Valialkin
ca8b5745b5 lib/promscrape: reduce memory allocations in promLabelsString() function
This should help with reducing memory usage in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
2020-11-04 10:38:44 +02:00
Aliaksandr Valialkin
f3f62ab04e lib/storage: do not report about the need of free disk space if parts cannot be merged due to too big write amplification 2020-11-03 15:32:02 +02:00
Aliaksandr Valialkin
e0a91ef163 app/vmagent/remotewrite: drop packets only on 409 status code, since there are other valid 4xx status codes, which shouldnt result in packet drop 2020-11-03 14:25:10 +02:00
Aliaksandr Valialkin
c87fb9191e lib/storage: remove unneeded fmt.Sprintf 2020-11-03 14:20:31 +02:00
John Belmonte
51e661ecfe
add short_version label to vm_app_version metric (#877)
* add short_version label to vm_app_version metric

use case:  Version panel of Grafana dashboard should use a live query, but currently it uses a template query which becomes stale.  Grafana is not able to preform regex substitution on labels.

* Update metrics.go

* fix compile
2020-11-03 14:10:42 +02:00
Aliaksandr Valialkin
cd071357d8 docs/MetricsQL.md: add missing whitespace 2020-11-02 23:49:56 +02:00
Aliaksandr Valialkin
61579680bb vendor: make vendor-update 2020-11-02 22:03:38 +02:00
Aliaksandr Valialkin
fe289331dd lib/storage: remove obsolete code 2020-11-02 19:11:59 +02:00
Aliaksandr Valialkin
d396c265a6 CHANGELOG.md: cut v1.45.0 2020-11-02 02:43:12 +02:00
Aliaksandr Valialkin
31918f60b2 vendor: make vendor-update 2020-11-02 02:41:02 +02:00
Aliaksandr Valialkin
d62ec1cb01 CHANGELOG.md: add a link to https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825 2020-11-02 02:28:55 +02:00
Aliaksandr Valialkin
5e75c389e6 app/vmselect/promql: allow dropping trailing sample only for default_rollup function
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/850
2020-11-02 02:10:59 +02:00
Aliaksandr Valialkin
c0f3be824d lib/promscrape: properly handle response body after 301 redirect
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/869
2020-11-02 01:09:52 +02:00
Aliaksandr Valialkin
ca566dce39 CHANGELOG.md: mention about packets drop in vmagent like Prometheus does 2020-11-02 00:46:49 +02:00
Aliaksandr Valialkin
0b35da159c app/vmagent/remotewrite: drop packets if remote storage returns 4xx status code
This makes consistent the behaviour with Prometheus.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/873
2020-11-02 00:45:09 +02:00
Aliaksandr Valialkin
cb71af216a app/vmselect/promql: go fmt 2020-11-02 00:15:29 +02:00
Aliaksandr Valialkin
daacbc7e34 app/vmselect/promql: do not drop trailing datapoints for instant queries
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
2020-11-02 00:12:37 +02:00
S.F
f477cbe861
OpenBSD packaging files (#853) 2020-11-01 23:39:25 +02:00
Roman Khavronenko
50d44d5932
dashboard: add Storage full ETA panel (#858)
* dashboard: add `Storage full ETA` panel

The new panel suppose to help to estimate the time needed to run out of free
disk space.
Thx to @belm0 @hekmon

* disable legend for `Storage full ETA` panel
2020-11-01 23:37:31 +02:00
Aliaksandr Valialkin
68d004bc05 CHANGELOG.md: mention about recently added bugfixes 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
e277c3d07b lib/promscrape: add stream parse mode for efficient scraping of targets that expose millions of metrics 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
29e4e7f422 lib/storage: drop more samples outside the given retention during background merge
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/17
2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
b7638f04a7 app/vmagent: expose /api/v1/targets page according to https://prometheus.io/docs/prometheus/latest/querying/api/#targets
This page is exposed by vmagent and by a single-node VictoriaMetrics

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/643
2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
c539494b36 app/vmselect/promql: allow passing optional third argument to topk_* and bottomk_* functions in order to obtain sum of time series outside top/bottom K 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
d12c4914f0 lib/storage: properly handle the case when key="__name__" is passed to MetricName.AddTag* 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
64e2d66014 lib/storage: code cleanup after 5bfd4e6218 2020-11-01 23:35:06 +02:00
Sergey Klyuykov
4108e85efd
Fix InfluxDB support on docker-compose deployment. (#872)
* Added UDP protocol support for Graphite/Influx in docker-compose deployment.

This is necessary for Proxmox VE External Metric Server support.
https://pve.proxmox.com/wiki/External_Metric_Server

* Added `influxListenAddr` in docker-compose deployment.

This is necessary for Proxmox VE External Metric Server support.
https://pve.proxmox.com/wiki/External_Metric_Server

Additionally created Grafana Dashboard for monitoring Proxmox VE hosts.
https://grafana.com/grafana/dashboards/13307
2020-11-01 23:34:39 +02:00
Roman Khavronenko
f0bdc5716e
vmalert: skip automatically added labels on alerts restore (#871)
Label `alertgroup` was introduced in #611 and automatically added to generated
time series. By mistake, this new label wasn't correctly purged on restore event
and affected alert's ID uniqueness. This commit removes `alertgroup` label
in restore function.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/870
2020-10-30 08:18:20 +00:00
Nikolay
67059caa12
fixes panic at scrape error body formating, (#868)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/864
regression after body reuse improvements
2020-10-29 17:17:52 +03:00
Nikolay
de3fe22815
adds leading forward slash check for scrapeURL path (#855)
* fixes in-consistency with prometheus behaviour for scrape targets url path.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/835
2020-10-29 08:39:42 +03:00
Sergey Kulukov
055f152246 Added UDP protocol support for Graphite/Influx in docker-compose deployment.
This is necessary for Proxmox VE External Metric Server support.
https://pve.proxmox.com/wiki/External_Metric_Server
2020-10-28 20:26:55 +02:00
Roman Khavronenko
20311f6065
dashboard: clarify the purpose of Concurrent flushes on disk panel (#849)
Current description led to confusion at https://victoriametrics.slack.com/archives/CGZF1H6L9/p1603270014273800
2020-10-28 18:10:46 +00:00
kreedom
a51a7b2a20
vmbackup fix panic when no origin fs given (#859)
* use fsnil when no origin fs
2020-10-28 20:09:10 +02:00
Aliaksandr Valialkin
bca468bb55 CHANGELOG.md: mention about recently added changes 2020-10-20 14:32:14 +03:00
Aliaksandr Valialkin
0729cc36b2 lib/memory: do not print trailing zeroes in logs for -memory.allowedPercent command-line flag 2020-10-20 14:32:07 +03:00
Aliaksandr Valialkin
5bfd4e6218 app/vmstorage: support for -retentionPeriod smaller than one month
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/173
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/17
2020-10-20 14:31:44 +03:00
Aliaksandr Valialkin
920300643a docs/vmrestore.md: make docs-sync 2020-10-20 10:48:19 +03:00
kreedom
ef77120170
vmalert - add dryRun (#842)
vmalert: add `dryRun` flag for rules validation without running the service
2020-10-20 08:15:21 +01:00
Seva Poliakov
b3f3c078e5 Fix typo in vnrestore readme 2020-10-18 15:41:39 +03:00
faceair
84e3881c0b
disable response compression on websocket (#841) 2020-10-17 13:32:34 +03:00
Aliaksandr Valialkin
2ed069c3bc docs/MetricsQL.md: small clarifications 2020-10-17 12:01:43 +03:00
Aliaksandr Valialkin
28353e48ca app/vmselect/promql: an attempt to improve heuristics for dropping trailing data points in time series
Now trailing data points are additionally dropped for time series with a single raw sample

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
2020-10-17 10:44:34 +03:00
Aliaksandr Valialkin
01987f8c77 lib/storage: small code adjustements after d2960a20e0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/781
2020-10-17 01:16:54 +03:00