Nikolay
9ea1dca3dd
fixes DO service discovery labels ( #1389 )
...
adds test for digitalocean sd
2021-06-17 17:21:10 +03:00
Aliaksandr Valialkin
60bc35f550
docs/{vmgateway,vmbackupmanager}: explicitly mention that these components are a part of an enterprise package
2021-06-17 17:19:13 +03:00
Aliaksandr Valialkin
a207be3ffb
lib/storage: fix infinite loop introduced in aa9b56a046
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1244
2021-06-17 14:27:30 +03:00
Aliaksandr Valialkin
0efd37cec1
lib/{mergeset,storage}: reduce the number of fsync calls on data ingestion path on systems with many cpu cores
...
VictoriaMetrics maintains a buffer per CPU core for the ingested data. These buffers are flushed to disk every second.
These buffers are flushed to disk in parallel starting from the commit 56b6b893cehttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338#issuecomment-863046999 .
This commit merges the per-CPU buffers into bigger in-memory buffers before flushing them to disk.
This should reduce the rate of fsync syscalls and, consequently, the write disk IO on systems with many CPU cores.
This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338
See also https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1244
2021-06-17 13:51:42 +03:00
Aliaksandr Valialkin
51fc469642
app/vmagent/remotewrite: go fmt after 0a796f7c3a
2021-06-17 13:51:40 +03:00
Aliaksandr Valialkin
90c3606269
docs/vmagent.md: sync with app/vmagent/README.md via make docs-sync
2021-06-16 12:37:55 +03:00
Aliaksandr Valialkin
644102b03b
docs/CHANGELOG.md: document the changed -remoteWrite.queues value
...
This is a follow-up for 0a796f7c3ahttps://github.com/VictoriaMetrics/VictoriaMetrics/pull/1385
2021-06-16 12:37:55 +03:00
Zongyang
cf506e300d
Change default value of '-remoteWrite.queues' to cgroup.AvailableCPUs * 2 ( #1385 )
...
* Change default value of '-remoteWrite.queues' to cgroup.AvailableCPUS() * 2 to reduce scrape interval
Default value of vmagent option '-remotewrite.queues' is 4 and default
size of vmagent ScheudleUnmarshalWorkers is number of CPUs, when available
CPUs is much greater than 4, e.g 32, worker are competing push queues
which will increase scrape interval and may cause scrape timeout.
* Update README and flag description
Co-authored-by: xiaozy <xiaozy01@fenbi.com>
2021-06-16 12:37:55 +03:00
Roman Khavronenko
a15c947045
promql: fix increase_pure calculation for cases with stale series ( #1381 )
...
Due to staleness handling, increase_pure were using incorrect previous value
during calculation in cases where series disappears for period longer
than staleness period and then returns back. The fix suppose to account
for a real datapoint value before staleness takes place. The fix should
remove unexpected spikes while using `increase_pure` for staled series.
2021-06-15 17:37:51 +03:00
Aliaksandr Valialkin
234152d66e
docs/Single-server-VictoriaMetrics.md: mention that VictoriaMetrics works great with APM workloads (aka Application Performance Monitoring)
2021-06-15 17:33:41 +03:00
Aliaksandr Valialkin
b133de1e37
lib/storage: move deletedMetricIDs set from indexDB to Storage
...
This makes consitent the list of deleted metricIDs when it is used from both the current indexDB and the previous indexDB (aka extDB).
This should fix the issue, which could lead to storing new samples under deleted metricIDs after indexDB rotation.
See more details at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347#issuecomment-861232136 .
Thanks to @tangqipengleoo for the initial analysis and the pull request - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .
This commit resolves the issue in more generic way compared to https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .
The downside of the commit is the deletedMetricIDs set isn't cleaned from the metricIDs outside the retention. It needs app restart.
This should be OK in most cases.
2021-06-15 15:07:54 +03:00
Aliaksandr Valialkin
ebaf68bcb0
lib/protoparser: stop reading the input stream as soon as the callback provided by the caller returns error
...
This is a follow-up for af90c3c43b
2021-06-14 15:20:38 +03:00
faceair
2ea187e801
lib/protoparser: stop read when callback error ( #1380 )
2021-06-14 15:20:37 +03:00
Aliaksandr Valialkin
5f91a701fa
lib/promscrape: show the number of samples collected during the last scrape at /targets and /api/v1/targets pages
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1377
2021-06-14 14:04:35 +03:00
Aliaksandr Valialkin
fb8114ad9c
vendor: update github.com/klauspost/compress from v1.13.0 to v1.13.1
2021-06-14 13:42:05 +03:00
Roman Khavronenko
db39c4a7d1
dashboard: bump version requirements ( #1379 )
2021-06-14 13:32:32 +03:00
Aliaksandr Valialkin
5cd50d840f
docs/CHANGELOG.md: document the addition of DigitalOcean service discovery
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1367
2021-06-14 13:19:31 +03:00
Nikolay
e42da47608
adds digital ocean sd ( #1376 )
...
* adds digital ocean sd config
* adds digital ocean sd
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1367
* typo fix
2021-06-14 13:19:29 +03:00
Roman Khavronenko
1053d3e5a9
Dashboard cluster ( #1375 )
...
* dashboard: update vmagent dash
The update contains the following changes:
* display anonymous memory usage metric. This metric suppose to reflect
memory usage of the process which can't be freed by OS;
* add legends to all panels. This is important for cases when users share
the screenshots;
* modify panels for Grafana v8.0.0
* dashboard: update cluster dash
The update contains the following changes:
* move stats panels to Configuration row, so it can be collapsed;
* display anonymous memory usage metric. This metric suppose to reflect
memory usage of the process which can't be freed by OS;
* add legends to all panels. This is important for cases when users share
the screenshots;
* modify panels for Grafana v8.0.0
2021-06-14 13:03:54 +03:00
Aliaksandr Valialkin
df057177a0
lib/promscrape: increase the duration for reading the full response in stream parsing mode
...
Increase the duration from 10x to 30x of the configured `scrape_interval'.
This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1365
2021-06-14 12:29:46 +03:00
Aliaksandr Valialkin
074b11fa69
lib/protoparser: measure the duration for reading the whole block of data instead of a single read operation
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1365
2021-06-14 12:29:45 +03:00
Aliaksandr Valialkin
357dbe092c
docs/Cluster-VictoriaMetrics.md: add lists for command-line flags for cluster components
2021-06-14 12:21:22 +03:00
Aliaksandr Valialkin
87d221f78a
lib/protoparser/common: log the duration for reading a block of data in ReadLinesBlockExt on error
...
This may help debugging issues like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1365
2021-06-14 12:21:21 +03:00
Aliaksandr Valialkin
52efd5a05c
docs/vmalert.md: follow-up after 6d5a8c28cd
2021-06-14 11:43:02 +03:00
Roman Khavronenko
c5f493db8e
Vmalert docs ( #1372 )
...
* vmalert: mention what happens if `for` is set to 0 or omitted
* vmalert: add more context to docs
2021-06-14 11:43:01 +03:00
Aliaksandr Valialkin
541429a9af
docs/CHANGELOG.md: cut v1.61.1
2021-06-11 13:02:04 +03:00
Aliaksandr Valialkin
0672cfffa2
app/vmauth: properly handle http.ErrAbortHandler panic
...
This panic can be raised by the reverseProxy on aborted request to the backend.
So handle it (e.g. suppress) at reverseProxy.ServeHTTP call.
Do not suppress the panic at lib/httpserver generic HTTP handler,
since it may result in an inconsistent state left after the panicking handler.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1353
2021-06-11 12:54:37 +03:00
Aliaksandr Valialkin
ce10bdc82a
lib/storage: reset cache on disk during series deletion and during indexdb rotation
...
This should prevent from inconsistent behavior (aka partially missing data for some time series) after unclean shutdown.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347
2021-06-11 12:54:36 +03:00
Aliaksandr Valialkin
d5973d3180
docs/CHANGELOG.md: document the bugfix from 7adfe878e1
2021-06-11 11:30:07 +03:00
Roman Khavronenko
f3cb2158a3
vmalert: fix mistake with object reuse while parsing response ( #1370 )
...
* vmalert: fix mistake with object reuse while parsing response
During the refactoring, the wrong optimisations was applied in
parse function which caused metric fields reset. The change removes
optimisation.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1369
* vmalert: add test to cover multiple metrics in one response
2021-06-11 11:30:07 +03:00
John Belmonte
3e79f3994e
spelling fix: synonym ( #1363 )
2021-06-11 10:58:48 +03:00
Aliaksandr Valialkin
eb335d2c29
lib/storage: consistency renaming: getMaxRawRowsPerPartition -> getMaxRawRowsPerShard
2021-06-11 10:52:31 +03:00
Aliaksandr Valialkin
d06c0e7a94
lib/storage: reduce the amounts of memory which can be occupied by rawRow items during data ingestion on a system with many CPU cores
2021-06-11 10:49:02 +03:00
Aliaksandr Valialkin
56192f24b4
vendor: make vendor-update
2021-06-09 20:41:58 +03:00
Aliaksandr Valialkin
83adc2f3ac
docs/CHANGELOG.md: cut v1.61.0
2021-06-09 19:04:59 +03:00
Aliaksandr Valialkin
c7efd5b43f
docs/FAQ.md: add a chapter comparing VictoriaMetrics to QuestDB
2021-06-09 19:03:47 +03:00
Aliaksandr Valialkin
e8e7f03394
app/vmselect/promql: typo fix in the comment
2021-06-09 18:34:57 +03:00
Aliaksandr Valialkin
203d883b2b
docs/Articles.md: update the broken link to https://nordicapis.com/api-monitoring-with-prometheus-grafana-alertmanager-and-victoriametrics/
2021-06-09 16:40:39 +03:00
Aliaksandr Valialkin
247b2a5a08
app/vmauth: improve readability for a config with multiple src_paths
2021-06-09 15:38:09 +03:00
Aliaksandr Valialkin
1e13deaa2c
docs/CHANGELOG.md: document the enterprise bugfix for the target property in Graphite Render API
2021-06-09 13:51:32 +03:00
Aliaksandr Valialkin
490783696a
docs/CHANGELOG.md: document improvements in re-routing handling in vminsert
...
See the following commits:
* 1c09e71f5b0d067eb1122c6b917749
2021-06-09 13:42:12 +03:00
Aliaksandr Valialkin
520d62ade2
docs/vmagent.md: mention that vmagent supports scrape targets sharding
2021-06-09 12:30:54 +03:00
Aliaksandr Valialkin
f3749dedba
docs: document rules replay feature for vmalert
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/836
This is a follow-up for 2a259ef5e7
2021-06-09 12:30:54 +03:00
Roman Khavronenko
5aa7846900
vmalert: support rules backfilling (aka replay) ( #1358 )
...
* vmalert: support rules backfilling (aka `replay`)
vmalert can `replay` configured rules in the past
and backfill results via remote write protocol.
It supports MetricsQL/PromQL storage as data source,
and can backfill data to remote write compatible
storage.
Supports recording and alerting rules `replay`. See more
details in README.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/836
* vmalert: review fixes
* vmalert: readme fixes
2021-06-09 12:30:54 +03:00
Aliaksandr Valialkin
8ad445474a
docs/CHANGELOG.md: document the bugfix, which prevents panics for aborted http requests in vmauth
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1353
This is a follow-up for 6b29b955c0
2021-06-09 12:12:47 +03:00
Nikolay
2c1611d316
disables panic for net/httpAbortHandler ( #1355 )
2021-06-09 12:12:45 +03:00
k1rk
c6c789db8f
rename serviceHealth group name to vm-health ( #1360 )
...
this causes conflicts in `victoria-metrics-k8s-stack` chart =)
2021-06-09 02:26:21 +03:00
Aliaksandr Valialkin
8890dadd73
deployment/docker/docker-compose.yml: update Grafana from v7.5.2 to v8.0.0
...
See https://github.com/grafana/grafana/releases/tag/v8.0.0
2021-06-08 15:53:24 +03:00
Aliaksandr Valialkin
2f8baf2aa7
vendor: make vendor-update
2021-06-08 15:53:23 +03:00
Aliaksandr Valialkin
661f6f929b
deployment/docker: update Go builder from v1.16.4 to v1.16.5
...
See the fixed isses at https://github.com/golang/go/issues?q=milestone%3AGo1.16.5+label%3ACherryPickApproved
2021-06-08 15:44:00 +03:00
