Commit graph

975 commits

Author SHA1 Message Date
Aliaksandr Valialkin
7a35447031 lib/uint64set: small code cleanup and perf tuning
* Remember the last accessed bucket on Has() call.
* Inline fast paths inside Add() and Has() calls.
* Remove fragile code with maxUnsortedBuckets inside bucket32.
2020-03-25 15:29:59 +02:00
Aliaksandr Valialkin
19d93e1a2e deployment/docker: update Go builder from Go1.14.0 to Go1.14.1 2020-03-24 22:35:46 +02:00
Aliaksandr Valialkin
cce936de5b lib/uint64set: go fmt 2020-03-24 22:28:09 +02:00
Aliaksandr Valialkin
7cdac6634c lib/storage: serialize snapshot creation process with mutex
This guarantees that the snapshot contains all the recently added data
from inmemory buffers when multiple concurrent calls to Storage.CreateSnapshot are performed.
2020-03-24 22:27:28 +02:00
Aliaksandr Valialkin
c31b956355 lib/uint64set: added more tests 2020-03-24 22:27:26 +02:00
Aliaksandr Valialkin
8fa9066b98 docs/CaseStudies.md: added a case study from MHI Vestas Offshore Wind 2020-03-14 13:22:41 +02:00
Aliaksandr Valialkin
31a533656e lib/storage: remove obsolete code 2020-03-13 22:42:42 +02:00
Aliaksandr Valialkin
58cb7fc476 app/vmselect: adjust label_map() handling for corner cases
The following corner cases now supported:
* label_map(q, "label", "", "foo") - adds `label="foo"` to series with missing `label`
* label_map(q, "label", "foo", "") - removes `label="foo"` from series

All the unmatched labels are kept unchanged.
2020-03-13 18:41:52 +02:00
Aliaksandr Valialkin
5c5a30734e vendor: update github.com/VictoriaMetrics/metrics from v1.11.0 to v1.11.2
This fixes data race in Histogram
2020-03-13 12:40:15 +02:00
Aliaksandr Valialkin
bf1869d33d lib/promscrape: allow overriding external_labels as Prometheus does
Prometheus docs at https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config say:

> In communication with external systems, they are always applied only
> when a time series does not have a given label yet and are ignored otherwise.

Though this may result in consistency chaos when scrape targets override `external_labels`,
let's stick with Prometheus behavior for the sake of backwards compatibility.

There is last resort in vmagent with `-remoteWrite.label`, which consistently
sets the configured labels to all the metrics before sending them to remote storage.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/366
2020-03-12 20:25:27 +02:00
Aliaksandr Valialkin
0e7a71a245 app/vmselect: add label_map(q, label, srcValue1, dstValue1, ... srcValueN, dstValueN) function to MetricsQL
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/369
2020-03-12 19:13:56 +02:00
Aliaksandr Valialkin
39a977b5aa vendor: update google.golang.org/genproto from fc8f55426688 to da6875a35672 2020-03-12 18:12:46 +02:00
Aliaksandr Valialkin
9ef7bba17b vendor: update golang.org/x/tools from 26f6a1b6802d to 5e2df02acb1e 2020-03-12 18:08:02 +02:00
Aliaksandr Valialkin
d91790543f vendor: update github.com/aws/aws-sdk-go from v1.29.10 to v1.29.22 2020-03-12 17:55:11 +02:00
Aliaksandr Valialkin
fa4d70b428 vendor: update google.golang.org/api from v0.19.0 to v0.20.0 2020-03-12 17:52:02 +02:00
Aliaksandr Valialkin
83fe650ca1 vendor: update golang.org/x/sys from d5e6a3e2c0ae to 5c8b2ff67527 2020-03-12 17:46:39 +02:00
Aliaksandr Valialkin
e5c073a9a1 vendor: update github.com/klauspost/compress from v1.10.1 to v1.10.3 2020-03-12 17:33:11 +02:00
Aliaksandr Valialkin
fa7910fba1 lib/protoparser/csvimport: add missing metric vm_rows_invalid_total{type="csvimport"} 2020-03-12 15:28:10 +02:00
Aliaksandr Valialkin
41b532046f README.md: mention about alternative dashboard for cluster version - https://grafana.com/grafana/dashboards/11831 2020-03-12 15:11:55 +02:00
Aliaksandr Valialkin
50555d89d3 app/vmselect: add -search.maxStalenessInterval for tuning Prometheus data model closer to Influx-style data model 2020-03-11 16:44:03 +02:00
Aliaksandr Valialkin
375d5483fa lib/promscrape: remove possible races when registering and de-registering scrape workers for /targets page 2020-03-11 16:30:43 +02:00
Aliaksandr Valialkin
b46af9678e app/vmagent: mention that vmagent can filter data 2020-03-11 16:23:10 +02:00
Aliaksandr Valialkin
803f919c75 docs/Articles.md: add a link to https://stas.starikevich.com/posts/disk-usage-for-vm-versus-prometheus/ 2020-03-11 04:56:37 +02:00
Aliaksandr Valialkin
187fd89c70 lib/promscrape: consistently update /targets page after SIGHUP 2020-03-11 03:20:38 +02:00
Aliaksandr Valialkin
8939c19281 app/vmstorage: return 500 status code instead of 200 status code on internal errors inside /snapshot/* handlers 2020-03-10 23:54:27 +02:00
Aliaksandr Valialkin
b51e548b64 docs/vmagent.md: sync with app/vmagent/README.md 2020-03-10 21:54:35 +02:00
Aliaksandr Valialkin
f6410ff2bf app/vmselect: add optional max_rows_per_line query arg to /api/v1/export
This arg allows limiting the number of data points that may be exported on a single line.
2020-03-10 21:47:43 +02:00
Aliaksandr Valialkin
2f0a36044c app/{vmagent,vminsert}: add support for importing csv data via /api/v1/import/csv 2020-03-10 21:17:40 +02:00
Aliaksandr Valialkin
7545784a49 all: fix golangci-lint issues 2020-03-10 19:40:03 +02:00
Aliaksandr Valialkin
8a2ea0171a docs/FAQ.md: actualize answer about deduplication 2020-03-09 13:37:39 +02:00
Aliaksandr Valialkin
d70c9b9556 docs: add missing vmagent.png, which is used in vmagent.md 2020-03-09 13:37:38 +02:00
Aliaksandr Valialkin
3fc6599aa2 app/vmagent: properly apply -remoteWrite.sendTimeout to fasthttp.HostClient 2020-03-09 13:31:22 +02:00
Aliaksandr Valialkin
d39dd8aa69 lib/promscrape: do not retry idempotent requests when scraping targets
This should prevent from the following unexpected side-effects of idempotent request retries:
- increased actual timeout when scraping the target comparing to the configured scrape_timeout
- increased load on the target

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/357
2020-03-09 13:31:20 +02:00
Aliaksandr Valialkin
12789a4621 app/vmagent: do not allow non-supported fields in -remoteWrite.relabelConfig and file_sd_configs
This should reduce possible confusion like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/363
2020-03-06 20:19:34 +02:00
Aliaksandr Valialkin
47e986c26f app/vmagent: properly add labels set via -remoteWrite.label to metrics before sending them to -remoteWrite.url 2020-03-06 19:28:14 +02:00
Aliaksandr Valialkin
0d893eff36 Makefile: add build and test rules with enabled race detector. These rules have -race suffix
Fix also `unsafe pointer conversion` errors detected by Go1.14. See https://golang.org/doc/go1.14#compiler .
2020-03-05 12:05:16 +02:00
Aliaksandr Valialkin
197d2916ab README.md: add a link to https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/Articles 2020-03-04 20:06:22 +02:00
Aliaksandr Valialkin
7909964cf3 docs/Articles.md: add a link to https://www.percona.com/blog/2020/02/28/better-prometheus-rate-function-with-victoriametrics/ 2020-03-04 20:06:21 +02:00
Aliaksandr Valialkin
0176fc4206 app/vmagent/README.md: small fixes 2020-03-04 18:15:24 +02:00
Aliaksandr Valialkin
ac03be5a2c app/vmagent/README.md: typo fix 2020-03-04 18:05:43 +02:00
Aliaksandr Valialkin
9354b9177a app/vmagent/README.md: clarification 2020-03-04 18:04:06 +02:00
Aliaksandr Valialkin
4302555228 app/vmagent/README.md: add iot and edge monitoring use case 2020-03-04 18:01:40 +02:00
Aliaksandr Valialkin
ea5904fd76 app/vmagent/README.md: add use cases section 2020-03-04 17:42:57 +02:00
Aliaksandr Valialkin
ed355fe6b4 docs/CaseStudies: add Synthesio 2020-03-04 17:12:18 +02:00
Aliaksandr Valialkin
50190263c8 docs/Single-server-VictoriaMetrics.md: sync with README.md 2020-03-03 21:39:31 +02:00
Aliaksandr Valialkin
31a76a7b3a lib/promscrape: consistency renaming: stopCh -> globalStopCh 2020-03-03 20:08:22 +02:00
Aliaksandr Valialkin
f01d1bf4a8 app/vmagent: add -remoteWrite.maxDiskUsagePerURL for limiting the maximum disk usage for each -remoteWrite.url buffer
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/352
2020-03-03 19:49:20 +02:00
Aliaksandr Valialkin
808c17e250 app/vmagent/remotewrite: do not reset empty relabelCtx 2020-03-03 15:01:21 +02:00
Aliaksandr Valialkin
af19ca2483 app/vmagent: add -remoteWrite.urlRelabelConfig for applying individual relabeling for each -remoteWrite.url
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/320
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/308
2020-03-03 13:13:06 +02:00
Aliaksandr Valialkin
c3b239eb1a lib/protoparser/prometheus: allow trailing comma in tags list
The trailing comma is generated by cloudwatch exporter.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/350
2020-03-02 22:23:28 +02:00