251 commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	46127b432d	lib/bytesutil: add -internStringDisableCache and -internStringCacheExpireDuration command-line flags ... This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3872	2023-02-27 14:16:49 -08:00
Aliaksandr Valialkin	23871fb0bf	app/vmagent: add -remoteWrite.vmProtoCompressLevel command-line flag for tuning the compression level for VictoriaMetrics remote write protocol	2023-02-27 11:03:49 -08:00
Aliaksandr Valialkin	ffa327d6d1	app/vmagent: use the provided auth options when checking whether the remote storage supports VictoriaMetrics remote write protocol ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3847 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225	2023-02-26 12:07:47 -08:00
Aliaksandr Valialkin	a02cf92fd1	docs: update --help descriptions after recent changes	2023-02-23 19:02:27 -08:00
Aliaksandr Valialkin	c080443fef	app/vmagent: automatically detect whether the remote storage supports VictoriaMetrics remote write protocol ... Substitute -remoteWrite.useVMProto with -remoteWrite.forcePromProto command-line flag, which can be used for forcing Prometheus remote write protocol in cases when the remote storage supports VictoriaMetrics remote write protocol. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3847 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225	2023-02-23 17:36:55 -08:00
Aliaksandr Valialkin	9fbd45a22f	lib/promscrape/discovery/kuma: follow-up for 317fef95f9 ... - Do not generate __meta_server label, since it is unavailable in Prometheus. - Add a link to https://docs.victoriametrics.com/sd_configs.html#kuma_sd_configs to docs/CHANGELOG.md, so users could click it and read the docs without the need to search the corresponding docs. - Remove kumaTarget struct, since it is easier generating labels for discovered targets directly from the response returned by Kuma. This simplifies the code. - Store the generated labels for discovered targets inside atomic.Value. This allows reading them from concurrent goroutines without the need to use mutex. - Use synchronouse requests to Kuma instead of long polling, since there is a little sense in the long polling when the Kuma server may return 304 Not Modified response every -promscrape.kumaSDCheckInterval. - Remove -promscrape.kuma.waitTime command-line flag, since it is no longer needed when long polling isn't used. - Set default value for -promscrape.kumaSDCheckInterval to 30s in order to be consistent with Prometheus. - Remove unnecessary indirections for string literals, which are used only once, in order to improve code readability. - Remove unused fields from discoveryRequest and discoveryResponse. - Update tests. - Document why fetch_timeout and refresh_interval options are missing in kuma_sd_config. - Add docs to discoveryutils.RequestCallback and discoveryutils.ResponseCallback, since these are public types. Side notes: it is weird that Prometheus implementation for kuma_sd_configs sets `instance` label, since usually this label is set by the Prometheus itself to __address__ after the relabeling phase. See https://www.robustperception.io/life-of-a-label/ Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3389 See https://github.com/prometheus/prometheus/issues/7919 and https://github.com/prometheus/prometheus/pull/8844 as a reference implementation in Prometheus	2023-02-22 17:51:51 -08:00
Alexander Marshalov	317fef95f9	add kuma_sd_config for Kuma Control Plane targets discovery (#3389) (#3840)	2023-02-22 13:59:56 +01:00
Artem Navoiev	6b4ccc17c2	docs: fix typo OpentTSDB -> OpenTSDB (#3854) ... Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2023-02-22 11:24:26 +01:00
Aliaksandr Valialkin	f4f1f2f976	docs/vmagent.md: remove the claim that VictoriaMetrics remote write protocol reduces the network bandwidth usage by up to 10x comparing to Prometheus remote write protocol ... The 10x savings are reproduced only on artificial data. The savings on production data are usually in the range 2x-4x.	2023-02-20 19:11:31 -08:00
Aliaksandr Valialkin	a678cbe62e	docs/vmagent.md: mention that Mimir doesnt support backfilling	2023-02-20 19:11:30 -08:00
Aliaksandr Valialkin	76f2c70be3	app/vmagent: add support for VictoriaMetrics remote write protocol, which allows saving up to 10x on network bandwidth costs under high load ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225	2023-02-20 19:11:30 -08:00
Aliaksandr Valialkin	f40661e7b7	docs/vmagent.md: clarify that automatically generated metrics contain all the target-specific labels, including instance and job	2023-02-09 11:04:51 -08:00
Aliaksandr Valialkin	0890adde67	docs: update command-line descriptions after 73256fe438	2023-01-27 00:00:37 -08:00
Roman Khavronenko	95ee86b600	docs: specify the time window for series_limit (#3708) ... Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-01-25 09:30:20 -08:00
Aliaksandr Valialkin	28f66f0079	docs: update the list of command-line flags according to the latest changes	2023-01-25 09:20:24 -08:00
Artem Navoiev	37c52ccaf4	vmagent: add minimal scrape file exampe for vmagent quick start (#3627) ... * vmagent: add minimal scrape file exampe for vmagent quick start Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * replace example with link to your prometheus.yml in docker Signed-off-by: Artem Navoiev <tenmozes@gmail.com> Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2023-01-10 23:16:10 +01:00
Aliaksandr Valialkin	af263fe881	all: small improvements in error messages and command-line flag descriptions related to concurrency limiters	2023-01-07 00:11:44 -08:00
Karan Sharma	48f371a46c	lib/promscrape: add Prometheus-compatible service discovery for Nomad (#3549) ... Add nomad_sd_config support for service discovery	2023-01-05 23:03:58 +01:00
Aliaksandr Valialkin	0dea3b71da	lib/promscrape: pre-fetch metric_relabel_configs rules when debugging metric relabeling for a particular target ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407	2023-01-05 03:26:49 -08:00
Aliaksandr Valialkin	fd175ad80b	docs: update -help outputs for vm* tools	2023-01-03 23:27:06 -08:00
Aliaksandr Valialkin	fa13bbc48a	app/{vmagent,vminsert}: add support for streaming aggregation ... See https://docs.victoriametrics.com/stream-aggregation.html Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3460	2023-01-03 22:19:21 -08:00
Artem Navoiev	7d9c4bebc0	update links to grafana dashboards (#3534) ... docs: update links to grafana dashboards Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2022-12-25 17:36:20 +01:00
Aliaksandr Valialkin	5d30080555	lib/flagutil: support for TB and TiB suffixes for command-line flags, which accept byte sizes	2022-12-14 17:52:32 -08:00
Aliaksandr Valialkin	231569f89b	docs/vmagent.md: small formatting fix	2022-12-14 14:25:35 -08:00
Aliaksandr Valialkin	7a61bafe59	docs/vmagent.md: clarify that relabeling is actually a debugging at relabel debug section	2022-12-13 15:46:21 -08:00
Aliaksandr Valialkin	b01607e3fb	docs: clarify that single-node VictoriaMetrics also provides functionality for relabel debugging ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407	2022-12-10 20:49:52 -08:00
Aliaksandr Valialkin	a8b8e23d68	lib/promscrape: implement target-level and metric-level relabel debugging ... Target-level debugging is performed by clicking the 'debug' link at the corresponding target on either http://vmagent:8429/targets page or on http://vmagent:8428/service-discovery page. Metric-level debugging is perfromed at http://vmagent:8429/metric-relabel-debug page. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407 See https://docs.victoriametrics.com/vmagent.html#relabel-debug	2022-12-10 02:09:44 -08:00
Aliaksandr Valialkin	eeacbaf0b6	all: update Go builder from v1.19.3 to v1.19.4 ... See https://github.com/golang/go/issues?q=milestone%3AGo1.19.4+label%3ACherryPickApproved	2022-12-08 16:41:24 -08:00
Aliaksandr Valialkin	56b8980915	lib/promscrape: allow using sample_limit and series_limit options in stream parsing mode ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3458	2022-12-08 16:33:38 -08:00
Aliaksandr Valialkin	654e94f420	lib/promscrape: add exported_ prefix to metric names exported by scrape targets if they clash with automatically generated metrics ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3406	2022-11-28 18:37:09 -08:00
Aliaksandr Valialkin	045fec631b	docs/vmagent.md: typo fix	2022-11-24 17:19:45 -08:00
Aliaksandr Valialkin	20d758e3e4	all: add a link to https://docs.victoriametrics.com/enterprise.html into description for enterprise flags	2022-11-21 15:42:01 +02:00
Roman Khavronenko	9f8bf524ad	bump go version to 1.19.3 (#3327) ... Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-11-08 16:43:59 +01:00
Aliaksandr Valialkin	c52c23c272	docs/enterprise.md: describe all the enteprise features in a short doc at https://docs.victoriametrics.com/enterprise.html	2022-10-24 18:02:03 +03:00
Aliaksandr Valialkin	5269b1ad77	lib/promscrape: allow controlling staleness tracking on a per-scrape_config basis ... Add support for no_stale_markers option at scrape_config section. See https://docs.victoriametrics.com/sd_configs.html#scrape_configs and https://docs.victoriametrics.com/vmagent.html#prometheus-staleness-markers	2022-10-07 23:36:14 +03:00
Aliaksandr Valialkin	b47caa86db	all: update the minimum required Go verson from 1.19.1 to 1.19.2 ... This is needed because of security vulnerabilities found in Go 1.19.1 See https://go.dev/doc/devel/release#go1.19.2	2022-10-07 22:43:37 +03:00
Aliaksandr Valialkin	d7327d2f02	docs/vmagent.md: fix incorrect url for multitenant writes to VictoriaMetrics cluster	2022-10-01 18:51:37 +03:00
Aliaksandr Valialkin	0dc93cca7f	app/vmagent/remotewrite: allow specifying per--remoteWrite.url disk limits for persistent queue with pending data ... This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3071 Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2970	2022-10-01 18:40:59 +03:00
Aliaksandr Valialkin	63a94b1d54	docs/vmagent.md: formatting fixes	2022-10-01 13:36:24 +03:00
Aliaksandr Valialkin	5b83e6e14e	app/vmagent/README.md: apply typo fix from 1fda517af9	2022-10-01 12:31:06 +03:00
Nikolay	33f40f4a5f	app/vminsert: allows parsing tenant id from labels (#3009) ... * app/vminsert: allows parsing tenant id from labels it should help mitigate issues with vmagent's multiTenant mode, which works incorrectly at heavy load and it cannot handle more then 100 different tenants. This functional hidden with flag and do not change vminsert default behaviour https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2970 * Update docs/Cluster-VictoriaMetrics.md Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> * wip * app/vminsert/netstorage: clean remaining labels in order to free up GC * docs/Cluster-VictoriaMetrics.md: typo fix * wip * wip Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-09-30 18:35:53 +03:00
Dmytro Kozlov	6ab1cede62	lib/{httpserver,netutil}: allow to define min and max TLS version of the http server (#3109) ... * lib/{httpserver,netutil}: allow to define min and max TLS version of the http server * lib/httpserver: added descriptions about tls supported versions * lib/netutil: check minimal tls version, added supported tls versions to error * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-09-26 17:35:45 +03:00
Aliaksandr Valialkin	7c2474dac7	lib/protoparser/datadog: sanitize metric names by default in the same way as DataDog does ... This commit is based on the pull request https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3105 Thanks to @PerGon for the idea and initial implementation.	2022-09-26 13:57:23 +03:00
Aliaksandr Valialkin	fcffdba9dc	app/{vmagent,vminsert}: add -usePromCompatibleNaming command-line flag for normalizing metric names and label names in the ingested samples ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113 Thanks to @erkexzcx for the idea and the initial pull request at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3146	2022-09-26 13:11:39 +03:00
Aliaksandr Valialkin	bbecd27557	docs: update docs with explicitly marked enterprise command-line flags for VictoriaMetrics and vmagent	2022-09-15 13:22:56 +03:00
Roman Khavronenko	efea51a9ee	bump Go version to 1.19.1 (#3108) ... The reason is to cover vulnerability GO-2022-0969 Found in: net/http@go1.18.5 Fixed in: net/http@go1.19.1 More info: https://pkg.go.dev/vuln/GO-2022-0969 Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-09-14 12:29:19 +02:00
Aliaksandr Valialkin	ef7fdbb63c	docs/Single-server-VictoriaMetrics.md: add a note that cardinality explorer at cluster version of VictoriaMetrics may return lower than expected number of unique label values ... See the corresponding comment in the code: 5a6e617b5e/app/vmselect/netstorage/netstorage.go (L1039-L1045)	2022-09-06 14:46:35 +03:00
Aliaksandr Valialkin	3d12ee47f9	docs: mention that it is safe sharing the collected profiles from security PoV ... The collected profiles do not contain sensitive information	2022-08-24 14:07:36 +03:00
Aliaksandr Valialkin	c011fb0f30	docs/vmagent.md: fix alerting query when scraped samples are dropped because of exceeded series limit ... This is a follow-up after 7d26414b2e	2022-08-24 01:18:31 +03:00
Aliaksandr Valialkin	8550c44e31	app/vmagent: add ability to construct a label from multiple existing labels by referring them in the replacement field during relabeling ... For example: - target_label: composite-label replacement: "{{source_label1}}-{{source_label2}}"	2022-08-21 22:50:01 +03:00