80 commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	8703b2fa87	app/vmselect: small cleanup after 4f3f9950d0 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3807	2023-05-09 22:45:02 -07:00
Yury Molodov	ddc5197bce	vmui: add metric relabel debug (#3889) ... * feat: add metric relabel debug (#3807) * fix: add link to relabeling cookbook * lib/promrelabel: merge, fix conflicts * lib/promrelabel: fix diff * docs/vmui: add metric relabel playground --------- Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>	2023-05-08 14:59:35 -07:00
Dmytro Kozlov	85b01c4aa7	lib/promrelabel: make target url from labels on target relabel page (#3882) ... * lib/promrelabel: make target url from labels on target relabel page * wip --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-03-20 22:08:39 -07:00
Oleksandr Redko	0e1c395609	app,lib: fix typos in comments (#3804)	2023-02-13 09:32:35 -08:00
Aliaksandr Valialkin	1af6e0b233	lib/promrelabel: pass query args via query string at /metric-relabel-debug and /target-relabel-debug pages if their length doesnt exceed 1000 ... This allows copy-n-pasting the url to another browser window and seeing the same result. The limit in 1000 chars is selected in order to prevent from potential issues with systems which limit the url length such as Internet Explorer - see https://stackoverflow.com/questions/812925/what-is-the-maximum-possible-length-of-a-query-string If the limit is exceeded, then query args are sent via POST method and aren't visible in the url. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3580	2023-01-05 16:45:42 -08:00
Aliaksandr Valialkin	1d16cc9349	lib/promscrape: pre-fetch metric_relabel_configs rules when debugging metric relabeling for a particular target ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407	2023-01-05 03:28:14 -08:00
Aliaksandr Valialkin	ebb8aeb0cf	app/vmselect: remove dependency on lib/promscrape from app/vmselect	2023-01-03 23:27:36 -08:00
Aliaksandr Valialkin	d3f8298739	lib/bytesutil: add InternBytes() function as a shortcut to InternString(ToUnsafeString(..))	2023-01-03 22:15:49 -08:00
Aliaksandr Valialkin	a7942c6c0d	lib/promrelabel: allow calling Match on nil IfExpression ... This simplifies the caller side of IfExpression	2022-12-30 16:47:59 -08:00
Aliaksandr Valialkin	7888712185	lib/promrelabel: make fmt after d3de110070	2022-12-21 20:25:37 -08:00
Aliaksandr Valialkin	cc482b89a3	lib/promrelabel: add support for keepequal and dropequal relabeling actions ... These actions are supported by Prometheus starting from v2.41.0 See https://github.com/prometheus/prometheus/pull/11564 , https://github.com/prometheus/prometheus/issues/11556 and https://github.com/prometheus/prometheus/issues/3756 Side note: It's a pity that Prometheus developers decided inventing `keepequal` and `dropequal` relabeling actions instead of adding support for `keep_if_equal` and `drop_if_equal` relabeling actions supported by VictoriaMetrics since June 2020 - see 2a39ba639d .	2022-12-21 20:06:09 -08:00
Aliaksandr Valialkin	97b41e727c	lib/promscrape: implement target-level and metric-level relabel debugging ... Target-level debugging is performed by clicking the 'debug' link at the corresponding target on either http://vmagent:8429/targets page or on http://vmagent:8428/service-discovery page. Metric-level debugging is perfromed at http://vmagent:8429/metric-relabel-debug page. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407 See https://docs.victoriametrics.com/vmagent.html#relabel-debug	2022-12-10 02:25:56 -08:00
Aliaksandr Valialkin	6910b1de2e	all: typo fix: the the -> the	2022-12-03 21:53:07 -08:00
Aliaksandr Valialkin	be6da5053f	lib/promscrape: optimize service discovery speed ... - Return meta-labels for the discovered targets via promutils.Labels instead of map[string]string. This improves the speed of generating meta-labels for discovered targets by up to 5x. - Remove memory allocations in hot paths during ScrapeWork generation. The ScrapeWork contains scrape settings for a single discovered target. This improves the service discovery speed by up to 2x.	2022-11-29 21:26:23 -08:00
Aliaksandr Valialkin	4a6d5ab1b1	lib/promrelabel: go fmt after 5cec9706dc	2022-10-29 05:17:49 +03:00
Aliaksandr Valialkin	a72bf87e04	lib/promrelabel: add a test from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3251 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3251	2022-10-29 04:34:08 +03:00
Aliaksandr Valialkin	450a32970a	lib/envtemplate: allow referring env vars from other env vars via %{ENV_VAR} syntax ... This is a follow-up for 02096e06d0	2022-10-26 14:51:02 +03:00
Aliaksandr Valialkin	d0288ea417	all: log error when environment variables referred from -promscrape.config are missing ... This should prevent from using incorrect config files	2022-10-18 10:29:59 +03:00
Aliaksandr Valialkin	c4a3d8b169	lib/promrelabel: add relabeling tests when the source label is missing	2022-10-17 14:48:29 +03:00
Aliaksandr Valialkin	087393bcef	lib/promrelabel: remove unconditional sorting of the labels in ParsedConfigs.Apply(), since the sorting isnt needed in many places ... Sort labels explicitly after calling the ParsedConfigs.Apply() when needed. This reduces CPU usage when performing metric-level relabeling, where labels' sorting isn't needed.	2022-10-09 14:53:35 +03:00
Aliaksandr Valialkin	3987b0abd1	lib/promscrape: allow specifying full target url in __address__ label ... Previously the `__address__` label could contain only `host:port` part of the target url, while the scheme and metrics path were obtained from `__scheme__` and `__metrics_path__` labels. Now it is possible to set the full url in `__address__` label. This makes valid the following scrape config, which is frequently used by novice users: scrape_configs: - job_name: foo static_configs: - targets: - http://host1/metrics1 - https://host2/metrics2	2022-10-07 22:46:29 +03:00
Aliaksandr Valialkin	e5aa34b2e3	lib/promrelabel: export MustParseMetricWithLabels function, which can be used for simplifying tests	2022-10-01 16:15:00 +03:00
Aliaksandr Valialkin	969ae90941	lib/promrelabel: remove redundant memory allocations by using interned strings	2022-10-01 12:07:18 +03:00
Aliaksandr Valialkin	d8d455856c	lib/promrelabel: add a benchmark for realistic Kubernetes relabeling ... The benchmark name is BenchmarkApplyRelabelConfigs/kubernetes This benchmark has been copied from d521933053/model/relabel/relabel_test.go (L505) See also https://github.com/prometheus/prometheus/pull/11147	2022-10-01 12:07:18 +03:00
Aliaksandr Valialkin	39ba55dbb3	lib/promrelabel: go fmt	2022-09-30 12:28:27 +03:00
Aliaksandr Valialkin	9fc2817f41	lib/promrelabel: optimize action: replace for non-trivial regex values ... Cache `action: replace` results for non-trivial regexs and return them next time instead of performing CPU-intensive regex replacement. Optimize also `action: labelmap_all` and `action: replace_all` in the same way.	2022-09-30 12:28:25 +03:00
Aliaksandr Valialkin	f38c9db74d	lib/promrelabel: there is no need in calling regex.HasPrefix() after the optimization at 17289ff481	2022-09-30 12:28:25 +03:00
Aliaksandr Valialkin	fa46c28c5f	lib/promrelabel: optimize action: labelmap for non-trivial regexs	2022-09-30 12:28:25 +03:00
Aliaksandr Valialkin	f1eebc0a99	lib/promrelabel: properly parse regex with escaped $ at the end ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3131 Thanks to @dmitryk-dk for the initial fix at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3179	2022-09-30 08:20:57 +03:00
Aliaksandr Valialkin	4afa25fb38	lib/bytesutil: add FastStringTransformer and use it in the rest of the code where needed	2022-09-28 10:39:42 +03:00
Aliaksandr Valialkin	7f0b95b50a	lib/promrelabel: add SanitizeName() function for sanitizing Prometheus metric names and label names ... Optimize this function by using results cache for input strings. Use this function all over the code. This is a follow-up for fcffdba9dc Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113	2022-09-28 10:02:11 +03:00
Aliaksandr Valialkin	cead9c1e67	lib/promrelabel: call PromRegex.MatchString() on a slow path only if it contains non-empty literal prefix ... This should improve slow path speed for regexps without literal prefixes	2022-08-26 21:48:09 +03:00
Aliaksandr Valialkin	427d69e775	lib/promrelabel: optimize common regex mismatch cases for action: replace and action: labelmap	2022-08-26 15:48:11 +03:00
Aliaksandr Valialkin	da7697fda4	lib/promrelabel: use regexutil.PromRegex for regex matching in actions labeldrop,labelkeep,drop and keep ... This makes possible optimizing additional cases inside regexutil.PromRegex	2022-08-26 15:48:11 +03:00
Aliaksandr Valialkin	e1bd38fa97	lib/promrelabel: optimize matching for commonly used regex patterns in if option ... The following regex patterns are optimized: - literal string match, e.g. "foo" - prefix match, e.g. "foo.*" and "foo.+" - substring match, e.g. ".*foo.*" and ".+foo.+" - alternate values match, e.g. "foo|bar|baz"	2022-08-26 14:55:13 +03:00
Aliaksandr Valialkin	c49751adf8	lib/regexutil: add Simplify() function for simplifying the regular expression	2022-08-26 11:57:43 +03:00
Aliaksandr Valialkin	909e681024	lib/promrelabel: optimize action: {drop,keep,labeldrop,labelkeep} with anchored regex prefix ... The following commonly used relabeling rules must work faster now: - action: labeldrop regex: "^foo.+$" - action: labeldrop regex: "^bar.*"	2022-08-25 23:24:38 +03:00
Aliaksandr Valialkin	d60654eb0a	lib/promrelabel: optimize action: {labeldrop,labelkeep,keep,drop} with regex containing alternate values ... For example, the following relabeling rule must work much faster now: - action: labeldrop regex: "foo|bar|baz"	2022-08-24 17:55:54 +03:00
Aliaksandr Valialkin	1c7f402598	app/vmagent: add ability to construct a label from multiple existing labels by referring them in the replacement field during relabeling ... For example: - target_label: composite-label replacement: {{source_label1}}-{{source_label2}}	2022-08-21 22:49:24 +03:00
Roman Khavronenko	f31132b70b	lib/promrelabel: fix expected test result (#2957) ... follow-up after 68c4ec9472 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-08-08 13:48:14 +03:00
Aliaksandr Valialkin	9039f23bd1	lib/promrelabel: do not split regex into multiple lines if it contains groups ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2928	2022-08-08 03:16:15 +03:00
Aliaksandr Valialkin	979444b4ed	all: fix other typos in the same way as 6f4d9b2a48 does	2022-07-18 12:10:41 +03:00
Aliaksandr Valialkin	fe2269b999	all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package ... This package already has the same name, so there is no need in explicit name	2022-06-21 20:24:28 +03:00
Aliaksandr Valialkin	450aa0ae5a	lib/promrelabel: support action: graphite relabeling ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2737	2022-06-16 20:25:49 +03:00
Aliaksandr Valialkin	cc226e6ebe	docs/CHANGELOG.md: follow-up after 2177089f94	2022-06-01 14:57:39 +03:00
Roman Khavronenko	bca90d7148	promrelabel: add support of lowercase and uppercase relabeling actions (#2665) ... * promrelabel: add support of `lowercase` and `uppercase` relabeling actions https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2664 Signed-off-by: hagen1778 <roman@victoriametrics.com> * lib/storage: make golangci-lint happy Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: Nikolay <nik@victoriametrics.com>	2022-06-01 14:57:39 +03:00
Aliaksandr Valialkin	810dd74fb9	lib/promscrape: properly implement ScrapeConfig.clone() ... Previously ScrapeConfig.clone() was improperly copying promauth.Secret fields - their contents was replaced with `<secret>` value. This led to inability to use passwords and secrets in `-promscrape.config` file. The bug has been introduced in v1.77.0 in the commit 67b10896d2 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2551	2022-05-07 00:06:19 +03:00
Aliaksandr Valialkin	45385a5dc6	lib/promscrape: optimize getScrapeWork() function ... Reduce the number of memory allocations in this function. This improves its performance by up to 50%. This should improve service discovery speed when big number of potential targets with big number of meta-labels are generated by service discovery. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2270	2022-04-20 15:34:18 +03:00
Dmytro Kozlov	17552dba8b	lib/promscrape: Enable filters for endpoint and labels (#2466) ... * lib/promscrape: Enable filters for endpoint and labels * lib/promscrape: cleanup * lib/promscrape: update template * lib/promscrape: move logic filter logic to backend * lib/promscrape: updated placeholder * lib/promscrape: updated placeholder * lib/promscrape: use two different fields for filters, updated form, added error on parsing queries * lib/promscrape: rename functions * lib/promscrape: removed unused values * wip * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-19 18:27:44 +03:00
Aliaksandr Valialkin	2431c9cf81	lib/promrelabel: add support for conditional relabeling via if filter ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998	2022-02-24 02:35:13 +02:00