103 commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	2e3580905f	all: replace old https://docs.victoriametrics.com/relabeling.html url with the new one - https://docs.victoriametrics.com/relabeling/	2024-04-18 03:22:22 +02:00
Aliaksandr Valialkin	c81a633b02	all: replace the outdated url https://docs.victoriametrics.com/vmagent.html with the new one - https://docs.victoriametrics.com/vmagent/	2024-04-18 01:31:37 +02:00
Aliaksandr Valialkin	fb42380ef3	lib/protoparser/opentelemetry: follow-up after 47892b4a4c ... - Rename -opentelemetry.sanitizeMetrics command-line flag to more clear -opentelemetry.usePrometheusNaming - Clarify the description of the change at docs/CHANGELOG.md - Rename promrelabel.SanitizeLabelNameParts to more clear promrelabel.SplitMetricNameToTokens - Properly split metric names at '_' char in promerlabel.SplitMetricNameToTokens. - Add tests for various edge cases for Prometheus metric names' normalization according to the code at b865505850/pkg/translator/prometheus/normalize_name.go - Extract the code responsible for Prometheus metric names' normalization into a separate file (santize.go) Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6037 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6035	2024-04-03 02:25:29 +03:00
Aliaksandr Valialkin	918cccaddf	all: fix golangci-lint(revive) warnings after 0c0ed61ce7 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6001	2024-04-02 23:16:29 +03:00
Andrii Chubatiuk	47892b4a4c	opentelemetry: added cmd flag to sanitize metric names (#6035)	2024-03-29 13:51:24 +01:00
Aliaksandr Valialkin	cf2e80a869	lib/promrelabel: use clear() function inside CleanLabels()	2024-03-01 21:33:15 +02:00
Aliaksandr Valialkin	baaa88001e	lib/promrelabel: store the original labels before returning them them to promutils.PutLabels() ... This should reduce memory allocations. This is a follow-up for b09bd6c42a Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5389	2024-02-14 16:09:10 +02:00
Aliaksandr Valialkin	b09bd6c42a	lib/promrelabel: factor out applyInternal code into ApplyDebug and Apply functions ... This improves readability and maintanability Also remove memory allocation from SortLabels()	2024-02-14 14:27:08 +02:00
Aliaksandr Valialkin	95222b2079	all: upgrade Go builder from Go1.21.7 to Go1.22.0 ... See https://go.dev/doc/go1.22	2024-02-12 21:59:51 +02:00
Aliaksandr Valialkin	1f105dde98	all: allow dynamically reading *AuthKey flag values from files and urls ... Examples: 1) -metricsAuthKey=file:///abs/path/to/file - reads flag value from the given absolute filepath 2) -metricsAuthKey=file://./relative/path/to/file - reads flag value from the given relative filepath 3) -metricsAuthKey=http://some-host/some/path?query_arg=abc - reads flag value from the given url The flag value is automatically updated when the file contents changes.	2024-01-21 22:03:38 +02:00
Aliaksandr Valialkin	12f2c5679b	all: consistently clear prompbmarshal.Label by assigning an empty struct instead of zeroing Name and Value individually	2024-01-21 05:11:05 +02:00
Aliaksandr Valialkin	ac65c6b178	lib/promrelabel: add keep_if_contains and drop_if_contains relabeling actions	2023-11-29 12:22:43 +02:00
Nikolay	41f7940f97	lib/streamaggr: properly reference slice with labels (#5406) ... * lib/streamaggr: properly reference slice with labels by limiting slice capacity. It must fix issues with slice modification, in case of append new slice will be allocated, instead of modifying refrenced slice https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5402 * Reduce memory allocations when output_relabel_configs adds new labels to output samples --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-11-29 10:03:04 +02:00
Aliaksandr Valialkin	42dd71bb63	all: consistently use %w instead of %s in when error is passed to fmt.Errorf() ... This allows consistently using errors.Is() for verifying whether the given error wraps some other known error.	2023-10-25 21:24:03 +02:00
Aliaksandr Valialkin	fdae53a75b	lib/promrelabel: properly replace : char with _ in metric names when -usePromCompatibleNaming command-line flag is set ... This addresses https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113#issuecomment-1275077071 comment from @johnseekins	2023-08-14 16:14:42 +02:00
Aliaksandr Valialkin	63e3571e8c	lib/promrelabel: stop emitting DEBUG log lines when parsing if expressions ... These lines were accidentally left in the commit 62651570bb Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635	2023-08-14 15:24:31 +02:00
Zakhar Bessarab	1bd7637fe1	lib/promrelabel: fix relabeling if clause (#4816) ... * lib/promrelabel: fix relabeling if clause being applied to labels outside of current context Relabeling is applied to each metric row separately, but in order to lower amount of memory allocations it is reusing labels. Functions which are working on current metric row labels are supposed to use only current metric labels by using provided offset, but if clause matcher was using the whole labels set instead of local metrics. This leaded to invalid relabeling results such as one described here: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4806 Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> * docs/CHANGELOG.md: document the bugfix Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4806 --------- Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-08-11 06:37:48 -07:00
Aliaksandr Valialkin	ad08d9c884	lib/promrelabel: return correct string representation for IfExpression containing a single selector ... This is a follow-up for 62651570bb Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635	2023-07-24 19:33:14 -07:00
Aliaksandr Valialkin	62651570bb	lib/promrelabel: add support for a list of series selectors at IfExpression ... This makes possible specifying a list of series selectors at the following places: - Inside `if` option at relabeling rules - Inside `match` option at stream aggregation rules Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635	2023-07-24 17:08:52 -07:00
Aliaksandr Valialkin	4cb024d8a3	all: add support for or filters in series selectors ... This commit adds ability to select series matching distinct filters via a single series selector. For example, the following selector selects series with either {env="prod",job="a"} or {env="dev",job="b"} labels: {env="prod",job="a" or env="dev",job="b"} The `or` filter is supported in all the VictoriaMetrics tools now. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3997 Uses https://github.com/VictoriaMetrics/metricsql/pull/14	2023-07-16 00:06:33 -07:00
Aliaksandr Valialkin	1f2f74e70e	lib/promrelabel: use monospace font at textarea for writing relabel configs on /metric-relabel-debug and /target-relabel-debug pages ... This simplifies visual inspection of indentation in yaml configs	2023-05-18 20:48:41 -07:00
Aliaksandr Valialkin	318a87c36f	Revert "lib/promrelabel: show error message if labels not in prometheus exposition format (#4304)" ... This reverts commit 193a9c3328. Reason for revert: the commit doesn't fix the real issue with promutils.NewLabelsFromString() function, which must return error when improperly formatted Prometheus metric with labels is passed to it. See https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#text-format-example E.g. the promutils.NewLabelsFromString() must return error when the following strings are passed to it: - `{foo:"bar"}`, since `:` is disallowed in Prometheus text exposition format. The corect value is `{foo="bar"}` - `{"foo":"bar"}`, since label name shouldn't be quoted. The correct value is `{foo="bar"}`. The reverted commit introduces another set of bugs, which happily accept the following invalid input: - `{foo=~"bar"}` - `{foo!="bar"}` - `{foo!~"bar"}` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4284 See also https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4304	2023-05-12 16:07:37 -07:00
Dmytro Kozlov	193a9c3328	lib/promrelabel: show error message if labels not in prometheus exposition format (#4304) ... lib/promrelabel: show error message if labels not in prometheus exposition format https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4284	2023-05-12 10:42:56 +02:00
Aliaksandr Valialkin	ec3943d14a	app/vmselect: small cleanup after 4f3f9950d0 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3807	2023-05-08 14:57:11 -07:00
Yury Molodov	4f3f9950d0	vmui: add metric relabel debug (#3889) ... * feat: add metric relabel debug (#3807) * fix: add link to relabeling cookbook * lib/promrelabel: merge, fix conflicts * lib/promrelabel: fix diff * docs/vmui: add metric relabel playground --------- Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>	2023-04-26 11:53:29 +03:00
Dmytro Kozlov	e79cd24807	lib/promrelabel: make target url from labels on target relabel page (#3882) ... * lib/promrelabel: make target url from labels on target relabel page * wip --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-03-20 22:07:52 -07:00
Oleksandr Redko	9fff48c3e3	app,lib: fix typos in comments (#3804)	2023-02-13 13:27:13 +01:00
Aliaksandr Valialkin	1a28f0e5b3	lib/promrelabel: pass query args via query string at /metric-relabel-debug and /target-relabel-debug pages if their length doesnt exceed 1000 ... This allows copy-n-pasting the url to another browser window and seeing the same result. The limit in 1000 chars is selected in order to prevent from potential issues with systems which limit the url length such as Internet Explorer - see https://stackoverflow.com/questions/812925/what-is-the-maximum-possible-length-of-a-query-string If the limit is exceeded, then query args are sent via POST method and aren't visible in the url. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3580	2023-01-05 16:48:04 -08:00
Aliaksandr Valialkin	0dea3b71da	lib/promscrape: pre-fetch metric_relabel_configs rules when debugging metric relabeling for a particular target ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407	2023-01-05 03:26:49 -08:00
Aliaksandr Valialkin	5bca3a5be2	app/vmselect: remove dependency on lib/promscrape from app/vmselect	2023-01-03 23:28:27 -08:00
Aliaksandr Valialkin	add2c4bf07	lib/bytesutil: add InternBytes() function as a shortcut to InternString(ToUnsafeString(..))	2023-01-03 22:16:22 -08:00
Aliaksandr Valialkin	7b264b0c23	lib/promrelabel: allow calling Match on nil IfExpression ... This simplifies the caller side of IfExpression	2023-01-03 21:44:03 -08:00
Aliaksandr Valialkin	fa236c5a84	lib/promrelabel: make fmt after d3de110070	2022-12-21 20:24:57 -08:00
Aliaksandr Valialkin	31886aef3d	lib/promrelabel: add support for keepequal and dropequal relabeling actions ... These actions are supported by Prometheus starting from v2.41.0 See https://github.com/prometheus/prometheus/pull/11564 , https://github.com/prometheus/prometheus/issues/11556 and https://github.com/prometheus/prometheus/issues/3756 Side note: It's a pity that Prometheus developers decided inventing `keepequal` and `dropequal` relabeling actions instead of adding support for `keep_if_equal` and `drop_if_equal` relabeling actions supported by VictoriaMetrics since June 2020 - see 2a39ba639d .	2022-12-21 20:04:55 -08:00
Aliaksandr Valialkin	a8b8e23d68	lib/promscrape: implement target-level and metric-level relabel debugging ... Target-level debugging is performed by clicking the 'debug' link at the corresponding target on either http://vmagent:8429/targets page or on http://vmagent:8428/service-discovery page. Metric-level debugging is perfromed at http://vmagent:8429/metric-relabel-debug page. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407 See https://docs.victoriametrics.com/vmagent.html#relabel-debug	2022-12-10 02:09:44 -08:00
Aliaksandr Valialkin	14660d4df5	all: typo fix: the the -> the	2022-12-03 21:53:01 -08:00
Aliaksandr Valialkin	f325410c26	lib/promscrape: optimize service discovery speed ... - Return meta-labels for the discovered targets via promutils.Labels instead of map[string]string. This improves the speed of generating meta-labels for discovered targets by up to 5x. - Remove memory allocations in hot paths during ScrapeWork generation. The ScrapeWork contains scrape settings for a single discovered target. This improves the service discovery speed by up to 2x.	2022-11-29 21:26:00 -08:00
Aliaksandr Valialkin	869e0f9f85	lib/promrelabel: go fmt after 5cec9706dc	2022-10-29 05:17:10 +03:00
Aliaksandr Valialkin	5cec9706dc	lib/promrelabel: add a test from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3251 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3251	2022-10-29 04:33:38 +03:00
Aliaksandr Valialkin	518c340ae3	lib/envtemplate: allow referring env vars from other env vars via %{ENV_VAR} syntax ... This is a follow-up for 02096e06d0	2022-10-26 14:49:33 +03:00
Aliaksandr Valialkin	069401a304	all: log error when environment variables referred from -promscrape.config are missing ... This should prevent from using incorrect config files	2022-10-18 10:47:16 +03:00
Aliaksandr Valialkin	b6e8c1403a	lib/promrelabel: add relabeling tests when the source label is missing	2022-10-17 14:47:52 +03:00
Aliaksandr Valialkin	50f5eae0e0	lib/promrelabel: remove unconditional sorting of the labels in ParsedConfigs.Apply(), since the sorting isnt needed in many places ... Sort labels explicitly after calling the ParsedConfigs.Apply() when needed. This reduces CPU usage when performing metric-level relabeling, where labels' sorting isn't needed.	2022-10-09 14:51:16 +03:00
Aliaksandr Valialkin	f9df0cae16	lib/promscrape: allow specifying full target url in __address__ label ... Previously the `__address__` label could contain only `host:port` part of the target url, while the scheme and metrics path were obtained from `__scheme__` and `__metrics_path__` labels. Now it is possible to set the full url in `__address__` label. This makes valid the following scrape config, which is frequently used by novice users: scrape_configs: - job_name: foo static_configs: - targets: - http://host1/metrics1 - https://host2/metrics2	2022-10-07 22:43:04 +03:00
Aliaksandr Valialkin	3a98ef2f5f	lib/promrelabel: export MustParseMetricWithLabels function, which can be used for simplifying tests	2022-10-01 16:05:51 +03:00
Aliaksandr Valialkin	e8a64f6e7a	lib/promrelabel: remove redundant memory allocations by using interned strings	2022-10-01 11:50:21 +03:00
Aliaksandr Valialkin	73dc17ef64	lib/promrelabel: add a benchmark for realistic Kubernetes relabeling ... The benchmark name is BenchmarkApplyRelabelConfigs/kubernetes This benchmark has been copied from d521933053/model/relabel/relabel_test.go (L505) See also https://github.com/prometheus/prometheus/pull/11147	2022-10-01 10:38:22 +03:00
Aliaksandr Valialkin	171dd14aa3	lib/promrelabel: go fmt	2022-09-30 12:28:55 +03:00
Aliaksandr Valialkin	a18d6d5ccc	lib/promrelabel: optimize action: replace for non-trivial regex values ... Cache `action: replace` results for non-trivial regexs and return them next time instead of performing CPU-intensive regex replacement. Optimize also `action: labelmap_all` and `action: replace_all` in the same way.	2022-09-30 12:25:05 +03:00
Aliaksandr Valialkin	146021a076	lib/promrelabel: there is no need in calling regex.HasPrefix() after the optimization at 17289ff481	2022-09-30 10:49:18 +03:00