104 commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	7a2a2f173e	lib/logstorage: work-in-progress	2024-05-24 03:07:07 +02:00
Aliaksandr Valialkin	0c87cbe338	all: replace old https://docs.victoriametrics.com/relabeling.html url with the new one - https://docs.victoriametrics.com/relabeling/	2024-04-18 03:23:15 +02:00
Aliaksandr Valialkin	0211a04a52	all: replace the outdated url https://docs.victoriametrics.com/vmagent.html with the new one - https://docs.victoriametrics.com/vmagent/	2024-04-18 01:32:57 +02:00
Aliaksandr Valialkin	bb9bb600b3	lib/protoparser/opentelemetry: follow-up after 47892b4a4c ... - Rename -opentelemetry.sanitizeMetrics command-line flag to more clear -opentelemetry.usePrometheusNaming - Clarify the description of the change at docs/CHANGELOG.md - Rename promrelabel.SanitizeLabelNameParts to more clear promrelabel.SplitMetricNameToTokens - Properly split metric names at '_' char in promerlabel.SplitMetricNameToTokens. - Add tests for various edge cases for Prometheus metric names' normalization according to the code at b865505850/pkg/translator/prometheus/normalize_name.go - Extract the code responsible for Prometheus metric names' normalization into a separate file (santize.go) Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6037 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6035	2024-04-03 03:09:52 +03:00
Aliaksandr Valialkin	00f59d6ddf	all: fix golangci-lint(revive) warnings after 0c0ed61ce7 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6001	2024-04-03 03:00:45 +03:00
Andrii Chubatiuk	0799834aaa	opentelemetry: added cmd flag to sanitize metric names (#6035)	2024-04-03 02:31:39 +03:00
Aliaksandr Valialkin	7533070a52	lib/promrelabel: use clear() function inside CleanLabels()	2024-03-01 21:34:47 +02:00
Aliaksandr Valialkin	06da06dac0	lib/promrelabel: store the original labels before returning them them to promutils.PutLabels() ... This should reduce memory allocations. This is a follow-up for b09bd6c42a Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5389	2024-02-14 16:09:38 +02:00
Aliaksandr Valialkin	990a46c478	lib/promrelabel: factor out applyInternal code into ApplyDebug and Apply functions ... This improves readability and maintanability Also remove memory allocation from SortLabels()	2024-02-14 14:27:44 +02:00
Aliaksandr Valialkin	f5680a6857	all: upgrade Go builder from Go1.21.7 to Go1.22.0 ... See https://go.dev/doc/go1.22	2024-02-12 22:14:00 +02:00
Aliaksandr Valialkin	885ee160c2	all: allow dynamically reading *AuthKey flag values from files and urls ... Examples: 1) -metricsAuthKey=file:///abs/path/to/file - reads flag value from the given absolute filepath 2) -metricsAuthKey=file://./relative/path/to/file - reads flag value from the given relative filepath 3) -metricsAuthKey=http://some-host/some/path?query_arg=abc - reads flag value from the given url The flag value is automatically updated when the file contents changes.	2024-01-22 01:23:23 +02:00
Aliaksandr Valialkin	e15f07d989	all: consistently clear prompbmarshal.Label by assigning an empty struct instead of zeroing Name and Value individually	2024-01-22 01:11:59 +02:00
Aliaksandr Valialkin	f0215afee3	lib/promrelabel: add keep_if_contains and drop_if_contains relabeling actions ... (cherry picked from commit ac65c6b178)	2023-12-01 14:00:20 +01:00
Nikolay	9505d48070	lib/streamaggr: properly reference slice with labels (#5406) ... * lib/streamaggr: properly reference slice with labels by limiting slice capacity. It must fix issues with slice modification, in case of append new slice will be allocated, instead of modifying refrenced slice https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5402 * Reduce memory allocations when output_relabel_configs adds new labels to output samples --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com> (cherry picked from commit 41f7940f97)	2023-12-01 14:00:18 +01:00
Aliaksandr Valialkin	36a1fdca6c	all: consistently use %w instead of %s in when error is passed to fmt.Errorf() ... This allows consistently using errors.Is() for verifying whether the given error wraps some other known error.	2023-10-26 09:44:40 +02:00
Aliaksandr Valialkin	4b1f01e45d	lib/promrelabel: properly replace : char with _ in metric names when -usePromCompatibleNaming command-line flag is set ... This addresses https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113#issuecomment-1275077071 comment from @johnseekins	2023-08-14 16:18:17 +02:00
Aliaksandr Valialkin	e8fe00d39e	lib/promrelabel: stop emitting DEBUG log lines when parsing if expressions ... These lines were accidentally left in the commit 62651570bb Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635	2023-08-14 16:18:17 +02:00
Zakhar Bessarab	15b1810dc8	lib/promrelabel: fix relabeling if clause (#4816) ... * lib/promrelabel: fix relabeling if clause being applied to labels outside of current context Relabeling is applied to each metric row separately, but in order to lower amount of memory allocations it is reusing labels. Functions which are working on current metric row labels are supposed to use only current metric labels by using provided offset, but if clause matcher was using the whole labels set instead of local metrics. This leaded to invalid relabeling results such as one described here: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4806 Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> * docs/CHANGELOG.md: document the bugfix Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4806 --------- Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-08-11 06:44:46 -07:00
Aliaksandr Valialkin	3179d0528b	lib/promrelabel: return correct string representation for IfExpression containing a single selector ... This is a follow-up for 62651570bb Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635	2023-07-24 19:33:04 -07:00
Aliaksandr Valialkin	6e43664e24	lib/promrelabel: add support for a list of series selectors at IfExpression ... This makes possible specifying a list of series selectors at the following places: - Inside `if` option at relabeling rules - Inside `match` option at stream aggregation rules Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635	2023-07-24 17:09:59 -07:00
Aliaksandr Valialkin	a7fdc3fcc7	all: add support for or filters in series selectors ... This commit adds ability to select series matching distinct filters via a single series selector. For example, the following selector selects series with either {env="prod",job="a"} or {env="dev",job="b"} labels: {env="prod",job="a" or env="dev",job="b"} The `or` filter is supported in all the VictoriaMetrics tools now. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3997 Uses https://github.com/VictoriaMetrics/metricsql/pull/14	2023-07-15 23:56:18 -07:00
Aliaksandr Valialkin	c30f0e51d7	lib/promrelabel: use monospace font at textarea for writing relabel configs on /metric-relabel-debug and /target-relabel-debug pages ... This simplifies visual inspection of indentation in yaml configs	2023-05-18 20:49:47 -07:00
Aliaksandr Valialkin	b24da0f901	Revert "lib/promrelabel: show error message if labels not in prometheus exposition format (#4304)" ... This reverts commit 193a9c3328. Reason for revert: the commit doesn't fix the real issue with promutils.NewLabelsFromString() function, which must return error when improperly formatted Prometheus metric with labels is passed to it. See https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#text-format-example E.g. the promutils.NewLabelsFromString() must return error when the following strings are passed to it: - `{foo:"bar"}`, since `:` is disallowed in Prometheus text exposition format. The corect value is `{foo="bar"}` - `{"foo":"bar"}`, since label name shouldn't be quoted. The correct value is `{foo="bar"}`. The reverted commit introduces another set of bugs, which happily accept the following invalid input: - `{foo=~"bar"}` - `{foo!="bar"}` - `{foo!~"bar"}` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4284 See also https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4304	2023-05-12 17:01:23 -07:00
Dmytro Kozlov	24386f68db	lib/promrelabel: show error message if labels not in prometheus exposition format (#4304) ... lib/promrelabel: show error message if labels not in prometheus exposition format https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4284	2023-05-12 16:53:59 -07:00
Aliaksandr Valialkin	8703b2fa87	app/vmselect: small cleanup after 4f3f9950d0 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3807	2023-05-09 22:45:02 -07:00
Yury Molodov	ddc5197bce	vmui: add metric relabel debug (#3889) ... * feat: add metric relabel debug (#3807) * fix: add link to relabeling cookbook * lib/promrelabel: merge, fix conflicts * lib/promrelabel: fix diff * docs/vmui: add metric relabel playground --------- Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>	2023-05-08 14:59:35 -07:00
Dmytro Kozlov	85b01c4aa7	lib/promrelabel: make target url from labels on target relabel page (#3882) ... * lib/promrelabel: make target url from labels on target relabel page * wip --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-03-20 22:08:39 -07:00
Oleksandr Redko	0e1c395609	app,lib: fix typos in comments (#3804)	2023-02-13 09:32:35 -08:00
Aliaksandr Valialkin	1af6e0b233	lib/promrelabel: pass query args via query string at /metric-relabel-debug and /target-relabel-debug pages if their length doesnt exceed 1000 ... This allows copy-n-pasting the url to another browser window and seeing the same result. The limit in 1000 chars is selected in order to prevent from potential issues with systems which limit the url length such as Internet Explorer - see https://stackoverflow.com/questions/812925/what-is-the-maximum-possible-length-of-a-query-string If the limit is exceeded, then query args are sent via POST method and aren't visible in the url. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3580	2023-01-05 16:45:42 -08:00
Aliaksandr Valialkin	1d16cc9349	lib/promscrape: pre-fetch metric_relabel_configs rules when debugging metric relabeling for a particular target ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407	2023-01-05 03:28:14 -08:00
Aliaksandr Valialkin	ebb8aeb0cf	app/vmselect: remove dependency on lib/promscrape from app/vmselect	2023-01-03 23:27:36 -08:00
Aliaksandr Valialkin	d3f8298739	lib/bytesutil: add InternBytes() function as a shortcut to InternString(ToUnsafeString(..))	2023-01-03 22:15:49 -08:00
Aliaksandr Valialkin	a7942c6c0d	lib/promrelabel: allow calling Match on nil IfExpression ... This simplifies the caller side of IfExpression	2022-12-30 16:47:59 -08:00
Aliaksandr Valialkin	7888712185	lib/promrelabel: make fmt after d3de110070	2022-12-21 20:25:37 -08:00
Aliaksandr Valialkin	cc482b89a3	lib/promrelabel: add support for keepequal and dropequal relabeling actions ... These actions are supported by Prometheus starting from v2.41.0 See https://github.com/prometheus/prometheus/pull/11564 , https://github.com/prometheus/prometheus/issues/11556 and https://github.com/prometheus/prometheus/issues/3756 Side note: It's a pity that Prometheus developers decided inventing `keepequal` and `dropequal` relabeling actions instead of adding support for `keep_if_equal` and `drop_if_equal` relabeling actions supported by VictoriaMetrics since June 2020 - see 2a39ba639d .	2022-12-21 20:06:09 -08:00
Aliaksandr Valialkin	97b41e727c	lib/promscrape: implement target-level and metric-level relabel debugging ... Target-level debugging is performed by clicking the 'debug' link at the corresponding target on either http://vmagent:8429/targets page or on http://vmagent:8428/service-discovery page. Metric-level debugging is perfromed at http://vmagent:8429/metric-relabel-debug page. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407 See https://docs.victoriametrics.com/vmagent.html#relabel-debug	2022-12-10 02:25:56 -08:00
Aliaksandr Valialkin	6910b1de2e	all: typo fix: the the -> the	2022-12-03 21:53:07 -08:00
Aliaksandr Valialkin	be6da5053f	lib/promscrape: optimize service discovery speed ... - Return meta-labels for the discovered targets via promutils.Labels instead of map[string]string. This improves the speed of generating meta-labels for discovered targets by up to 5x. - Remove memory allocations in hot paths during ScrapeWork generation. The ScrapeWork contains scrape settings for a single discovered target. This improves the service discovery speed by up to 2x.	2022-11-29 21:26:23 -08:00
Aliaksandr Valialkin	4a6d5ab1b1	lib/promrelabel: go fmt after 5cec9706dc	2022-10-29 05:17:49 +03:00
Aliaksandr Valialkin	a72bf87e04	lib/promrelabel: add a test from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3251 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3251	2022-10-29 04:34:08 +03:00
Aliaksandr Valialkin	450a32970a	lib/envtemplate: allow referring env vars from other env vars via %{ENV_VAR} syntax ... This is a follow-up for 02096e06d0	2022-10-26 14:51:02 +03:00
Aliaksandr Valialkin	d0288ea417	all: log error when environment variables referred from -promscrape.config are missing ... This should prevent from using incorrect config files	2022-10-18 10:29:59 +03:00
Aliaksandr Valialkin	c4a3d8b169	lib/promrelabel: add relabeling tests when the source label is missing	2022-10-17 14:48:29 +03:00
Aliaksandr Valialkin	087393bcef	lib/promrelabel: remove unconditional sorting of the labels in ParsedConfigs.Apply(), since the sorting isnt needed in many places ... Sort labels explicitly after calling the ParsedConfigs.Apply() when needed. This reduces CPU usage when performing metric-level relabeling, where labels' sorting isn't needed.	2022-10-09 14:53:35 +03:00
Aliaksandr Valialkin	3987b0abd1	lib/promscrape: allow specifying full target url in __address__ label ... Previously the `__address__` label could contain only `host:port` part of the target url, while the scheme and metrics path were obtained from `__scheme__` and `__metrics_path__` labels. Now it is possible to set the full url in `__address__` label. This makes valid the following scrape config, which is frequently used by novice users: scrape_configs: - job_name: foo static_configs: - targets: - http://host1/metrics1 - https://host2/metrics2	2022-10-07 22:46:29 +03:00
Aliaksandr Valialkin	e5aa34b2e3	lib/promrelabel: export MustParseMetricWithLabels function, which can be used for simplifying tests	2022-10-01 16:15:00 +03:00
Aliaksandr Valialkin	969ae90941	lib/promrelabel: remove redundant memory allocations by using interned strings	2022-10-01 12:07:18 +03:00
Aliaksandr Valialkin	d8d455856c	lib/promrelabel: add a benchmark for realistic Kubernetes relabeling ... The benchmark name is BenchmarkApplyRelabelConfigs/kubernetes This benchmark has been copied from d521933053/model/relabel/relabel_test.go (L505) See also https://github.com/prometheus/prometheus/pull/11147	2022-10-01 12:07:18 +03:00
Aliaksandr Valialkin	39ba55dbb3	lib/promrelabel: go fmt	2022-09-30 12:28:27 +03:00
Aliaksandr Valialkin	9fc2817f41	lib/promrelabel: optimize action: replace for non-trivial regex values ... Cache `action: replace` results for non-trivial regexs and return them next time instead of performing CPU-intensive regex replacement. Optimize also `action: labelmap_all` and `action: replace_all` in the same way.	2022-09-30 12:28:25 +03:00