Aliaksandr Valialkin
27f790458b
lib/promrelabel: properly replace : char with _ in metric names when -usePromCompatibleNaming command-line flag is set
...
This addresses https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113#issuecomment-1275077071 comment from @johnseekins
2023-08-17 13:54:59 +02:00
Zakhar Bessarab
26cb1e00bf
lib/promrelabel: fix relabeling if clause ( #4816 )
...
* lib/promrelabel: fix relabeling if clause being applied to labels outside of current context
Relabeling is applied to each metric row separately, but in order to lower amount of memory allocations it is reusing labels.
Functions which are working on current metric row labels are supposed to use only current metric labels by using provided offset, but if clause matcher was using the whole labels set instead of local metrics.
This leaded to invalid relabeling results such as one described here: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4806
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
* docs/CHANGELOG.md: document the bugfix
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4806
---------
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-08-12 12:31:48 -07:00
Aliaksandr Valialkin
1af6e0b233
lib/promrelabel: pass query args via query string at /metric-relabel-debug and /target-relabel-debug pages if their length doesnt exceed 1000
...
This allows copy-n-pasting the url to another browser window and seeing the same result.
The limit in 1000 chars is selected in order to prevent from potential issues with systems
which limit the url length such as Internet Explorer - see https://stackoverflow.com/questions/812925/what-is-the-maximum-possible-length-of-a-query-string
If the limit is exceeded, then query args are sent via POST method and aren't visible in the url.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3580
2023-01-05 16:45:42 -08:00
Aliaksandr Valialkin
1d16cc9349
lib/promscrape: pre-fetch metric_relabel_configs rules when debugging metric relabeling for a particular target
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407
2023-01-05 03:28:14 -08:00
Aliaksandr Valialkin
ebb8aeb0cf
app/vmselect: remove dependency on lib/promscrape from app/vmselect
2023-01-03 23:27:36 -08:00
Aliaksandr Valialkin
d3f8298739
lib/bytesutil: add InternBytes() function as a shortcut to InternString(ToUnsafeString(..))
2023-01-03 22:15:49 -08:00
Aliaksandr Valialkin
a7942c6c0d
lib/promrelabel: allow calling Match on nil IfExpression
...
This simplifies the caller side of IfExpression
2022-12-30 16:47:59 -08:00
Aliaksandr Valialkin
7888712185
lib/promrelabel: make fmt after d3de110070
2022-12-21 20:25:37 -08:00
Aliaksandr Valialkin
cc482b89a3
lib/promrelabel: add support for keepequal and dropequal relabeling actions
...
These actions are supported by Prometheus starting from v2.41.0
See https://github.com/prometheus/prometheus/pull/11564 ,
https://github.com/prometheus/prometheus/issues/11556
and https://github.com/prometheus/prometheus/issues/3756
Side note:
It's a pity that Prometheus developers decided inventing `keepequal` and `dropequal`
relabeling actions instead of adding support for `keep_if_equal` and `drop_if_equal` relabeling
actions supported by VictoriaMetrics since June 2020 - see 2a39ba639d
2022-12-21 20:06:09 -08:00
Aliaksandr Valialkin
97b41e727c
lib/promscrape: implement target-level and metric-level relabel debugging
...
Target-level debugging is performed by clicking the 'debug' link at the corresponding target
on either http://vmagent:8429/targets page or on http://vmagent:8428/service-discovery page.
Metric-level debugging is perfromed at http://vmagent:8429/metric-relabel-debug page.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407
See https://docs.victoriametrics.com/vmagent.html#relabel-debug
2022-12-10 02:25:56 -08:00
Aliaksandr Valialkin
6910b1de2e
all: typo fix: the the -> the
2022-12-03 21:53:07 -08:00
Aliaksandr Valialkin
be6da5053f
lib/promscrape: optimize service discovery speed
...
- Return meta-labels for the discovered targets via promutils.Labels
instead of map[string]string. This improves the speed of generating
meta-labels for discovered targets by up to 5x.
- Remove memory allocations in hot paths during ScrapeWork generation.
The ScrapeWork contains scrape settings for a single discovered target.
This improves the service discovery speed by up to 2x.
2022-11-29 21:26:23 -08:00
Aliaksandr Valialkin
4a6d5ab1b1
lib/promrelabel: go fmt after 5cec9706dc
2022-10-29 05:17:49 +03:00
Aliaksandr Valialkin
a72bf87e04
lib/promrelabel: add a test from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3251
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3251
2022-10-29 04:34:08 +03:00
Aliaksandr Valialkin
450a32970a
lib/envtemplate: allow referring env vars from other env vars via %{ENV_VAR} syntax
...
This is a follow-up for 02096e06d0
2022-10-26 14:51:02 +03:00
Aliaksandr Valialkin
d0288ea417
all: log error when environment variables referred from -promscrape.config are missing
...
This should prevent from using incorrect config files
2022-10-18 10:29:59 +03:00
Aliaksandr Valialkin
c4a3d8b169
lib/promrelabel: add relabeling tests when the source label is missing
2022-10-17 14:48:29 +03:00
Aliaksandr Valialkin
087393bcef
lib/promrelabel: remove unconditional sorting of the labels in ParsedConfigs.Apply(), since the sorting isnt needed in many places
...
Sort labels explicitly after calling the ParsedConfigs.Apply() when needed.
This reduces CPU usage when performing metric-level relabeling, where labels' sorting isn't needed.
2022-10-09 14:53:35 +03:00
Aliaksandr Valialkin
3987b0abd1
lib/promscrape: allow specifying full target url in __address__ label
...
Previously the `__address__` label could contain only `host:port` part of the target url,
while the scheme and metrics path were obtained from `__scheme__` and `__metrics_path__`
labels. Now it is possible to set the full url in `__address__` label.
This makes valid the following scrape config, which is frequently used by novice users:
scrape_configs:
- job_name: foo
static_configs:
- targets:
- http://host1/metrics1
- https://host2/metrics2
2022-10-07 22:46:29 +03:00
Aliaksandr Valialkin
e5aa34b2e3
lib/promrelabel: export MustParseMetricWithLabels function, which can be used for simplifying tests
2022-10-01 16:15:00 +03:00
Aliaksandr Valialkin
969ae90941
lib/promrelabel: remove redundant memory allocations by using interned strings
2022-10-01 12:07:18 +03:00
Aliaksandr Valialkin
d8d455856c
lib/promrelabel: add a benchmark for realistic Kubernetes relabeling
...
The benchmark name is BenchmarkApplyRelabelConfigs/kubernetes
This benchmark has been copied from d521933053/model/relabel/relabel_test.go (L505)https://github.com/prometheus/prometheus/pull/11147
2022-10-01 12:07:18 +03:00
Aliaksandr Valialkin
39ba55dbb3
lib/promrelabel: go fmt
2022-09-30 12:28:27 +03:00
Aliaksandr Valialkin
9fc2817f41
lib/promrelabel: optimize action: replace for non-trivial regex values
...
Cache `action: replace` results for non-trivial regexs and return them next time
instead of performing CPU-intensive regex replacement.
Optimize also `action: labelmap_all` and `action: replace_all` in the same way.
2022-09-30 12:28:25 +03:00
Aliaksandr Valialkin
f38c9db74d
lib/promrelabel: there is no need in calling regex.HasPrefix() after the optimization at 17289ff481
2022-09-30 12:28:25 +03:00
Aliaksandr Valialkin
fa46c28c5f
lib/promrelabel: optimize action: labelmap for non-trivial regexs
2022-09-30 12:28:25 +03:00
Aliaksandr Valialkin
f1eebc0a99
lib/promrelabel: properly parse regex with escaped $ at the end
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3131
Thanks to @dmitryk-dk for the initial fix at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3179
2022-09-30 08:20:57 +03:00
Aliaksandr Valialkin
4afa25fb38
lib/bytesutil: add FastStringTransformer and use it in the rest of the code where needed
2022-09-28 10:39:42 +03:00
Aliaksandr Valialkin
7f0b95b50a
lib/promrelabel: add SanitizeName() function for sanitizing Prometheus metric names and label names
...
Optimize this function by using results cache for input strings.
Use this function all over the code.
This is a follow-up for fcffdba9dchttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113
2022-09-28 10:02:11 +03:00
Aliaksandr Valialkin
cead9c1e67
lib/promrelabel: call PromRegex.MatchString() on a slow path only if it contains non-empty literal prefix
...
This should improve slow path speed for regexps without literal prefixes
2022-08-26 21:48:09 +03:00
Aliaksandr Valialkin
427d69e775
lib/promrelabel: optimize common regex mismatch cases for action: replace and action: labelmap
2022-08-26 15:48:11 +03:00
Aliaksandr Valialkin
da7697fda4
lib/promrelabel: use regexutil.PromRegex for regex matching in actions labeldrop,labelkeep,drop and keep
...
This makes possible optimizing additional cases inside regexutil.PromRegex
2022-08-26 15:48:11 +03:00
Aliaksandr Valialkin
e1bd38fa97
lib/promrelabel: optimize matching for commonly used regex patterns in if option
...
The following regex patterns are optimized:
- literal string match, e.g. "foo"
- prefix match, e.g. "foo.*" and "foo.+"
- substring match, e.g. ".*foo.*" and ".+foo.+"
- alternate values match, e.g. "foo|bar|baz"
2022-08-26 14:55:13 +03:00
Aliaksandr Valialkin
c49751adf8
lib/regexutil: add Simplify() function for simplifying the regular expression
2022-08-26 11:57:43 +03:00
Aliaksandr Valialkin
909e681024
lib/promrelabel: optimize action: {drop,keep,labeldrop,labelkeep} with anchored regex prefix
...
The following commonly used relabeling rules must work faster now:
- action: labeldrop
regex: "^foo.+$"
- action: labeldrop
regex: "^bar.*"
2022-08-25 23:24:38 +03:00
Aliaksandr Valialkin
d60654eb0a
lib/promrelabel: optimize action: {labeldrop,labelkeep,keep,drop} with regex containing alternate values
...
For example, the following relabeling rule must work much faster now:
- action: labeldrop
regex: "foo|bar|baz"
2022-08-24 17:55:54 +03:00
Aliaksandr Valialkin
1c7f402598
app/vmagent: add ability to construct a label from multiple existing labels by referring them in the replacement field during relabeling
...
For example:
- target_label: composite-label
replacement: {{source_label1}}-{{source_label2}}
2022-08-21 22:49:24 +03:00
Roman Khavronenko
f31132b70b
lib/promrelabel: fix expected test result ( #2957 )
...
follow-up after 68c4ec9472
2022-08-08 13:48:14 +03:00
Aliaksandr Valialkin
9039f23bd1
lib/promrelabel: do not split regex into multiple lines if it contains groups
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2928
2022-08-08 03:16:15 +03:00
Aliaksandr Valialkin
979444b4ed
all: fix other typos in the same way as 6f4d9b2a48 does
2022-07-18 12:10:41 +03:00
Aliaksandr Valialkin
fe2269b999
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package
...
This package already has the same name, so there is no need in explicit name
2022-06-21 20:24:28 +03:00
Aliaksandr Valialkin
450aa0ae5a
lib/promrelabel: support action: graphite relabeling
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2737
2022-06-16 20:25:49 +03:00
Aliaksandr Valialkin
cc226e6ebe
docs/CHANGELOG.md: follow-up after 2177089f94
2022-06-01 14:57:39 +03:00
Roman Khavronenko
bca90d7148
promrelabel: add support of lowercase and uppercase relabeling actions ( #2665 )
...
* promrelabel: add support of `lowercase` and `uppercase` relabeling actions
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2664
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* lib/storage: make golangci-lint happy
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2022-06-01 14:57:39 +03:00
Aliaksandr Valialkin
810dd74fb9
lib/promscrape: properly implement ScrapeConfig.clone()
...
Previously ScrapeConfig.clone() was improperly copying promauth.Secret fields -
their contents was replaced with `<secret>` value.
This led to inability to use passwords and secrets in `-promscrape.config` file.
The bug has been introduced in v1.77.0 in the commit 67b10896d2https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2551
2022-05-07 00:06:19 +03:00
Aliaksandr Valialkin
45385a5dc6
lib/promscrape: optimize getScrapeWork() function
...
Reduce the number of memory allocations in this function. This improves its performance by up to 50%.
This should improve service discovery speed when big number of potential targets with big number of meta-labels
are generated by service discovery.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2270
2022-04-20 15:34:18 +03:00
Dmytro Kozlov
17552dba8b
lib/promscrape: Enable filters for endpoint and labels ( #2466 )
...
* lib/promscrape: Enable filters for endpoint and labels
* lib/promscrape: cleanup
* lib/promscrape: update template
* lib/promscrape: move logic filter logic to backend
* lib/promscrape: updated placeholder
* lib/promscrape: updated placeholder
* lib/promscrape: use two different fields for filters, updated form, added error on parsing queries
* lib/promscrape: rename functions
* lib/promscrape: removed unused values
* wip
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-19 18:27:44 +03:00
Aliaksandr Valialkin
2431c9cf81
lib/promrelabel: add support for conditional relabeling via if filter
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998
2022-02-24 02:35:13 +02:00
Aliaksandr Valialkin
053e85ff3d
all: typo fix: unexected -> unexpected
2021-12-20 17:40:13 +02:00
Aliaksandr Valialkin
d40441947a
app: allow specifying http and https urls in the following command-line flags
...
* -promscrape.config
* -relabelConfig
* -remoteWrite.relabelConfig
* -remoteWrite.urlRelabelConfig
2021-12-03 00:11:47 +02:00
