Aliaksandr Valialkin
885ee160c2
all: allow dynamically reading *AuthKey flag values from files and urls
...
Examples:
1) -metricsAuthKey=file:///abs/path/to/file - reads flag value from the given absolute filepath
2) -metricsAuthKey=file://./relative/path/to/file - reads flag value from the given relative filepath
3) -metricsAuthKey=http://some-host/some/path?query_arg=abc - reads flag value from the given url
The flag value is automatically updated when the file contents changes.
2024-01-22 01:23:23 +02:00
Aliaksandr Valialkin
e15f07d989
all: consistently clear prompbmarshal.Label by assigning an empty struct instead of zeroing Name and Value individually
2024-01-22 01:11:59 +02:00
Aliaksandr Valialkin
f0215afee3
lib/promrelabel: add keep_if_contains and drop_if_contains relabeling actions
...
(cherry picked from commit ac65c6b178
2023-12-01 14:00:20 +01:00
Nikolay
9505d48070
lib/streamaggr: properly reference slice with labels ( #5406 )
...
* lib/streamaggr: properly reference slice with labels
by limiting slice capacity. It must fix issues with slice modification, in case of append new slice will be allocated, instead of modifying refrenced slice
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5402
* Reduce memory allocations when output_relabel_configs adds new labels to output samples
---------
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
(cherry picked from commit 41f7940f97
2023-12-01 14:00:18 +01:00
Aliaksandr Valialkin
36a1fdca6c
all: consistently use %w instead of %s in when error is passed to fmt.Errorf()
...
This allows consistently using errors.Is() for verifying whether the given error wraps some other known error.
2023-10-26 09:44:40 +02:00
Aliaksandr Valialkin
4b1f01e45d
lib/promrelabel: properly replace : char with _ in metric names when -usePromCompatibleNaming command-line flag is set
...
This addresses https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113#issuecomment-1275077071 comment from @johnseekins
2023-08-14 16:18:17 +02:00
Aliaksandr Valialkin
e8fe00d39e
lib/promrelabel: stop emitting DEBUG log lines when parsing if expressions
...
These lines were accidentally left in the commit 62651570bbhttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635
2023-08-14 16:18:17 +02:00
Zakhar Bessarab
15b1810dc8
lib/promrelabel: fix relabeling if clause ( #4816 )
...
* lib/promrelabel: fix relabeling if clause being applied to labels outside of current context
Relabeling is applied to each metric row separately, but in order to lower amount of memory allocations it is reusing labels.
Functions which are working on current metric row labels are supposed to use only current metric labels by using provided offset, but if clause matcher was using the whole labels set instead of local metrics.
This leaded to invalid relabeling results such as one described here: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4806
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
* docs/CHANGELOG.md: document the bugfix
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4806
---------
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-08-11 06:44:46 -07:00
Aliaksandr Valialkin
3179d0528b
lib/promrelabel: return correct string representation for IfExpression containing a single selector
...
This is a follow-up for 62651570bbhttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635
2023-07-24 19:33:04 -07:00
Aliaksandr Valialkin
6e43664e24
lib/promrelabel: add support for a list of series selectors at IfExpression
...
This makes possible specifying a list of series selectors at the following places:
- Inside `if` option at relabeling rules
- Inside `match` option at stream aggregation rules
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635
2023-07-24 17:09:59 -07:00
Aliaksandr Valialkin
a7fdc3fcc7
all: add support for or filters in series selectors
...
This commit adds ability to select series matching distinct filters via a single series selector.
For example, the following selector selects series with either {env="prod",job="a"}
or {env="dev",job="b"} labels:
{env="prod",job="a" or env="dev",job="b"}
The `or` filter is supported in all the VictoriaMetrics tools now.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3997
Uses https://github.com/VictoriaMetrics/metricsql/pull/14
2023-07-15 23:56:18 -07:00
Aliaksandr Valialkin
c30f0e51d7
lib/promrelabel: use monospace font at textarea for writing relabel configs on /metric-relabel-debug and /target-relabel-debug pages
...
This simplifies visual inspection of indentation in yaml configs
2023-05-18 20:49:47 -07:00
Aliaksandr Valialkin
b24da0f901
Revert "lib/promrelabel: show error message if labels not in prometheus exposition format ( #4304 )"
...
This reverts commit 193a9c3328https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#text-format-example
E.g. the promutils.NewLabelsFromString() must return error when the following strings are passed to it:
- `{foo:"bar"}`, since `:` is disallowed in Prometheus text exposition format. The corect value is `{foo="bar"}`
- `{"foo":"bar"}`, since label name shouldn't be quoted. The correct value is `{foo="bar"}`.
The reverted commit introduces another set of bugs, which happily accept the following invalid input:
- `{foo=~"bar"}`
- `{foo!="bar"}`
- `{foo!~"bar"}`
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4284
See also https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4304
2023-05-12 17:01:23 -07:00
Dmytro Kozlov
24386f68db
lib/promrelabel: show error message if labels not in prometheus exposition format ( #4304 )
...
lib/promrelabel: show error message if labels not in prometheus exposition format
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4284
2023-05-12 16:53:59 -07:00
Aliaksandr Valialkin
8703b2fa87
app/vmselect: small cleanup after 4f3f9950d0
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3807
2023-05-09 22:45:02 -07:00
Yury Molodov
ddc5197bce
vmui: add metric relabel debug ( #3889 )
...
* feat: add metric relabel debug (#3807 )
* fix: add link to relabeling cookbook
* lib/promrelabel: merge, fix conflicts
* lib/promrelabel: fix diff
* docs/vmui: add metric relabel playground
---------
Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>
2023-05-08 14:59:35 -07:00
Dmytro Kozlov
85b01c4aa7
lib/promrelabel: make target url from labels on target relabel page ( #3882 )
...
* lib/promrelabel: make target url from labels on target relabel page
* wip
---------
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-03-20 22:08:39 -07:00
Oleksandr Redko
0e1c395609
app,lib: fix typos in comments ( #3804 )
2023-02-13 09:32:35 -08:00
Aliaksandr Valialkin
1af6e0b233
lib/promrelabel: pass query args via query string at /metric-relabel-debug and /target-relabel-debug pages if their length doesnt exceed 1000
...
This allows copy-n-pasting the url to another browser window and seeing the same result.
The limit in 1000 chars is selected in order to prevent from potential issues with systems
which limit the url length such as Internet Explorer - see https://stackoverflow.com/questions/812925/what-is-the-maximum-possible-length-of-a-query-string
If the limit is exceeded, then query args are sent via POST method and aren't visible in the url.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3580
2023-01-05 16:45:42 -08:00
Aliaksandr Valialkin
1d16cc9349
lib/promscrape: pre-fetch metric_relabel_configs rules when debugging metric relabeling for a particular target
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407
2023-01-05 03:28:14 -08:00
Aliaksandr Valialkin
ebb8aeb0cf
app/vmselect: remove dependency on lib/promscrape from app/vmselect
2023-01-03 23:27:36 -08:00
Aliaksandr Valialkin
d3f8298739
lib/bytesutil: add InternBytes() function as a shortcut to InternString(ToUnsafeString(..))
2023-01-03 22:15:49 -08:00
Aliaksandr Valialkin
a7942c6c0d
lib/promrelabel: allow calling Match on nil IfExpression
...
This simplifies the caller side of IfExpression
2022-12-30 16:47:59 -08:00
Aliaksandr Valialkin
7888712185
lib/promrelabel: make fmt after d3de110070
2022-12-21 20:25:37 -08:00
Aliaksandr Valialkin
cc482b89a3
lib/promrelabel: add support for keepequal and dropequal relabeling actions
...
These actions are supported by Prometheus starting from v2.41.0
See https://github.com/prometheus/prometheus/pull/11564 ,
https://github.com/prometheus/prometheus/issues/11556
and https://github.com/prometheus/prometheus/issues/3756
Side note:
It's a pity that Prometheus developers decided inventing `keepequal` and `dropequal`
relabeling actions instead of adding support for `keep_if_equal` and `drop_if_equal` relabeling
actions supported by VictoriaMetrics since June 2020 - see 2a39ba639d
2022-12-21 20:06:09 -08:00
Aliaksandr Valialkin
97b41e727c
lib/promscrape: implement target-level and metric-level relabel debugging
...
Target-level debugging is performed by clicking the 'debug' link at the corresponding target
on either http://vmagent:8429/targets page or on http://vmagent:8428/service-discovery page.
Metric-level debugging is perfromed at http://vmagent:8429/metric-relabel-debug page.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407
See https://docs.victoriametrics.com/vmagent.html#relabel-debug
2022-12-10 02:25:56 -08:00
Aliaksandr Valialkin
6910b1de2e
all: typo fix: the the -> the
2022-12-03 21:53:07 -08:00
Aliaksandr Valialkin
be6da5053f
lib/promscrape: optimize service discovery speed
...
- Return meta-labels for the discovered targets via promutils.Labels
instead of map[string]string. This improves the speed of generating
meta-labels for discovered targets by up to 5x.
- Remove memory allocations in hot paths during ScrapeWork generation.
The ScrapeWork contains scrape settings for a single discovered target.
This improves the service discovery speed by up to 2x.
2022-11-29 21:26:23 -08:00
Aliaksandr Valialkin
4a6d5ab1b1
lib/promrelabel: go fmt after 5cec9706dc
2022-10-29 05:17:49 +03:00
Aliaksandr Valialkin
a72bf87e04
lib/promrelabel: add a test from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3251
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3251
2022-10-29 04:34:08 +03:00
Aliaksandr Valialkin
450a32970a
lib/envtemplate: allow referring env vars from other env vars via %{ENV_VAR} syntax
...
This is a follow-up for 02096e06d0
2022-10-26 14:51:02 +03:00
Aliaksandr Valialkin
d0288ea417
all: log error when environment variables referred from -promscrape.config are missing
...
This should prevent from using incorrect config files
2022-10-18 10:29:59 +03:00
Aliaksandr Valialkin
c4a3d8b169
lib/promrelabel: add relabeling tests when the source label is missing
2022-10-17 14:48:29 +03:00
Aliaksandr Valialkin
087393bcef
lib/promrelabel: remove unconditional sorting of the labels in ParsedConfigs.Apply(), since the sorting isnt needed in many places
...
Sort labels explicitly after calling the ParsedConfigs.Apply() when needed.
This reduces CPU usage when performing metric-level relabeling, where labels' sorting isn't needed.
2022-10-09 14:53:35 +03:00
Aliaksandr Valialkin
3987b0abd1
lib/promscrape: allow specifying full target url in __address__ label
...
Previously the `__address__` label could contain only `host:port` part of the target url,
while the scheme and metrics path were obtained from `__scheme__` and `__metrics_path__`
labels. Now it is possible to set the full url in `__address__` label.
This makes valid the following scrape config, which is frequently used by novice users:
scrape_configs:
- job_name: foo
static_configs:
- targets:
- http://host1/metrics1
- https://host2/metrics2
2022-10-07 22:46:29 +03:00
Aliaksandr Valialkin
e5aa34b2e3
lib/promrelabel: export MustParseMetricWithLabels function, which can be used for simplifying tests
2022-10-01 16:15:00 +03:00
Aliaksandr Valialkin
969ae90941
lib/promrelabel: remove redundant memory allocations by using interned strings
2022-10-01 12:07:18 +03:00
Aliaksandr Valialkin
d8d455856c
lib/promrelabel: add a benchmark for realistic Kubernetes relabeling
...
The benchmark name is BenchmarkApplyRelabelConfigs/kubernetes
This benchmark has been copied from d521933053/model/relabel/relabel_test.go (L505)https://github.com/prometheus/prometheus/pull/11147
2022-10-01 12:07:18 +03:00
Aliaksandr Valialkin
39ba55dbb3
lib/promrelabel: go fmt
2022-09-30 12:28:27 +03:00
Aliaksandr Valialkin
9fc2817f41
lib/promrelabel: optimize action: replace for non-trivial regex values
...
Cache `action: replace` results for non-trivial regexs and return them next time
instead of performing CPU-intensive regex replacement.
Optimize also `action: labelmap_all` and `action: replace_all` in the same way.
2022-09-30 12:28:25 +03:00
Aliaksandr Valialkin
f38c9db74d
lib/promrelabel: there is no need in calling regex.HasPrefix() after the optimization at 17289ff481
2022-09-30 12:28:25 +03:00
Aliaksandr Valialkin
fa46c28c5f
lib/promrelabel: optimize action: labelmap for non-trivial regexs
2022-09-30 12:28:25 +03:00
Aliaksandr Valialkin
f1eebc0a99
lib/promrelabel: properly parse regex with escaped $ at the end
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3131
Thanks to @dmitryk-dk for the initial fix at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3179
2022-09-30 08:20:57 +03:00
Aliaksandr Valialkin
4afa25fb38
lib/bytesutil: add FastStringTransformer and use it in the rest of the code where needed
2022-09-28 10:39:42 +03:00
Aliaksandr Valialkin
7f0b95b50a
lib/promrelabel: add SanitizeName() function for sanitizing Prometheus metric names and label names
...
Optimize this function by using results cache for input strings.
Use this function all over the code.
This is a follow-up for fcffdba9dchttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113
2022-09-28 10:02:11 +03:00
Aliaksandr Valialkin
cead9c1e67
lib/promrelabel: call PromRegex.MatchString() on a slow path only if it contains non-empty literal prefix
...
This should improve slow path speed for regexps without literal prefixes
2022-08-26 21:48:09 +03:00
Aliaksandr Valialkin
427d69e775
lib/promrelabel: optimize common regex mismatch cases for action: replace and action: labelmap
2022-08-26 15:48:11 +03:00
Aliaksandr Valialkin
da7697fda4
lib/promrelabel: use regexutil.PromRegex for regex matching in actions labeldrop,labelkeep,drop and keep
...
This makes possible optimizing additional cases inside regexutil.PromRegex
2022-08-26 15:48:11 +03:00
Aliaksandr Valialkin
e1bd38fa97
lib/promrelabel: optimize matching for commonly used regex patterns in if option
...
The following regex patterns are optimized:
- literal string match, e.g. "foo"
- prefix match, e.g. "foo.*" and "foo.+"
- substring match, e.g. ".*foo.*" and ".+foo.+"
- alternate values match, e.g. "foo|bar|baz"
2022-08-26 14:55:13 +03:00
Aliaksandr Valialkin
c49751adf8
lib/regexutil: add Simplify() function for simplifying the regular expression
2022-08-26 11:57:43 +03:00
