Commit graph

8016 commits

Author SHA1 Message Date
hagen1778
cb1e618a16
app/vmauth: properly initialize URLPrefix in tests
It is assumed that URLPrefix.busOriginal will be initialized
durin Unmarshal of the config. But in tests we set fields manually,
so this field never get initialized properly.

Fixes the error `panic: runtime error: integer divide by zero`
at `vmauth.getLeastLoadedBackendURL`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-08 21:10:11 +01:00
hagen1778
0b7ce70df4
app/vmctl: support TLS configuration for VictoriaMetrics destination
VictoriaMetrics destination is specified via `--vm-*` cmd-line flags
and is used in opentsdb, influx, prometheus, remote-read modes.

updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5426

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-08 20:47:36 +01:00
hagen1778
83a8c24281
app/vmctl: follow-up b9f7c3169a
* fix typos in flags description
* move the change to #tip section in changelog

b9f7c3169a
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-08 20:04:02 +01:00
Khushi Jain
b9f7c3169a
app/vmctl : Provide TLS config options for vm native protocol (#5824)
Co-authored-by: Khushi Jain <khushi.jain@nokia.com>
2024-03-08 19:52:55 +01:00
Zakhar Bessarab
25eeb2b16c
docs: fix typo in flags description (#5942)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-03-08 11:22:51 +01:00
Github Actions
90c7c67793
Automatic update operator docs from VictoriaMetrics/operator@9caa896 (#5941) 2024-03-08 10:52:10 +04:00
Aliaksandr Valialkin
98b31b7f7c
docs/vmauth.md: update -help output after e08b91baafc95da090f75e9c29a27d8f62a2b76e 2024-03-07 01:37:39 +02:00
Aliaksandr Valialkin
0d8bec9c6c
docs/CHANGELOG.md: typo fixes 2024-03-07 01:35:34 +02:00
Aliaksandr Valialkin
cb259116b4
lib/promauth: set the Host header to tlsServerName if itsn't empty
If tlsServerName isn't empty, then it is likely the https request is sent to IP instead of hostname.
In this case the request will fail, since Go automatically sets the Host header to the IP instead
of the desired hostname at tlsServerName. So set the Host header to tlsServerName if itsn't empty.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5802
2024-03-07 01:22:01 +02:00
Aliaksandr Valialkin
c0a93cf183
docs/vmauth.md: typo fixes after 7b2b980181 2024-03-07 01:08:33 +02:00
Aliaksandr Valialkin
7b2b980181
app/vmauth: allow discovering backend ips behind shared hostname and spreading load among the discovered ips
This is done with the `discover_backend_ips` option at `user` and `url_map` level.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5707
2024-03-07 01:02:16 +02:00
Aliaksandr Valialkin
76ef84fcae
app/vmauth: add src_headers option at url_map, which allows routing incoming requests to different backends depending on request headers 2024-03-06 21:56:32 +02:00
Aliaksandr Valialkin
d8688c9e82
vendor: run make vendor-update 2024-03-06 21:24:42 +02:00
Aliaksandr Valialkin
8efe12d66e
app/vmauth: simplify configuration for src_query_args
Use the shorter form:

src_query_args:
- arg1=value1
- arg2=value2

instead of

src_query_args:
- name: arg1
  value: value2
- name: arg2
  value: value2

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878
2024-03-06 21:19:45 +02:00
Aliaksandr Valialkin
97dd7e26ad
deployment/docker: update Go builder from Go1.21.7 to Go1.22.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.22.1+label%3ACherryPickApproved
2024-03-06 21:04:11 +02:00
Aliaksandr Valialkin
b2efacb624
docs/vmauth.md: mention that request query args can used for routing decisions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878
2024-03-06 20:58:29 +02:00
Aliaksandr Valialkin
61d1af8050
app/vmauth: add ability to route requests based on HTTP query args via src_query_args option
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878
2024-03-06 20:52:25 +02:00
Aliaksandr Valialkin
9c1331a38a
app/vmauth: small code cleanup for working with auth tokens 2024-03-06 20:05:59 +02:00
Aliaksandr Valialkin
5582a24ecf
lib/streamaggr: add tests for keep_metric_names and drop_input_labels options 2024-03-06 18:34:04 +02:00
Aliaksandr Valialkin
96f913c83e
app/vmauth: use slices.Contains() instead of hasInt() 2024-03-06 17:35:55 +02:00
Yury Molodov
76a6f806ae
vmui: configure npm cache path to resolve EACCES issues on macOS. (#5928) 2024-03-06 14:06:41 +02:00
Github Actions
5b42f15ccc
Automatic update operator docs from VictoriaMetrics/operator@403a78a (#5932) 2024-03-06 12:55:35 +01:00
Aliaksandr Valialkin
b4b38f782c
app/vmagent/remotewrite: clarify the reason behind the default value for -remoteWrite.queues in the same way as the reason for -maxConcurrentInserts is defined at 73f5fb0f0c 2024-03-06 13:43:08 +02:00
Aliaksandr Valialkin
b33b620af6
app/vmselect/prometheus: do not drop match[] filters if -search.ignoreExtraFiltersAtLabelsAPI flag is set
The `match[]` filter is mandatory at /api/v1/series, so it mustn't be dropped here.

There is no sense in dropping `match[]` filter together with `extra_label` and `extra_filters[]`
at /api/v1/labels and /api/v1/label/.../values if -search.ignoreExtraFiltersAtLabelsAPI commnad-line flag is set,
since:
- the `match[]` filter triggers slow path at these APIs;
- the `extra_label` and `extra_filters[]` filters narrow down the number of matched time series,
  so they improve performance comparing to the case when only `match[]` filter is left,
  while `extra_label` and `extra_filters[]` filters are dropped.

This is a follow-up for 0b7a23a91d
2024-03-06 13:31:51 +02:00
Daria Karavaieva
e036433b8b
redirect alias (#5934) 2024-03-06 11:28:31 +01:00
Yury Molodov
3971ce0625
vmui: improve tracing styles (#5926)
Improved trace display for better visual separation of branches:
* Increased left padding for each element
* Added padding for the last element in the branch
2024-03-06 10:36:13 +01:00
hagen1778
73f5fb0f0c
lib/writeconcurrencylimiter: mention dependency on CPU cores for -maxConcurrentInserts flag
The change also removes misleading `default` value from README for `maxConcurrentInserts`
cmd-line flag.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-05 18:55:38 +01:00
Github Actions
c2ff1cfd30
Automatic update operator docs from VictoriaMetrics/operator@f028fdf (#5929) 2024-03-05 17:18:23 +01:00
hagen1778
f781c42ea4
dashboards: add more context to cluster dashboard panels
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-05 15:00:49 +01:00
hagen1778
1c6230c977
docs: clarify deduplication is needed in multi-retention setup
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-05 08:02:48 +01:00
Aliaksandr Valialkin
da611ad628
app/{vmagent,vminsert}: add -streamAggr.dropInputSamples command-line flag for dropping the specified labels from input samples before deduplication and streaming aggregation 2024-03-05 02:15:01 +02:00
Aliaksandr Valialkin
ed523b5bbc
app/{vminsert,vmagent}: allow using -streamAggr.dedupInterval without -streamAggr.config
This allows performing online de-duplication of incoming samples
2024-03-05 00:45:30 +02:00
Aliaksandr Valialkin
22d63ac7cd
lib/streamaggr: do not reset aggregation state after the aggregation took longer than the configured interval
It is better from user PoV preserving this state until the next flush
2024-03-04 20:03:06 +02:00
Aliaksandr Valialkin
32653db7d5
lib/streamaggr: add missing "s" suffix in the warning message when the de-duplication or aggregation couldnt be finished in a timely manner 2024-03-04 19:37:58 +02:00
Aliaksandr Valialkin
6319d029a8
lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial 2024-03-04 19:12:28 +02:00
Aliaksandr Valialkin
074abd5bee
Revert "lib/streamaggr: do not flush dedup shards in parallel"
This reverts commit eb40395a1c.

Reason for revert: it has been appeared that the performance gain on multiple CPU cores
wasn't visible because the benchmark was generating incorrect pushSample.key.

See a207e0bf687d65f5198207477248d70c69284296
2024-03-04 19:12:28 +02:00
Aliaksandr Valialkin
e70177c5fb
lib/streamaggr: properly generate pushSample.key in benchmarks 2024-03-04 19:12:27 +02:00
Aliaksandr Valialkin
b232968bb4
lib/streamaggr: reduce the number of pointers at "total" aggregation state
This should reduce load on GC when scanning heap objects.
2024-03-04 19:12:27 +02:00
Aliaksandr Valialkin
d42667fc41
lib/streamaggr: use multiple job label values in BenchmarkAggregatorsPush instead of single value
This should make the benchmark closer to production cases
2024-03-04 19:12:26 +02:00
Aliaksandr Valialkin
f5bbffd45f
lib/streamaggr: use multiple job labels in BenchmarkAggregatorsPush 2024-03-04 19:12:26 +02:00
Github Actions
a1e9af3abe
Automatic update operator docs from VictoriaMetrics/operator@c8ff654 (#5918) 2024-03-04 17:09:14 +01:00
Aliaksandr Valialkin
eb40395a1c
lib/streamaggr: do not flush dedup shards in parallel
This significantly increases CPU usage on systems with many CPU cores, while doesn't reduce flush latency too much
2024-03-04 17:00:20 +02:00
Aliaksandr Valialkin
946814afee
lib/streamaggr: reduce memory allocations when registering new series in deduplication and aggregation structs 2024-03-04 17:00:19 +02:00
Aliaksandr Valialkin
925f60841f
lib/streamaggr: make aggregate.runFlusher() more roubst and clear 2024-03-04 17:00:19 +02:00
Aliaksandr Valialkin
aa5e7e268c
lib/streamaggr: properly drop samples on the first incomplete interval
Previously samples were dropped on the first incomplete interval and the next complete interval.
Also make sure that the de-duplication is performed just before flushing the aggregate state.
This should help the case then dedup_interval = interval.
2024-03-04 17:00:18 +02:00
hagen1778
0ab1069363
dashboards: update links in various panels
* use docs.victoriametrics.com instead of github docs
* add links to common terms used in VictoriaMetrics

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-04 15:43:31 +01:00
Aliaksandr Valialkin
86494518da
lib/streamaggr: explicitly call resetSeries after flushSeries
This makes the code less fragile
2024-03-04 06:01:18 +02:00
Aliaksandr Valialkin
ac3cf3f357
lib/streamaggr: enable time alignment for aggregate flushed to multiples of interval
For example, if `interval: 1m`, then data flush occurs at the end of every minute,
while `interval: 1h` leads to data flush at the end of every hour.

Add `no_align_flush_to_interval` option, which can be used for disabling the alignment.
2024-03-04 05:42:58 +02:00
Aliaksandr Valialkin
2b8253185b
docs/stream-aggregation.md: add troubleshooting section with solutions for common problems in streaming aggregation 2024-03-04 03:04:46 +02:00
Aliaksandr Valialkin
138a4d1c2b
lib/streamaggr: ignore the first sample in new time series during staleness_interval seconds after the stream aggregation start for total and increase outputs 2024-03-04 01:49:26 +02:00