8353 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Aliaksandr Valialkin	e7e9721476	docs/vmauth.md: typo fixes after 7b2b980181	2024-03-07 01:08:40 +02:00
Aliaksandr Valialkin	93ef217857	app/vmauth: allow discovering backend ips behind shared hostname and spreading load among the discovered ips ... This is done with the `discover_backend_ips` option at `user` and `url_map` level. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5707	2024-03-07 01:02:38 +02:00
Aliaksandr Valialkin	b2e182ca62	app/vmauth: add src_headers option at url_map, which allows routing incoming requests to different backends depending on request headers	2024-03-06 21:56:42 +02:00
Aliaksandr Valialkin	f5dd9b423e	vendor: run make vendor-update	2024-03-06 21:24:55 +02:00
Aliaksandr Valialkin	5bd4325135	app/vmauth: simplify configuration for src_query_args ... Use the shorter form: src_query_args: - arg1=value1 - arg2=value2 instead of src_query_args: - name: arg1 value: value2 - name: arg2 value: value2 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878	2024-03-06 21:24:53 +02:00
Aliaksandr Valialkin	1df7e5b69d	deployment/docker: update Go builder from Go1.21.7 to Go1.22.1 ... See https://github.com/golang/go/issues?q=milestone%3AGo1.22.1+label%3ACherryPickApproved	2024-03-06 21:04:44 +02:00
Aliaksandr Valialkin	0929f255f0	docs/vmauth.md: mention that request query args can used for routing decisions ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878	2024-03-06 20:58:10 +02:00
Aliaksandr Valialkin	87d59cdfa5	app/vmauth: add ability to route requests based on HTTP query args via src_query_args option ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878	2024-03-06 20:52:42 +02:00
Aliaksandr Valialkin	247d2f3118	app/vmauth: small code cleanup for working with auth tokens	2024-03-06 20:06:25 +02:00
Aliaksandr Valialkin	5817bf1c59	lib/streamaggr: add tests for keep_metric_names and drop_input_labels options	2024-03-06 20:06:23 +02:00
Aliaksandr Valialkin	03c65c9619	app/vmauth: use slices.Contains() instead of hasInt()	2024-03-06 17:36:02 +02:00
Aliaksandr Valialkin	1d7efc4c64	app/vmagent/remotewrite: clarify the reason behind the default value for -remoteWrite.queues in the same way as the reason for -maxConcurrentInserts is defined at 73f5fb0f0c	2024-03-06 13:57:53 +02:00
Aliaksandr Valialkin	2711732944	app/vmselect/prometheus: do not drop match[] filters if -search.ignoreExtraFiltersAtLabelsAPI flag is set ... The `match[]` filter is mandatory at /api/v1/series, so it mustn't be dropped here. There is no sense in dropping `match[]` filter together with `extra_label` and `extra_filters[]` at /api/v1/labels and /api/v1/label/.../values if -search.ignoreExtraFiltersAtLabelsAPI commnad-line flag is set, since: - the `match[]` filter triggers slow path at these APIs; - the `extra_label` and `extra_filters[]` filters narrow down the number of matched time series, so they improve performance comparing to the case when only `match[]` filter is left, while `extra_label` and `extra_filters[]` filters are dropped. This is a follow-up for 0b7a23a91d	2024-03-06 13:57:20 +02:00
Daria Karavaieva	98b56476d7	redirect alias (#5934)	2024-03-06 13:54:48 +02:00
Yury Molodov	56e61bb54b	vmui: improve tracing styles (#5926) ... Improved trace display for better visual separation of branches: * Increased left padding for each element * Added padding for the last element in the branch	2024-03-06 13:54:07 +02:00
hagen1778	a8d42ce4ed	lib/writeconcurrencylimiter: mention dependency on CPU cores for -maxConcurrentInserts flag ... The change also removes misleading `default` value from README for `maxConcurrentInserts` cmd-line flag. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2024-03-06 13:44:16 +02:00
Github Actions	cfce708eb9	Automatic update operator docs from VictoriaMetrics/operator@f028fdf (#5929)	2024-03-06 13:34:37 +02:00
hagen1778	764fc566ff	dashboards: add more context to cluster dashboard panels ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2024-03-06 13:34:10 +02:00
hagen1778	a4cdfaea28	docs: clarify deduplication is needed in multi-retention setup ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2024-03-06 13:33:40 +02:00
hagen1778	a364de3cfe	lib/writeconcurrencylimiter: mention dependency on CPU cores for -maxConcurrentInserts flag ... The change also removes misleading `default` value from README for `maxConcurrentInserts` cmd-line flag. Signed-off-by: hagen1778 <roman@victoriametrics.com> (cherry picked from commit 73f5fb0f0c) Signed-off-by: hagen1778 <roman@victoriametrics.com>	2024-03-05 18:56:38 +01:00
Aliaksandr Valialkin	27b9e8ed3e	app/{vmagent,vminsert}: add -streamAggr.dropInputSamples command-line flag for dropping the specified labels from input samples before deduplication and streaming aggregation	2024-03-05 02:27:27 +02:00
hagen1778	b612c0536e	docs: re-fresh vmctl docs ... * add recommendation about network bandiwdth between vmctl, source and destination * use more relevant time series selector in examples - see https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5835 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2024-03-05 00:49:48 +02:00
Aliaksandr Valialkin	c38c45d71f	app/{vminsert,vmagent}: allow using -streamAggr.dedupInterval without -streamAggr.config ... This allows performing online de-duplication of incoming samples	2024-03-05 00:47:23 +02:00
Aliaksandr Valialkin	4352544d61	lib/streamaggr: do not reset aggregation state after the aggregation took longer than the configured interval ... It is better from user PoV preserving this state until the next flush	2024-03-04 20:03:45 +02:00
Aliaksandr Valialkin	9f81450b38	lib/streamaggr: add missing "s" suffix in the warning message when the de-duplication or aggregation couldnt be finished in a timely manner	2024-03-04 19:38:39 +02:00
Aliaksandr Valialkin	d10932bd99	lib/streamaggr: benchmark only flush routines in BenchmarkDedupAggrFlushSerial and BenchmarkAggregatorsFlushSerial	2024-03-04 19:13:50 +02:00
Aliaksandr Valialkin	36ee08cad4	Revert "lib/streamaggr: do not flush dedup shards in parallel" ... This reverts commit eb40395a1c. Reason for revert: it has been appeared that the performance gain on multiple CPU cores wasn't visible because the benchmark was generating incorrect pushSample.key. See a207e0bf687d65f5198207477248d70c69284296	2024-03-04 19:13:50 +02:00
Aliaksandr Valialkin	9728aaf5d9	lib/streamaggr: properly generate pushSample.key in benchmarks	2024-03-04 19:13:49 +02:00
Aliaksandr Valialkin	93a057e4e6	lib/streamaggr: reduce the number of pointers at "total" aggregation state ... This should reduce load on GC when scanning heap objects.	2024-03-04 19:13:49 +02:00
Aliaksandr Valialkin	9e00d8ad60	lib/streamaggr: use multiple job label values in BenchmarkAggregatorsPush instead of single value ... This should make the benchmark closer to production cases	2024-03-04 19:13:48 +02:00
Aliaksandr Valialkin	9773ad200e	lib/streamaggr: use multiple job labels in BenchmarkAggregatorsPush	2024-03-04 19:13:48 +02:00
Github Actions	b530f6bf73	Automatic update operator docs from VictoriaMetrics/operator@c8ff654 (#5918)	2024-03-04 19:12:59 +02:00
Aliaksandr Valialkin	482560a1f3	lib/streamaggr: do not flush dedup shards in parallel ... This significantly increases CPU usage on systems with many CPU cores, while doesn't reduce flush latency too much	2024-03-04 17:01:42 +02:00
Aliaksandr Valialkin	d7252fce79	lib/streamaggr: reduce memory allocations when registering new series in deduplication and aggregation structs	2024-03-04 17:01:41 +02:00
Aliaksandr Valialkin	402dc14ec0	lib/streamaggr: make aggregate.runFlusher() more roubst and clear	2024-03-04 17:01:41 +02:00
Aliaksandr Valialkin	2ffef39bb3	lib/streamaggr: properly drop samples on the first incomplete interval ... Previously samples were dropped on the first incomplete interval and the next complete interval. Also make sure that the de-duplication is performed just before flushing the aggregate state. This should help the case then dedup_interval = interval.	2024-03-04 17:01:40 +02:00
hagen1778	51745ec5ff	dashboards: update links in various panels ... * use docs.victoriametrics.com instead of github docs * add links to common terms used in VictoriaMetrics Signed-off-by: hagen1778 <roman@victoriametrics.com>	2024-03-04 17:00:54 +02:00
Aliaksandr Valialkin	c2dae136b3	lib/streamaggr: explicitly call resetSeries after flushSeries ... This makes the code less fragile	2024-03-04 06:23:36 +02:00
Aliaksandr Valialkin	48a425898a	lib/streamaggr: enable time alignment for aggregate flushed to multiples of interval ... For example, if `interval: 1m`, then data flush occurs at the end of every minute, while `interval: 1h` leads to data flush at the end of every hour. Add `no_align_flush_to_interval` option, which can be used for disabling the alignment.	2024-03-04 06:23:35 +02:00
Aliaksandr Valialkin	3ba9b2225e	docs/stream-aggregation.md: add troubleshooting section with solutions for common problems in streaming aggregation	2024-03-04 03:04:59 +02:00
Aliaksandr Valialkin	d80deaeaf4	lib/streamaggr: ignore the first sample in new time series during staleness_interval seconds after the stream aggregation start for total and increase outputs	2024-03-04 03:04:58 +02:00
Aliaksandr Valialkin	5e9cbfd4db	lib/streamaggr: flush dedup state and aggregation state in parallel on all the available CPU cores ... This should reduce the time needed for aggregation state flush on systems with many CPU cores	2024-03-04 01:22:41 +02:00
Aliaksandr Valialkin	1e741ed6db	lib/streamaggr: add a benchmark for flushing dedup state	2024-03-04 01:22:40 +02:00
Aliaksandr Valialkin	5205972b83	lib/streamaggr: add a benchmark for measuring the performance of aggregator.flush	2024-03-04 01:22:40 +02:00
Aliaksandr Valialkin	8daf7a3f43	lib/streamaggr: add a benchmark for de-duplicating of 1M samples	2024-03-04 01:22:39 +02:00
Aliaksandr Valialkin	d4a425af87	lib/prompbmarshal: use clear() instead of a loop for clearing tss inside ResetTimeSeries()	2024-03-03 23:40:47 +02:00
Aliaksandr Valialkin	b958135677	lib/promutils: optimize LabelsCompressor.Decompress by using a specialized labelsMap struct instead of sync.Map ... The labelsMap struct employs the fact that label indexes are condensed around 0, so it stores the referred labels in a slice instead of map and uses slice index as label key. This allows increasing the LabelsCompressor.Decompress performance by up to 3x. This also reduces the latency of data flush in stream aggregation.	2024-03-03 23:25:27 +02:00
Aliaksandr Valialkin	492c6c3ff5	docs/CHANGELOG.md: typo fix	2024-03-02 04:52:41 +02:00
Aliaksandr Valialkin	d0e6541f35	docs/stream-aggregation.md: typo fixes	2024-03-02 04:35:37 +02:00
Aliaksandr Valialkin	b912a45220	docs/stream-aggregation.md: remove superflouous output_relabel_configs from the config example for histogram aggregation	2024-03-02 03:36:08 +02:00