Commit graph

5002 commits

Author SHA1 Message Date
f41gh7
6d0420b454
make docs-update-version 2024-11-29 17:45:18 +01:00
Nikolay
430163d01a
app/vmagent: fixes multitenant token parse
Previously, vmagent produced parsing error for 'multitenant' auth token
value for the cases:
* data ingestion with enableMultitentEndpoints
* data scrapping at promscrape

 It's inconsistent to the other VictoriaMetrics components.
Since 'multitenant' is well-known token value for multitenancy via
labels. And vmagent is intended to be compatible with vminsert ingestion
endpoints.

 This commit replaces NewToken with NewTokenPossibleMultitenant function
for token parsing. It allows to use multitenant value for it. And it
makes token values consistent for the all components.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7694
2024-11-29 14:04:07 +01:00
f41gh7
f404e0b3a5
app/vmauth: fixes ip_filters init for uauthorized_user
Previously ip_filters wasn't properly inited for this part of config.
It resulted to bypass requests for this section.

 This commit properly inits `ip_filter`.
2024-11-29 11:01:00 +01:00
f41gh7
1d4a4eb8b0
app/vmauth: add ip_filter deny metrics
Previously, all requests rejected by `ip_filter` were silently aborted.

This commit adds new metrics:
* vmauth_user_ip_denies_total
* vmauth_global_ip_denies_total
* vmauth_unauthorized_user_ip_denies_total

 It adds observability to this feature and allow to measure rejected requests.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6883

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-29 11:00:32 +01:00
Github Actions
30903d9361
Automatic update helm docs from VictoriaMetrics/helm-charts@f5f1b9c (#7691)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-29 10:30:34 +01:00
Hui Wang
000a918f38
app/vmalert-tool: print an error message if no rule group is found under 'rule_files'
Both vmalert and vmalert-tool support multiple `rule_files` and use
directory as a file, so it's ok if some files don't contain any rule
group. But vmalert-tool should warn the user if no rule group is found
in any of the `rule_files`.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7663
2024-11-29 10:29:04 +01:00
Nikolay
9c35807368
app/vmauth: add removeXFFHTTPHeaderValue flag
Previously, there was no option to replace value of `X-Forwarded-For`
HTTP Header. It was only possible to completely remove it. It's not good
solution, since backend may require this information. But using direct
value of this header is insecure. And requires complex knowledge of
infrastruce at backend side (see spoofing X-Forwarded-For articles).

This commit adds new flag, that replaces content of `X-Forwarded-For`
HTTP Header value with current `RemoteAddress` of client that send
request.
It should be used if `vmauth` is directly attached to the internet.

 Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6883

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-29 10:25:47 +01:00
Nikolay
92512cbe54
app/vmauth: add real_ip_header to ip_filters
This commit allows vmauth to obtain client IP address from HTTP Headers.
Main scenario for it is vmauth located behind reverse-proxy.

 It adds both global and per user configuration settings: -httpRealIPHeader and `real_ip_header` config option.

vmauth try to obtain IP from header if this setting is set. If header is not exists, vmauth fallbacks to `remoteAddress`.

Commit also updates incorrect benchmarks and align test package naming for ip_filters

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6883

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-28 14:55:35 +01:00
Yury Molodov
dec9a2f023
vmui: fix predefined panels
### Describe Your Changes

- Fixes the handling of the `showLegend` flag.  
- Fixes the handling of `alias`.  
- Adds support for alias templates, allowing dynamic substitutions like
`{{label_name}}`.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7565
2024-11-28 13:47:37 +01:00
Github Actions
9c6a4d915d
Automatic update helm docs from VictoriaMetrics/helm-charts@a5bc12b (#7685)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-28 13:25:10 +01:00
Fred Navruzov
6fa3283d04
docs/vmanomaly - release 1.18.5 (#7684)
### Describe Your Changes

docs/vmanomaly - release 1.18.5 doc updates

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-28 13:34:45 +02:00
Nikolay
20d5314833
app/vmauth: adds dryRun flag for config validation
This flag only validates configuration file pointed by `auth.config`
flag.

 Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7505

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-27 13:51:48 +01:00
Yury Molodov
10c42668a1
vmui/logs: optimize memory consumption (#7524)
### Describe Your Changes

- **Memory Optimization**: Reduced memory consumption on the "Group" and
"JSON" tabs by approximately 30%.

- **Table Pagination**: Added pagination to the "Table" view with an
option to select the number of rows displayed (from 10 to 1000 items per
page, with a default of 1000). This change significantly reduced memory
usage by approximately 75%.

Related to #7185

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-27 13:49:06 +01:00
Andrii Chubatiuk
60d587f55b
docs/vmgateway: added to documentation a case, when vm_access claim value is of string type 2024-11-27 11:13:59 +01:00
Viet Hung Nguyen
466cbee433
docs: fix rule unittest rule_files point to alerts.yaml (#7664)
### Describe Your Changes

Fix wrong path to rules_file in vmalert-tool doc

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Viet Hung Nguyen <hvn@familug.org>
Co-authored-by: Hui Wang <haley@victoriametrics.com>
2024-11-27 08:04:21 +01:00
Hui Wang
cdbe69e62b
docs: add multitenat endpoints usage for vmalert multitenancy (#7665)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-27 08:01:11 +01:00
Zhu Jiekun
7374a8813d
lib/promscrape/discovery: properly apply the resource_group filter for Azure service discovery
Previously, this filter did not apply to virtual
machine scale sets, causing all virtual machines to be discovered.

 This commit conditionally adds `resource_group` filter for Azure service discovery on virtual
machine scale sets. 

 Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7630.
2024-11-26 19:06:43 +01:00
Roman Khavronenko
8ab1261750
app/vmauth: dump requests that failed the route rules to stderr (#7649)
Additional info from the dump can be used to debug rotuing rules.

https://pkg.go.dev/net/http/httputil#DumpRequest

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-26 10:36:27 +01:00
Artem Navoiev
b1324360b8
docs: update formatting on cloud billing page
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-11-26 09:52:53 +01:00
matty
c3fa806b2f
docs: Bump LTS versions (#7658)
### Describe Your Changes

LTS versions are out of date, linking to old releases. This updates them
to the latest.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-26 09:45:06 +01:00
Artem Navoiev
3adc7abf8f
docs: add cloud billing page (#7655)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-11-25 11:03:15 -08:00
Github Actions
2962e35c0b
Automatic update helm docs from VictoriaMetrics/helm-charts@ebc6d67 (#7653)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-25 18:07:27 +01:00
Andrei Baidarov
727bc02a5c
vmagent: set up a timeout for tcp connection establishment during k8s discovery
Previously, default dial timeout was used for kubernetes API server connection.

 This commit changes it for custom dialer used by the all VictoriaMetrics components. It has lower connection timeout (30s by default). 


 Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7127

---------
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-11-25 18:02:09 +01:00
hagen1778
71d774f76d
docs: re-structure multitenancy docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-25 13:10:30 +01:00
hagen1778
530b731101
docs: update otel docs
* cross-link related doc chapters and guides about otel
* mention different URL format for cluster version of VM

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-25 13:10:14 +01:00
hagen1778
0e6731323a
docs: update link for tsdb stats word
The link was pointing to itself, which doesn't make sense.
Changed it to point to https://docs.victoriametrics.com/url-examples/#apiv1statustsdb

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-25 12:27:26 +01:00
Nikolay
bdac00f674
app/vmselect: fixes multitenant cache init
Previously multitenant cache was inited before flag.Parse call. It
didn't allow to change cache expiration value and default value was
always used.

 This commit adds cache init at the first time cache was called.

 Also this commit adds small cache improvements:
* chore for cleanup cache, it now uses common pattern for in-place items
filtering
* fail cache request fast if item is already expired

---------
Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-25 11:51:30 +01:00
Andrei Baidarov
037808dad5
app/vmselect: fix panic/incorrect tenant in key
This is a follow-up after 3120dc2

- Consistently use key for rollupCache in multitenant mode cache keys use different authTokens. Previously it could lead to panic in rare cases when cache state was inconsistent.
- Do not share `err` variable across goroutines for `processBlock` function. It could lead to data races.

Related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7549
---------
Signed-off-by: Andrei Baidarov <abaidarov@yandex.ru>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-11-25 11:50:49 +01:00
Hui Wang
508bafced3
vmalert-tool: exit immediately with error message if no test file found in -files (#7627) 2024-11-25 11:43:50 +01:00
Github Actions
0b2b96422d
Automatic update helm docs from VictoriaMetrics/helm-charts@6a5b14b (#7642)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-25 10:08:51 +01:00
dmitry-shur
2328656f87
docs: fixed typo in FAQ (#7643)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-25 09:43:00 +01:00
Github Actions
9d5ef1cdd1
Automatic update helm docs from VictoriaMetrics/helm-charts@a582432 (#7641)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-25 09:10:51 +01:00
hagen1778
c9837de9cd
docs: add vmui section to cluster docs
It looks like users get confused with vmui availability in vmselect.
Adding a note about it to cluster docs should improve situation.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7631

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-24 14:18:26 +01:00
Artem Navoiev
6c6e469bfb
docs: cloud mention silver level in support
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-11-23 11:08:18 +01:00
Hui Wang
6ff1de89a9
vmalert: fix alert states restoration (#7624)
Previously, when the alert got resolved shortly before the vmalert
process shuts down, this could result in false alerts.

This change switches vmalert to use MetricsQL function during alerts state restore, which makes it
incompatible for state restoration with PromQL.

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-22 09:11:31 +01:00
Github Actions
4319d9f2b0
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@a5e2cb4 (#7622) 2024-11-21 19:32:27 +01:00
hagen1778
a15fcac1b6
docs: update differences between vmagent and Prometheus in FAQ
* mention stream aggregation
* rm statement that Prometheus can only pull data, which is not true anymore
* mention absence of backfilling limitations

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-21 16:30:59 +01:00
Ivan Yurochko
5dd879cd17
lib/streamaggr: add ignore_first_sample_interval param for streamaggr cfg (#7313)
### Describe Your Changes

As of right now by default aggregated output in streaming aggregation
takes a staleness interval and only starts sending first samples after
the staleness interval passes. We have a use case where we prefer to
start sending data as soon as we have any. This adds the option to
configure when we start sending first samples

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7116

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-11-21 16:20:22 +01:00
Github Actions
84b4b5f3e5
Automatic update helm docs from VictoriaMetrics/helm-charts@b0095d3 (#7618)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-21 16:04:37 +01:00
hagen1778
a4b3ce9641
docs: mention Raw Query tab in vmui
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7024
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-21 16:03:05 +01:00
Yury Molodov
cd0ad293fe
vmui: add Raw Query tab (#7575)
### Describe Your Changes

1. **Add new `Raw Query` tab**  
A new `Raw Query` tab has been added to the
[vmui](https://docs.victoriametrics.com/#vmui) interface for displaying
raw data. The tab uses the `/api/v1/export` API endpoint. Related issue:
[#7024](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7024)


2. **Fix rendering of isolated points on the graph**  
Previously, isolated points (not connected to other points on the left
or right) were not visible on the graph. Now, they are rendered
correctly.
 

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-21 15:52:50 +01:00
Nikolay
bb399518db
app/vmselect: properly return binary pow function result (#7619)
Previously, for `^` aka pow function calls, VictoriaMetrics returned `1`
if left arg was Nan. For example, given query=`(hour()==2)^1` returns 1
for NaN produced by hour() == 2 function. It added additional non-exist
datapoints to the timeseries.

This commit port bugfix from `metricql` package and adds test for it.
Now, VictoriaMetrics
correctly returns `NaN` for such cases.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7359

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-21 15:16:28 +01:00
Hui Wang
1bd927e3fe
vmalert: remove deprecated cmd-line flags -datasource.lookback, `da… (#6779)
…tasource.queryTimeAlignment` and `remoteRead.ignoreRestoreErrors`

Those flags were all deprecated before
[v1.101.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.101.0).
2024-11-21 13:58:09 +01:00
Github Actions
f07574a78e
Automatic update operator docs from VictoriaMetrics/operator@b3e19a2 (#7608)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-11-21 09:48:14 +01:00
Github Actions
689196048f
Automatic update helm docs from VictoriaMetrics/helm-charts@8a9669e (#7616)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-21 09:47:46 +01:00
Will Jordan
a19a4f34ff
lib/tenantmetrics: improves CounterMap performance with large numbers of tenants
Previously, map for storing tenant metrics was re-created to each newly ingested tenant. It has significant performance impact for systems with large number of tenants.

 This commit addresses this issue by changing algorithm of creating tenant metric records at map. Instead of map re-creation, it uses `sync.Map` primitive.

Benchmark results:

```
goos: linux
goarch: amd64
pkg: github.com/VictoriaMetrics/VictoriaMetrics/lib/tenantmetrics
cpu: AMD Ryzen 9 5900X 12-Core Processor
                                            │ lib/tenantmetrics/orig.bench │     lib/tenantmetrics/new.bench     │
                                            │            sec/op            │    sec/op     vs base               │
CounterMapGrowth/n=100,nProcs=GOMAXPROCS-24                  1943.2µ ±  5%   248.0µ ± 11%  -87.24% (p=0.001 n=7)
CounterMapGrowth/n=100-24                                    434.63µ ±  5%   98.82µ ± 16%  -77.26% (p=0.001 n=7)
CounterMapGrowth/n=1000-24                                   32.719m ± 20%   1.425m ±  5%  -95.65% (p=0.001 n=7)
CounterMapGrowth/n=10000-24                                 3653.60m ±  5%   18.00m ±  2%  -99.51% (p=0.001 n=7)
geomean                                                       17.83m         890.4µ        -95.00%
```

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7482

---
Co-authored-by: Artem Fetishev <rtm@victoriametrics.com>
2024-11-20 18:42:47 +01:00
Alexander Frolov
7454d938cc
app/vmagent: respect Pushgateway protocol in multi-tenant vmagent handler (#7571)
### Describe Your Changes

fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636 for
multi-tenant vmagent handler

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-20 16:34:25 +01:00
Hui Wang
580fb3ad85
doc: clarify rule group type (#7595)
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7585
2024-11-20 16:23:13 +01:00
Renato Monteiro
4485877a83
docs: fix small grammatical error in CONTRIBUTING.md (#7596) 2024-11-20 16:22:42 +01:00
Github Actions
a8997e97de
Automatic update operator docs from VictoriaMetrics/operator@c2b7a5a (#7598)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-11-20 16:21:49 +01:00