Commit graph

4968 commits

Author SHA1 Message Date
Hui Wang
6ff1de89a9
vmalert: fix alert states restoration (#7624)
Previously, when the alert got resolved shortly before the vmalert
process shuts down, this could result in false alerts.

This change switches vmalert to use MetricsQL function during alerts state restore, which makes it
incompatible for state restoration with PromQL.

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-22 09:11:31 +01:00
Github Actions
4319d9f2b0
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@a5e2cb4 (#7622) 2024-11-21 19:32:27 +01:00
hagen1778
a15fcac1b6
docs: update differences between vmagent and Prometheus in FAQ
* mention stream aggregation
* rm statement that Prometheus can only pull data, which is not true anymore
* mention absence of backfilling limitations

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-21 16:30:59 +01:00
Ivan Yurochko
5dd879cd17
lib/streamaggr: add ignore_first_sample_interval param for streamaggr cfg (#7313)
### Describe Your Changes

As of right now by default aggregated output in streaming aggregation
takes a staleness interval and only starts sending first samples after
the staleness interval passes. We have a use case where we prefer to
start sending data as soon as we have any. This adds the option to
configure when we start sending first samples

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7116

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-11-21 16:20:22 +01:00
Github Actions
84b4b5f3e5
Automatic update helm docs from VictoriaMetrics/helm-charts@b0095d3 (#7618)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-21 16:04:37 +01:00
hagen1778
a4b3ce9641
docs: mention Raw Query tab in vmui
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7024
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-21 16:03:05 +01:00
Yury Molodov
cd0ad293fe
vmui: add Raw Query tab (#7575)
### Describe Your Changes

1. **Add new `Raw Query` tab**  
A new `Raw Query` tab has been added to the
[vmui](https://docs.victoriametrics.com/#vmui) interface for displaying
raw data. The tab uses the `/api/v1/export` API endpoint. Related issue:
[#7024](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7024)


2. **Fix rendering of isolated points on the graph**  
Previously, isolated points (not connected to other points on the left
or right) were not visible on the graph. Now, they are rendered
correctly.
 

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-21 15:52:50 +01:00
Nikolay
bb399518db
app/vmselect: properly return binary pow function result (#7619)
Previously, for `^` aka pow function calls, VictoriaMetrics returned `1`
if left arg was Nan. For example, given query=`(hour()==2)^1` returns 1
for NaN produced by hour() == 2 function. It added additional non-exist
datapoints to the timeseries.

This commit port bugfix from `metricql` package and adds test for it.
Now, VictoriaMetrics
correctly returns `NaN` for such cases.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7359

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-21 15:16:28 +01:00
Hui Wang
1bd927e3fe
vmalert: remove deprecated cmd-line flags -datasource.lookback, `da… (#6779)
…tasource.queryTimeAlignment` and `remoteRead.ignoreRestoreErrors`

Those flags were all deprecated before
[v1.101.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.101.0).
2024-11-21 13:58:09 +01:00
Github Actions
f07574a78e
Automatic update operator docs from VictoriaMetrics/operator@b3e19a2 (#7608)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-11-21 09:48:14 +01:00
Github Actions
689196048f
Automatic update helm docs from VictoriaMetrics/helm-charts@8a9669e (#7616)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-21 09:47:46 +01:00
Will Jordan
a19a4f34ff
lib/tenantmetrics: improves CounterMap performance with large numbers of tenants
Previously, map for storing tenant metrics was re-created to each newly ingested tenant. It has significant performance impact for systems with large number of tenants.

 This commit addresses this issue by changing algorithm of creating tenant metric records at map. Instead of map re-creation, it uses `sync.Map` primitive.

Benchmark results:

```
goos: linux
goarch: amd64
pkg: github.com/VictoriaMetrics/VictoriaMetrics/lib/tenantmetrics
cpu: AMD Ryzen 9 5900X 12-Core Processor
                                            │ lib/tenantmetrics/orig.bench │     lib/tenantmetrics/new.bench     │
                                            │            sec/op            │    sec/op     vs base               │
CounterMapGrowth/n=100,nProcs=GOMAXPROCS-24                  1943.2µ ±  5%   248.0µ ± 11%  -87.24% (p=0.001 n=7)
CounterMapGrowth/n=100-24                                    434.63µ ±  5%   98.82µ ± 16%  -77.26% (p=0.001 n=7)
CounterMapGrowth/n=1000-24                                   32.719m ± 20%   1.425m ±  5%  -95.65% (p=0.001 n=7)
CounterMapGrowth/n=10000-24                                 3653.60m ±  5%   18.00m ±  2%  -99.51% (p=0.001 n=7)
geomean                                                       17.83m         890.4µ        -95.00%
```

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7482

---
Co-authored-by: Artem Fetishev <rtm@victoriametrics.com>
2024-11-20 18:42:47 +01:00
Alexander Frolov
7454d938cc
app/vmagent: respect Pushgateway protocol in multi-tenant vmagent handler (#7571)
### Describe Your Changes

fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636 for
multi-tenant vmagent handler

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-20 16:34:25 +01:00
Hui Wang
580fb3ad85
doc: clarify rule group type (#7595)
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7585
2024-11-20 16:23:13 +01:00
Renato Monteiro
4485877a83
docs: fix small grammatical error in CONTRIBUTING.md (#7596) 2024-11-20 16:22:42 +01:00
Github Actions
a8997e97de
Automatic update operator docs from VictoriaMetrics/operator@c2b7a5a (#7598)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-11-20 16:21:49 +01:00
Github Actions
d3316b333d
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@c4d6669 (#7602) 2024-11-20 16:21:12 +01:00
Hui Wang
71f521fc0c
vmalert: revert the default value of -remoteWrite.maxQueueSize from… (#7570)
… `1_000_000` to `100_000`

It was bumped in
[v1.104.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.104.0),
which increases memory usage and is not needed for most setups. See
[this
issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7471).
2024-11-20 16:20:51 +01:00
Github Actions
6bf49aef03
Automatic update operator docs from VictoriaMetrics/operator@1e7c2c5 (#7582)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: Haleygo <39937150+Haleygo@users.noreply.github.com>
2024-11-19 16:40:51 +01:00
Github Actions
a8c5035d3d
Automatic update helm docs from VictoriaMetrics/helm-charts@131846f (#7581)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-19 16:40:20 +01:00
Mathias Palmersheim
43a45cf4e3
Updated vmanomaly product link (#7580)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

fixes #7579 7579
2024-11-18 15:56:48 -06:00
Fred Navruzov
a534df6cf3
docs/vmanomaly: add self-monitoring section (#7558)
### Describe Your Changes

- Added self-monitoring guide for `vmanomaly`.
- Added cross-referencing on other pages.
- Slight improvements in wording on related pages
- Update references to v1.18.4
- [x] publish Grafana dashboard to
https://grafana.com/orgs/victoriametrics/dashboards:
https://grafana.com/grafana/dashboards/22337-victoriametrics-vmanomaly/

@AndrewChubatiuk , JFYI if it somehow impacts your work on supporting
`vmanomaly` in operator.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-18 20:14:46 +02:00
JAYICE
a696ef21df
docs: correct the wrong description of vector support in the docs (#7566)
### Describe Your Changes

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7557

Vector only supports OpenTelemetry as a source.
2024-11-18 16:30:43 +01:00
f41gh7
d4d3ec877e
{deployment/,docs/}: point app versions to the lastest v1.106.1 release
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-18 15:07:30 +01:00
f41gh7
4ade6f9d25
docs/changelog: mention 1.106.1,1.102.7 and 1.97.12 releases
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-18 14:33:04 +01:00
f41gh7
4b74baf696
CHANGELOG.md: cut v1.106.1 release
remove upcoming v1.107.0 release from changelog
2024-11-15 23:38:06 +01:00
f41gh7
0abefae46c
CHANGELOG.md: cut v1.107.0 release 2024-11-15 18:06:48 +01:00
Zakhar Bessarab
11af902b22
lib/storage/downsampling: handle dedup separately from downsampling rules with filters
Previously, dedup was added as a downsampling rule with 0s offset to all downsmapling rules with filters. That enforced a metric name lookup even in cases it is not needed.
For example, the following configuration: `-dedup.minScrapeInterval=10s -downsampling.period={__name__=~"node.*"}:1h:1m` would be parsed as: `{__name__=~"node.*"}:1h:1m {}:0s:10s`

This commit changes this logic and treats dedup as a separate case. This allows to perform metric name lookups only in cases when timestamp of current partition can be eligible to use some of downsampling filters. Newer parts will not trigger metric name lookup and will apply deduplication directly.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7440
---------
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-11-15 17:56:47 +01:00
Nikolay
1985110de2
lib/storage: properly check for minMissingTimestamps
After changes at commit 787b9cd. Minimal timestamps for extDB check was performed without context of the index search prefix.
It worked fine for Single node version, but for cluster version a different prefix was used for
metricID search requests. It may lead to incomplete results, if minimal missing timestamp was cached
for the tenant with different ingestion patterns.

 Minimal reproducible case is:
- metrics were ingested for tenants 0 and 1
- at some point in time metrics ingestion for tenant 1 stopped
- index records have the following timestamps layout:
 tenant 0:  1,2,3,4,5,6
 tenant 1:  1,2,3,4
- after indexDB rotation, containsTimeRange lookups may produce
  incorrect results:
 time range request for tenant 1 - 5:6 caches 5 as min timestamp
 request for the same or smaller time range for tenant 0 now returns
empty results.

Second case:
- requests for the tenant without metrics always updates atomic value with incorrect minimal time range for other tenants.

 This commit replaces single atomic with map of search prefix keys. It should have slight performance overhead,
but work consistently for cluster version. minMissingTimestamp is cached by prefix search key, which included tenantID.

 Since it will be only populated at runtime, it doesn't hold unused tenants for queries.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7417
2024-11-15 16:25:13 +01:00
Andrei Baidarov
479ae93e04
app/vmselect: fixes possible panics for multitenant queries
This commit fixes panic for multitenant requests and empty storage node responses for tenants api.

 It also optimizes `populateSqTenantTokensIfNeeded` function calls, by making it only once for query request. Previously it was incorrectly called multiple times per each storage node request.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7549
---------
Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-11-15 16:22:30 +01:00
Aliaksandr Valialkin
fc0b6c62fe
docs: refer to the https://itnext.io/how-do-open-source-solutions-for-logs-work-elasticsearch-loki-and-victorialogs-9f7097ecbc2f in the appropriate places 2024-11-15 14:29:01 +01:00
Aliaksandr Valialkin
47a52f5f40
docs/Articles.md: fix broken link to Open-sourcing VictoriaMetrics article 2024-11-15 14:15:13 +01:00
andriibeee
5d85968659
app/vmauth: fix unauthorized_user routing inconsistency
This commit makes vmauth respect the routing config for unauthorized
requests for requests that despite having Authorization header failed to
authorize successfully.

 It covers the following use-cases:
- vmauth is used at load-balanacer and must forward requests as is. There is no any authorization configs.
- vmauth has authorization config, but it must forward requests with invalid credential tokens to some other backend.

related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7543

---------
Signed-off-by: Andrii <andriibeee@gmail.com>
2024-11-15 12:28:25 +01:00
Fred Navruzov
a335ed23c7
docs/vmanomaly - patch release 1.18.3 (#7540)
### Describe Your Changes

`vmanomaly` docs update with patch release 1.18.3

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-14 19:59:13 +02:00
Zhu Jiekun
0a6d58b4ca
app/vmctl: add command-line flag list to doc
This commit  adds documentation for  **Command-line flags** of vmctl.

---
Related issue: 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7521
2024-11-14 11:23:48 +01:00
Hui Wang
304996bc08
docs/vmalert: clarify some vmalert flags
Some flags are shared between datasourceURL and remoteReadURL, some
flags are not valid for victoriaLogs as the datasource.
2024-11-14 11:21:35 +01:00
Crazygit
5eb6a0f9da
docs/vlogs: fixes typo for VL-Extra-Field
`VL-Extra-Field` should changed to `VL-Extra-Fields` according to code
2024-11-14 11:20:30 +01:00
Artem Navoiev
1cf5cf05db
docs: vmbackupmanager specify schedule
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-11-14 10:43:37 +01:00
Fred Navruzov
5696a087b8
docs/vmanomaly - release 1.18.2 (#7533)
### Describe Your Changes

docs update for `vmanomaly` release 1.18.2

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-13 18:52:20 +02:00
Aliaksandr Valialkin
7e02cb484c
deployment/docker: update VictoriaLogs from v0.42.0-victorialogs to v1.0.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.0.0-victorialogs
2024-11-12 19:15:57 +01:00
Aliaksandr Valialkin
99607e2f3b
docs/VictoriaLogs/CHANGELOG.md: mention that VictoriaLogs gained all the planned features 2024-11-12 19:07:09 +01:00
Aliaksandr Valialkin
1bf58b2f13
docs/VictoriaLogs: cut v1.0.0-victorialogs 2024-11-12 18:31:59 +01:00
Aliaksandr Valialkin
4837dc6e09
docs/VictoriaLogs/querying/vlogscli.md: replace vlogsql with vlogscli everywhere in the docs
`vlogsql` was the initial name of the command-line utility for interactive querying of logs.
Later it was renamed to `vlogscli`.

Thanks to @grinapo for the bugreport at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7512
2024-11-12 18:21:00 +01:00
Aliaksandr Valialkin
33a4f275b1
app/vmauth: properly inherit user-level options at url_map when url_prefix isnt set at the user level
The following user-level options must be unconditionally inherited by url_map, since this is what most users expect:

- retry_status_codes
- load_balancing_policy
- drop_src_path_prefix_parts
- discover_backend_ips

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7519
2024-11-12 17:53:19 +01:00
Hui Wang
32b89447ae
dashboards: add file label filter to vmalert dashboard panels (#7515)
Previously, metrics from groups with the same name but in different
files could be mixed in the results.

e.g. the evaluation time
[here](https://grafana.maas.victoriametrics.com/d/LzldHAVnz/victoriametrics-vmalert?orgId=1&var-ds=PE8D8DB4BEE4E4B22&var-job=All&var-instance=All&var-file=%2Fetc%2Fvmalert%2Fconfig%2Fvm-per-tenant-rulefiles-0%2Fmaas-tenant-1011-maas-1011-vm-health.yaml&var-group=All&var-topk=5&editPanel=23)
is the total for multiple groups from different tenants.
2024-11-12 09:00:39 -07:00
Github Actions
254d9e2729
Automatic update helm docs from VictoriaMetrics/helm-charts@ae2dabc (#7522)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-12 09:00:19 -07:00
Github Actions
8b0f5b2315
Automatic update operator docs from VictoriaMetrics/operator@593d523 (#7510)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: Amper <495795+Amper@users.noreply.github.com>
2024-11-12 08:59:51 -07:00
Zhu Jiekun
58ae3772fe
vmctl: add docs migrating from cluster to single with --vm-native-disable-binary-protocol (#7520)
### Describe Your Changes

when migrating from cluster to single, the native export API attach
`vm_account_id` and `vm_project_id` labels.
`--vm-native-disable-binary-protocol` is a workaround to remove them,
which is available since v1.93.0 but did not documented in vmctl doc.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4716

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-12 08:59:04 -07:00
Artem Fetishev
58dae07b7a
tests: cover key concepts with more tests (#7516)
More key concept tests

-  Verify how the time range points are calculated
-  Vefify that a range query is equivalent to many instant queries

Fix docs accordingly.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2024-11-12 08:57:54 -07:00
Fred Navruzov
47b7487b5f
docs: vmanomaly - release 1.18.1 (#7517)
### Describe Your Changes

doc updates for vmanomaly v1.18.1

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-12 10:20:29 +02:00