Commit graph

4774 commits

Author SHA1 Message Date
Aliaksandr Valialkin
252ff71a49
docs/VictoriaLogs/data-ingestion/README.md: document that Rsyslog and Syslog-ng can be used for ingesting logs into VictoriaLogs 2024-06-25 15:15:53 +02:00
Aliaksandr Valialkin
0a88a55ae0
docs/VictoriaLogs/CHANGELOG.md: cut v0.23.0-victorialogs release 2024-06-25 14:59:31 +02:00
Aliaksandr Valialkin
f24123a776
lib/logstorage: parse syslog structured data into separate fields in order to simplify further querying of this data 2024-06-25 14:54:25 +02:00
Aliaksandr Valialkin
1716c4e609
lib/logstorage: properly parse timezone offset at TryParseTimestampRFC3339Nano()
The TryParseTimestampRFC3339Nano() must properly parse RFC3339 timestamps with timezone offsets.

While at it, make tryParseTimestampISO8601 function private in order to prevent
from improper usage of this function from outside the lib/logstorage package.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6508
2024-06-25 14:54:24 +02:00
hagen1778
26d58f8768
docs: add missing ticket links to the recent releases
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-25 14:54:24 +02:00
hagen1778
841305da8a
deployment: fix incorrect link to vmui from VictoriaMetrics plugin in Grafana
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 0dcb3acbf7)
2024-06-25 13:58:59 +02:00
Nikolay
bf1464fc33
dashboards: add dashboard and alerts for vmauth (#6491)
Signed-off-by: f41gh7 <nik@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit 14b9ef1e4d)
2024-06-25 11:17:04 +02:00
hagen1778
8237b63fe3
docs: make docs-sync
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit cc6a1da0d7)
2024-06-25 11:17:04 +02:00
Roman Khavronenko
e3da10a7c1
docs: add section for docs modification (#6528)
While there, remove the obsolete files from the previous version of
docs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 87edfa61c1)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-25 11:17:03 +02:00
Aliaksandr Valialkin
a5445e09c2
Revert "app/vmselect: fix the way of counting raw samples in single query (#6464)"
This reverts commit 5ecf439078.

Reason for revert: the previous logic was correct.

The purpose of `-search.maxSamplesPerQuery` command-line flag is to limit the amounts of CPU resources,
which could be taken by a single query - see https://docs.victoriametrics.com/#resource-usage-limits .
VictoriaMetrics processes samples in blocks during querying - it reads the block, then unpacks it,
then filters out samples outside the selected time range. This means that it _spends CPU time_
on reading and unpacking of _all the samples_ in every block on the requested time range,
even if only a single sample per each block matches the given time range.

The previous logic was effectively limiting CPU time a single query could take.
The new logic fails limiting CPU time a single query could take in some pathological cases
when only a small fraction of samples per each requested block fit the requested time range.
This allows performing multiplication DoS-attacks by querying very narrow time ranges over historical blocks,
which tend to be full. For example, if the `-search.maxSamplesPerQuery` equals to a billion,
and the query requests a single sample out of 8K samples per each block, this means that the query
may unpack a billion of such blocks without exceeding the limit, e.g. it may unpack and process 8K*1e9=8e12 samples.
This is not what the resource usage limits were created for originally - see https://docs.victoriametrics.com/#resource-usage-limits

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5851
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6464
2024-06-25 02:55:43 +02:00
Aliaksandr Valialkin
813a7aec6c
docs: return back spellcheck fixes from the commit 92b22581e6, which were accidentally reverted in 45cf83cd8f
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6470
2024-06-25 02:33:59 +02:00
Aliaksandr Valialkin
7f2b491748
Revert "docs: Add spellcheck workflow (#6470)"
This reverts commit 92b22581e6.

Reason for revert: too complex and slow approach for spellchecking task.

This approach may significantly slow down development pace. It also may take non-trivial
amounts of additional time and resources at CI/CD because of all this npm shit at cspell directory.

Note to @arkid15r : the idea with the ability to run spellchecker on all the VictoriaMetrics
codebase is great. But this shouldn't be mandatory pre-commit check. It is enough to have
a Makefile rule like `make spellcheck`, which could be run manually whenever spellcheck is needed
(e.g. once per month or once per quarter).
2024-06-25 02:28:38 +02:00
Aliaksandr Valialkin
7026498359
lib/httpserver: revert 9b7e532172
Reason for revert: this commit doesn't resolve real security issues,
while it complicates the resulting code in subtle ways (aka security circus).

Comparison of two strings (passwords, auth keys) takes a few nanoseconds.
This comparison is performed in non-trivial http handler, which takes thousands
of nanoseconds, and the request handler timing is non-deterministic because of Go runtime,
Go GC and other concurrently executed goroutines. The request handler timing is even
more non-deterministic when the application is executed in shared environments
such as Kubernetes, where many other applications may run on the same host and use
shared resources of this host (CPU, RAM bandwidth, network bandwidth).

Additionally, it is expected that the passwords and auth keys are passed via TLS-encrypted connections.
Establishing TLS connections takes additional non-trivial time (millions of nanoseconds),
which depends on many factors such as network latency, network congestion, etc.

This makes impossible to conduct timing attack on passwords and auth keys in VictoriaMetrics components.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6423/files
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6392
2024-06-25 01:51:06 +02:00
Aliaksandr Valialkin
7de6f5b4ce
lib/logstorage: work-in-progress 2024-06-25 00:44:57 +02:00
Aliaksandr Valialkin
8dd69fd7ae
docs/CHANGELOG.md: cut v1.102.0-rc2 release 2024-06-24 23:22:45 +02:00
Nikolay
7ab046cf88
vendor update: updates metrics to v1.34.0 (#6523)
mention change for
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6457

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 1af13208c2)
2024-06-24 14:52:32 +02:00
Ivan Yatskevich
50343768fd
Publish Alertmanager configuration docs for Managed VictoriaMetrics (#6428)
### Describe Your Changes

With the recent release of Managed VictoriaMetrics users are able to
create and execute Alerting & Recording rules and send notifications via
hosted Alertmanager.

So, we're publishing Alertmanager configuration docs for Managed
VictoriaMetrics.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 1106d91741)
2024-06-24 12:50:15 +02:00
Arkadii Yakovets
c454084eb1
docs: Add spellcheck workflow (#6470)
### Describe Your Changes

This is a follow-up PR containing workflow related part of the initial
[PR#6362](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6362).
It adds spell checking control based on
[cspell](https://github.com/streetsidesoftware/cspell). The related
configs are consolidated under `cspell` directory.

You can add VictoriaMetrics specific terms into `cspell/custom-dict.txt`
(it contains ~30 items atm). All other absent commonly used terms should
be added directly to respective
[cspell-dicts](https://github.com/streetsidesoftware/cspell-dicts/blob/main/CONTRIBUTING.md).

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>
(cherry picked from commit 92b22581e6)
2024-06-24 12:50:13 +02:00
Andrii Chubatiuk
50783fca4d
app/vmagent: add max_scrape_size to scrape config (#6434)
Related to
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6429

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 1e83598be3)
2024-06-20 14:00:22 +02:00
Hui Wang
a393b993d6
vmalert: exit replay mode with non-zero code if generated samples are… (#6513)
… not successfully written into remoteWrite url

address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6512

(cherry picked from commit bc37b279aa)
2024-06-20 14:00:19 +02:00
Aliaksandr Valialkin
d5224f3363
lib/logstorage: work-in-progress 2024-06-20 03:10:37 +02:00
Zakhar Bessarab
886f545f81
lib/fs/fscore: do not trim content from path (#6503)
### Describe Your Changes

Trimming content which is loaded from an external pass leads to obscure
issues in case user-defined input contained trimmed chars. For example.
user-defined password "foo\n" will become "foo" while user will expect
it to contain a new line.

---
For example, a user defines a password which ends with `\n`. This often
happens when user Kubernetes secrets and manually encodes value as
base64-encoded string.

In this case vmauth configuration might look like:
```
users:
  - url_prefix:
      - http://vminsert:8480/insert/0/prometheus/api/v1/write
    name: foo
    username: foo
    password: "foobar\n"
```

vmagent configuration for this setup will use the following flags:
```
-remoteWrite.url=http://vmauth:8427/
-remoteWrite.basicAuth.passwordFile=/tmp/vmagent-password
-remoteWrite.basicAuth.username="foo"
```
Where `/tmp/vmagent-password` is a file with `foobar\n` password.

Before this change such configuration will result in `401 Unauthorized`
response received by vmagent since after file content will become
`foobar`.

---
An example with Kubernetes operator which uses a secret to reference the
same password in multiple configurations.

<details>
  <summary>See full manifests</summary>

`Secret`:
```
apiVersion: v1
data:
  name: Zm9v # foo
  password: Zm9vYmFy # foobar\n
  username: Zm9v= # foo
kind: Secret
metadata:
  name: vmuser
```

`VMUser`:
```
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
  name: vmagents
spec:
  generatePassword: false
  name: vmagents
  targetRefs:
  - crd:
      kind: VMAgent
      name: some-other-agent
      namespace: example
  username: foo
  # note - the secret above is referenced to provide password
  passwordRef:
    name: vmagent
    key: password
```

`VMAgent`:
```
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
  name: example
spec:
  selectAllByDefault: true
  scrapeInterval: 5s
  replicaCount: 1
  remoteWrite:
    - url: "http://vmauth-vmauth-example:8427/api/v1/write"
      # note - the secret above is referenced as well
      basicAuth:
        username:
          name: vmagent
          key: username
        password:
          name: vmagent
          key: password
```

</details>

Since both config target exactly the same `Secret` object it is expected
to work, but apparently the result will be `401 Unauthrized` error.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 201fd6de1e)
2024-06-19 10:37:12 +02:00
Nihal
8fd46caa22
victoria-metrics: constant-time comparison of credentials like authkeys and basic auth credentials (#6423)
Changes for constant-time comparison of credentials like authkeys and
basic auth credentials.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6392

---------

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>
(cherry picked from commit 9b7e532172)
2024-06-19 10:37:09 +02:00
Hui Wang
36f6a08f07
doc: add missing cmd flag description (#6504)
(cherry picked from commit 2259e3cac7)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-18 15:29:11 +02:00
Yury Molodov
88650abf97
vmui/logs: add bar chart (#6461)
- Added a bar chart displaying the number of log entries over a time
range.
   #6404
- When `_msg` is empty, all fields are displayed in a single line.
- Added double quotes when copying pairs: `key: "value"`.
- Minor style adjustments.

(cherry picked from commit 32fbffedd9)
2024-06-18 15:28:56 +02:00
Hui Wang
5be2f2c4e4
vmalert-tool: support file path with hierarchical patterns and regexp… (#6501)
…es, and http url in unittest cmd-line flag `-files`

(cherry picked from commit 3b8970802e)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-18 14:17:42 +02:00
Hui Wang
b09da16899
vmalert-tool: exit normally when no rule is defined under rule group (#6502)
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6500

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit d62f303e53)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-18 14:17:23 +02:00
Aliaksandr Valialkin
8650809435
docs/keyConcepts.md: document optional timeout query arg at /api/v1/query and /api/v1/query_range
This is a follow-up for ac9c2a796f

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6020
2024-06-18 12:32:04 +02:00
Aliaksandr Valialkin
0ee360b48c
docs/VictoriaLogs/CHANGELOG.md: typo fix in the release date for v0.20.2-victorialogs 2024-06-18 00:28:49 +02:00
Aliaksandr Valialkin
ee114ca59c
app/vlinsert: properly parse timestamps with nanosecond precision at /insert/jsonline HTTP endpoint
This has been broken in 2b6a634ec0
2024-06-18 00:24:11 +02:00
Aliaksandr Valialkin
7658cc36a6
docs/VictoriaLogs/CHANGELOG.md: cut v0.20.1-victorialogs 2024-06-17 23:29:28 +02:00
Aliaksandr Valialkin
c10a646d19
app/vlinsert/syslog: allow accepting syslog messages with different configs at different ports 2024-06-17 23:16:58 +02:00
Aliaksandr Valialkin
f74e6b0674
app/vlinsert: properly parse length-delimited syslog messages sent over TCP according to RFC5425 2024-06-17 22:30:28 +02:00
jackyin
4a6bf7f218
app/vmui: copy button shows undefined (#6495)
### Describe Your Changes
fix #6421
some aggregation func don't return \_\_name\_\_ value
2024-06-17 22:30:28 +02:00
Roman Khavronenko
df7e300071
app/vmselect/promql: check for ranged vectors in aggr funcs if implicit conversions are disabled (#6450)
Check for ranged vector arguments in aggregate expressions when
`-search.disableImplicitConversion` or `-search.logImplicitConversion`
are enabled.
 For example, `sum(up[5m])` will fail to execute if these flags are set.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [*] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6149adbe10)
2024-06-17 14:25:43 +02:00
Aliaksandr Valialkin
938862946c
docs/VictoriaLogs: mention which collectors support sending logs to VictoriaLogs in syslog format 2024-06-17 13:13:51 +02:00
Aliaksandr Valialkin
1750991119
lib/logstorage: work-in-progress 2024-06-17 12:13:25 +02:00
Zhu Jiekun
9c3dabccac
docs: [vmagent] Add CHANGELOG for Statsd support in v1.102.0-rc1 (#6494)
### Describe Your Changes
Add CHANGELOG for Statsd support in v1.102.0-rc1
- CHANGELOG is missing for #5053.
- It would be better to include it so that users can be aware of when
this feature is released, rather than attempting to use it with lower
version VM components. See discussion
[here](https://victoriametrics.slack.com/archives/CGZF1H6L9/p1718462859065049?thread_ts=1718451117.668789&cid=CGZF1H6L9).

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-06-17 12:11:50 +02:00
Fred Navruzov
73fbce30bd
docs/vmanomaly - add config example for min_dev arg (#6490)
### Describe Your Changes

Added config example for `min_dev_from_expected` arg; also, small
styling fixes and alignments

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-06-17 12:11:50 +02:00
hagen1778
43c53271bf
docs: update wording for 6e395048d3
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit da4fbf61a4)
2024-06-14 16:31:42 +02:00
Hui Wang
5ecf439078
app/vmselect: fix the way of counting raw samples in single query (#6464)
The limit is specified with command-line flag
`-search.maxSamplesPerQuery`.
Previously, samples might be over-counted and query can't be fixed by
reducing time range.
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5851

(cherry picked from commit 6e395048d3)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-14 16:24:31 +02:00
Andrii Chubatiuk
8ca1813bd2
lib/flagutil: use month limit for duration flag for parsed duration assessment (#6486)
use maxMonths limit for parsed duration flag value

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6330

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit faf67aa8b5)
2024-06-14 15:21:32 +02:00
jackyin
f69495cd5f
app/vmalert: fix VMAlert oauth2 error (#6478)
Properly set ClientSecret param for notifier.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6471

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5223981fed)
2024-06-14 15:21:30 +02:00
hagen1778
64df1383e7
app/vmalert: properly configure authentication with S3 when -s3.configFilePath is specified.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-14 14:22:35 +02:00
Andrii Chubatiuk
abc233a902
lib/backup/s3remote: fixed credsFilePath flag (#6488)
properly use credsFilePath flag value

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6353

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit e678a9aa51)
2024-06-14 14:14:58 +02:00
Zakhar Bessarab
a91a2b97ed
docs/guides/otel-guide: update sentence to make it less obscure (#6483)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit f11261dc44)
2024-06-14 14:14:56 +02:00
Andrii Chubatiuk
779436bd9c
app/vmalert: fixed path prefixes for system routes (#6435)
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6433

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit eea361defb)
2024-06-14 14:14:54 +02:00
Artem Navoiev
6c587eefbf
docs: add more redirects
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit d1baa41b6a)
2024-06-14 13:26:47 +02:00
Fred Navruzov
455d761de9
docs/vmanomaly: fix-typos-in-v1.13-preset (#6484)
### Describe Your Changes

small fix of typos in v1.13 presets (vmanomaly docs)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 1cf28598c6)
2024-06-14 13:26:44 +02:00
Roman Khavronenko
5df50e5645
lib/streamaggr: prevent rate_sum and rate_avg from producing NaNs (#6482)
### Describe Your Changes

* check if `lastValue` was seen at least twice with different
timestamps. Otherwise, the difference between last timestamp and
previous timestamp could be `0` and will result into `NaN` calculation
* check if there items left in lastValue map after staleness cleanup.
Otherwise, `rate_avg` could have produce `NaN` result.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 51d19485bb)
2024-06-14 13:26:42 +02:00
Github Actions
5683610bf8
Automatic update Grafana datasource docs from VictoriaMetrics/grafana-datasource@bc562ef (#6481)
(cherry picked from commit 2d575cd27a)
2024-06-14 13:26:40 +02:00
Fred Navruzov
aa85b772bb
docs/vmanomaly: update preset guide and cross-links (#6480)
### Describe Your Changes

Updates and cross-references for `preset` mode of `vmanomaly` [in
docs](https://docs.victoriametrics.com/anomaly-detection/presets/)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 2184c0c730)
2024-06-14 13:26:37 +02:00
Daria Karavaieva
ae39b02496
docs/vmanomaly: preset guide (#6151)
- added instructions of how to run presets
- description of Node Exporter indicators covered

(cherry picked from commit c32b8d39dd)
2024-06-14 13:26:29 +02:00
Aliaksandr Valialkin
b455d356fc
docs/CONTRIBUTING.md: add checks about backwards compatibility and historical conventions 2024-06-13 17:08:54 +02:00
Aliaksandr Valialkin
7af2b7905f
docs/CONTRIBUTING.md: various typo fixes 2024-06-13 17:02:05 +02:00
Zakhar Bessarab
ac16d1dc1b
lib/promscrape: increase default value for promscrape.maxDroppedTargets to 10_000 (#6459)
### Describe Your Changes
This limit can be increased since after
4513893ead
tracking of dropped targets uses much less memory per entry.

See:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6381#issuecomment-2156708228

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

(cherry picked from commit 34071ac660)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-13 09:28:16 +02:00
LHHDZ
41e4135371
app/vmauth: fix discovering backend IPs when url_prefix contains hostname with srv+ prefix (#6401)
This change fixes the following panic:
```
2024-06-04T11:16:52.899Z        warn    app/vmauth/auth_config.go:353   cannot discover backend SRV records for http://srv+localhost:8080: lookup localhost on 10.100.10.4:53: server misbehaving; use it literally
panic: runtime error: integer divide by zero

goroutine 9 [running]:
github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver.handlerWrapper.func1()
        /Users/lhhdz/wd/projects/go/VictoriaMetrics/lib/httpserver/httpserver.go:291 +0x58
panic({0x103115100?, 0x10338d700?})
        /Users/lhhdz/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.3.darwin-arm64/src/runtime/panic.go:770 +0x124
main.getLeastLoadedBackendURL({0x0?, 0x22?, 0x1400014757b?}, 0x1400013c120?)
        /Users/lhhdz/wd/projects/go/VictoriaMetrics/app/vmauth/auth_config.go:473 +0x210
main.(*URLPrefix).getBackendURL(0x140000aa080)
        /Users/lhhdz/wd/projects/go/VictoriaMetrics/app/vmauth/auth_config.go:312 +0xb8
```

---------

Co-authored-by: Haley Wang <haley@victoriametrics.com>
2024-06-12 11:47:44 +02:00
Aliaksandr Valialkin
27efb9499b
docs/VictoriaLogs/logsql-examples.md: added an example on how to search for a phrase with whitespace 2024-06-11 19:13:47 +02:00
Aliaksandr Valialkin
9135b404d9
lib/logstorage: work-in-progress 2024-06-11 17:51:01 +02:00
Fred Navruzov
ea90d91a30
docs/vmanomaly - fix 404 img links & remove outdated models (#6460)
### Describe Your Changes

Fix 404 relative img links for v1.13.0 update of vmanomaly docs

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 9a355c3512)
2024-06-11 17:05:10 +02:00
Fred Navruzov
0219d34b21
docs/vmanomaly - release 1.13.0 preparation (#6436)
### Describe Your Changes

[vmanomaly docs](https://docs.victoriametrics.com/anomaly-detection/)
update for changes, introduced in v1.13.0

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 1feb5d04d7)
2024-06-11 17:05:07 +02:00
Nikolay
ae76794a19
follow-up docs update after 77f22fdb8d (#6454)
Signed-off-by: f41gh7 <nik@victoriametrics.com>
(cherry picked from commit 33d07e915f)
2024-06-11 17:05:05 +02:00
Github Actions
c96b334b3e
Automatic update operator docs from VictoriaMetrics/operator@310a860 (#6449)
(cherry picked from commit 020d3f3d88)
2024-06-11 17:05:01 +02:00
Lapo Luchini
dd311aa76f
Small fix in MetricsQL.md (#6452)
### Describe Your Changes

value→valid

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Lapo Luchini <lapo@lapo.it>
(cherry picked from commit 14f586238f)
2024-06-11 17:04:59 +02:00
Aliaksandr Valialkin
37a8cc0b12
lib/logstorage: work-in-progress 2024-06-10 18:42:31 +02:00
Yury Molodov
2300e30ff3
vmui/logs: add markdown support (#6292)
Add support for markdown format and emoji for the `_msg` field in the
"Group" view.
Add markdown rendering toggle. Disabled by default. Value is stored in
`localStorage`.
2024-06-10 16:39:25 +02:00
hagen1778
fdf0a936f0
vmctl: rm --vm-disable-progress-bar flag
It is better to remove deprecated flag completely, so vmctl will
fail if this flag is used and user can immediately fix the issue.

Before, flag was ignored and it is worse then fail fast.

follow-up after 8b46bb0c41 (diff-2bfab3db5cc1baf4c6d3ff6b19901926e3bdf4411ec685dac973e5fcff1c723b)

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 8d95522529)
2024-06-10 14:05:58 +02:00
Nikolay
0ce7f38e1c
app/vmauth: adds idleConnTimeout flag, retry trivial errors (#6388)
* adds idleConnTimeout flag, which must reduce probability of `broken
pipe` and `connection reset` errors.
* one-time retry trivial network requests for the same backend

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit d44058bcd6)
2024-06-10 12:41:51 +02:00
Dmytro Kozlov
a4bdc14bc5
vmctl: disable progress bar for prometheus snapshot migrations (#6385)
* deprecate `--vm-disable-progress-bar` in favour of `--disable-progress-bar`
* new `--disable-progress-bar` consistently disables usage of progress bar
for all migration modes.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6367

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 8b46bb0c41)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-10 12:41:44 +02:00
Hui Wang
028a80613f
lib/httpserver: allow reloadAuthKey and configAuthKey to override htt… (#6338)
…pAuth.*

address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6329,
makes `reloadAuthKey`, `configAuthKey`, `flagsAuthKey`, `pprofAuthKey`
behavior the same way,
but keys like `-snapshotAuthKey`, `-forceMergeAuthKey` are still
protected by httpAuth.*. All the available key are listed in
https://docs.victoriametrics.com/single-server-victoriametrics/#security.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 61dce6f2a1)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-10 12:41:29 +02:00
Andrii Chubatiuk
6fd314d8ba
vmagent: updated dashboard and alert for stream aggregation (#6427)
### Describe Your Changes

Added streaming aggregation section to vmagent dashboards
Added alert for streaming aggregation and deduplication flush timeouts
Removed deprecated compose versions from compose files

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 2da45a8368)
2024-06-10 12:37:22 +02:00
Artem Navoiev
d6b56a1460
docs victorialogs: add alias for quickstart for VL to be consistent with other quickstarts
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 318e9e9de0)
2024-06-10 12:37:19 +02:00
Artem Navoiev
a383e00e48
add alias for vmalert-tools
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 8f254232bb)
2024-06-10 12:37:16 +02:00
Aliaksandr Valialkin
792dd697f4
docs/CHANGELOG.md: document v1.93.15 LTS release
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.15
2024-06-07 23:43:20 +02:00
Aliaksandr Valialkin
f61bf790a4
docs/LTS-releases.md: update v1.97.4 LTS release to v1.97.5
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.5
2024-06-07 20:16:55 +02:00
Aliaksandr Valialkin
37ca3a2b0c
docs/CHANGELOG.md: add changelog for v1.97.5 LTS release
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.5
2024-06-07 20:03:35 +02:00
Aliaksandr Valialkin
3f883559e2
docs/CHANGELOG.md: cut v1.202.0-rc1 release 2024-06-07 16:53:48 +02:00
Aliaksandr Valialkin
a38759f66a
docs: run make docs-sync after c740a8042e
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6362
2024-06-07 16:45:53 +02:00
Aliaksandr Valialkin
32aa0751a1
lib/streamaggr: follow-up for 7cb894a777
- Use bytesutil.InternString() instead of strings.Clone() for inputKey and outputKey in aggregatorpushSamples().
  This should reduce string allocation rate, since strings can be re-used between aggrState flushes.
- Reduce memory allocations at dedupAggrShard by storing dedupAggrSample by value in the active series map.
- Remove duplicate call to bytesutil.InternBytes() at Deduplicator, since it is already called inside dedupAggr.pushSamples().
- Add missing string interning at rateAggrState.pushSamples().

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6402
2024-06-07 16:35:53 +02:00
Roman Khavronenko
78121642df
lib/streamaggr: reduce number of inuse objects (#6402)
The main change is getting rid of interning of sample key. It was
discovered that for cases with many unique time series aggregated by
vmagent interned keys could grow up to hundreds of millions of objects.
This has negative impact on the following aspects:
1. It slows down garbage collection cycles, as GC has to scan all inuse
objects periodically. The higher is the number of inuse objects, the
longer it takes/the more CPU it takes.
2. It slows down the hot path of samples aggregation where each key
needs to be looked up in the map first.

The change makes code more fragile, but suppose to provide performance
optimization for heavy-loaded vmagents with stream aggregation enabled.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-06-07 16:35:52 +02:00
Roman Khavronenko
fae589bb83
lib/promrelabel: speedup label match by __name__ (#6432)
The change adds a fastpath for `equalValue` comparisons against
`__name__` label by avoiding calls to `toCanonicalLabelName` func. This
speedups matches by metric name like `'foo'`. See bench stats below:
```
benchcmp old.txt new.txt

benchmark                                           old ns/op     new ns/op     delta
BenchmarkIfExpression/equal_label:_last-10          35.6          35.1          -1.18%
BenchmarkIfExpression/equal_label:_middle-10        18.3          17.3          -5.41%
BenchmarkIfExpression/equal_label:_first-10         1.20          1.24          +2.74%
BenchmarkIfExpression/equal___name__:_last-10       10.1          4.96          -50.75%
BenchmarkIfExpression/equal___name__:_middle-10     5.79          3.16          -45.41%
BenchmarkIfExpression/equal___name__:_first-10      1.17          1.05          -9.76%
```

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-07 16:35:52 +02:00
Andrii Chubatiuk
93cd08f15f
lib/streamaggr: metrics to track dropped, nan samples and samples lag (#6358)
### Describe Your Changes

Added streamaggr metrics to:
 - `vm_streamaggr_samples_lag_seconds` - samples lag
- `vm_streamaggr_ignored_samples_total{reason="nan"}` - ignored NaN
samples
- `vm_streamaggr_ignored_samples_total{reason="too_old"}` - ignored old
samples

(cherry picked from commit 185fac03b3)
2024-06-06 19:22:45 +02:00
Aliaksandr Valialkin
53382ae837
lib/logstorage: work-in-progress 2024-06-06 12:27:11 +02:00
Aliaksandr Valialkin
6d986f731b
docs/VictoriaLogs/CHANGELOG.md: document the a68c2c0f17 2024-06-06 12:21:47 +02:00
Github Actions
2e3c039113
Automatic update operator docs from VictoriaMetrics/operator@32f7674 (#6414) 2024-06-06 12:12:58 +02:00
Aliaksandr Valialkin
c6b6900152
docs/VictoriaLogs/LogsQL.md: typo fix: replace from _json with from _msg where appropriate 2024-06-05 10:15:56 +02:00
Aliaksandr Valialkin
be4aefd133
docs/VictoriaLogs/LogsQL.md: substitute TAG with APP-NAME[PROCID] in rfc3164 syslog message format at unpack_syslog pipe docs 2024-06-05 10:13:58 +02:00
Aliaksandr Valialkin
a200fb433a
lib/logstorage: allow using eval keyword instead of math keyword in math pipe 2024-06-05 10:08:08 +02:00
Aliaksandr Valialkin
f8336783c8
docs/VictoriaLogs/LogsQL.md: typo fix: substitute "'" with "" in front of John` 2024-06-05 09:51:29 +02:00
Aliaksandr Valialkin
9e79a632c1
docs/VictoriaLogs/LogsQL.md: typo fix: its -> it 2024-06-05 09:46:35 +02:00
Zhu Jiekun
024ab786ce
docs: [deployment] update CHANGELOG.md to include go version change 1.22.4 (#6412)
### Describe Your Changes
Update CHANGELOG to include go version change introduced in
43cf221681

Also see:
- https://go.dev/doc/devel/release
-
https://github.com/golang/go/issues?q=milestone%3AGo1.22.4+label%3ACherryPickApproved

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-06-05 09:46:34 +02:00
Aliaksandr Valialkin
b45e466a1b
lib/logstorage: work-in-progress 2024-06-05 03:18:25 +02:00
hagen1778
607007add9
docs: add technical articles by @jiekun
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-05 03:16:41 +02:00
pludov
2efd97a63c
lib/fs: support NFS implementations that return EEXIST instead of ENOTEMPTY (#6398)
### Describe Your Changes

Fix for issue #6396: according to rmdir manpage, ENOTEMPTY and EEXIST
should be treated equally

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6396

### Checklist

The following checks are **mandatory**:

- [x ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Ludovic Pollet <ludovic.pollet@exfo.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 3ddae77c63)
2024-06-04 15:30:48 +02:00
Aliaksandr Valialkin
1a1ee87b46
docs/VictoriaLogs/LogsQL.md: mention unpack_syslog pipe in the Transformations section 2024-06-04 03:17:14 +02:00
Aliaksandr Valialkin
7d31574071
docs/VictoriaLogs/Roadmap.md: remove LogsQL transformations from roadmap, since they are already implemented 2024-06-04 02:21:23 +02:00
Aliaksandr Valialkin
39d238592c
docs/VictoriaLogs/logsql-examples.md: typo fix 2024-06-04 02:17:59 +02:00
Aliaksandr Valialkin
53928a44d1
docs/VictoriaLogs/logsql-examples.md: typo fix 2024-06-04 02:10:56 +02:00
Aliaksandr Valialkin
b7b3a9e9a3
lib/logstorage: work-in-progress 2024-06-04 01:50:55 +02:00
hagen1778
d6096a477f
app/vmalert: rm extra response for unsupported path
Unsupported path is already handled by `lib/httpserver`.
This prevents from misleading errors in logs caused by double-writing response headers.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit a5f81f67fd)
2024-06-03 12:53:38 +02:00
Hui Wang
39286afae3
vmalert-tool: fix float values template in input_series (#6395)
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6391

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit e3e40cb848)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-03 11:53:58 +02:00
Zakhar Bessarab
ff1bf76237
deployment/docker: add scratch-based images (#6386)
### Describe Your Changes

Scratch based images will be using a separate tag: "(version)-scratch"
and will be built for the same architecture as regular images.
This is useful for environments with higher security standards. In this
case using alpine as base layer requires updating images more frequently
in order to get the latest updates for the base image, even in case the
user did not need to update VictoriaMetrics version.

Tested that scratch images work for:
- vmagent - enterprise with kafka and opensource
- cluster
- single-node

No issues observed so far.

cc: @tenmozes

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 7dc9124ba7)
2024-06-03 11:53:45 +02:00
Github Actions
32a9c62481
Automatic update operator docs from VictoriaMetrics/operator@d936bb7 (#6378)
(cherry picked from commit 53422797a7)
2024-06-03 11:53:39 +02:00
hagen1778
73c9981335
chore: follow-up after c740a8042e
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6d8e02f278)
2024-06-03 11:53:37 +02:00
Arkadii Yakovets
a6655322b1
docs: fix docs/ and README.md spelling errors (#6362)
Fixes `docs/` and `README.md` typos and errors.

Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>

(cherry picked from commit c740a8042e)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-03 11:53:33 +02:00
Nikolay
908a50f79d
app/vmalert: adds idleConnTimeout flags and retry trivial network errors (#6382)
* "*.idleConnTimeout" flags must reduce probability of `write: broken
pipe` and `read: connection reset by peer` errors Those errors may occur
if remote server closes TCP socket for connection, while it's still
exist at client.
* single time retries for `write: broken pipe` and `read: connection
reset by peer` must handle a case for incorrectly configured timeouts at
middleware proxies, mitigate minor network issues.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5661

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit b97916276f)
2024-06-03 11:52:58 +02:00
Aliaksandr Valialkin
b994e64567
docs/VictoriaLogs: typo fixes 2024-05-30 16:35:25 +02:00
Aliaksandr Valialkin
2fd8cf15c4
deployment/docker: update VictoriaLogs Docker image from v0.14.0-victorialogs to v0.15.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.15.0-victorialogs
2024-05-30 16:27:53 +02:00
Aliaksandr Valialkin
540bbb63a2
lib/logstorage: work-in-progress 2024-05-30 16:19:36 +02:00
Zakhar Bessarab
b08ee34061
docs/metricsql: fix typo in link (#6384)
### Describe Your Changes

Fix typo in link.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-05-30 16:17:20 +02:00
Fred Navruzov
b200c4263a
docs/vmanomaly: improve FAQ (#6369)
### Describe Your Change
More explicit [vmanomaly
FAQ](https://docs.victoriametrics.com/anomaly-detection/faq/index.html),
based on common Q&A from recent communications with users


### Checklist

The following checks are **mandatory**:

- ✔️ My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-05-30 16:16:56 +02:00
Dmytro Kozlov
617ec1fbec
docs: update managed create deployment description (#6376)
### Describe Your Changes

Updated description how to create deployment in the cloud soulution
2024-05-30 16:16:18 +02:00
yumeiyin
95b8cf76f8
chore: remove redundant words (#6348)
(cherry picked from commit 9289c7512d)
2024-05-29 14:37:04 +02:00
Roman Khavronenko
189af53142
lib/storage: filter deleted label names and values from `/api/v1/labe… (#6342)
…ls` and `/api/v1/label/.../values`

Check for deleted metrics when `match[]` filter matches small number of
time series (optimized path).

The issue was introduced
[v1.81.0](https://docs.victoriametrics.com/changelog_2022/#v1810).

Related issue
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6300 Updates
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2978

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit b984f4672e)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-29 14:37:00 +02:00
Github Actions
9d301809b6
Automatic update operator docs from VictoriaMetrics/operator@cf753a6 (#6373)
(cherry picked from commit 14ba7b237d)
2024-05-29 14:04:24 +02:00
Aliaksandr Valialkin
e83fd4a117
lib/logstorage: work-in-progress 2024-05-29 01:52:34 +02:00
Aliaksandr Valialkin
6ddc618731
docs/VictoriaLogs: typo fixes 2024-05-28 22:48:57 +02:00
Aliaksandr Valialkin
2395c7e65e
deployment/docker: update VictoriaLogs image from v0.12.1-victorialogs to v0.13.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.13.0-victorialogs
2024-05-28 22:16:30 +02:00
Alexander Marshalov
7d532a31fb
Update base Alpine image to 3.20.0 to avoid security risks (#6370)
fixes: CVE-2023-42366, CVE-2023-42363, CVE-2024-4603, CVE-2024-2511,
CVE-2024-24788, CVE-2024-24787
2024-05-28 22:16:29 +02:00
Aliaksandr Valialkin
79c03fc35f
lib/logstorage: work-in-progress 2024-05-28 19:29:50 +02:00
Aliaksandr Valialkin
d429c0b83e
deployment: update VictoriaLogs image from v0.12.0-victorialogs to v0.12.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.12.1-victorialogs
2024-05-26 02:08:52 +02:00
Aliaksandr Valialkin
afa597ce2a
lib/logstorage: work-in-progress 2024-05-26 01:56:12 +02:00
Aliaksandr Valialkin
6427b3c3c0
lib/logstorage: work-in-progress 2024-05-25 22:59:21 +02:00
Aliaksandr Valialkin
f02c5cb79a
docs/VictoriaLogs/CHANGELOG.md: cut v0.12.0-victorialogs 2024-05-25 21:40:20 +02:00
Aliaksandr Valialkin
03fe4c8963
lib/logstorage: work-in-progress 2024-05-25 21:36:24 +02:00
Aliaksandr Valialkin
3152df2bce
lib/logstorage: work-in-progress 2024-05-25 00:31:55 +02:00
hagen1778
63b83d62e8
deployment/alerts: add new alerting rules TooLongLabelValues and TooLongLabelNames to notify about truncation of label values or names respectively.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 1be1e9a7a4)
2024-05-24 16:08:41 +02:00
Nikolay
5025ede7bc
lib/mergeset: adds tracking for indexdb records drop (#6297)
It allows to create alert for possible item drops at indexdb. It may
happen, if ingested metric size exceeds max indexdb item size.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 69d244e6fb)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-24 16:08:34 +02:00
Zakhar Bessarab
85eee7de0a
app/vmselect: update flag description (#6347)
Update wording to highlight that cache is not persistent if flag is
value is empty. Previously, it was not clear if cache is not used at all
or just not persistent.
2024-05-24 15:58:54 +02:00
Aliaksandr Valialkin
db48642302
deployment: update Docker image tag for VictoriaLogs from v0.8.0-victorialogs to v0.10.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.10.0-victorialogs
2024-05-24 03:16:06 +02:00
Aliaksandr Valialkin
e5ca4fff76
docs/VictoriaLogs/CHANGELOG.md: cut v0.10.0-victorialogs 2024-05-24 03:08:29 +02:00
Aliaksandr Valialkin
7a2a2f173e
lib/logstorage: work-in-progress 2024-05-24 03:07:07 +02:00
Github Actions
8025cfdb73
Automatic update operator docs from VictoriaMetrics/operator@4b059a4 (#6332) 2024-05-24 03:05:39 +02:00
hagen1778
d691d515f1
docs: fix changelog formatting
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-24 03:05:39 +02:00
Aliaksandr Valialkin
ae8b2bcf2e
app/vlselect: fix loading web UI 2024-05-22 23:25:00 +02:00
Aliaksandr Valialkin
2befafa264
docs/VictoriaLogs/LogsQL.md: mention that it is better from performance PoV parsing JSON and logfmt logs before the ingestion into VictoriaLogs 2024-05-22 23:04:48 +02:00
Aliaksandr Valialkin
181b5c5437
docs/VictoriaLogs/LogsQL.md: typo fixes 2024-05-22 22:24:56 +02:00
Aliaksandr Valialkin
802d7edd0e
docs/VictoriaLogs/CHANGELOG.md: cut v0.8.0-victorialogs 2024-05-22 22:07:38 +02:00
Nikolay
dfbd2f8ff7
lib/storage: change default value for maxLabelValueLen to 1024 (#6313)
* It must reduce memory usage for misbehaving clients. Since
VictoriaMetrics stores sparse index inmemory.
* Reduce disk space usage for indexdb.
* Prevent possible indexDB items drops.
* It may trigger slow insert and new timeseries registration due to
default value for flag change

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6176

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-05-22 21:55:21 +02:00
Alexander Marshalov
0b70c4c1f1
[vmlogs] fixed time parsing with millisecond precision time (#6293) (#6295)
fix for #6293

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-05-22 21:54:50 +02:00
Yury Molodov
252a196405
vmui/logs: fix parsing long _msg values (#6310)
This PR fixes an issue where parsing long `_msg` values caused errors,
resulting in some log records not being displayed.

The error occurred due to partial processing of strings. In some cases,
a long record could be split into multiple chunks, causing only part of
the record to be processed instead of the entire entry.

#6281

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-05-22 21:44:49 +02:00
Aliaksandr Valialkin
04d0dd2542
lib/logstorage: work-in-progress 2024-05-22 21:01:28 +02:00
hagen1778
89819f2054
dashboards: use $__interval variable for offsets and look-behind windows in annotations
This should improve precision of `restarts` and `version change` annotations when
 zooming-in/zooming-out on the dashboards.

 The change also makes `restarts` dashboard visible on the panels, so user can disable it from
 displaying if needed. This could be useful when restarts overlap with version change events.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 9dd9b4442f)
2024-05-22 16:40:08 +02:00
Roman Khavronenko
f3e893f699
lib/backup: add -s3TLSInsecureSkipVerify command-line flag (#6318)
* The new flag can be used for for skipping TLS certificates
verification when connecting to S3 endpoint. Affects vmbackup,
vmrestore, vmbackupmanager.

* replace deprecated `EndpointResolver` with `BaseEndpoint`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1056

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit ac836bcf6c)
2024-05-22 16:40:06 +02:00
Daria Karavaieva
aea042bc9d
docs/vmanomaly: backtest parameter fix (#6327)
### Fix backtest parameters

Fix date parameters names in vmanomaly backtest scheduler component
documentation.

(cherry picked from commit c40f355496)
2024-05-22 16:40:01 +02:00
Hui Wang
5b8c3fc9d0
app/vmalert: support DNS SRV record in -remoteWrite.url (#6299)
part of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6053,
supports [DNS SRV](https://en.wikipedia.org/wiki/SRV_record) address in
`-remoteWrite.url` command-line option.

(cherry picked from commit d7b5062917)
2024-05-22 10:53:22 +02:00
Hui Wang
0b9e59e67e
vmalert: speed up reloading rules from object storage by verifying ob… (#755)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6210
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 974b7783ee)
2024-05-21 16:37:27 +02:00
Github Actions
26306fec80
Automatic update operator docs from VictoriaMetrics/operator@84833cc (#6316)
(cherry picked from commit 23619a3adf)
2024-05-21 16:37:25 +02:00
hagen1778
0dd3fec2b7
deployment/dashboards: fix AnnotationQueryRunner error in Grafana
The error appears when executing annotations query against Prometheus backend
because the query itself hasn't specified look-behind window (which is allowed
in VictoriaMetrics query engine).

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6309
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit c746ba154d)
2024-05-21 16:37:23 +02:00
Artem Navoiev
f4743dcef2
docs: victorialogs mention that our bench suite can be run against Loki as well
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit cee4bfebd7)
2024-05-21 16:37:21 +02:00
Yury Molodov
33eaa18c14
vmui: fix URL params handling for navigation (#6284)
This PR fixes the handling of URL parameters to ensure correct browser
navigation using the back and forward buttons.

#6126

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5516#issuecomment-1867507232
(cherry picked from commit f14497f1cd)
2024-05-20 14:46:41 +02:00
Yury Molodov
97c3c946a7
vmui/logs: change time range to start and end query args (#6296)
change time range limitation from `_time` in the expression to `start`
and `end` query args.

(cherry picked from commit a6a599cbdc)
2024-05-20 14:46:39 +02:00
Roman Khavronenko
3e8b5e74d5
lib/streamaggr: skip empty aggregators (#6307)
Prevent excessive resource usage when stream aggregation config file
contains no matchers by prevent pushing data into Aggregators object.
Before this change a lot of extra work was invoked without reason.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 7ce052b32d)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-20 14:46:36 +02:00
Github Actions
d86d768aec
Automatic update operator docs from VictoriaMetrics/operator@c8534a5 (#6305) 2024-05-20 11:04:36 +02:00
Aliaksandr Valialkin
10dd940c54
docs/VictoriaLogs: typo fixes 2024-05-20 04:48:42 +02:00
Aliaksandr Valialkin
18982e1ab7
docs/VictoriaLogs/LogsQL.md: typo fix 2024-05-20 04:27:44 +02:00
Aliaksandr Valialkin
b02d085a52
docs/VictoriaLogs/LogsQL.md: typo fix 2024-05-20 04:24:34 +02:00
Aliaksandr Valialkin
400f43dd48
deployment: update VictoriaLogs from v0.6.0-victorialogs to v0.8.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.8.0-victorialogs
2024-05-20 04:17:51 +02:00
Aliaksandr Valialkin
582e7d5439
lib/logstorage: work-in-progress 2024-05-20 04:09:15 +02:00
viperstars
ab78f3c89d
app/vmagent/remotewrite: skip sending empty block to downstream server (#6241)
Occasionally, vmagent sends empty blocks to downstream servers. If a
downstream server returns an unexpected response, vmagent gets stuck in
a retry loop. While vmagent handles 400 and 409 errors, there are
various prometheus remote write implementations that return different
error codes. For example, vector returns a 422 error. To mitigate the
risk of vmagent getting stuck in a retry loop, it is advisable to skip
sending empty blocks to downstream servers.

Co-authored-by: hao.peng <hao.peng@smartx.com>
Co-authored-by: Zhu Jiekun <jiekun.dev@gmail.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 3661373cc2)
2024-05-17 14:57:07 +02:00
Andrii Chubatiuk
23c7193ca4
docs: added information about difference between default_rollup and last_over_time (#6282)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6279
(cherry picked from commit ce3cfd720e)
2024-05-17 14:57:04 +02:00
Yury Molodov
5bfbfe6ad2
vmui: remove redundant requests on the Explore Cardinality page (#6263)
Remove redundant requests on the Explore Cardinality page.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6240
(cherry picked from commit be291c36f7)
2024-05-17 14:56:55 +02:00
Yury Molodov
0edef9105b
vmui: fix calendar display (#6255)
Fix the calendar display issue occurring with the `UTC+00:00` timezone

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6239
(cherry picked from commit 4ad577cc6f)
2024-05-17 14:56:53 +02:00
Andrii Chubatiuk
fe332c3419
app/vmagent: add global aggregator (#6268)
Add global stream aggregation for VMAgent

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5467
(cherry picked from commit f153f54d11)
2024-05-17 14:01:31 +02:00
Nikolay
ee4a94a371
follow-up for c6c5a5a186 (#6265)
* adds datadog extensions for statsd:
  - multiple packed values (v1.1)
  - additional types distribution, histogram

* adds type check and append metric type to the labels with special tag
name `__statsd_metric_type__`. It simplifies streaming aggregation
config.

* remove statsd support from cluster, since cluster doesn't support
stream aggregation.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit b2765c45d0)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-17 13:49:24 +02:00
Roman Khavronenko
a5c427bac4
app/vmalert/datasource: reduce number of allocations when parsing instant responses (#6272)
Allocations are reduced by implementing custom json parser via fastjson
lib.
The change also re-uses `promInstant` object in attempt to reduce number
of
allocations when parsing big responses, as usually happens with heavy
recording rules.

```
name                                old allocs/op  new allocs/op  delta
ParsePrometheusResponse/Instant-10     9.65k ± 0%     5.60k ± 0%   ~     (p=1.000 n=1+1)

```

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 4f0525852f)
2024-05-16 09:35:58 +02:00
Aliaksandr Valialkin
473c7a344c
docs/VictoriaLogs/CHANGELOG.md: cut v0.7.0-victorialogs
(cherry picked from commit 6fdba8599d)
2024-05-16 09:35:57 +02:00
Aliaksandr Valialkin
28626db066
lib/logstorage: work-in-progress
(cherry picked from commit 0aa19a2837)
2024-05-16 09:35:55 +02:00
Nikolay
2b2fdffd77
app/vmauth: explicitly unregister metrics set for auth config (#6252)
it's needed to remove Summary metric type from the global state of
metrics package. metrics package tracks each bucket of summary and
periodically swaps old buckets with new.

Simple set unregister is not enough to release memory used by Set

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6247
(cherry picked from commit 6a6e34ab8e)
2024-05-14 09:28:37 +02:00
Aliaksandr Valialkin
8af6c80bf2
docs/VictoriaLogs/CHANGELOG.md: cut v0.6.1-victorialogs 2024-05-14 03:07:02 +02:00
Aliaksandr Valialkin
b1ee7bca1a
lib/logstorage: work-in-progress 2024-05-14 03:06:02 +02:00
Aliaksandr Valialkin
f52275bbd7
lib/logstorage: work-in-progress
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6258
2024-05-14 01:49:58 +02:00
Hui Wang
ec56f4625e
storage: correctly apply -inmemoryDataFlushInterval when it's set t… (#6221)
…o minimum supported value 1s
pendingRowsFlushInterval was bumped to 2s in
73f0a805e2

(cherry picked from commit 4c80b17027)
2024-05-13 16:50:02 +02:00
Andrii Chubatiuk
d9cddf1ad8
lib/streamaggr: added rate and rate_avg output (#6243)
Added `rate` and `rate_avg` output
Resource usage is the same as for increase output, tested on a benchmark

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 9c3d44c8c9)
2024-05-13 16:49:39 +02:00
Andrii Chubatiuk
ec2273b247
app/vmagent: removed deprecated -remoteWrite.multitenantURL flag support (#6253)
Removed deprecated `-remoteWrite.multitenantURL` flag to simplify global
stream aggregation

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 680b8c25c8)
2024-05-13 16:49:33 +02:00
Aliaksandr Valialkin
3c645c4a79
deployment: update VictoriaLogs Docker image from v0.5.2-victorialogs to v0.6.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.6.0-victorialogs
2024-05-12 23:23:05 +02:00
Aliaksandr Valialkin
b24daf7712
docs/VictoriaLogs/LogsQL.md: cross-reference uniq pipe with uniq_values stats function 2024-05-12 16:48:12 +02:00
Aliaksandr Valialkin
3ba65c5804
docs/VictoriaLogs/CHANGELOG.md: cut v0.6.0-victorialogs 2024-05-12 16:38:09 +02:00
Aliaksandr Valialkin
147704aab0
lib/logstorage: initial implementation of pipes in LogsQL
See https://docs.victoriametrics.com/victorialogs/logsql/#pipes
2024-05-12 16:36:01 +02:00
Aliaksandr Valialkin
0d55ef3658
docs/vmauth.md: small fixes after proofreading 2024-05-12 12:35:32 +02:00
Aliaksandr Valialkin
536d87cd51
app/vmselect/promql: properly estimate the needed amounts of memory for executing aggregate function over rollup function in incremental mode
Incremental aggregation processes only GOMAXPROCS time series at a time, so its' memory usage doesn't depend
on the number of input time series.

The issue has been introduced in 5138eaeea0

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2024-05-12 10:14:27 +02:00
Aliaksandr Valialkin
65128ea814
README.md: mention that -tlsCertFile and -tlsKeyFile options aren't needed when automatic issuing of tls certificates is enabled with -tlsAutoCertHosts flag 2024-05-12 09:49:16 +02:00
Aliaksandr Valialkin
0c89adf589
docs/Single-server-VictoriaMetrics.md and docs/README.md: sync with README.md with make docs-sync after the commit c6c5a5a186 2024-05-12 09:41:39 +02:00
Roman Khavronenko
0bed453737
Feature allow configuring disableOnDiskQueue and dropSamplesOnOverload per url (#6248)
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html):
allow configuring `-remoteWrite.disableOnDiskQueue` and
`-remoteWrite.dropSamplesOnOverload` cmd-line flags per each
`-remoteWrite.url`. See this [pull
request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6065).
Thanks to @rbizos for implementaion!
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add
labels `path` and `url` to metrics
`vmagent_remotewrite_push_failures_total` and
`vmagent_remotewrite_samples_dropped_total`. Now number of failed pushes
and dropped samples can be tracked per `-remoteWrite.url`.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Raphael Bizos <r.bizos@criteo.com>
(cherry picked from commit 87fd400dfc)
2024-05-10 14:32:23 +02:00
Github Actions
ab5b233c3a
Automatic update operator docs from VictoriaMetrics/operator@0829591 (#6250)
(cherry picked from commit c87ce86d96)
2024-05-10 14:32:18 +02:00
Github Actions
ac33ef71ca
Automatic update operator docs from VictoriaMetrics/operator@4a51b37 (#6245)
(cherry picked from commit a8d0c1a62d)
2024-05-10 14:29:22 +02:00
Roman Khavronenko
7be6fcd8fd
lib/streamaggr: set correct suffix <output>_prometheus (#6228)
Set correct suffix `<output>_prometheus` for aggregation outputs
`increase_prometheus` and `total_prometheus`
Before, outputs `total` and `total_prometheus` or `increase` and
`increase_prometheus` had the same suffix.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 8a03e987cb)
2024-05-10 14:29:01 +02:00
Andrii Chubatiuk
f3d65ba902
streamaggr: made labels compressor shared (#6173)
Though labels compressor is quite resource intensive, each aggregator
and deduplicator instance has it's own compressor. Made it shared across
all aggregators to consume less resources while using multiple
aggregators.

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
(cherry picked from commit a9283e06a3)
2024-05-10 14:28:59 +02:00
Zhu Jiekun
139f909cdb
chore: [deployment] upgrade from go 1.22.2 to 1.22.3 to include security fixes (#6238)
### Describe Your Changes

upgrade from go 1.22.2 to 1.22.3 to include security fixes. Also see:
- https://go.dev/doc/devel/release
-
https://github.com/golang/go/issues?q=milestone%3AGo1.22.3+label%3ACherryPickApproved

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Jiekun <jiekun.dev@gmail.com>
(cherry picked from commit 02851d7800)
2024-05-10 14:28:56 +02:00
Zhu Jiekun
10063e98a6
feature: [vmagent] Add service discovery support for Vultr (#6068)
### Describe Your Changes
related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6041

#### Added
- Added service discovery support for Vultr.

#### Docs
- `CHANGELOG.md`, `sd_configs.md`, `vmagent.md` are updated.

#### Note
- Useful links:
- Vultr API:
https://www.vultr.com/api/#tag/instances/operation/list-instances
    - Vultr client SDK: https://github.com/vultr/govultr
- Prometheus SD:
https://github.com/prometheus/prometheus/tree/main/discovery/vultr

---
### Checklist

The following checks are mandatory:

- [X] I have read the [Contributing
Guidelines](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/CONTRIBUTING.md)
- [x] All commits are signed and include `Signed-off-by` line. Use `git
commit -s` to include `Signed-off-by` your commits. See this
[doc](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) about
how to sign your commits.
- [x] Tests are passing locally. Use `make test` to run all tests
locally.
- [x] Linting is passing locally. Use `make check-all` to run all
linters locally.

Further checks are optional for External Contributions:

- [X] Include a link to the GitHub issue in the commit message, if issue
exists.
- [x] Mention the change in the
[Changelog](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/CHANGELOG.md).
Explain what has changed and why. If there is a related issue or
documentation change - link them as well.

  Tips for writing a good changelog message::

* Write a human-readable changelog message that describes the problem
and solution.
* Include a link to the issue or pull request in your changelog message.
* Use specific language identifying the fix, such as an error message,
metric name, or flag name.
* Provide a link to the relevant documentation for any new features you
add or modify.

- [ ] After your pull request is merged, please add a message to the
issue with instructions for how to test the fix or try the feature you
added. Here is an
[example](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4048#issuecomment-1546453726)
- [x] Do not close the original issue before the change is released.
Please note, in some cases Github can automatically close the issue once
PR is merged. Re-open the issue in such case.
- [x] If the change somehow affects public interfaces (a new flag was
added or updated, or some behavior has changed) - add the corresponding
change to documentation.

Signed-off-by: Jiekun <jiekun.dev@gmail.com>
(cherry picked from commit 17e3d019d2)
2024-05-10 14:28:51 +02:00
Oleg
76af930e4a
Statsd protocol compatibility (#5053)
In this PR I added compatibility with [statsd
protocol](https://github.com/b/statsd_spec) with tags to be able to send
metrics directly from statsd clients to vmagent or directly to VM.
For example its compatible with
[statsd-instrument](https://github.com/Shopify/statsd-instrument) and
[dogstatsd-ruby](https://github.com/DataDog/dogstatsd-ruby) gems

Related issues: #5052, #206, #4600

(cherry picked from commit c6c5a5a186)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-10 14:27:31 +02:00
Github Actions
cc68484f95
Automatic update operator docs from VictoriaMetrics/operator@2789953 (#6237)
(cherry picked from commit 55c7dafb35)
2024-05-10 13:14:24 +02:00
Alexander Marshalov
2922b2a2e0
fix typo in scrape config examples (#6234)
(cherry picked from commit 3d4988ecf6)
2024-05-10 13:14:22 +02:00
Github Actions
bb667fd3f7
Automatic update operator docs from VictoriaMetrics/operator@6271553 (#6233)
(cherry picked from commit 134dcaef33)
2024-05-10 13:14:19 +02:00
hagen1778
33db552a39
docs: mention influxListenAddr in URLs format doc
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 2561a132ee)
2024-05-10 13:14:09 +02:00
Andrii Chubatiuk
e26b55db1e
app/vmagent/remotewrite: do not cleanup timeseries which are used in multiple remote write contexts (#6206)
When at least one remote write has deduplication configured it cleans up
timeseries while they can be in use by another remote write without
deduplication

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6205
---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 879771808b)
2024-05-06 12:10:45 +02:00
Yury Molodov
75af52c1d0
vmui: fix issue preventing first query trace expansion (#6197)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6186
(cherry picked from commit 046a4a5ecf)
2024-04-30 18:39:22 +02:00
hagen1778
798a446d5a
docs: mention actual version in update nodes
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit c0050beadc)
2024-04-30 18:38:20 +02:00
Hui Wang
abd29c15ab
docs: update vmalert and vmagent docs (#6207)
* restore and actualize doc section explaining duplicated labels error
* rm misleading comment about post-aggregation in stream aggregation

(cherry picked from commit e3c226cf92)
2024-04-30 10:30:19 +02:00
Github Actions
f22b2a0563
Automatic update operator docs from VictoriaMetrics/operator@8f025b3 (#6200)
Co-authored-by: Hui Wang <haley@victoriametrics.com>
(cherry picked from commit e908effd22)
2024-04-30 10:30:10 +02:00
hagen1778
d87c8757cf
dashboards: add new panel Concurrent selects to vmstorage row
The panel will show how many ongoing select queries are processed by vmstorage
and should help to identify resource bottlenecks. See panel description for more details.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit d386a68b59)
2024-04-30 10:30:08 +02:00
hagen1778
0d77a55961
deployment: update per-tenant-statistic dashboard to be compatible with Grafana 10
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 9e18724036)
2024-04-30 10:30:06 +02:00
hagen1778
a0698d92c3
deployment: update backupmanager dashboard to be compatible with Grafana 10
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5917ac003e)
2024-04-30 10:30:04 +02:00
hagen1778
40af0fa179
deployment: update operator dashboard to be compatible with Grafana 10
- Use TimeSeries panel instead of deprecated Graph
- Update panel styles
- Fix version panel

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f0c4d372bd)
2024-04-30 10:30:02 +02:00
hagen1778
8f48747802
dashboards: add Network Usage panel to Resource Usage row
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4478
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 8606b48ce5)
2024-04-30 10:29:58 +02:00
Dima Lazerka
d821e13c24
deployment/dashboards: properly show version for non-stable docker images (#6150)
re: .*-(?:tags|heads)-(.*)-(?:0|dirty)-.*

cases:
victoria-metrics-20240419-160209-heads-enterprise-single-node-0-g08f933ab0c
enterprise-single-node

victoria-metrics-20240201-133950-tags-v1.97.1-enterprise-0-g760a8733b
v1.97.1-enterprise

victoria-metrics-20240419-160209-heads-rotation-part-2-0-ge2367b6d1-dirty-848b54cd
rotation-part-2-0-ge2367b6d1

victoria-metrics-20240419-160209-heads-lts-1.93-enterprise-search-contention-0-g30ef4aad21-amd64
lts-1.93-enterprise-search-contention

victoria-metrics-20240425-150852-tags-v1.101.0-enterprise-0-g718138c64
v1.101.0-enterprise

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Dzmitry Lazerka <dlazerka@gmail.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 564463259a)
2024-04-30 10:29:56 +02:00
Zakhar Bessarab
88677f179c
dashboards/victoria-metrics-single: allow selecting multiple instance values (#5870)
Allowing to select multiple instance IPs makes it much easier to view
metrics for longer periods of time in dynamic environments such as
Kubernetes. In k8s update will also cause IP to change making it harder
to use dashboard to check the status.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5869

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6b493582da)
2024-04-30 10:29:54 +02:00
Github Actions
954abe78a3
Automatic update operator docs from VictoriaMetrics/operator@bbf847c (#6189)
(cherry picked from commit 3a669b4b87)
2024-04-30 10:29:29 +02:00
hagen1778
310d100ed7
docs/CHANGELOG.md: cut v1.101.0
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5334f0c2ce)
2024-04-25 16:25:38 +02:00
hagen1778
5b82f615ff
docs: add change for c0e4ccb7b5
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit a62551f773)
2024-04-25 16:02:57 +02:00
Hui Wang
7fdea4b31c
app/vmselect: implement cmd-line flags -search.disableImplicitConversions and -search.logImplicitConversions (#6180)
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4338
support disable or log [implicit
conversions](https://docs.victoriametrics.com/metricsql/#implicit-query-conversions)
for subquery with cmd-line flags `-search.disableImplicitConversion` and
`-search.logImplicitConversion`

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit dd0d2c77c8)
2024-04-25 13:08:05 +02:00
Yury Molodov
14c0c06526
vmui: improve error message for server response issues (#6177)
Updates error messages for better clarity and guidance on server
response issues.

(cherry picked from commit 57b7d16259)
2024-04-25 13:08:02 +02:00
Yury Molodov
669cbcb92e
vmui: trigger auto-suggestion at any cursor position (#6155)
- Implemented auto-suggestion triggers for mid-string cursor positions
in vmui.
- Improved the suggestion list positioning to appear directly beneath
the active text editing area.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5864
(cherry picked from commit 6193fa3dcf)
2024-04-25 13:08:00 +02:00
hagen1778
d4e901e212
Revert "docs: removed code-style highlighting for commanad-line flags of VM components (#6147)"
This reverts commit 9bedbcfa2f.

(cherry picked from commit 6aaf1768f4)
2024-04-25 13:07:58 +02:00
hagen1778
6f6e9e1e51
docs/changelog: mention downsampling fixes for ENT version of VM
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 2d9bbe1934)
2024-04-24 17:08:28 +02:00
hagen1778
59b3f21708
Revert "app/vmbackup: introduce new flag type URL (#6152)"
This reverts commit 029060af60.

(cherry picked from commit 679844feaf)
2024-04-24 17:08:26 +02:00
Roman Khavronenko
ff73b66182
app/vmbackup: introduce new flag type URL (#6152)
The new flag type is supposed to be used for specifying URL values which
could contain sensitive information such as auth tokens in GET params or
HTTP basic authentication.

The URL flag also allows loading its value from files if `file://`
prefix is specified. As example, the new flag type was used in
app/vmbackup as it requires specifying `authKey` param for making the
snapshot.

See related issue
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5973

Thanks to @wasim-nihal for initial implementation
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6060

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 029060af60)
2024-04-24 17:08:24 +02:00
Github Actions
ff0a1eb3e6
Automatic update operator docs from VictoriaMetrics/operator@6e1a876 (#6179)
(cherry picked from commit a14acd4fcc)
2024-04-24 17:08:21 +02:00
Artem Navoiev
4f07d32548
docs: vmalert remove new lines in configuration section
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 592a9fe9f2)
2024-04-24 17:08:19 +02:00
Artem Navoiev
f22cfdc616
docs: vmalert remove new lines in configuration section
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit aeba3f39db)
2024-04-24 17:08:18 +02:00
hagen1778
fcff6cc12a
docs/changelog: mention icnreased CPU usage in 1.100.1 release for ENT distributions
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 22cfab1ea2)
2024-04-24 17:08:13 +02:00
Github Actions
f2897fb859
Automatic update operator docs from VictoriaMetrics/operator@e343a8e (#6168)
(cherry picked from commit 301bd387d4)
2024-04-23 14:52:33 +02:00
Andrii Chubatiuk
29f9b8ef5d
deployment/docker: allow cross-platform building on arm64 platform (#6158)
Added x86_64 libraries to allow building cross-platform images on arm64

(cherry picked from commit 7958f38864)
2024-04-23 14:52:30 +02:00
Roman Khavronenko
2566e19306
app/vmalert: fix links with anchors in vmalert's UI (#6146)
Starting from v1.99.0 vmalert could ignore anchors pointing to specific
rule groups if `search` param was present in URL.
This change makes anchors compatible with `search` param in UI.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5f487c7090)
2024-04-22 15:05:23 +02:00
Github Actions
d1ee860d46
Automatic update operator docs from VictoriaMetrics/operator@7e2ba6f (#6165)
(cherry picked from commit 6dbb4c1671)
2024-04-22 15:05:17 +02:00
Denys Holius
cad2cb2b56
docs: removed code-style highlighting for commanad-line flags of VM components (#6147)
Using `sh` or `console` formatting doesn't do word-breaking on render. This makes flags description
harder to read, as users need to scroll the web page horizontally.
Removing the formatting renders the description with normal word-breaking.

(cherry picked from commit 9bedbcfa2f)
2024-04-22 15:04:49 +02:00
hagen1778
342290275e
app/streamaggr: follow-up after c0e4ccb7b5
* rm vmagent mentions from vminsert flags
* improve documentation wording, add links to related sections
* mention `ignore_first_intervals` in the stream aggr options
* update flags description
* add basic test for config parsing validation

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit bae3874e6a)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-22 14:39:23 +02:00
Andrii Chubatiuk
131367fb59
lib/streamaggr: add option to ignore first N aggregation intervals (#6137)
Stream aggregation may yield inaccurate results if it processes incomplete data.
This issue can arise when data is sourced from clients that maintain a queue of unsent data, such as Prometheus or vmagent.
 If the queue isn't fully cleared within the aggregation interval, only a portion of the time series may be included in that period, leading to distorted calculations.
To mitigate this we add an option to ignore first N aggregation intervals. It is expected, that client queues
will be cleared during the time while aggregation ignores first N intervals and all subsequent aggregations
will be correct.

(cherry picked from commit c0e4ccb7b5)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-22 14:34:36 +02:00
Aliaksandr Valialkin
ff122798b1
docs/MetricsQL.md: add links to raw samples chapter at https://docs.victoriametrics.com/keyconcepts/ 2024-04-21 00:22:03 +02:00
Aliaksandr Valialkin
946757fa5e
docs/CONTRIBUTING.md: small re-wording 2024-04-21 00:08:02 +02:00
Aliaksandr Valialkin
c254a889f7
docs: remove duplicate information regarding contribution guidelines to VictoriaMetrics
Substitute duplicate information with the link to https://docs.victoriametrics.com/contributing/ across all the docs.
2024-04-21 00:01:33 +02:00
Aliaksandr Valialkin
aa496d8827
docs/CONTRIBUTING.md: more formatting fixes and clarifications 2024-04-20 23:49:13 +02:00
Aliaksandr Valialkin
59648c880a
docs/CONTRIBUTING.md: small corrections after a402847eb6 2024-04-20 23:22:04 +02:00
Aliaksandr Valialkin
b16dbab137
Move CONTRIBUTING.md to docs/
It is better from visibility PoV if the CONTRIBUTING.md file is visible at https://docs.victoriametrics.com/contributing/ .
This page can be indexed by search engines and searched by our users later.
It is also easier to provide a link to this page now comparing to the old link, which is much harder to remember:
https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/CONTRIBUTING.md

While at it, syncrhonize docs/CONTRIBUTING.md with .github/PULL_REQUEST_TEMPLATE/pull_request_template.md ,
so they do not contain duplicate information, which can be outdated over time. Now all the relevant information
is located at docs/CONTRIBUTING.md, while .github/PULL_REQUEST_TEMPLATE/pull_request_template.md refers this doc.

This is a follow-up for c006db1798

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6040
2024-04-20 23:12:53 +02:00
Bartosz Fenski
b36661539f
typo (#6159)
s/infromation/information/
2024-04-20 20:49:16 +02:00
Github Actions
ac7746d7fa
Automatic update Grafana datasource docs from VictoriaMetrics/grafana-datasource@5b8a0ba (#6156) 2024-04-20 20:48:44 +02:00
Aliaksandr Valialkin
fba3c10ed1
app/vmselect/promql: add support for matching against multiple numeric constants via q == (c1,...,cN) and q != (c1,...,cN) syntax 2024-04-19 17:57:09 +02:00
Aliaksandr Valialkin
5504b6f2bd
docs/CHANGELOG.md: document v1.97.4 LTS release
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.4
2024-04-19 13:10:37 +02:00
Aliaksandr Valialkin
a40a900b43
docs/CHANGELOG.md: document v1.93.14 LTS release
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.14
2024-04-19 12:52:38 +02:00
Aliaksandr Valialkin
a674029111
docs/LTS-releases.md: change the latest v1.93.x LTS release from v1.93.13 to v1.93.14
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.14
2024-04-19 12:52:36 +02:00
Aliaksandr Valialkin
5b29be1f4d
app/vmagent/remotewrite: add support for replication additionally to sharding when both -remoteWrite.shardByURL and -remoteWrite.shardByURLReplicas=RF command-line flags are set
This allows setting up data replication among failure domains if the replication factor is smaller than the number of failure domains.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6054

See https://docs.victoriametrics.com/vmagent/#sharding-among-remote-storages
2024-04-19 11:37:04 +02:00
Github Actions
0fde6f4b8f
Automatic update operator docs from VictoriaMetrics/operator@3d7bcba (#6149) 2024-04-19 11:36:38 +02:00
Github Actions
66f3f90f74
Automatic update operator docs from VictoriaMetrics/operator@781dbba (#6143) 2024-04-19 11:35:58 +02:00
Github Actions
a0b1c95787
Automatic update operator docs from VictoriaMetrics/operator@731de81 (#6139)
Co-authored-by: Alexander Marshalov <_@marshalov.org>
2024-04-19 11:35:21 +02:00
Github Actions
3cb3a9c950
Automatic update operator docs from VictoriaMetrics/operator@7f6c6d8 (#6138) 2024-04-19 11:34:50 +02:00
Hui Wang
e0d47ab6af
vmalert: avoid blocking APIs when alerting rule uses template functio… (#6129)
* vmalert: avoid blocking APIs when alerting rule uses template function `query`

* app/vmalert: small refactoring

* simplify labels and templates expanding
* simplify `newAlert` interface
* fix `TestGroupStart` which mistakenly skipped annotations
and response labels check

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* reduce alerts lock time when restore

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-04-19 11:30:40 +02:00
Aliaksandr Valialkin
f3967737b0
docs/Cluster-VictoriaMetrics.md: typo fix after dadeb6620a: folling -> following 2024-04-19 00:24:52 +02:00
Aliaksandr Valialkin
90794e84bc
app/vmselect/netstorage: add support for the ability to set cross-group replication factor at vmselect
The cross-group replication factor can be set via `-globalReplicationFactor` command-line flag at vmselect.
In this case vmselect continues returning full responses if up to globalReplicationFactor-1 groups are unavailable.

See https://docs.victoriametrics.com/cluster-victoriametrics/#vmstorage-groups-at-vmselect for details.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6054
2024-04-19 00:16:04 +02:00
Aliaksandr Valialkin
e2bc49534b
all: replace old https://docs.victoriametrics.com/PerTenantStatistic.html url with the new one - https://docs.victoriametrics.com/pertenantstatistic/ 2024-04-18 03:36:23 +02:00
Aliaksandr Valialkin
4f1eb03441
all: replace old https://docs.victoriametrics.com/Quick-Start.html url with the new one - https://docs.victoriametrics.com/quick-start/ 2024-04-18 03:33:01 +02:00
Aliaksandr Valialkin
b1477aa2be
all: replace old https://docs.victoriametrics.com/scrape_config_examples.html url with the new one - https://docs.victoriametrics.com/scrape_config_examples/ 2024-04-18 03:29:54 +02:00
Aliaksandr Valialkin
87641fa7e7
all: replace old https://docs.victoriametrics.com/Troubleshooting.html url with the new one - https://docs.victoriametrics.com/troubleshooting/ 2024-04-18 03:27:18 +02:00
Aliaksandr Valialkin
0c87cbe338
all: replace old https://docs.victoriametrics.com/relabeling.html url with the new one - https://docs.victoriametrics.com/relabeling/ 2024-04-18 03:23:15 +02:00
Aliaksandr Valialkin
10f3540bc6
all: replace old https://docs.victoriametrics.com/Articles.html url with the new one - https://docs.victoriametrics.com/articles/ 2024-04-18 03:20:13 +02:00
Aliaksandr Valialkin
987c4a2da5
all: replace old https://docs.victoriametrics.com/CaseStudies.html url with the new one - https://docs.victoriametrics.com/casestudies/ 2024-04-18 03:16:54 +02:00
Aliaksandr Valialkin
b933001d2d
all: replace old https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html url with the new one - https://docs.victoriametrics.com/single-server-victoriametrics/ 2024-04-18 03:11:49 +02:00
Aliaksandr Valialkin
b0129fab30
all: replace old https://docs.victoriametrics.com/url-examples.html url with the new one - https://docs.victoriametrics.com/url-examples/ 2024-04-18 03:02:11 +02:00
Aliaksandr Valialkin
a21d1fcf57
all: replace old https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html url with the new one - https://docs.victoriametrics.com/cluster-victoriametrics/ 2024-04-18 02:56:28 +02:00
Aliaksandr Valialkin
819569beda
all: replace old https://docs.victoriametrics.com/FAQ.html url with the new one - https://docs.victoriametrics.com/faq/ 2024-04-18 02:46:30 +02:00
Aliaksandr Valialkin
ed5744f072
all: replace old https://docs.victoriametrics.com/CHANGELOG_YYYY.html url with the new one - https://docs.victoriametrics.com/changelog_YYYY/ 2024-04-18 02:42:16 +02:00
Aliaksandr Valialkin
9fa1e5d843
all: replace old https://docs.victoriametrics.com/CHANGELOG.html url with the new one - https://docs.victoriametrics.com/changelog/ 2024-04-18 02:39:09 +02:00
Aliaksandr Valialkin
baf5c8d6d0
all: replace old https://docs.victoriametrics.com/keyConcepts.html url with the new one - https://docs.victoriametrics.com/keyconcepts/ 2024-04-18 02:34:09 +02:00
Aliaksandr Valialkin
513e69c55e
all: replace old https://docs.victoriametrics.com/sd_configs.html url with the new one - https://docs.victoriametrics.com/sd_configs/ 2024-04-18 02:28:26 +02:00
Aliaksandr Valialkin
728bcf0585
all: replace old https://docs.victoriametrics.com/stream-aggregation.html url with the new one - https://docs.victoriametrics.com/stream-aggregation/ 2024-04-18 02:20:00 +02:00
Aliaksandr Valialkin
64938732e3
all: replace old https://docs.victoriametrics.com/MetricsQL.html url with the new one - https://docs.victoriametrics.com/metricsql/ 2024-04-18 02:15:33 +02:00
Aliaksandr Valialkin
2ec18a6634
all: replace old https://docs.victoriametrics.com/vmgateway.html url with the new one - https://docs.victoriametrics.com/vmgateway/ 2024-04-18 02:08:53 +02:00
Aliaksandr Valialkin
a501e237d2
all: replace old https://docs.victoriametrics.com/vmbackupmanager.html url with the new one - https://docs.victoriametrics.com/vmbackupmanager/ 2024-04-18 02:04:39 +02:00
Aliaksandr Valialkin
a85602bb2d
all: replace old https://docs.victoriametrics.com/vmrestore.html url with the new one - https://docs.victoriametrics.com/vmrestore/ 2024-04-18 02:01:18 +02:00
Aliaksandr Valialkin
6fe21520ce
all: replace old https://docs.victoriametrics.com/vmbackup.html url with the new one - https://docs.victoriametrics.com/vmbackup/ 2024-04-18 01:58:00 +02:00
Aliaksandr Valialkin
c7866f3e19
all: replace old https://docs.victoriametrics.com/vmctl.html url with the new one - https://docs.victoriametrics.com/vmctl/ 2024-04-18 01:54:16 +02:00
Aliaksandr Valialkin
728aceebfe
all: replace old https://docs.victoriametrics.com/vmauth.html url with the new one - https://docs.victoriametrics.com/vmauth/ 2024-04-18 01:50:16 +02:00
Aliaksandr Valialkin
a99005eff6
all: replace old https://docs.victoriametrics.com/vmalert.html url with the new one - https://docs.victoriametrics.com/vmalert/ 2024-04-18 01:44:54 +02:00
Aliaksandr Valialkin
0211a04a52
all: replace the outdated url https://docs.victoriametrics.com/vmagent.html with the new one - https://docs.victoriametrics.com/vmagent/ 2024-04-18 01:32:57 +02:00
Aliaksandr Valialkin
e615bcced7
lib/storage: improve performance for /api/v1/label/labelName/values when match[] contains only a single filter on labelName
This speeds up auto-suggestion for metric names in VMUI and Grafana, which use the following query in this case:

  /api/v1/label/__name__/values?match[]={__name__=~"*.some_value.*"}

When the user types `some_value` in the query input field.
2024-04-18 01:18:03 +02:00
Aliaksandr Valialkin
164032cd9b
lib/httpserver: add support for automatic issuing of TLS certificates via Lets Encrypt service
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5949
2024-04-17 23:53:51 +02:00
Aliaksandr Valialkin
30c96ba8d7
app/{vminsert,vmselect}: support for srv+addr scheme for specifying DNS SRV addresses at -storageNode flag
The new scheme is consistent with SRV urls introduced at b426d10847 and dc326f70b4

Deprecte the old scheme: `dns+srv:addr` by removing it from the docs.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6053
2024-04-17 23:15:05 +02:00
Aliaksandr Valialkin
9f33b3760d
docs/vmagent.md: typo fixes after dc326f70b4
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6053
2024-04-17 21:13:01 +02:00
Aliaksandr Valialkin
284d99e269
app/vmagent: support for DNS SRV urls at -remoteWrite.url, scrape target urls and service discovery urls
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6053
2024-04-17 20:56:23 +02:00
Aliaksandr Valialkin
e627810146
app/vmauth: add support for configuring backends via DNS SRV urls 2024-04-17 20:56:21 +02:00
Aliaksandr Valialkin
6cc80ab621
app/vmauth: add support for client TLS sertificates for backend requests over https
While at it, also add support for TLS ServerName for backend requests over https
2024-04-17 17:13:12 +02:00
Aliaksandr Valialkin
eb870d9de3
docs/vmauth.md: add Authorization and Routing chapters 2024-04-17 16:17:21 +02:00
Aliaksandr Valialkin
6cdf01297e
app/vmauth: follow-up for b155b20de4
- Use exact matching by default for the query arg value provided via arg=value syntax at src_query_args.
  Regex matching can be enabled by using =~ instead of = . For example, arg=~regex.
  This ensures that the exact matching works as expected without the need to escape special regex chars.

- Add helper functions for creating QueryArg, Header and Regex structs in tests.
  This improves maintainability of the tests.

- Remove url.QueryUnescape() call on the url in TestCreateTargetURLSuccess(), since this is bogus approach.
  The url.QueryUnescape() must be applied to individual query args, and it mustn't be applied to the whole url,
  since in this case it may perform invalid unescaping in the context of the url, or make the resulting url invalid.

While at it, properly marshal all the fields inside UserInfo config to yaml in tests.
Previously Header and QueryArg structs were improperly marshaled because the custom MarshalYAML
is called only on pointers to Header and QueryArg structs. This improves test coverage.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6070
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6115
2024-04-17 14:53:20 +02:00
Aliaksandr Valialkin
54c9f9c5ea
docs/Cluster-VictoriaMetrics.md: clarify per-node workload increase when one of vmstorage node is unavailable in the cluster
The docs update is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6099#issuecomment-2060856417
2024-04-17 14:52:49 +02:00
Yury Molodov
e4d6ded0f5
vmui: preserve Select value if it matches list on blur (#6101) 2024-04-17 14:52:30 +02:00
Roman Khavronenko
256c561005
app/vmauth: support regex matching in src_query_args (#6115)
Support regex matching when routing incoming requests based on HTTP query args
via `src_query_args` option at `url_map`.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6070

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-17 14:52:02 +02:00
Aliaksandr Valialkin
c6f0a37125
docs/Cluster-VictoriaMetrics.md: remove incorrect information about the increase of workload on the remaining vmstorage nodes when one vmstorage node is unavailable
If one out of 5 vmstorage nodes is unavailable, then the remaining 4 vmstorage nodes will recieve 1/5=20% increase of workload, not 5%.
If one out of 10 vmstorage nodes is unavailable, then the remaining 9 vmstorage nodes will receive 1/10=10% increase of workload, not 1%.

This is a follow-up for 458338afa5

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6099
2024-04-17 14:49:59 +02:00
Zakhar Bessarab
3f618c0485
app/vmauth: do not increment backend_errors when hitting concurrency limit (#6078)
* app/vmauth: do not increment backend_errors when hitting concurrency limit

Previously, both "vmauth_concurrent_requests_limit_reached_total" and "vmauth_user_request_backend_errors_total" were incremented.
This was based on the assumption that if concurrency limit is hit the backend must be failing to handle the request thus meaning an error.

This assumption does not work in case the endpoint can be overloaded by the misbehaving client sending too many requests within the timeframe.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5565

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-04-17 12:34:33 +02:00
Github Actions
60a447bb76
Automatic update operator docs from VictoriaMetrics/operator@99fbc98 (#6122) 2024-04-17 12:15:39 +02:00
Aliaksandr Valialkin
2347699126
docs/CHANGELOG.md: move the description for the bugfix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6110 from v1.100.1 to the tip section
The bugfix isn't included in v1.100.1 release.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6110
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6111
This is a follow-up for 7308bad777
2024-04-17 12:13:01 +02:00
Zakhar Bessarab
691181867e
docs/cluster: update cluster resizing info (#6099)
* docs/cluster: update cluster resizing info

- add example of resources distribution
- add info about how to handle uneven disk usage after adding a new storage node

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Update docs/Cluster-VictoriaMetrics.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Update docs/Cluster-VictoriaMetrics.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit 458338afa5)
2024-04-17 09:40:36 +02:00
Github Actions
a8444cb54b
Automatic update operator docs from VictoriaMetrics/operator@13f6dac (#6119)
(cherry picked from commit aaa18e565d)
2024-04-17 09:40:34 +02:00
hagen1778
3bddb10af8
docs: mention HTTP sink configuration example for Vector
Follow-up 16eeb4e

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 4f55aa29db)
2024-04-17 09:40:32 +02:00
Devin Buhl
a09da7a42c
victorialogs: mention vector supports http/json stream (#6114)
https://github.com/vectordotdev/vector/issues/18883#issuecomment-1771424716
(cherry picked from commit 16eeb4eb33)
2024-04-17 09:40:27 +02:00
yudrywet
d89229c838
chore: fix some typos in comments (#6103)
Signed-off-by: yudrywet <yudeyao@yeah.net>

(cherry picked from commit 43835704b7)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-16 09:58:30 +02:00
Alexander Marshalov
c61eb6d620
vmalert: support any status code from the range 200-299 from alertmanager as successful (#6111)
* any status code from the range 200-299 from alertmanager to vmalert is not considered an error from now on (#6110)

* add changelog

(cherry picked from commit 7308bad777)
2024-04-16 09:58:09 +02:00
Github Actions
20f16aed1a
Automatic update operator docs from VictoriaMetrics/operator@cf48a99 (#6113)
(cherry picked from commit 7db8ba41e7)
2024-04-16 09:58:07 +02:00
Dmytro Kozlov
097cc8a2a8
docs: fix typo in the curl command (#6109)
(cherry picked from commit 7b20de4674)
2024-04-16 09:58:05 +02:00
Github Actions
4859f5eeed
Automatic update operator docs from VictoriaMetrics/operator@73a1996 (#6100)
(cherry picked from commit cba2f6dce1)
2024-04-16 09:57:44 +02:00
hagen1778
b81a66ceea
docs: add 1.100.1 release date
Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit e39a1a98f5)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-16 09:57:41 +02:00
Hui Wang
9c0da1ef3d
deployment: bump victoriametrics components to v1.100.1 (#6096) 2024-04-12 12:19:21 +02:00
Aliaksandr Valialkin
4179ebdce7
deployment: upgrade VictoriaLogs from v0.5.1-victorialogs to v0.5.2-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.2-victorialogs
2024-04-11 09:53:36 +02:00
Aliaksandr Valialkin
14448966d9
docs/VictoriaLogs/CHANGELOG.md: cut v0.5.2-victorialogs release 2024-04-11 09:42:22 +02:00
Aliaksandr Valialkin
67af029b7f
docs/VictoriaLogs/CHANGELOG.md: document the bugfix at 2205de2391
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6042
2024-04-11 09:42:20 +02:00
Aliaksandr Valialkin
8cb768a7a5
docs/CHANGELOG.md: cut v1.100.1 release 2024-04-11 09:35:27 +02:00
Aliaksandr Valialkin
14a6dee206
docs/CHANGELOG.md: mention that the bug with incorrect registration of new entries in the IndexDB has been introduced in v1.99.0 2024-04-11 09:35:26 +02:00
Zakhar Bessarab
da8bb106aa
lib/mergeset: fix flushing incorrect set of inmemoryBlocks (#6089)
Follow-up for bace9a2501

Related:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6069
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5959

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-04-11 09:26:48 +02:00
Github Actions
26ed8d0571
Automatic update operator docs from VictoriaMetrics/operator@17082f0 (#6090) 2024-04-11 09:15:21 +02:00
Zakhar Bessarab
956ff263fc
docs/changelog: add entry for bd398974334678a29972475b0566e0bd2434d197
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-04-11 09:12:12 +02:00
7c4c6f7ee6
docs: fix a typo in docs/operator/resources/vmalert.md (#6088) 2024-04-11 09:09:26 +02:00
Aliaksandr Valialkin
e46150c3bf
docs/CHANGELOG.md: remove description of the reverted change at c79bf3925c
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5985
2024-04-04 16:56:04 +03:00
Aliaksandr Valialkin
b8d5fe0584
deployment/docker: update VictoriaLogs tag from v0.5.0-victorialogs to v0.5.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.1-victorialogs
2024-04-04 16:52:06 +03:00
Aliaksandr Valialkin
a549fc2941
docs/VictoriaLogs/CHANGELOG.md: cut v0.5.1-victorialogs release 2024-04-04 16:41:38 +03:00
Aliaksandr Valialkin
c55afb9a18
docs/VictoriaLogs/CHANGELOG.md: document the fix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5920
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5927
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5920

This is a follow-up for 46fd0ed693
2024-04-04 16:39:39 +03:00
Aliaksandr Valialkin
fe28388c5d
deployment: update VictoriaMetrics docer image from v1.99.0 to v1.100.0
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.100.0
2024-04-04 15:29:12 +03:00
Aliaksandr Valialkin
18e521dbff
docs: change old url from https://docs.victoriametrics.com/enterprise.html to new url https://docs.victoriametrics.com/enterprise/ 2024-04-04 15:22:38 +03:00
Aliaksandr Valialkin
68a9bcaa1e
docs/Single-server-VictoriaMetrics.md: remove misleading filters word from the text, which recommends evaluating downsampling in VictoriaMetrics enterprise 2024-04-04 14:58:45 +03:00
Aliaksandr Valialkin
6cf8d2144e
docs: clarify that downsampling drops all the samples except the last one on every downsampling interval 2024-04-04 14:50:24 +03:00
Aliaksandr Valialkin
34687cfec4
docs/CHANGELOG.md: typo fix: remove duplicate samples word 2024-04-04 14:46:21 +03:00
Aliaksandr Valialkin
6332a1c447
docs/CHANGELOG.md: advise upgrading from v1.99.0 to v1.100.0 because of the issue at v1.99.0
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5959
2024-04-04 14:41:31 +03:00
Aliaksandr Valialkin
283f4b96f6
docs/CHANGELOG.md: move custom backup interval feature to tip, since it wasnt included in v1.100.0 release
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5966
This is a follow-up for baa869742208f311d84e800ea527a7f07eb1ca18
2024-04-04 12:33:49 +03:00
Zakhar Bessarab
27f5eb38bb
Vmbackupmanager: add support of custom backup interval (#742)
* app/vmbackupmanager: add support of defining custom backup interval

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5966
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: update vmbackupmanager docs

Update docs after https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5966

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-04-04 12:33:48 +03:00
Aliaksandr Valialkin
530d7b40bd
docs/CHANGELOG.md: add release date for v1.100.0 2024-04-04 12:28:31 +03:00
Aliaksandr Valialkin
ddb1b5b70f
docs/CHANGELOG.md: cut v1.100.0 2024-04-04 03:47:32 +03:00
Aliaksandr Valialkin
d61f6c89a1
app/{vmagent,vminsert}: accept Prometheus remote write protocol requests at /prometheus/api/v1/push additionally to /api/v1/push
This is a follow-up for 7ccdb57ea4
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5990
2024-04-04 02:17:10 +03:00
nemobis
fdfdc9f8f1
Add note about final deduplication space needs (#5996)
Addresses #5975

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-04-04 02:09:12 +03:00
Yury Molodov
2d8777ac1e
vmui: fix step update on input blur in Firefox/Safari (#6034)
* vmui: fix step value application on input blur

* Update docs/CHANGELOG.md

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-04-04 01:53:15 +03:00
Yury Molodov
da642b56a1
vmui: fix trigger auto-suggestion (#6033)
* vmui: fix ui freeze on query paste #5923

* vmui: fix auto-suggestion trigger issue after whitespace char #5866

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-04-04 01:49:17 +03:00
Aliaksandr Valialkin
ecf3dfde9a
deployment: update Go builder from 1.22.1 to 1.22.2
See https://github.com/golang/go/issues?q=milestone%3AGo1.22.2+label%3ACherryPickApproved
2024-04-04 01:44:58 +03:00
Aliaksandr Valialkin
ecd782c75e
app/vmagent: follow-up for b3b29ba6ac
- Automatically reload changed TLS root CA pointed by -remoteWrite.tlsCAFile command-line flag
- Automatically reload changed TLS root CA configured via oauth2.tsl_config.ca_file option at -promscrape.config
- Document the change as a feature instead of a bug at docs/CHANGELOG.md
- Simplify the code at lib/promauth, which is responsible for reloading changed TLS root CA files.
- Simplify the usage of lib/promauth.Config.NewRoundTripper() - now it accepts the base http.Transport
  instead of a callback, which can change the internal http.Transport.
- Reuse the default tls config if lib/promauth.Config doesn't contain tls-specific configs.
  This should reduce memory usage a bit when tls isn't used for scraping big number of targets.
- Do not re-read TLS root CA files on every processed request. Re-read them once per second.
  This should reduce CPU usage when scraping big number of targets over https.
- Do not store cert.pem and key.pem files in TestTLSConfigWithCertificatesFilesUpdate, since they can be loaded
  from byte slices via crypto/tls.X509KeyPair().
- Remove obsolete comparisons of string representations for authConfig and proxyAuthConfig at areEqualScrapeConfigs().

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5725
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5526
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2171
2024-04-04 01:26:38 +03:00
Aliaksandr Valialkin
1c6dbe5714
docs/CHANGELOG.md: add - in front of -logInvalidAuthTokens command-line flag in order to be consistent with command-line flag naming
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6029
2024-04-04 00:42:37 +03:00
Zakhar Bessarab
bb2d029526
vmgateway: add an ability to log invalid auth tokens (#743)
* app/vmgateway: add an ability to log invalid auth tokens

This is useful for debugging to make it easier for user to find issues in token contents.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6029
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: add info about new vmgateway flag

- add changelog entry
- add info about logInvalidAuthTokens flag

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vmgateway/filters/auth: improve reject reason visibility

Explicitly return a rejection reason for request when "logInvalidAuthTokens" is enabled.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-04-04 00:42:09 +03:00
Aliaksandr Valialkin
2be627f1fe
app/vmalert: document that -rule.stripFilePath command-line flag is available only in enterprise version of vmalert 2024-04-04 00:41:35 +03:00
Thomas
7526f1d3a5
chore(docs): fix vmalertmanager typo (#6056)
Fixes: #6055

Signed-off-by: Thomas Way <thomas@6f.io>
Co-authored-by: Alexander Marshalov <_@marshalov.org>
2024-04-04 00:39:53 +03:00
Github Actions
ecee306e7a
Automatic update operator docs from VictoriaMetrics/operator@92cdca3 (#6052) 2024-04-04 00:39:19 +03:00
Zakhar Bessarab
da4352fe7b
lib/{promauth,promscrape}: automatically refresh root CA certificates after changes on disk (#5725)
* lib/{promauth,promscrape}: automatically refresh root CA certificates after changes on disk

Added a custom `http.RoundTripper` implementation which checks for root CA content changes and updates `tls.Config` used by `http.RoundTripper` after detecting CA change.

Client certificate changes are not tracked by this implementation since `tls.Config` already supports passing certificate dynamically by overriding `tls.Config.GetClientCertificate`.

This change implements dynamic reload of root CA only for streaming client used for scraping. Blocking client (`fasthttp.HostClient`) does not support using custom transport so can't use this implementation.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5526

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promauth/config: update NewRoundTripper API

Update API to allow user to update only parameters required for transport.

Add warning log when reloading Root CA failed.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promauth/config: fix mutex acquire logic

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promauth/config: replace RWMutex with regular mutex to simplify the code

- remove additional mutex used for getRootCABytes - require callee to use mutex
- replace RWMutex with regular mutex

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promauth/config: refactor

- hold the mutex lock to avoid round tripper being re-created twice
- move recreation logic into separate func to simplify the code

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-04-04 00:34:43 +03:00
Aliaksandr Valialkin
2e1c8bb436
docs/CHANGELOG.md: typo fix: resonses -> responses 2024-04-03 03:20:09 +03:00
Aliaksandr Valialkin
62c1545f97
docs/CHANGELOG.md: remove duplicate line left after 1e0a079143 2024-04-03 03:19:48 +03:00
hagen1778
6a3bf16475
docs: update vmalert troubleshooting docs
* rm recommendation to keep look-behind window empty, as it is not correct
* mention the change of default value for `-search.latencyOffset`

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-03 03:15:11 +03:00
Aliaksandr Valialkin
bb9bb600b3
lib/protoparser/opentelemetry: follow-up after 47892b4a4c
- Rename -opentelemetry.sanitizeMetrics command-line flag to more clear -opentelemetry.usePrometheusNaming
- Clarify the description of the change at docs/CHANGELOG.md
- Rename promrelabel.SanitizeLabelNameParts to more clear promrelabel.SplitMetricNameToTokens
- Properly split metric names at '_' char in promerlabel.SplitMetricNameToTokens.
- Add tests for various edge cases for Prometheus metric names' normalization
  according to the code at b865505850/pkg/translator/prometheus/normalize_name.go
- Extract the code responsible for Prometheus metric names' normalization into a separate file (santize.go)

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6037
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6035
2024-04-03 03:09:52 +03:00
Aliaksandr Valialkin
a5f65756f8
docs: follow-up for ac9c2a796f
Remove description for -search.maxExportDuration and -search.maxStatusRequestDuration command-line flags
from the 'Resource usage limits' chapter, since these flags are rarely used for limiting resource usage
and they are already documented in the 'List of command-line flags' chapter.
2024-04-03 03:08:27 +03:00
Aliaksandr Valialkin
5874189797
app/vmselect/graphite: follow-up for 23ab865035
- Fix docs for new functions at app/vmselect/graphite/functions.json
- Properly drain series lists on errors in aggregateSeriesListsGeneric() and aggregateSeriesList()
- Add links to docs for the added functions at docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5809
2024-04-03 03:07:15 +03:00
Aliaksandr Valialkin
f1b5784432
app/vmauth: add ability to authorize via any opaque HTTP request header value
This can be done via `auth_token` option at -auth.config - see https://docs.victoriametrics.com/vmauth/#auth-config
2024-04-03 02:52:39 +03:00
Aliaksandr Valialkin
2e9ee89cf1
app/vmauth: add ability to read auth tokens from multiple http request headers
This is needed for VictoriaMetrics Cloud, where the same token could be passed either
via Authorization or via X-Amz-Firehose-Access-Key header - see 4487dac30b (r140500722)

This is a follow-up for 4487dac30b

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6009
2024-04-03 02:52:20 +03:00
Aliaksandr Valialkin
456180b9d9
app/vmauth: follow-up for bc90f4aae6
- Allow specifying only a single HTTP header for reading auth tokens via -httpAuthHeader command-line flag.
  This is better from security PoV, since this prevents from accidental reading of auth token from undesired
  HTTP header. By default the -httpAuthHeader equals to Authorization. When it is overridden, then
  auth token isn't read from Authorization header - it is read only from the specified header.

- Document the -httpAuthHeader command-line flag at https://docs.victoriametrics.com/vmauth/#reading-auth-tokens-from-other-http-headers

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6009
2024-04-03 02:51:28 +03:00
Aliaksandr Valialkin
faa2ba828a
app/vmagent: simplify code after 509df44d03
- Simplify the code in order to improve its maintenance
- Properly pass tenant ID when processing multi-tenant opentelemetry request at vmagent

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6016
2024-04-03 02:50:46 +03:00
Fred Navruzov
28f7b2218a
docs/vmanomaly: typos fix (#6047) 2024-04-03 02:45:35 +03:00
Fred Navruzov
d83e05a3e9
docs/vmanomaly: v1.12 updates & fixes (#6046)
* docs/vmanomaly: v1.12.0 & link updates

* add autotuned description to model section

* - update refs of vmanomaly on enterprise and vmalert pages
- add diagrams for model types
- update self-monitoring section

* - fix typos
- remove .index.html from links
2024-04-03 02:44:58 +03:00
Aliaksandr Valialkin
e51190a34c
Revert "app/vmselect: make vmselect resilient to absence of cache folder (#5987)"
This reverts commit cb23685681.

Reason for revert: the "fix" may hide programming bugs related to incorrect creation of folders
before their use. This may complicate detecting and fixing such bugs in the future.

There are the following fixes for the issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5985 :
- To configure the OS to do not drop data from the system-wide temporary directory (aka /tmp).
- To run VictoriaMetrics with -cacheDataPath command-line flag, which points to the directory,
  which cannot be removed automatically by the OS.

The case when the user accidentally deletes the directory with some files created by VictoriaMetrics
shouldn't be considered as expected, so VictoriaMetrics shouldn't try resolving this case automatically.
It is much better from operation and debuggability PoV is to crash with the clear `directory doesn't exist` error
in this case.
2024-04-03 02:44:00 +03:00
Aliaksandr Valialkin
db6d5d6349
docs/CHANGELOG.md: mention that the bug with improper use of -search.maxExportDuration instead of -search.maxLabelsAPIDuration has been introduced in v1.99.0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5992
This is a follow-up for bc79f7196d
2024-04-03 02:43:40 +03:00
Aliaksandr Valialkin
83652eea1d
docs/VictoriaLogs/CHANGELOG.md: improve the description of the bugfix from 43b5d8bc7a, so it can be googled by users 2024-04-03 02:43:21 +03:00
Aliaksandr Valialkin
7edb5f77f1
app/vmagent: properly shutdown when -maxIngestionRate limit is reached
The remotewrite.Stop() expects that there are no pending calls to TryPush().
This means that the ingestionRateLimiter.Register() must be unblocked inside TryPush() when calling remotewrite.Stop().
Provide remotewrite.StopIngestionRateLimiter() function for unblocking the rate limiter before calling the remotewrite.Stop().

While at it, move the rate limiter into lib/ratelimiter package, since it has two users.
Also move the description of the feature to the correct place at docs/CHANGELOG.md.
Also cross-reference -remoteWrite.rateLimit and -maxIngestionRate command-line flags.

This is a follow-up for 02bccd1eb9
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5900
2024-04-03 02:41:11 +03:00
Aliaksandr Valialkin
6719415750
docs/managed-victoriametrics/alerting-vmalert-managed-victoria-metrics.md: user proper image paths according to docs/assets/README.md 2024-04-03 02:40:51 +03:00
Aliaksandr Valialkin
7d98632c9d
docs/managed-victoriametrics: use proper names for the linked images according to docs/assets/README.md
This is a follow-up for db3709c87d

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5989
2024-04-03 02:40:31 +03:00
Aliaksandr Valialkin
eff9676508
docs/CHANGELOG.md: remove the update notes regarding converting custom HTTP header keys to canonical form
Custom HTTP headers are set via net/http.Header.Set or net/http.Header.Add functions.
These functions always convert header keys to canonical form. So the change at b577413d3b
isn't visible to users of VictoriaMetrics components.

There is no need in documenting this change at docs/CHANGELOG.md, since it doesn't give any useful information to users.

This is a follow-up for e6dd52b04c
2024-04-03 02:39:08 +03:00
Zakhar Bessarab
3d15a31c6d
lib/storage: add ability to use downsampling for the given series filter (#733)
* lib/storage: add ability to use downsampling for the given series filter

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: add information about downsampling filters

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: fix MetricsQL filter

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/downsampling: treat missing downsampling filter as a bug

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/part_header: verify correctness of downsampling filters when opening partition

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/downsampling: save only appliable rules in part metadata

Filter and save only rules which are appliable to partition based on MinTimestamp of stored data.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/downsampling: update log messages for final dedup

Properly specify a reason of re-running deduplication for partition.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage: consistently use MaxTimestamp to determine deduplication/downsampling rules

Using MinTimestamp leads to applying downsampling to parts which are only partially covered by downsampling rule.
For example, partition covers range [1000-2000]. At t=2100 and rule offset 500 data with t=2100-500 => 1600 must be downsampled. The range check against MinTimestamp evaluates to true even though partition contains range which must not be downsampled - [1600:2000].

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Follow-up

- Apply the first matching downsampling period if multiple filters match the given time series.
  This allows fine-tuning the downsampling config for the specific needs.
- Take into account downsampling filters during search queries.
- Reduce the difference between community and enterprise branches. This should simplify further maintenance of these branches.
- Properly parse series filters with colons inside them.
- Document the feature at docs/CHANGELOG.md.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4960

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-04-03 02:38:37 +03:00
Andrii Chubatiuk
0799834aaa
opentelemetry: added cmd flag to sanitize metric names (#6035) 2024-04-03 02:31:39 +03:00
Aliaksandr Valialkin
3f79e54a51
app/vmagent/remotewrite: follow-up for 166b97b8d0 and b6bd9a97a3
- Make the configuration more clear by accepting the list of ignored labels during sharding
  via a dedicated command-line flag - -remoteWrite.shardByURL.ignoreLabels.
  This prevents from overloading the meaning of -remoteWrite.shardByURL.labels command-line flag.

- Removed superfluous memory allocation per each processed sample if sharding by remote storage is enabled.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5938
2024-04-03 00:50:48 +03:00
Dmytro Kozlov
111a8e5fd0
docs: describe timeout query argument (#6020)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-04-03 00:02:00 +03:00
Hui Wang
f410520cfa
docs: fix golangci-lint check (#6036)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-04-02 23:52:50 +03:00
Hui Wang
fdb6eb1071
vmalert: fix sending alert messages (#6028)
* vmalert: fix sending alert messages
1. fix `endsAt` field in messages that send to alertmanager, previously rule with small interval could never be triggered;
2. fix behavior of `-rule.resendDelay`, before it could prevent sending firing message when rule state is volatile.

* docs: update changelog notes

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-04-02 23:51:06 +03:00
Aliaksandr Valialkin
93b389eb70
docs/MetricsQL.md: typo fix: outlier_iqr_over_time(memory_usage_bytes[1h]) triggers when memory_usage_bytes goes outside the usual value range for the last hour, not the last 24 hours
This is a follow-up for ea81f6fc36
2024-04-02 23:48:58 +03:00
hagen1778
467c1ba617
docs: mention new guide How to use OpenTelemetry metrics with VictoriaMetrics in docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-02 23:47:44 +03:00
Nikolay
2ee8e462f8
docs/opentelemetry: adds opentemetry get started guide (#5861)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Andrii Chubatiuk <andrew.chubatiuk@gmail.com>
2024-04-02 23:47:06 +03:00
Github Actions
5d25fe206d
Automatic update operator docs from VictoriaMetrics/operator@c336013 (#6023) 2024-04-02 23:46:00 +03:00
Github Actions
ee95e9d14d
Automatic update operator docs from VictoriaMetrics/operator@ac29c88 (#6022) 2024-04-02 23:45:41 +03:00
Zakhar Bessarab
7c1ee69205
lib/storage/table: wait for merges to be completed when closing a table (#5965)
* lib/storage/table: properly wait for force merges to be completed during shutdown

Properly keep track of running background merges and wait for merges completion when closing the table.
Previously, force merge was not in sync with overall storage shutdown which could lead to holding ptw ref.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: add changelog entry

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-04-02 21:25:30 +03:00
Andrii Chubatiuk
9c92cc2759
vmauth: support other auth header names besides Authorization (#6009) 2024-04-02 18:41:00 +03:00
Andrii Chubatiuk
914b23f1e8
app/{vmagent,vminsert}: fixed firehose response (#6016) 2024-04-02 18:03:12 +03:00
hagen1778
1c597e9dba
docs: follow-up after 623d257faf
623d257faf
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 2e843a8ed9)
2024-03-29 14:30:25 +01:00
Jiekun
1e0a079143
app/vmalert: respect batch size limit for remote write on shutdown (#6039)
During shutdown period of vmalert, remotewrite client retrieve all pending time series from buffer queue, compose them into 1 batch and execute remote write.

This final batch may exceed the limit of -remoteWrite.maxBatchSize, and be rejected by the receiver (gateway, vmcluster or others).

This changes ensures that even during shutdown vmalert won't exceed the max batch size limit for remote write
destination.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6025

(cherry picked from commit 623d257faf)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-29 14:30:24 +01:00
hagen1778
36c1ca9949
app/vmagent: follow-up 166b97b8d0
* add tests for sharding function
* update flags description
* add changelog note

166b97b8d0
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit b6bd9a97a3)
2024-03-29 14:10:17 +01:00
Roman Khavronenko
548bf31dd2
app/vmselect: make vmselect resilient to absence of cache folder (#5987)
vmselect uses a cache folder in file system for two purposes:
1. Storing rollup cache results on shutdown;
2. Storing temporary search results from vmstorage during query executions.

It could happen that cache folder is deleted accidentally by user, or by OS
during cleanup routines. This would cause vmselect to:
1. panic on /metrics call, because `MustGetFreeSpace` will fail;
2. return query error user, as it won't be able to store temporary search results.

The changes in this commit are the following:
1. Make `MustGetFreeSpace` to try re-creating the cache folder if it is missing;
2. Make vmselect to try re-creating the cache folder if it can't persist tmp search
results.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5985

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
(cherry picked from commit cb23685681)
2024-03-26 15:27:32 +01:00
Artem Navoiev
bf226879ec
add more redirects
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit b08cbd0400)
2024-03-25 15:42:54 +01:00
Artem Navoiev
19a11c12d1
fix typo in kyiv city name
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit b569fa0b2c)
2024-03-25 15:42:53 +01:00
Nikolay
60d64edd6e
app/vlselect: follow-up for 0514091948 (#6004)
removes println lines

(cherry picked from commit 43b5d8bc7a)
2024-03-25 15:42:53 +01:00
Alexander Marshalov
0b3744effa
[vmagent] added ingestion rate limiting with new flag -maxIngestionRate (#5900)
* [vmagent] added ingestion rate limiting with new flag `-maxIngestionRate`. This flag can be used to limit the number of samples ingested by vmagent per second. If the limit is exceeded, the ingestion rate will be throttled.

* fix changelog

* fix review comment

(cherry picked from commit 02bccd1eb9)
2024-03-25 15:42:52 +01:00
Nikolay
3ea0b87399
docs/managed: adds alertmanager configuration examples (#5989)
* docs/managed: adds alertmanager configuration examples

* apply review suggestions

(cherry picked from commit db3709c87d)
2024-03-25 15:42:52 +01:00
hagen1778
a1b666c35e
docs: add missing API version to VMSingle example
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 93a29fce4e)
2024-03-25 15:42:51 +01:00
hagen1778
2b05559293
docs: mention Query Analyzer in docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 21d9393c9e)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-25 15:42:51 +01:00
Yury Molodov
30778e0d0f
vmui: fix the _time filter insertion for all queries in VictoriaLogs UI #5920 (#5927)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5920
(cherry picked from commit 46fd0ed693)
2024-03-25 15:42:33 +01:00
Dmytro Kozlov
9817ecdcc9
app/vmctl: break explore phase in vm-native mode by time intervals
When `--vm-native-step-interval` is specified, explore phase will be executed
within specified intervals. Discovered metric names will be associated with
time intervals at which they were discovered. This suppose to reduce number
of requests vmctl makes per metric name since it will skip time intervals
when metric name didn't exist.

This should also reduce probability of exceeding complexity limits
for number of selected series in one request during explore phase.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5369
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5f8b91186a)
2024-03-25 15:42:24 +01:00
hagen1778
9fcd34287b
docs: follow-up for 70eaa06f08
Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit bc79f7196d)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-25 15:33:11 +01:00
Aliaksandr Valialkin
cd222d6502
lib/streamaggr: ignore out of order samples for last output
This is a follow-up for 6a465f6e29

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5931
2024-03-18 01:03:58 +02:00
Aliaksandr Valialkin
eecc5e8463
lib/storage: wait for up to 60 seconds before deciding to delete metricID entries from indexdb if metricID->metricName entry is missing during search
The metricID->metricName entry can remain invisible for search for some time after registering new metricName.
This is expected condition. So wait for up to 60 seconds in the hope that the metricID->metricName
entry will become visible before deleting all the entries from indexdb, which are associated with the given metricID.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5959
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5948

See also 20812008a7
2024-03-18 00:37:11 +02:00
Aliaksandr Valialkin
111d0aa2bf
app/{vmagent,vminsert}: add an ability to ignore input samples outside the current aggregation interval for stream aggregation
See https://docs.victoriametrics.com/stream-aggregation.html#ignoring-old-samples
2024-03-17 23:30:46 +02:00
Aliaksandr Valialkin
e70b644f1f
lib/streamaggr: ignore out of order samples when calculating increase, increase_prometheus, total and total_prometheus outputs
Out of order samples may result in unexpected spikes for these outputs.
So it is better to ignore such samples.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5931
2024-03-17 23:24:14 +02:00
Aliaksandr Valialkin
1f753a049a
lib/streamaggr: follow-up for 15e33d56f1
- Properly set pushSample.timestamp when flushing de-duplicated samples to stream aggregation
  This is needed for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5931

- Re-classify this change as feature instead of bugfix at docs/CHANGELOG.md

- Verify de-duplication logic for samples with different timestamps

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5643
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5939
2024-03-17 23:23:57 +02:00
Aliaksandr Valialkin
c3dbcc7c0e
docs/CHANGELOG.md: clarify that -datasource.lookback commnad-line flag is no-op in the upcoming release
Document the solution - to switch to eval_delay option at group config.

This is a follow-up for e80b44f19d
2024-03-17 23:23:31 +02:00
Aliaksandr Valialkin
b481448c53
docs/CHANGELOG.md: document the bugfix from cb259116b4
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5802
2024-03-17 23:23:11 +02:00
Aliaksandr Valialkin
7c4d7dc6dd
lib/promauth: properly set Host header in requests to scrape targets.
The `Host` header must be set via net/http.Request.Host field, since net/http.Client
ignores this header if it is set via Request.Header.Set().

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5969
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5970
2024-03-17 23:22:54 +02:00
hagen1778
efddda945e
app/vmctl: fix arguments order in httputils.TLSConfig
follow-up after 9d5bf5ba5d

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-17 23:22:08 +02:00
Github Actions
eb82f329cb
Automatic update operator docs from VictoriaMetrics/operator@ae6e1b6 (#5964) 2024-03-17 23:16:52 +02:00
Daria Karavaieva
dbdbbacf93
docs/vmanomaly: fix 404 links (#5968) 2024-03-17 23:16:15 +02:00
hagen1778
770dd7affd
docs: mention missing vmalert change for memory usage reduction in 1.97.3
521f9ffb43
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-17 23:15:23 +02:00
hagen1778
41a1efbea8
docs: follow-up 15e33d56f1
Update documentation according to changes in deduplication logic.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-17 23:07:51 +02:00
Andrii Chubatiuk
a58a81d80b
lib/streamaggr: pick sample with bigger timestamp or value on deduplicator (#5939)
Apply the same deduplication logic as in https://docs.victoriametrics.com/#deduplication
This would require more memory for deduplication, since we need to track timestamp
for each record. However, deduplication should become more consistent.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5643

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-03-17 23:06:37 +02:00
Github Actions
61dc9f5c8d
Automatic update operator docs from VictoriaMetrics/operator@de88675 (#5958) 2024-03-17 20:48:56 +02:00
Zakhar Bessarab
5f1f1f2982
docs: explicitly mention VMUI is available in cluster (#5955)
It is confusing for cluster users to find that VMUI is available at vmselect as it is only mentioned in the list of URLs. Explicit mention of vmselect URL in docs will make it easier to discover.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-03-17 20:48:07 +02:00
hagen1778
7b2f19a63e
docs: mention bug investigation in 1.99
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-17 20:46:48 +02:00
Hui Wang
349564fd82
vmalert: deprecate cmd-line flag -datasource.lookback (#5877)
* vmalert: deprecate cmd-line flag `-datasource.lookback`

* fix lint

* review fixes

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit e80b44f19d)
2024-03-12 16:17:40 +01:00
Aliaksandr Valialkin
abf347ec0f
docs: replace speed up with more clear accelerate wording 2024-03-12 03:03:15 +02:00
Aliaksandr Valialkin
f92d4609e2
lib/storage: optimize /api/v1/labels and /api/v1/label/.../values when match[] contains metric name
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2978
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5055
2024-03-12 03:01:47 +02:00
nemobis
14b913eb51
docs: fix typo in stalenes (#5950)
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-03-12 02:59:43 +02:00
Github Actions
221f0a000b
Automatic update operator docs from VictoriaMetrics/operator@9b1a6e6 (#5946) 2024-03-12 02:57:08 +02:00
Artem Navoiev
1348a822f8
docs:vmbackup fix typo sped -> speed
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-03-12 02:55:08 +02:00
Zakhar Bessarab
217893921c
docs: fix typo in flags description (#5942)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-03-12 02:45:46 +02:00
Github Actions
b2b034ddb7
Automatic update operator docs from VictoriaMetrics/operator@9caa896 (#5941) 2024-03-10 13:00:35 +02:00
Github Actions
9d57092a58
Automatic update operator docs from VictoriaMetrics/operator@403a78a (#5932) 2024-03-10 13:00:22 +02:00
hagen1778
86abbe5b67
app/vmctl: support TLS configuration for VictoriaMetrics destination
VictoriaMetrics destination is specified via `--vm-*` cmd-line flags
and is used in opentsdb, influx, prometheus, remote-read modes.

updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5426

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 0b7ce70df4)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-08 21:15:23 +01:00
hagen1778
b1fe25792c
app/vmctl: follow-up b9f7c3169a
* fix typos in flags description
* move the change to #tip section in changelog

b9f7c3169a
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 83a8c24281)
2024-03-08 21:15:09 +01:00
Khushi Jain
98c0909a12
app/vmctl : Provide TLS config options for vm native protocol (#5824)
Co-authored-by: Khushi Jain <khushi.jain@nokia.com>
(cherry picked from commit b9f7c3169a)
2024-03-08 21:15:09 +01:00
Aliaksandr Valialkin
b21fb355fa
docs/vmauth.md: update -help output after e08b91baafc95da090f75e9c29a27d8f62a2b76e 2024-03-07 01:37:48 +02:00
Aliaksandr Valialkin
b9788d5cf8
docs/CHANGELOG.md: typo fixes 2024-03-07 01:35:41 +02:00
Aliaksandr Valialkin
d9e3670627
lib/promauth: set the Host header to tlsServerName if itsn't empty
If tlsServerName isn't empty, then it is likely the https request is sent to IP instead of hostname.
In this case the request will fail, since Go automatically sets the Host header to the IP instead
of the desired hostname at tlsServerName. So set the Host header to tlsServerName if itsn't empty.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5802
2024-03-07 01:23:40 +02:00
Aliaksandr Valialkin
e7e9721476
docs/vmauth.md: typo fixes after 7b2b980181 2024-03-07 01:08:40 +02:00
Aliaksandr Valialkin
93ef217857
app/vmauth: allow discovering backend ips behind shared hostname and spreading load among the discovered ips
This is done with the `discover_backend_ips` option at `user` and `url_map` level.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5707
2024-03-07 01:02:38 +02:00
Aliaksandr Valialkin
b2e182ca62
app/vmauth: add src_headers option at url_map, which allows routing incoming requests to different backends depending on request headers 2024-03-06 21:56:42 +02:00
Aliaksandr Valialkin
5bd4325135
app/vmauth: simplify configuration for src_query_args
Use the shorter form:

src_query_args:
- arg1=value1
- arg2=value2

instead of

src_query_args:
- name: arg1
  value: value2
- name: arg2
  value: value2

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878
2024-03-06 21:24:53 +02:00
Aliaksandr Valialkin
1df7e5b69d
deployment/docker: update Go builder from Go1.21.7 to Go1.22.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.22.1+label%3ACherryPickApproved
2024-03-06 21:04:44 +02:00
Aliaksandr Valialkin
0929f255f0
docs/vmauth.md: mention that request query args can used for routing decisions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878
2024-03-06 20:58:10 +02:00
Aliaksandr Valialkin
87d59cdfa5
app/vmauth: add ability to route requests based on HTTP query args via src_query_args option
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878
2024-03-06 20:52:42 +02:00
Aliaksandr Valialkin
2711732944
app/vmselect/prometheus: do not drop match[] filters if -search.ignoreExtraFiltersAtLabelsAPI flag is set
The `match[]` filter is mandatory at /api/v1/series, so it mustn't be dropped here.

There is no sense in dropping `match[]` filter together with `extra_label` and `extra_filters[]`
at /api/v1/labels and /api/v1/label/.../values if -search.ignoreExtraFiltersAtLabelsAPI commnad-line flag is set,
since:
- the `match[]` filter triggers slow path at these APIs;
- the `extra_label` and `extra_filters[]` filters narrow down the number of matched time series,
  so they improve performance comparing to the case when only `match[]` filter is left,
  while `extra_label` and `extra_filters[]` filters are dropped.

This is a follow-up for 0b7a23a91d
2024-03-06 13:57:20 +02:00
Daria Karavaieva
98b56476d7
redirect alias (#5934) 2024-03-06 13:54:48 +02:00
Yury Molodov
56e61bb54b
vmui: improve tracing styles (#5926)
Improved trace display for better visual separation of branches:
* Increased left padding for each element
* Added padding for the last element in the branch
2024-03-06 13:54:07 +02:00
Github Actions
cfce708eb9
Automatic update operator docs from VictoriaMetrics/operator@f028fdf (#5929) 2024-03-06 13:34:37 +02:00
hagen1778
a4cdfaea28
docs: clarify deduplication is needed in multi-retention setup
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-06 13:33:40 +02:00
hagen1778
a364de3cfe
lib/writeconcurrencylimiter: mention dependency on CPU cores for -maxConcurrentInserts flag
The change also removes misleading `default` value from README for `maxConcurrentInserts`
cmd-line flag.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 73f5fb0f0c)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-05 18:56:38 +01:00
Aliaksandr Valialkin
27b9e8ed3e
app/{vmagent,vminsert}: add -streamAggr.dropInputSamples command-line flag for dropping the specified labels from input samples before deduplication and streaming aggregation 2024-03-05 02:27:27 +02:00
hagen1778
b612c0536e
docs: re-fresh vmctl docs
* add recommendation about network bandiwdth between vmctl, source and destination
* use more relevant time series selector in examples - see https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5835

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-05 00:49:48 +02:00
Aliaksandr Valialkin
c38c45d71f
app/{vminsert,vmagent}: allow using -streamAggr.dedupInterval without -streamAggr.config
This allows performing online de-duplication of incoming samples
2024-03-05 00:47:23 +02:00
Github Actions
b530f6bf73
Automatic update operator docs from VictoriaMetrics/operator@c8ff654 (#5918) 2024-03-04 19:12:59 +02:00
Aliaksandr Valialkin
48a425898a
lib/streamaggr: enable time alignment for aggregate flushed to multiples of interval
For example, if `interval: 1m`, then data flush occurs at the end of every minute,
while `interval: 1h` leads to data flush at the end of every hour.

Add `no_align_flush_to_interval` option, which can be used for disabling the alignment.
2024-03-04 06:23:35 +02:00
Aliaksandr Valialkin
3ba9b2225e
docs/stream-aggregation.md: add troubleshooting section with solutions for common problems in streaming aggregation 2024-03-04 03:04:59 +02:00
Aliaksandr Valialkin
492c6c3ff5
docs/CHANGELOG.md: typo fix 2024-03-02 04:52:41 +02:00
Aliaksandr Valialkin
d0e6541f35
docs/stream-aggregation.md: typo fixes 2024-03-02 04:35:37 +02:00
Aliaksandr Valialkin
b912a45220
docs/stream-aggregation.md: remove superflouous output_relabel_configs from the config example for histogram aggregation 2024-03-02 03:36:08 +02:00
Aliaksandr Valialkin
0d5d46f9db
lib/streamaggr: huge pile of changes
- Reduce memory usage by up to 5x when de-duplicating samples across big number of time series.
- Reduce memory usage by up to 5x when aggregating across big number of output time series.
- Add lib/promutils.LabelsCompressor, which is going to be used by other VictoriaMetrics components
  for reducing memory usage for marshaled []prompbmarshal.Label.
- Add `dedup_interval` option at aggregation config, which allows setting individual
  deduplication intervals per each aggregation.
- Add `keep_metric_names` option at aggregation config, which allows keeping the original
  metric names in the output samples.
- Add `unique_samples` output, which counts the number of unique sample values.
- Add `increase_prometheus` and `total_prometheus` outputs, which ignore the first sample
  per each newly encountered time series.
- Use 64-bit hashes instead of marshaled labels as map keys when calculating `count_series` output.
  This makes obsolete https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5579
- Expose various metrics, which may help debugging stream aggregation:
  - vm_streamaggr_dedup_state_size_bytes - the size of data structures responsible for deduplication
  - vm_streamaggr_dedup_state_items_count - the number of items in the deduplication data structures
  - vm_streamaggr_labels_compressor_size_bytes - the size of labels compressor data structures
  - vm_streamaggr_labels_compressor_items_count - the number of entries in the labels compressor
  - vm_streamaggr_flush_duration_seconds - a histogram, which shows the duration of stream aggregation flushes
  - vm_streamaggr_dedup_flush_duration_seconds - a histogram, which shows the duration of deduplication flushes
  - vm_streamaggr_flush_timeouts_total - counter for timed out stream aggregation flushes,
    which took longer than the configured interval
  - vm_streamaggr_dedup_flush_timeouts_total - counter for timed out deduplication flushes,
    which took longer than the configured dedup_interval
- Actualize docs/stream-aggregation.md

The memory usage reduction increases CPU usage during stream aggregation by up to 30%.

This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5850
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5898
2024-03-02 03:15:43 +02:00
hagen1778
8ce65e93d0
docs: mention docs link for VictoriaLogs docker env
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-01 21:34:15 +02:00
Aliaksandr Valialkin
7559b6eb97
docs: update -help output after recent changes in VictoriaMetrics components 2024-03-01 05:31:13 +02:00
Aliaksandr Valialkin
0dbcc07d04
docs/CHANGELOG.md: typo fix 2024-03-01 04:41:39 +02:00
Aliaksandr Valialkin
5f0dd8743c
docs: bump the latest VictoriaMetrics release from v1.98.0 to v1.99.0
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.99.0
2024-03-01 04:24:11 +02:00
Aliaksandr Valialkin
be1446f720
docs/LTS-releases.md: update latest LTS release to v1.97.3 and v1.93.13
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.3
and https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.13
2024-03-01 04:21:25 +02:00
Aliaksandr Valialkin
2540b8fdf8
deployment: update VictoriaLogs docker image from v0.4.2-victorialogs to v0.5.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.0-victorialogs
2024-03-01 04:18:59 +02:00
Aliaksandr Valialkin
d66680fc47
docs/CHANGELOG.md: cut v1.99.0 release 2024-03-01 03:33:44 +02:00
Aliaksandr Valialkin
dc91c07384
docs/CHANGELOG.md: document v1.97.3 LTS release 2024-03-01 03:31:19 +02:00
Aliaksandr Valialkin
e4a1d6774e
docs: consistently use https://docs.victoriametrics.com/lts-releases/ link for LTS releases 2024-03-01 02:52:41 +02:00
Aliaksandr Valialkin
0fac8d6e87
docs/CHANGELOG.md: document v1.93.13 LTS release
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.13
2024-03-01 02:41:05 +02:00
Aliaksandr Valialkin
bde2de3298
docs/VictoriaLogs/CHANGELOG.md: typo fix: bufix -> bugfix 2024-03-01 02:41:04 +02:00
Aliaksandr Valialkin
9a470239cf
docs/VictoriaLogs/CHANGELOG.md: cut v0.5.0-victorialogs 2024-03-01 01:43:35 +02:00
Aliaksandr Valialkin
d84cdf98da
app/{vmselect,vlselect}/vmui: run make vmui-update vmui-logs-update after e130f29659
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5862
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5152
2024-03-01 00:51:49 +02:00
Yury Molodov
c71c2b37c5
vmui: add gap display option for charts #5152 (#5862) 2024-03-01 00:51:48 +02:00
Aliaksandr Valialkin
728fb59e0d
docs: update docs after 0b7a23a91d 2024-03-01 00:18:46 +02:00
Aliaksandr Valialkin
6aea1e5093
app/{vmagent,vminsert}: follow-up for 434a5803e7
Document the /opentelemetry/v1/metrics endpoint instead of /opentelemetry/api/v1/push,
since the /v1/metrics suffix is hardcoded in OpenTelemetry protocol specification.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5871
2024-02-29 18:05:12 +02:00
Nikolay
abb773120d
app/{vmagent,vminsert}: adds /v1/metrics suffix for opentelemetry route path (#5871)
* app/{vmagent,vminsert}: adds /v1/metrics suffix for opentelemetry route path
it must fix compatibility with opentemetry-collector [spec](https://opentelemetry.io/docs/specs/otlp/\#otlphttp-request)
this suffix is hard-coded and cannot be changed with collector configuration

* Apply suggestions from code review

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-29 17:59:57 +02:00
Github Actions
5e20be845f
Automatic update operator docs from VictoriaMetrics/operator@7e8f811 (#5902) 2024-02-29 17:53:11 +02:00
Alexander Marshalov
b8c588a7e2
fixed broken link in docs to google pub sub (#5901) 2024-02-29 17:11:46 +02:00
Hui Wang
25e454be4c
metricsql: fix label_join() when dst_label is equal to one of the `… (#5886)
* metricsql: fix label_join() when `dst_label` is equal to one of the `src_label`

* Update app/vmselect/promql/transform.go

* Update docs/CHANGELOG.md

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-29 16:02:48 +02:00
Aliaksandr Valialkin
cfeae6fffb
docs/VictoriaLogs/CHANGELOG.md: document the bugfix at a5795f533d
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5897
2024-02-29 15:34:57 +02:00
Aliaksandr Valialkin
7310605c27
docs/CHANGELOG.md: clarify the description for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5859
This is a follow-up for 3c74aa6b3d
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5888
2024-02-29 15:28:52 +02:00
Aliaksandr Valialkin
0d2d1897c2
app/vmbackupmanager: follow-up after f2885a5c41dfaeecfecba0aceb8ae65b15e15d1c
- Document the change at docs/CHANGELOG.md
- Give more clear name for maxHealthChecksRetries constant
- Improve logging when storage isn't reachable
- Use http.NewRequestWithContext() instead of setting up a request without context
  and then creating a new request with context vi req.WithContext() call.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5859
Updates https://github.com/VictoriaMetrics/VictoriaMetrics-enterprise/pull/735
2024-02-29 15:17:57 +02:00
Hui Wang
99acc5bbba
caseStudies: add NetEaseCloudMusic post (#5891) 2024-02-29 14:58:15 +02:00
Artem Navoiev
3d9f947e42
docs: update statistic per tenant, add use cases section (#5892)
* docs: update statistic per tenant, add use cases section

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* prettify

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* Update docs/PerTenantStatistic.md

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-29 14:55:55 +02:00
Aliaksandr Valialkin
38b42489c6
docs/anomaly-detection/guides: bump alertmanager version from v0.25.0 to v0.27.0 after 3723c809a1 2024-02-29 14:51:39 +02:00
Aliaksandr Valialkin
7832d0800e
app/{vminsert,vmagent}: follow-up after 67a55b89a4
- Document the ability to read OpenTelemetry data from Amazon Firehose at docs/CHANGELOG.md

- Simplify parsing Firehose data. There is no need in trying to optimize the parsing with fastjson
  and byte slice tricks, since OpenTelemetry protocol is really slooow because of over-engineering.
  It is better to write clear code for better maintanability in the future.

- Move Firehose parser from /lib/protoparser/firehose to lib/protoparser/opentelemetry/firehose,
  since it is used only by opentelemetry parser.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5893
2024-02-29 14:47:20 +02:00
Github Actions
33a65d6988
Automatic update operator docs from VictoriaMetrics/operator@9f1c910 (#5895) 2024-02-29 13:56:31 +02:00
Aliaksandr Valialkin
4e5cbc5dc6
deployment/docker: downgrade Go builder from 1.22.0 to 1.21.7
Go1.22.0 contains the bug https://github.com/golang/go/issues/65705 ,
which prevents vmagent from normal operation.
2024-02-29 13:52:58 +02:00
Aliaksandr Valialkin
cb9b46be54
app/vmselect/prometheus: add -search.ignoreExtraFiltersAtLabelsAPI command-line flag for ignoring extra_filters and extra_label args at /api/v1/labels, /api/v1/label/.../values and /api/v1/series 2024-02-29 13:00:24 +02:00
lzfhust
2821993c9c
add xiaohongshu in Case studies (#5885) 2024-02-29 02:40:00 +02:00
Aliaksandr Valialkin
7d619deacc
docs/Release-Guide.md: https://github.com/VictoriaMetrics/victoriametrics-lts-rpm should contain the latest release
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5811
2024-02-27 13:14:25 +02:00
Roman Khavronenko
baa32adc47
deployment: create a separate env for VictoriaLogs (#5857)
* deployment: create a separate env for VictoriaLogs

The new environment consists of the following components:
* VictoriaLogs
* fluentbit for collecting logs and sending to VictoriaLogs
* VictoriaMetrics for scraping and storing metrics from fluentbit and VictoriaLogs
* Grafana with VictoriaLogs datasource for monitoring

-----------------

The motivation for creating a separate environment is to simplify existing environments
and make it easier to update or modify them in future.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-27 13:13:42 +02:00
Aliaksandr Valialkin
dc5b1e4dc1
lib/httpserver: return back the default value for -http.connTimeout to 2 minutes
It has been appeared that there are VictoriaMetrics users, who rely on the fact that
VictoriaMetrics components were closing incoming connections to -httpListenAddr every 2 minutes
by default. So let's return back this value by default in order to fix the breaking change
made at d8c1db7953 .

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1304#issuecomment-1961891450 .
2024-02-24 00:20:11 +02:00
Aliaksandr Valialkin
2f0b2b942b
docs/Troubleshooting.md: add Too much disk space used chapter` 2024-02-24 00:19:51 +02:00
hagen1778
0ac2df6e66
docs: mention missing change 521f9ffb430edf6aea7720a966b5b079cf15e34e
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-24 00:19:13 +02:00
Aliaksandr Valialkin
83217b7473
app/vmselect: add -search.maxLabelsAPIDuration and -search.maxLabelsAPISeries options for fine-tuning CPU and RAM usage for /api/v1/series , /api/v1/labels and /api/v1/label/.../values
This commit returns back limits for these endpoints, which have been removed at 5d66ee88bd ,
since it has been appeared that missing limits result in high CPU usage, while the introduced concurrency limiter
results in failed lightweight requests to these endpoints because of timeout when heavyweight requests are executed.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5055
2024-02-23 02:56:58 +02:00
Yury Molodov
140aaafad0
vmui: add a time picker to the "Logs Explorer" page (#5808)
* vmui: add a time picker to the "Logs Explorer" page #5673

* Update app/vmui/packages/vmui/src/pages/ExploreLogs/hooks/useFetchLogs.ts

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-23 01:38:54 +02:00
Yury Molodov
b2b9f6e900
vmui: fix display Popper.tsx (#5842)
* vmui: fix display Popper.tsx

* vmui/docs: fix display Popper.tsx

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-23 01:33:12 +02:00
Aliaksandr Valialkin
50ea081d75
docs/Single-server-VictoriaMetrics.md: sync with docs/README.md after 79b57f625c 2024-02-23 01:28:42 +02:00
Aliaksandr Valialkin
fbae1533fc
docs/CHANGELOG.md: document d68bb658ce
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5833
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5834
2024-02-23 01:26:02 +02:00
hagen1778
225fd781a5
docs: re-classify change of default http timeout to bugfix
The change introduced in d8c1db7953 (diff-2bfab3db5cc1baf4c6d3ff6b19901926e3bdf4411ec685dac973e5fcff1c723b)
was backported to v1.97.2. Therefore, it is a `bugfix` and should be explicitly
mentioned in the changelog of v1.97.2

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-23 01:17:40 +02:00
Nikolay
22762d7a69
app/vmselect: change export/csv timestamp format for rfc3339 to respect milliseconds (#5853)
* app/vmselect: adds milliseconds to the csv export response for rfc3339
* milliseconds is a standard prescion for VictoriaMetrics query request responses
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5837

* app/victoria-metrics: adds tests for csv export/import
follow-up after 3541a8d0cf96dd4f8563624c4aab6816615d0756

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-02-23 01:16:08 +02:00
Fred Navruzov
c9d8627676
- v1.11 doc updates (#5852)
- fix dead links
2024-02-23 01:11:05 +02:00
Dan Dascalescu
6b5e8e7089
docs: CSV RFC3339 format uses server timezone (#5839) 2024-02-23 01:07:55 +02:00
Aliaksandr Valialkin
a982ab6bfb
app/vmstorage: expose vm_snapshots metric, which shows the current number of snapshots
While at it, refresh docs about snapshots - https://docs.victoriametrics.com/#how-to-work-with-snapshots
2024-02-23 01:07:04 +02:00
Aliaksandr Valialkin
843f3ec94e
docs/MetricsQL.md: improve text formatting for better readability 2024-02-23 01:05:49 +02:00
Aliaksandr Valialkin
477fdc21aa
app/vmselect/promql: add count_values_over_time() MetricsQL function
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5847
2024-02-23 01:05:31 +02:00
Alexander Marshalov
8322425364
[lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5814)
* [lib/promutils, lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5801)

* fixed tests

* fixed test

* Revert "fixed test"

This reverts commit 8a29764806.

* Revert "fixed tests"

This reverts commit 9ce13d1042.

* Revert "[lib/promutils, lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5801)"

This reverts commit a7a04bd4

* [lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5801)

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-02-23 00:58:26 +02:00
Artem Navoiev
b46014c8ab
docs: change header from h1 to h2 1.97.2. The markdown requires the proper structure in hierachy of title so h1 can not be a child of h1,h2... only be a separate item, in our structure title is the parent h1
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-02-23 00:42:15 +02:00
Github Actions
e2bbab072a
Automatic update operator docs from VictoriaMetrics/operator@d88157b (#5845) 2024-02-23 00:41:10 +02:00
Github Actions
f08f33cd5a
Automatic update operator docs from VictoriaMetrics/operator@c393852 (#5844) 2024-02-23 00:40:38 +02:00
Github Actions
786679135b
Automatic update operator docs from VictoriaMetrics/operator@4791fd1 (#5843) 2024-02-23 00:40:04 +02:00
Roman Khavronenko
2e172b9361
vmctl : Provide TLS config options for Open TSDB datasource #5797 (#5832)
Originally implemented here https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5797

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: khushijain21 <khushij393@gmail.com>
(cherry picked from commit bb1279bfc4)
2024-02-20 16:27:52 +01:00
Daria Karavaieva
550b589790
Vmanomaly Quickstart Fix absolute links (#5831)
* links fix

* typo fix

(cherry picked from commit 4034d081f4)
2024-02-20 16:27:52 +01:00
Daria Karavaieva
a68518c0e0
Vmanomaly QuickStart (#5800)
* first edit

* typo 1

* typo 2

* fixes 3

* fixes 4

* fixes 5

* fixes, cross links

* v1.10 config

* models why, self-monitoring fix

* config next steps

* fixes

* minor fix

(cherry picked from commit b2baf7d472)
2024-02-20 16:27:52 +01:00
hagen1778
c065287757
docs: move recent changes to Tip
These changes were mistakenly put to existing release

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit dc25c30fdc)
2024-02-20 16:27:51 +01:00
hagen1778
4474c23aed
app/vmalert: consistently sort groups by name and filename on /groups page
This should prevent non-deterministic sorting for groups with identical names.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit e2dad3a2ac)
2024-02-20 13:51:31 +01:00
hagen1778
6c63fd831d
app/vmalert: follow-up after b60dcbe11f
* support case-insensitive search
* reflect search condition in URL, so link can be sharable
* support filtering on /alerts page
* fix collapseAll/expandAll logic to respect only shown entries
* add changelog

b60dcbe11f
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 11b03d9fc8)
2024-02-20 13:35:02 +01:00
Artem Navoiev
e0569a355b
docs: mention slack inviter and slack channel (#5817)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 5b652bccad)
2024-02-20 13:35:01 +01:00
Aliaksandr Valialkin
1de59610c8
docs/VictoriaLogs/CHANGELOG.md: document cafd6f08b3
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5400
2024-02-18 23:18:12 +02:00
Aliaksandr Valialkin
c4b16e3749
docs/VictoriaLogs/CHANGELOG.md: document 333bda8702
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5447
2024-02-18 23:13:30 +02:00
Aliaksandr Valialkin
b58c429044
app/vlselect: follow-up for 451d2abf50
- Consistently return the first `limit` log entries if the total size of found log entries doesn't exceed 1Mb.
  See app/vlselect/logsql/sort_writer.go . Previously random log entries could be returned with each request.
- Document the change at docs/VictoriaLogs/CHANGELOG.md
- Document the `limit` query arg at docs/VictoriaLogs/querying/README.md
- Make the change less intrusive.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5674
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5778
2024-02-18 23:06:08 +02:00
Dmytro Kozlov
2d674f98d4
Enable the limit query param for the /select/logsql/query (#5778)
* app/vlselect: add limit for logs query

* app/vlselect: CHANGELOG.md

* app/vlselect: stop search process if limit is reached, update logic, remove default limit

* app/vlselect: fix tests

* app/vlselect: fix filter tests

* app/vlselect: fix tests
2024-02-18 22:59:16 +02:00
Aliaksandr Valialkin
82e38e1627
lib/promscrape: add support for enable_compression option in the same way as Prometheus does
Updates https://github.com/prometheus/prometheus/pull/13166
Updates https://github.com/prometheus/prometheus/issues/12319

Do not document enable_compression option at docs/sd_configs.md, since vmagent already supports
more clear disable_compression option - see https://docs.victoriametrics.com/vmagent/#scrape_config-enhancements
2024-02-18 19:42:09 +02:00
Aliaksandr Valialkin
f0db7d474f
lib/promscrape/discovery/kuma: add support for client_id option
See https://github.com/prometheus/prometheus/pull/13278
2024-02-18 19:19:55 +02:00
Aliaksandr Valialkin
690f6c833c
vendor: update github.com/VictoriaMetrics/metricsql from v0.72.1 to v0.73.0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5383
2024-02-18 18:43:41 +02:00
Aliaksandr Valialkin
c1a3876e95
docs/MetricsQL.md: properly document how MetricsQL selects the lookbehind window
- rate(m) isn't equivalent to rate(m[1i]) when step is smaller than the interval between samples.
- default_rollup(m) isn't equivalent to default_rollup(m[1i]) when step is smaller than the interval between samples.

These changes have been made in v1.85.3 as a part of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3483 .
See the corresponding commit - 9fa3f1dc57 .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5816
2024-02-18 13:48:45 +02:00
Aliaksandr Valialkin
8053ff9135
docs/LTS-releases.md: cosmetic fixes 2024-02-17 18:09:43 +02:00
Aliaksandr Valialkin
55bba932d4
docs/CHANGELOG.md: document f8207e33a2 2024-02-17 17:55:01 +02:00
Aliaksandr Valialkin
cf96c3416d
docs/CHANGELOG.md: add missing for in the description of the TLS configuration features for vmctl
This is a follow-up for 6a07cb1bdb and f973711e56
2024-02-17 17:55:01 +02:00
Aliaksandr Valialkin
efbead081c
docs/Single-server-VictoriaMetrics.md: enumerate all the VictoriaMetrics components 2024-02-17 17:54:34 +02:00
Aliaksandr Valialkin
9f7c9e9033
docs/LTS-releases.md: add a dedicated page describing LTS lines of releases for VictoriaMetrics 2024-02-17 17:53:56 +02:00
hagen1778
e53f53aaf5
app/vmctl: follow-up after 0c293a66ec
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f973711e56)
2024-02-16 15:31:58 +01:00
Khushi Jain
9ce7f21a63
app/vmctl : support TLS config options for remote read mode (#5798)
(cherry picked from commit 0c293a66ec)
2024-02-16 15:31:58 +01:00
hagen1778
025e52adad
app/vmctl: follow-up after 7cd1b7d047
* cleanup code
* update docs

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6a07cb1bdb)
2024-02-16 15:31:58 +01:00
Khushi Jain
02c8b5015c
app/vmctl : support TLS config options for InfluxDB datasource (#5783)
* vmctl: TLS flags for influx DB

* added httputils function

* Add changelog and doc

---------

Co-authored-by: Khushi Jain <khushi.jain@nokia.com>
(cherry picked from commit 7cd1b7d047)
2024-02-16 15:31:57 +01:00
Fred Navruzov
d8de87aeb0
docs: vmanomaly - updates of v1.10.0 and model type section (#5813)
* - apply v1.10 changes
- chapter on model types (uni/multivariate and rolling)

* - update self-monitoring labels description
- fix typos

* fix duplicated text and link rendering

(cherry picked from commit 172e196ac9)
2024-02-16 15:31:56 +01:00
hagen1778
5e4732cc2d
docs: update formatting for usage examples
- Use `sh` format for examples
- Reduce length of progress bars

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 3170ad3f44)
2024-02-16 15:31:56 +01:00
Aliaksandr Valialkin
33b2553c78
app/vmstorage: expose vm_last_partition_parts metrics, which may help identifying performance issues related to the increased number of parts in the last partition 2024-02-15 14:52:53 +02:00
Aliaksandr Valialkin
6fd9331b7f
docs/CHANGELOG.md: document v1.93.12 LTS release 2024-02-14 20:17:52 +02:00
Aliaksandr Valialkin
c617379789
docs/CHANGELOG.md: document v1.97.2 LTS release 2024-02-14 18:50:59 +02:00
Aliaksandr Valialkin
77a514f2cc
all: update Docker image tag for VictoriaMetrics components from v1.97.1 to v1.98.0
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.98.0
2024-02-14 17:14:55 +02:00
Aliaksandr Valialkin
06da06dac0
lib/promrelabel: store the original labels before returning them them to promutils.PutLabels()
This should reduce memory allocations.

This is a follow-up for b09bd6c42a

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5389
2024-02-14 16:09:38 +02:00
Aliaksandr Valialkin
acc37ff28a
docs/CHANGELOG.md: cut v1.98.0 release 2024-02-14 16:00:59 +02:00
Aliaksandr Valialkin
9df42f000d
docs/CHANGELOG.md: various typo cleanups 2024-02-14 02:45:25 +02:00
Yury Molodov
b08a23c4a5
vmui: improve the context for autocomplete #5736 #5737 #5739 (#5804)
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-14 02:36:06 +02:00
Aliaksandr Valialkin
5f2905d120
app/vmselect: add sum_eq_over_time, sum_gt_over_time and sum_le_over_time functions to MetricsQL
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4641
2024-02-13 23:40:30 +02:00
Nikolay
0a2cc0e873
app/vmauth: properly release memory during config reload (#5805)
* app/vmauth: properly release memory during config reload
previously metrics package hold a refrence for channels for users concurrent requests.
it case of churn at `name`  field of users configuration, new metric was created. But previous one wasn't deleted.
It prevented full parsed configuration from being garbace collected.

now all config related metrics are bound to corresponding metrics.Set and unregistered during config reload process.

It also must fix an issue with incorrect values for current concurrent user requests

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4690

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-13 20:49:57 +02:00
Aliaksandr Valialkin
89aceb2b2f
docs: add link to https://docs.victoriametrics.com/scrape_config_examples/ to docs about configuring target scraping at vmagent and single-node VictoriaMetrics 2024-02-13 20:48:04 +02:00
Aliaksandr Valialkin
2c44ea89fd
docs/vmbackupmanager.md: mention -license command-line flag instead of deprecated -eula 2024-02-13 20:41:33 +02:00
Aliaksandr Valialkin
3f74cadba4
docs/vmauth.md: add Config reload chapter, which explains how to reload -auth.config at vmauth
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1194
2024-02-13 20:36:59 +02:00
Aliaksandr Valialkin
f6cb2021c3
docs/enterprise.md: remove the mention of deprecated -eula command-line flag 2024-02-13 18:47:07 +02:00
Aliaksandr Valialkin
dcacf209d9
vendor: update github.com/VictoriaMetrics/metricsql from v0.70.1 to v0.71.0
This adds an ability to propagate label filters across label_set() and alias() functions.

This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827#issuecomment-1654095358
2024-02-13 06:37:09 +02:00
Aliaksandr Valialkin
67091537ae
app/vmagent/remotewrite: add -remoteWrite.tlsHandshakeTimeout command-line flag for tuning tls handshake timeout to -remoteWrite.url
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1699
2024-02-13 02:46:24 +02:00
Aliaksandr Valialkin
84e439c469
docs/vmauth.md: add missing dot 2024-02-13 01:09:11 +02:00
Aliaksandr Valialkin
6bc70a883d
app/vmauth: add support for mTLS-based routing of incoming requests to different backends depending on the subject field in the TLS certificate provided by the user
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1547
2024-02-13 01:04:19 +02:00
Aliaksandr Valialkin
ebcbca0e00
docs/keyConcepts.md: do not duplicate the list of supported data ingestion protocols - just refer to the original list at https://docs.victoriametrics.com/#how-to-import-time-series-data
This should simplify keeping docs in sync
2024-02-12 22:55:27 +02:00
Aliaksandr Valialkin
8793817fbe
docs/vmbackup.md: remove the unneeded -storageDataPath command-line from the example for making server-side copy of the backup 2024-02-12 22:24:41 +02:00
Aliaksandr Valialkin
f5680a6857
all: upgrade Go builder from Go1.21.7 to Go1.22.0
See https://go.dev/doc/go1.22
2024-02-12 22:14:00 +02:00
Artem Navoiev
5815ea4160
docs vmbackupmanager update flags
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-02-12 20:16:00 +02:00
Aliaksandr Valialkin
12cf2d8b44
docs/enterprise.md: remove duplicate Enterprise word in the same sentence 2024-02-12 20:13:30 +02:00
Aliaksandr Valialkin
0f46a49912
docs/Single-server-VictoriaMetrics.md: substitute duplicate cases studies list with the link to the original list 2024-02-12 20:13:01 +02:00
Aliaksandr Valialkin
8ea4b8a3b4
docs: remove misleading either from the description of or label filters 2024-02-12 20:12:27 +02:00
Github Actions
c952c678b2
Automatic update operator docs from VictoriaMetrics/operator@e261c37 (#5788) 2024-02-12 20:11:07 +02:00
Roman Khavronenko
433c3726b2
app/vmalert: support filtering for /api/v1/rule like Prometheus does (#5787)
Follow-up after 62e5e2a4c8

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 8850c7431d)
2024-02-09 14:36:15 +01:00
Victor Amorim dos Santos
56b1d8e9ed
app/vmalert: support type param for filtering /api/v1/rules response by rule type (#5749)
Co-authored-by: Hui Wang <haley@victoriametrics.com>
(cherry picked from commit 62e5e2a4c8)
2024-02-09 14:36:14 +01:00
Aliaksandr Valialkin
46de5d7eb1
docs: update docs after ae8a867924
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1470
2024-02-09 04:20:09 +02:00
Aliaksandr Valialkin
cf64597878
all: add support for specifying multiple -httpListenAddr options 2024-02-09 03:22:49 +02:00
Aliaksandr Valialkin
5bbf95f9b4
docs/CHANGELOG.md: typo fixes 2024-02-08 21:18:52 +02:00
Aliaksandr Valialkin
ae7da12280
lib/httpserver: do not close client connections every 2 minutes by default
Closing client connections every 2 minutes doesn't help load balancing -
this just leads to "jumpy" connections between multiple backend servers,
e.g. the load isn't spread evenly among backend servers, and instead jumps
between the servers every 2 minutes.

It is still possible periodically closing client connections by specifying non-zero -http.connTimeout command-line flag.

This should help with https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1304#issuecomment-1636997037

This is a follow-up for d387da142e
2024-02-08 21:10:54 +02:00
Aliaksandr Valialkin
9216d00a6c
docs/{vmagent,vmalert}: mention that /-/reload endpoint may be protected with -reloadAuthKey command-line flag
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1172
2024-02-08 18:49:21 +02:00
Aliaksandr Valialkin
0d313dda8a
docs/Cluster-VictoriaMetrics.md: document that /api/v1/query?series_lector[d] returns raw samples
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1148
2024-02-08 18:32:47 +02:00
Aliaksandr Valialkin
fc8328a87f
docs: sync with enterprise branch 2024-02-08 17:25:26 +02:00
Aliaksandr Valialkin
428109a427
docs: update -help output after 83e55456e2 2024-02-08 17:22:42 +02:00
Artem Navoiev
fbe3f51fee
docs: change ndjson links to https://jsonlines.org/ as original one was hacked (#5782)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-02-08 17:22:31 +02:00
hagen1778
2aa86313bc
docs: sync flags description for vmbackup and vmbackupmanager
Sync description after changes in 83e55456e2

Remove extra text in flags description for vmbackupmanager as it breaks layout
when rendered at docs.victoriametrics.com

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-08 17:12:53 +02:00
Aliaksandr Valialkin
44052f4fd5
docs/CHANGELOG.md: add a link to docs describing -disableReroutingOnUnavailable command-line flag 2024-02-08 17:05:27 +02:00
hagen1778
a2063e0e7a
docs: follow-up after 83e55456e2
83e55456e2
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit e1926f286b)
2024-02-08 15:58:34 +01:00
Khushi Jain
a076cb4a93
app/vmbackup: support client-side TLS configuration for create/delete snapshot API (#5738)
(cherry picked from commit 83e55456e2)
2024-02-08 15:58:34 +01:00
Aliaksandr Valialkin
d0b55e359a
docs/vmagent.md: add debugging scrape targets chapter 2024-02-08 16:32:54 +02:00
Aliaksandr Valialkin
582d431df2
app/victoria-metrics: properly send staleness markers on victoriametrics shutdown if -selfScrapeInterval > 0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/943
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
2024-02-08 15:30:40 +02:00
Aliaksandr Valialkin
202d8e2c40
docs: update -help output after 61d9df4c36
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/834
2024-02-08 14:50:56 +02:00
Aliaksandr Valialkin
b18e608016
app/vmselect: add ability to reset rollup result cache on startup by passing -search.resetRollupResultCacheOnStartup command-line flag
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/834
2024-02-08 14:42:15 +02:00
Aliaksandr Valialkin
84a3f4719d
docs/CHANGELOG.md: clarify the bugfix description for e1bf8440eb 2024-02-08 13:01:26 +02:00
Aliaksandr Valialkin
ae2a9c8195
lib/mergeset: prevent from possible too big indexBlockSize panic
This panic could occur when samples with too long label values are ingested into VictoriaMetrics.
This could result in too long fistItem and commonPrefix values at blockHeader (up to 64kb each).
This may inflate the maximum index block size by 4 * maxIndexBlockSize.
2024-02-08 12:55:58 +02:00
hagen1778
9b173c2f01
dashboards: follow-up 4369bc1df2
* add more details to changelog
* simplify panels description
* remove capacity planning recommendation, as it proves it incompetent

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-08 12:55:42 +02:00
Hui Wang
0cd0ddc1c1
deployment/dashboards: fix Storage full ETA panels (#5747)
During background downsampling, rate(vm_deduplicated_samples_total{type="merge"}) could be much bigger than 
rate(vm_rows_added_to_storage_total) and it could last quite some time,
 which causes negative values of Storage full ETA and confuses users, see playground.

Instead of trying to get more accurate results during downsampling, I think it's ok to ignore 
vm_deduplicated_samples_total at all, it's more reasonable to see Storage full ETA increase after downsampling.

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-08 12:54:31 +02:00
Aliaksandr Valialkin
def5573f92
app/vmselect/promql: properly handle precision errors in rollup functions
changes(), increases_over_time() and resets() shouldn't take into account
value changes, which may occur because of precision errors.

The maximum guaranteed precision for raw samples stored in VictoriaMetrics is 12 decimal digits.
So do not count relative changes for values if they are smaller than 1e-12 comparing to the value.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/767
2024-02-08 02:33:17 +02:00
Aliaksandr Valialkin
f4252cd9aa
docs/CHANGELOG.md: properly document the change at b74006e2ca
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5774
2024-02-07 22:06:37 +02:00
Nihal
95287d269b
[vmsingle/vminsert]: change http success response code to 200 for -/reload request handler (#5776)
* change vmsingle's response code to 200 for reload request handler

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* change vmsingle's response code to 200 for the reload request handler

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* change vmsingle's response code to 200 for the reload request handler. See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5774

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

---------

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>
2024-02-07 22:06:24 +02:00
Github Actions
3c069607b3
Automatic update operator docs from VictoriaMetrics/operator@38dfad5 (#5772) 2024-02-07 21:55:11 +02:00
Aliaksandr Valialkin
5d9e0ab71e
docs/CHANGELOG.md: support empty command-line flag values in short array notation
For example, -fooDuration=',10s,' is now supported - it sets three command-line flag values:

- the first and the last one are set to the default value for `-fooDuration`
- the second one is set to 10s
2024-02-07 20:55:01 +02:00
Aliaksandr Valialkin
949481fe48
docs/relabeling.md: add examples on how to rename scraped metrics and how to add/change labels in scraped metrics 2024-02-07 20:29:04 +02:00
Aliaksandr Valialkin
caf706fcc0
all: update Go builder from Go1.21.6 to Go1.21.7
See https://github.com/golang/go/issues?q=milestone%3AGo1.21.7+label%3ACherryPickApproved
2024-02-07 04:01:05 +02:00
Aliaksandr Valialkin
18e1dfd7a8
docs/scrape_config_examples.md: add an example for scraping cadvisor in Kubernetes 2024-02-07 03:23:01 +02:00
Aliaksandr Valialkin
898414856f
docs/Cluster-VictoriaMetrics.md: document the /datadog/api/beta/sketches endpoint
This is a follow-up for a1d1ccd6f2

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5584
2024-02-07 01:36:08 +02:00
Aliaksandr Valialkin
82f4e4e070
app/{vmagent,vminsert}: follow-up after a1d1ccd6f2
- Document the change at docs/CHANGELOG.md
- Copy changes from docs/Single-server-VictoriaMetrics.md to README.md
- Add missing handler for processing multitenant requests ( https://docs.victoriametrics.com/vmagent/#multitenancy )
- Substitute github.com/stretchr/testify dependency with 3 lines of code in the added tests
- Comment unclear code at lib/protoparser/datadogsketches/parser.go , so @AndrewChubatiuk could update it
  and add permalinks to the original source code there.
- Various code cleanups

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5584
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3091
2024-02-07 01:31:52 +02:00
Daria Karavaieva
0ff3e6f3bc
Vmanomaly Guide - dashboard and query change (#5771)
* dashboard fix

* query fix

* changed screenshots

* minor fixes
2024-02-07 01:31:15 +02:00
hagen1778
eb90f2170a
docs/vmalert: mention step param when comapring rule results to raw queries
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-07 01:30:32 +02:00
Andrii Chubatiuk
c634859c4f
support datadog /api/beta/sketches API (#5584)
Co-authored-by: Andrew Chubatiuk <andrew.chubatiuk@motional.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-07 01:30:00 +02:00
Yury Molodov
5778acf9eb
vmui: improve select component functionality (#5755)
* vmui: fix select closing on click outside (#5728)

* vmui: clear entered text in select after selecting a value (#5727)

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-06 22:50:28 +02:00
Yury Molodov
0b5f5d456c
vmui: fix handling invalid timezone (#5758)
* vmui: fix handling invalid timezone (#5732)

* vmui: switch browser timezone flag to isValid

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-06 22:48:00 +02:00
Yury Molodov
0cf17068b8
vmui: fix graph dragging (#5769)
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-06 22:32:43 +02:00
Aliaksandr Valialkin
ebd0f1b72e
vendor: update github.com/VictoriaMetrics/metricsql from v0.70.0 to v0.70.1
This should help with the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5604
2024-02-06 21:52:55 +02:00
Aliaksandr Valialkin
dc340a83a9
docs/scrape_config_examples.md: document that full urls can be used as in the targets list 2024-02-06 19:49:23 +02:00
Aliaksandr Valialkin
2fec67632f
docs/scrape_config_examples.md: add missing to specify phrase 2024-02-06 19:43:30 +02:00
Aliaksandr Valialkin
0504771db1
docs/CHANGELOG.md: add link to mTLS feature request 2024-02-06 19:32:51 +02:00
Aliaksandr Valialkin
bace92fab6
lib/httpserver: add support for mTLS for requests to -httpListenAddr 2024-02-06 17:47:27 +02:00
Aliaksandr Valialkin
79990cc89b
docs/CHANGELOG.md: move descriptions for recently added features from v1.97.1 to tip 2024-02-06 16:31:44 +02:00
Aliaksandr Valialkin
22524353ed
docs/scrape_config_examples.md: add examples for typical scrape_config usage 2024-02-06 16:01:34 +02:00
Aliaksandr Valialkin
01312003be
docs/Cluster-VictoriaMetrics.md: add a warning that -disableReroutingOnUnavailable command-line flag may result in long pause for data ingestion when some of vmstorage nodes are unavailable for long time 2024-02-06 16:01:07 +02:00
Aliaksandr Valialkin
bffcb40537
docs/enterprise.md: prioritize enterprise technical support over other enterprise features 2024-02-06 16:00:21 +02:00
hagen1778
8bc18b5a4c
docs/vmalert: update limit description
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-06 16:00:02 +02:00
hagen1778
702cc4514a
docs/vmalert: mention limit option in group params
This param was supported for long time but was missing in the docs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-06 15:59:28 +02:00
Fred Navruzov
eaf75a565f
update changelog with 1.0.0 helm chart ref (#5763) 2024-02-06 15:58:49 +02:00
Aliaksandr Valialkin
2bad79a97e
docs/sd_configs.md: improve readability a bit 2024-02-05 15:33:23 +02:00
Github Actions
c0b2ccb6ca
Automatic update operator docs from VictoriaMetrics/operator@bfc521d (#5761)
Co-authored-by: Hui Wang <haley@victoriametrics.com>
2024-02-05 15:33:22 +02:00
Aliaksandr Valialkin
63a43331a3
docs/Cluster-VictoriaMetrics.md: document -disableReroutingOnUnavailable command-line flag
This is a follow-up for 88f0d1572e
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5713
2024-02-05 15:17:09 +02:00
Aliaksandr Valialkin
67d166f181
docs/Cluster-VictoriaMetrics.md: move the Improve re-routing performance during restart to more appropriate place
Previoulsy it was mistakenly inserted between `No downtime strategy` and 'Minimum downtime strategy' chapters.

This is a follow-up for 37997abd14
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5293
2024-02-05 14:44:06 +02:00
Aliaksandr Valialkin
e2c58a9db1
docs/Cluster-VictoriaMetrics.md: mention -metrics.exposeMetadata command-line flag in Monitoring section 2024-02-05 11:48:05 +02:00
Aliaksandr Valialkin
103f527037
docs/CHANGELOG.md: add a link to all VictoriaMetrics dashboards for Grafana 2024-02-05 11:45:39 +02:00
Aliaksandr Valialkin
6af029fd9b
docs/Troubleshooting.md: clarify the section about GOGC tuning
This is a follow-up for 487a94565b
2024-02-05 11:45:39 +02:00
Aliaksandr Valialkin
b105a5a1f5
docs/CHANGELOG.md: add missing links to the corresponding dashboards 2024-02-05 11:45:06 +02:00
Aliaksandr Valialkin
19f9203c7a
docs: mention -metrics.exposeMetadata command-line flag in Monitoring sections
This is a follow-up for 326a77c697
2024-02-05 11:44:39 +02:00
Artem Navoiev
e3138a356c
docs: fix aliasis for url-example
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-02-05 11:43:58 +02:00
Fred Navruzov
be41ffb925
docs: vmanomaly - fix 1.9.2 references in pull cmd and docs (#5756)
* fix 1.9.2 references in pull cmd and docs
* better readability of upgrade note
2024-02-05 11:43:41 +02:00
hagen1778
bdbab7bed5
dashboards/all: add new panel CPU spent on GC
It should help identifying cases when too much CPU is spent on garbage collection,
 and advice users on how this can be addressed.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-05 11:42:28 +02:00
hagen1778
2206309439
dashboards: add Targets scraped/s
A new stat panel shows the number of targets scraped by the vmagent per-second.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-05 10:51:35 +02:00
Aliaksandr Valialkin
9f23231926
docs/CHANGELOG.md: document v1.93.11 LTS release 2024-02-01 18:21:35 +02:00
Aliaksandr Valialkin
d63bc81f29
docs/CHANGELOG.md: document v1.87.14 LTS release 2024-02-01 17:09:03 +02:00
Aliaksandr Valialkin
24437b6d70
all: update VictoriaMetrics Docker image from v1.97.0 to v1.97.1
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.1
2024-02-01 16:15:14 +02:00
Aliaksandr Valialkin
dab3da9ee8
docs: mark v1.97.x as long-term support release 2024-02-01 15:16:52 +02:00
Aliaksandr Valialkin
e1ceb64432
docs/CHANGELOG.md: cut v1.97.1 2024-02-01 15:08:53 +02:00
Aliaksandr Valialkin
eff4dc3685
docs: sync -help output after recent changes 2024-02-01 15:06:19 +02:00
Aliaksandr Valialkin
d8f508cc21
docs/Single-server-VictoriaMetrics.md: clarify Security chapter a bit 2024-02-01 14:45:10 +02:00
Aliaksandr Valialkin
ee327cfd5d
docs/Single-server-VictoriaMetrics.md: run make docs-sync after 49d5e7fef5
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5262
2024-02-01 14:42:07 +02:00
Aliaksandr Valialkin
6c56f49f9c
lib/prompbmarshal: return back custom protobuf marshaler for lib/prompbmarshal.WriteRequest
The easyproto-based marshaler is 2x slower than the previous custom marshaler,
so let's stick with it. This improves the performance for sending data to remote storage at vmagent
and reduces CPU usage to pre-v1.97.0 levels.
2024-02-01 06:34:46 +02:00
Aliaksandr Valialkin
cc626ae3b5
lib/promauth: follow-up for fca3b14b7b
- Simplify the code for handling BasicAuthConfig at lib/promauth/config.go
- Move the description of the change into correct place at docs/CHANGELOG.md
- Put tests for username in front of tests for password at lib/promauth/config_test.go

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5720
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5511
2024-01-31 19:47:53 +02:00
Nihal
bcd094ac8b
Support for username_file in scrape config (basic_auth) similar to Prometheus for having config compatibility (#5720)
* adding support for username_file in basic_auth of scrape config

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* adding support for username_file in basic_auth of scrape config. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5511

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* adding support for username_file in basic_auth of scrape config

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* adding support for username_file in basic_auth of scrape config

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* adding support for username_file in basic_auth of scrape config

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

---------

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>
2024-01-31 19:47:50 +02:00
Aliaksandr Valialkin
4f5cb17042
app/vmselect/netstorage: properly handle the case when an empty brsPool points to the end of brs.brs
This case is possible after a new brsPool is allocated. The fix is to verify whether len(brsPool) >= len(brs.brs)
before trying to append a new item to brsPool and sharing its contents with brs.brs.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5733
2024-01-31 10:31:51 +02:00
hagen1778
151247c9b9
dashboards/single: fix typo in query for version annotation
The typo falsely produced many version change events.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-31 10:28:24 +02:00
Aliaksandr Valialkin
2033fe4caf
app/vmselect/promql: really keep metric names when keep_metric_names modifier is applied to binary operator
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5556
2024-01-31 02:33:06 +02:00
Aliaksandr Valialkin
a0043e214f
docs/vmauth.md: typo fix after 68be182075 2024-01-31 00:13:56 +02:00
Aliaksandr Valialkin
f1effc1133
all: update VictoriaMetrics Docker image from v1.96.0 to v1.97.0
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.0
2024-01-30 23:37:42 +02:00
Aliaksandr Valialkin
6d766abde7
deployment/docker: upgrade base Docker image from Alpine 3.19.0 to 3.19.1
See https://www.alpinelinux.org/posts/Alpine-3.19.1-released.html
2024-01-30 22:47:29 +02:00
Aliaksandr Valialkin
966f1ad07f
docs/CHANGELOG.md: cut v1.97.0 2024-01-30 22:45:10 +02:00
Dan Dascalescu
d2c38e2588
docs: t...over_time functions return fractional seconds (#5715)
* docs: t...over_time functions return fractional seconds

* Apply suggestions from code review

---------

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-30 22:20:16 +02:00
Roman Khavronenko
02e609b141
app/vmselect: set proper timestamp for cached instant responses (#5723)
* app/vmselect: set proper timestamp for cached instant responses

The change updates `getSumInstantValues` to prefer timestamp
from the most recent results. Before, timestamp from cached series
was used.

The old behavior had negative impact on recording rules as they
were getting responses with shifted timestamps in past.
Subsequent recording or alerting rules fetching results of these
recording rules could get no result due to staleness interval.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5659
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-30 22:20:16 +02:00
Yury Molodov
587c5fec2d
vmui: add the ability to expand all tracing entries (#5677) (#5726) 2024-01-30 21:13:13 +02:00
Github Actions
317a2551ee
Automatic update Grafana datasource docs from VictoriaMetrics/grafana-datasource@40e4e15 (#5729) 2024-01-30 21:07:50 +02:00
Aliaksandr Valialkin
1a071c5228
docs/CHANGELOG.md: document the enhancement, which reduces initial memory usage when vmagent scrapes targets with large responses
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5567
2024-01-30 20:51:29 +02:00
Aliaksandr Valialkin
dc9f4f0907
docs/CHANGELOG.md: refer to the related pull request for the bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945 2024-01-30 20:21:51 +02:00
Aliaksandr Valialkin
16787415b3
docs/CHANGELOG.md: document the bugfix addressed by the commit bc7cf4950b
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945
2024-01-30 20:16:32 +02:00
Artem Navoiev
17a4d310d0
docs: add alias for keyconcepts
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-01-30 18:39:59 +02:00
Aliaksandr Valialkin
09c388a8e4
lib/promscrape: use the standard net/http.Client instead of fasthttp.Client for scraping targets in non-streaming mode
While fasthttp.Client uses less CPU and RAM when scraping targets with small responses (up to 10K metrics),
it doesn't work well when scraping targets with big responses such as kube-state-metrics.
In this case it could use big amounts of additional memory comparing to net/http.Client,
since fasthttp.Client reads the full response in memory and then tries re-using the large buffer
for further scrapes.

Additionally, fasthttp.Client-based scraping had various issues with proxying, redirects
and scrape timeouts like the following ones:

- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5425
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2794
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1017

This should help reducing memory usage for the case when target returns big response
and this response is scraped by fasthttp.Client at first before switching to stream parsing mode
for subsequent scrapes. Now the switch to stream parsing mode is performed on the first scrape
after reading the response body in memory and noticing that its size exceeds the value passed
to -promscrape.minResponseSizeForStreamParse command-line flag.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5567

Overrides https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4931
2024-01-30 18:39:55 +02:00
Aliaksandr Valialkin
36be93f53c
docs/keyConcepts.md: clarify the information about which data is returned by instant and range queries
Do not use `raw samples` term there, since it adds more confusion than clarity:
the `raw samples` refers to real samples stored in the database, while neither range nor instant queries
do not return raw samples - they both return *calculated* samples at *the given* timestamps.

This is a follow-up for b5978ed8f9

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5710
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5708
2024-01-29 18:22:27 +01:00
Fred Navruzov
ff67f4d3fb
- fix link/version of helm chart in update request (#5716) 2024-01-29 18:22:11 +01:00
Artem Navoiev
00db646851
docs: add backward compaitble redicrt for url examples page
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-01-29 18:21:03 +01:00
Fred Navruzov
8043cf35d8
- update versions to 1.9.2 (#5714)
- update guide asset urls to flat
2024-01-29 18:20:38 +01:00
hagen1778
131afea8ab
docs: specify results of Instant and Range queries
Mention explicitly what are value and timestamp field in returned
results from Instant and Range queries.

Updates
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5710
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5708

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-29 18:20:02 +01:00
Fred Navruzov
790ee9af6d
update guide and changelog to 1.9.1 (#5706) 2024-01-29 17:23:06 +01:00
hagen1778
216f268c1a
docs: follow-up after 491287ed15
* port un-synced changed from docs/readme to readme
* consistently use `sh` instead of `console` highlight, as it looks like
a more appropriate syntax highlight
* consistently use `sh` instead of `bash`, as it is shorter
* consistently use `yaml` instead of `yml`

See syntax codes here https://gohugo.io/content-management/syntax-highlighting/

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-29 17:06:26 +01:00
hagen1778
3baeface89
docs: follow-up after 491287ed15
491287ed15
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-29 16:59:13 +01:00
Artem Navoiev
0d210308c3
docs: remove witdh from images, remove <p>, remove <div> (#5705)
* docs: remove witdh from images, remove <p>, remove <div>

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* docs: remove <div> clarify language in code blocks

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-01-29 16:50:51 +01:00
Daria Karavaieva
469bbbb2fa
version 1.9.1 update, dashboard viz flag (#5704) 2024-01-29 16:48:55 +01:00
Aliaksandr Valialkin
f5559c038c
lib/storage: do not check the limit for -search.maxUniqueTimeseries when performing /api/v1/labels and /api/v1/label/.../values requests
This limit has little sense for these APIs, since:

- Thses APIs frequently result in scanning of all the time series on the given time range.
  For example, if extra_filters={datacenter="some_dc"} .

- Users expect these APIs shouldn't hit the -search.maxUniqueTimeseries limit,
  which is intended for limiting resource usage at /api/v1/query and /api/v1/query_range requests.

Also limit the concurrency for /api/v1/labels, /api/v1/label/.../values
and /api/v1/series requests in order to limit the maximum memory usage and CPU usage for these API.
This limit shouldn't affect typical use cases for these APIs:

- Grafana dashboard load when dashboard labels should be loaded
- Auto-suggestion list load when editing the query in Grafana or vmui

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5055
2024-01-29 16:44:46 +01:00
Roman Khavronenko
9e9f170fe7
lib/streamaggr: skip unfinished aggregation state on shutdown by default (#5689)
Sending unfinished aggregate states tend to produce unexpected anomalies with lower values than expected.
The old behavior can be restored by specifying `flush_on_shutdown: true` setting in streaming aggregation config

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-26 22:45:45 +01:00
Roman Khavronenko
562edb72ea
app/vmalert: fix data race during hot-config reload (#5698)
* app/vmalert: fix data race during hot-config reload

During hot-reload, the logic evokes the group update and rules evaluation
interruption simultaneously. Falsely assuming that interruption happens before
the update. However, it could happen that group will be updated first and only
after the rules evaluation will be cancelled. Which will result in permanent
interruption for all rules within the group.

The fix caches the cancel context function into local variable first. And only after
performs the group update. With cached cancel function we can safely call it without
worrying that we cancel the evaluation for already updated group.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Revert "app/vmalert: fix data race during hot-config reload"

This reverts commit a4bb7e8932.

* app/vmalert: fix data race during hot-config reload

During hot-reload, the logic evokes the group update and rules evaluation
interruption simultaneously. Falsely assuming that interruption happens before
the update. However, it could happen that group will be updated first and only
after the rules evaluation will be cancelled. Which will result in permanent
interruption for all rules within the group.

The fix cancels the evaulation context before applying the update, making sure
that the context will be cancelled for old group always.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-26 22:43:02 +01:00
Yury Molodov
551f48466c
vmui: fix Enter key in query field (#5667) (#5681) 2024-01-26 22:38:51 +01:00
Artem Navoiev
d42908133c
docs: remove <p> for imanges (#5702)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-01-26 22:34:40 +01:00
Artem Navoiev
36fa314161
remove all <div> as far they obsolete and can break markdown (#5701)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-01-26 22:32:54 +01:00
Aliaksandr Valialkin
2b0123058a
docs: update -help output after bb7a419cc3 2024-01-26 22:29:22 +01:00