Commit graph

9071 commits

Author SHA1 Message Date
Aliaksandr Valialkin
f8779d1ed2
lib/streamaggr: follow-up for the commit c0e4ccb7b5
- Clarify docs for `Ignore aggregation intervals on start` feature.

- Make more clear the code dealing with ignoreFirstIntervals at aggregator.runFlusher() functions.
  It is better from readability and maintainability PoV using distinct a.flush() calls
  for distinct cases instead of merging them into a single a.flush() call.

- Take into account the first incomplete interval when tracking the number of skipped aggregation intervals,
  since this behaviour is easier to understand by the end users.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6137
2024-07-02 21:34:48 +02:00
Aliaksandr Valialkin
41f95d90f9
README.md: sync with docs/Cluster-VictoriaMetrics.md after the commit ae76794a19
This fixes `make docs-sync` results in cluster branch

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3923
2024-07-02 18:56:21 +02:00
hagen1778
e423c4b72c
docs: mention graphite.sanitizeMetricName in cluster docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit ffb49c677b)
2024-07-02 17:18:07 +02:00
Andrii Chubatiuk
252aa5a3ab
lib/protoparser/graphite: added -graphite.sanitizeMetricName flag (#6489)
### Describe Your Changes

Added flag to sanitize graphite metrics
fixes #6077

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 476faf5578)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-02 17:16:00 +02:00
Zakhar Bessarab
4ece1747d3
vmbackupmanager: fix state restore (#773)
* app/vmbackupmanager: fix state restore on startup

Fix improperly treating completed backups as "failed" in metrics.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/changelog: document the fix

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f3831bdd13)
2024-07-02 14:37:15 +02:00
LHHDZ
c8431c8e4d
app/vmauth: reader pool to reduce gc & mem alloc (#6533)
follow up https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6446

issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6445

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
(cherry picked from commit 4d66e042e3)
2024-07-02 14:37:15 +02:00
Github Actions
3fda4cc4de
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@e9bd079 (#6564)
(cherry picked from commit dd97dd6373)
2024-07-02 14:37:15 +02:00
hagen1778
309a767fc5
dashboards: fix wrong templating for vmauth
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit e45d80cd79)
2024-07-02 14:37:15 +02:00
Aliaksandr Valialkin
fed596b202
docs/VictoriaLogs/CHANGELOG.md: cut v0.27.0-victorialogs 2024-07-02 01:43:54 +02:00
Aliaksandr Valialkin
7426b40250
lib/logstorage: allow writing after N in front of before N at stream_context pipe 2024-07-02 01:39:45 +02:00
Aliaksandr Valialkin
0912a652d5
app/vlinsert/insertutils: flush the ingested logs from in-memory buffer to storage every second
Previously the in-memory buffer could remain unflushed for long periods of time under low ingestion rate.
The ingested logs weren't visible for search during this time.
2024-07-02 01:39:45 +02:00
Aliaksandr Valialkin
ab28a1f93e
app/vlinsert/syslog: add an ability to use log ingestion time as the _time field 2024-07-02 01:39:45 +02:00
Aliaksandr Valialkin
f4dd9bd988
docs/VictoriaLogs/CHANGELOG.md: use new url https://docs.victoriametrics.com/victorialogs/querying/ instead of old one https://docs.victoriametrics.com/VictoriaLogs/querying/
(cherry picked from commit 387b3b7fb7)
2024-07-01 16:40:44 +02:00
Hui Wang
085bc1f15c
vmui: increase max query tab from 4 to 10 (#6546)
(cherry picked from commit 9da78f1e0e)
2024-07-01 16:40:42 +02:00
Andrii Chubatiuk
65c742d976
deployment: remove snap packages support (#6543)
### Describe Your Changes

Removed snap packages support as it requires time for maintenance and
it's not popular at all

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 0a42c8fd8b)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 16:40:39 +02:00
Hui Wang
87cb132f53
app/vmselect/netstorage: do not retry request when complexity limit i… (#6469)
…s already exceeded

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 16:38:15 +02:00
Andrii Chubatiuk
937ae2ca90
lib/streamaggr: added stale samples metric, added metrics labels (#6462)
### Describe Your Changes

- added stale metrics counters for input and output samples
- added labels for aggregator metrics =>
`name="{rwctx}:{aggrId}:{aggrSuffix}"`
   - rwctx - global or number starting from 1
   - aggrid - aggregator id starting from 1
   - aggrSuffix - <interval>_(by|without)_label1_label2_labeln
   e.g: `name="global:1:1m_without_instance_pod"`

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 861852f262)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 15:01:49 +02:00
hagen1778
69625aa8a1
docs: mark without optional in stream aggr docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f8eea0f2c9)
2024-07-01 14:56:46 +02:00
Aliaksandr Valialkin
5143e93534
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.1-victorialogs 2024-07-01 02:33:02 +02:00
Aliaksandr Valialkin
208a624d4d
lib/logstorage: properly search for the surrounding logs in stream_context pipe
The set of log fields in the found logs may differ from the set of log fields present in the log stream.
So compare only the log fields in the found logs when searching for the matching log entry in the log stream.

While at it, return _stream field in the delimiter log entry, since this field is used by VictoriaLogs Web UI
for grouping logs by log streams.
2024-07-01 02:33:00 +02:00
Aliaksandr Valialkin
516c7b2ca0
docs/VictoriaLogs: remove "preview" warning - VictoriaLogs is ready for prod 2024-07-01 01:52:40 +02:00
Aliaksandr Valialkin
0e6ee55a95
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.0-victorialogs 2024-07-01 01:49:55 +02:00
Aliaksandr Valialkin
76a58ae08d
lib/logstorage: add ability to store sorted log position into a separate field with sort ... rank <fieldName> syntax 2024-07-01 01:46:03 +02:00
Aliaksandr Valialkin
d0dca7b8c5
lib/logstorage: add delimiter between log chunks returned from | stream_context pipe 2024-07-01 01:46:02 +02:00
Artem Navoiev
2ac2d0919d
operator: sync docs with operator repo, add alias
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-06-29 17:19:49 +02:00
Aliaksandr Valialkin
76053a0ef0
docs/VictoriaLogs: typo fixes 2024-06-28 19:26:32 +02:00
Aliaksandr Valialkin
75c7b2c07a
docs/VictoriaLogs/CHANGELOG.md: cut v0.25.0-victorialogs 2024-06-28 19:18:21 +02:00
Aliaksandr Valialkin
4b3477e62b
lib/logstorage: add stream_context pipe, which allows selecting surrounding logs for the matching logs 2024-06-28 19:15:19 +02:00
Aliaksandr Valialkin
c9fc8079c4
app/vlinsert/syslog: properly skip empty lines in Syslog protocol
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6548
2024-06-28 14:09:45 +02:00
Aliaksandr Valialkin
bb6424aeca
app/vlselect/logsql: add optional fields_limit query arg to /select/logsql/hits HTTP endpoint
This query arg is needed for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545
in order to return top N groups with the biggest number of hits.
2024-06-28 03:10:05 +02:00
Aliaksandr Valialkin
3eecc3de8c
docs/VictoriaLogs: typo fixes 2024-06-28 03:10:05 +02:00
Aliaksandr Valialkin
5a24ea6cc3
docs/VictoriaLogs: document how to export logs from VictoriaLogs
While at it, mention vmctl-like tool for migrating logs from other systems to VictoriaLogs at docs/VictoriaLogs/Roadmap.md

Thanks to the question from Xavier Pestel at https://www.linkedin.com/feed/update/urn:li:activity:7212093021301927937?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A7212093021301927937%2C7212171550651731969%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287212171550651731969%2Curn%3Ali%3Aactivity%3A7212093021301927937%29
2024-06-28 02:06:24 +02:00
Aliaksandr Valialkin
2f28819bb1
lib/logstorage: it is safe using | unroll pipe in live tailing
`| unroll` pipe can make multiple copies of rows from the input row.
This doesn't break live tailing, so allow `| unroll` pipe in live tailing.
2024-06-27 19:45:12 +02:00
Aliaksandr Valialkin
05365999ea
docs/VictoriaLogs/querying/README.md: clarify live tailing docs 2024-06-27 19:12:54 +02:00
Aliaksandr Valialkin
4391ef25e9
docs/VictoriaLogs/Roadmap.md: remove Web UI from roadmap, since it is already implemented
See https://docs.victoriametrics.com/victorialogs/querying/#web-ui

While at it, add DataDog protocol for logs into 'data ingestion protocols' roadmap.
2024-06-27 19:12:54 +02:00
Aliaksandr Valialkin
00899506f7
docs/VictoriaLogs/querying: remove unused and outdated vmui.webp image
The image has been deleted from the docs/VictoriaLogs/querying/README.md file
in the commit 87f1c8bd6c
2024-06-27 15:16:43 +02:00
Aliaksandr Valialkin
13df75a28b
docs/VictoriaLogs/querying/README.md: add a link to live tailing docs from querying log docs 2024-06-27 15:12:59 +02:00
Aliaksandr Valialkin
b26acec9a8
app/vlselect: properly return live tailing results 2024-06-27 15:06:15 +02:00
Aliaksandr Valialkin
dd62a2b9d6
lib/logstorage: work-in-progress 2024-06-27 14:21:03 +02:00
Aliaksandr Valialkin
0352d98d74
docs/goals.md: clarify that the usability of the existing functionality must be improved the first 2024-06-27 09:55:40 +02:00
Artem Navoiev
19c4dfd72c
dashboards: update statistic by tenant dashboard, fix billing disk usage pie panel (#6521)
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit c9f496bdd0)
2024-06-27 09:32:13 +02:00
Arkadii Yakovets
c4bae2ab77
github/workflows: run CodeQL analysis for both JavaScript and TypeScript (#6541)
### Describe Your Changes

Enables CodeQL TypeScript analysis.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>
(cherry picked from commit 8a982e88cd)
2024-06-27 09:32:11 +02:00
Arkadii Yakovets
5a199d7e8f
github/workflows: run CodeQL Go analysis for .go files only (#6540)
### Describe Your Changes

The current CodeQL setup runs Go analysis on irrelevant files (see [this
run](https://github.com/VictoriaMetrics/VictoriaMetrics/actions/runs/9380962792)
as an example that took around 3 minutes, [PR
link](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6419)).

The PR makes sure the Go CodeQL job is triggered by `**.go` path pattern
only.
It contains 2 commits -- first for a better diff, and second for
renaming the config file.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>
(cherry picked from commit c3f8af5636)
2024-06-27 09:32:09 +02:00
Josh Keife
bdebc128f7
Update vmalert.md (#6539)
Fix typo

### Describe Your Changes

Fix typo in the vmalert docs. In the docs it states rules at `VMAgent`'s
namespace when it should be `VMAlert`'s namespace.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 8d01dd0514)
2024-06-27 09:32:07 +02:00
Aliaksandr Valialkin
911891da95
docs/goals.md: clarify that the listed goals are related to development and support, not sales and marketing 2024-06-26 19:12:36 +02:00
Fred Navruzov
ed41f18c5a
docs/vmanomaly: quickstart page improvements (#6538)
### Describe Your Changes

Small improvements to a QuickStart guide of `vmanomaly`

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-06-26 19:12:35 +02:00
Andrii Chubatiuk
580d02c3f8
added IMDSv2 for YC SD (#6524)
### Describe Your Changes

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5513

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-06-26 19:12:35 +02:00
Andrii Chubatiuk
f79df2aa8b
app/vmauth: allow dropping host header (#6525)
### Describe Your Changes

Fixes #6453

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-06-26 19:12:35 +02:00
rtm0
48a5c4cb01
Fix Date metricid cache consistency under concurrent use (#6534)
### Describe Your Changes

Fix Date metricid cache consistency under concurrent use.
When one goroutine calls Has() and does not find the cache entry in the
immutable map it will acquire a lock and check the mutable map. And it
is possible that before that lock is acquired, the entry is moved from
the mutable map to the immutable map by another goroutine causing a
cache miss.

The fix is to check the immutable map again once the lock is acquired. 

### Checklist

The following checks are **mandatory**:

- [x ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <wwctrsrx@gmail.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-06-26 19:12:34 +02:00
Aliaksandr Valialkin
3a2c0b0a61
docs/goals.md: heading fixes 2024-06-26 16:41:21 +02:00