Commit graph

8688 commits

Author SHA1 Message Date
Artem Navoiev
1738328e75
add file placeholder grafana logs datasource docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-03 14:07:02 +02:00
Artem Navoiev
8e14d7bd00
remove unsued image
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-03 13:44:07 +02:00
Github Actions
a647fa47c8
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@dae8560 (#6568)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-03 04:20:47 -07:00
Aliaksandr Valialkin
2cdab5ba49
docs/MetricsQL.md: document which metric types are usually passed to which rollup functions
This should reduce incorrect usage of rollup functions like this one - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3974#issuecomment-2205574667
2024-07-03 11:36:31 +02:00
Aliaksandr Valialkin
916d285f9f
docs: update VictoriaMetrics release from v1.100.1 to v1.101.0 across all the docs
This is a follow-up for 5e8c087d42

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6194
2024-07-03 01:06:29 +02:00
Aliaksandr Valialkin
61d794c5e7
app/vmselect/promql: follow-up for dd0d2c77c8 and 6149adbe10
Use metricsql.IsLikelyInvalid() function for determining whether the given query is likely invalid,
e.g. there is high change the query is incorrectly written, so it will return unexpected results.

The query is invalid most of the time if it passes something other than series selector into rollup function.
For example:

- rate(sum(foo))
- rate(foo + bar)
- rate(foo > bar)

Improtant note: the query is considered valid if it misses the lookbehind window in square brackes inside rollup function,
e.g. rate(foo), since this is very convenient MetricsQL extention to PromQL, and this query returns the expected results
most of the time.

Other unsafe query types can be added in the future into metricsql.IsLikelyInvalid().

TODO: probably, the -search.disableImplicitConversion command-line flag must be set by default in the future releases of VictoriaMetrics.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4338
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6180
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6450
2024-07-03 00:47:10 +02:00
Aliaksandr Valialkin
f5518b2adc
deployment/docker: update Go builder from Go1.22.4 to Go1.22.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.22.5+label%3ACherryPickApproved
2024-07-03 00:07:09 +02:00
Aliaksandr Valialkin
f15ba5eac3
vendor: run make vendor-update 2024-07-03 00:00:09 +02:00
Aliaksandr Valialkin
f17b408643
lib/streamaggr: follow-up for the commit c0e4ccb7b5
- Clarify docs for `Ignore aggregation intervals on start` feature.

- Make more clear the code dealing with ignoreFirstIntervals at aggregator.runFlusher() functions.
  It is better from readability and maintainability PoV using distinct a.flush() calls
  for distinct cases instead of merging them into a single a.flush() call.

- Take into account the first incomplete interval when tracking the number of skipped aggregation intervals,
  since this behaviour is easier to understand by the end users.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6137
2024-07-02 21:24:50 +02:00
hagen1778
ffb49c677b
docs: mention graphite.sanitizeMetricName in cluster docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-02 17:17:54 +02:00
Andrii Chubatiuk
476faf5578
lib/protoparser/graphite: added -graphite.sanitizeMetricName flag (#6489)
### Describe Your Changes

Added flag to sanitize graphite metrics
fixes #6077

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-02 14:56:41 +02:00
Zakhar Bessarab
f3831bdd13
vmbackupmanager: fix state restore (#773)
* app/vmbackupmanager: fix state restore on startup

Fix improperly treating completed backups as "failed" in metrics.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/changelog: document the fix

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-02 14:35:51 +02:00
LHHDZ
4d66e042e3
app/vmauth: reader pool to reduce gc & mem alloc (#6533)
follow up https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6446

issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6445

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-07-02 14:32:32 +02:00
Github Actions
dd97dd6373
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@e9bd079 (#6564) 2024-07-02 14:10:59 +02:00
hagen1778
e45d80cd79
dashboards: fix wrong templating for vmauth
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-02 13:08:11 +02:00
Aliaksandr Valialkin
d506e47c5a
docs/VictoriaLogs/CHANGELOG.md: cut v0.27.0-victorialogs 2024-07-02 01:43:46 +02:00
Aliaksandr Valialkin
3b6c78c26c
lib/logstorage: allow writing after N in front of before N at stream_context pipe 2024-07-02 01:38:20 +02:00
Aliaksandr Valialkin
e11f0aa9ec
app/vlinsert/insertutils: flush the ingested logs from in-memory buffer to storage every second
Previously the in-memory buffer could remain unflushed for long periods of time under low ingestion rate.
The ingested logs weren't visible for search during this time.
2024-07-02 01:38:19 +02:00
Aliaksandr Valialkin
ba6f82069f
app/vlinsert/syslog: add an ability to use log ingestion time as the _time field 2024-07-02 01:38:19 +02:00
Hui Wang
dd802eaf1d
app/vmselect/netstorage: do not retry request when complexity limit i… (#6469)
…s already exceeded

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 16:41:34 +02:00
Aliaksandr Valialkin
387b3b7fb7
docs/VictoriaLogs/CHANGELOG.md: use new url https://docs.victoriametrics.com/victorialogs/querying/ instead of old one https://docs.victoriametrics.com/VictoriaLogs/querying/ 2024-07-01 16:31:42 +02:00
Hui Wang
9da78f1e0e
vmui: increase max query tab from 4 to 10 (#6546) 2024-07-01 15:52:19 +02:00
Andrii Chubatiuk
0a42c8fd8b
deployment: remove snap packages support (#6543)
### Describe Your Changes

Removed snap packages support as it requires time for maintenance and
it's not popular at all

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 15:51:32 +02:00
Andrii Chubatiuk
861852f262
lib/streamaggr: added stale samples metric, added metrics labels (#6462)
### Describe Your Changes

- added stale metrics counters for input and output samples
- added labels for aggregator metrics =>
`name="{rwctx}:{aggrId}:{aggrSuffix}"`
   - rwctx - global or number starting from 1
   - aggrid - aggregator id starting from 1
   - aggrSuffix - <interval>_(by|without)_label1_label2_labeln
   e.g: `name="global:1:1m_without_instance_pod"`

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 14:56:17 +02:00
hagen1778
f8eea0f2c9
docs: mark without optional in stream aggr docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 13:51:20 +02:00
Aliaksandr Valialkin
839596c00d
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.1-victorialogs 2024-07-01 02:32:51 +02:00
Aliaksandr Valialkin
6bb66cb3e9
lib/logstorage: properly search for the surrounding logs in stream_context pipe
The set of log fields in the found logs may differ from the set of log fields present in the log stream.
So compare only the log fields in the found logs when searching for the matching log entry in the log stream.

While at it, return _stream field in the delimiter log entry, since this field is used by VictoriaLogs Web UI
for grouping logs by log streams.
2024-07-01 02:29:50 +02:00
Aliaksandr Valialkin
e0ea83c466
docs/VictoriaLogs: remove "preview" warning - VictoriaLogs is ready for prod 2024-07-01 01:52:33 +02:00
Aliaksandr Valialkin
353e26cdfc
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.0-victorialogs 2024-07-01 01:49:49 +02:00
Artem Navoiev
4b863621b4
operator: sync docs with operator repo, add alias
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-01 01:46:25 +02:00
Aliaksandr Valialkin
bb0deb7ac4
lib/logstorage: add ability to store sorted log position into a separate field with sort ... rank <fieldName> syntax 2024-07-01 01:44:17 +02:00
Aliaksandr Valialkin
dc291d8980
lib/logstorage: add delimiter between log chunks returned from | stream_context pipe 2024-07-01 01:30:37 +02:00
Aliaksandr Valialkin
c9568d45dc
docs/VictoriaLogs: typo fixes 2024-06-28 19:26:24 +02:00
Aliaksandr Valialkin
9877c086b9
docs/VictoriaLogs/CHANGELOG.md: cut v0.25.0-victorialogs 2024-06-28 19:17:49 +02:00
Aliaksandr Valialkin
d4ca651547
lib/logstorage: add stream_context pipe, which allows selecting surrounding logs for the matching logs 2024-06-28 19:14:29 +02:00
Aliaksandr Valialkin
d7185f1b77
app/vlinsert/syslog: properly skip empty lines in Syslog protocol
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6548
2024-06-28 14:09:28 +02:00
Aliaksandr Valialkin
e8322147e9
app/vlselect/logsql: add optional fields_limit query arg to /select/logsql/hits HTTP endpoint
This query arg is needed for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545
in order to return top N groups with the biggest number of hits.
2024-06-28 03:08:40 +02:00
Aliaksandr Valialkin
59e639eed2
docs/VictoriaLogs: typo fixes 2024-06-28 02:21:19 +02:00
Aliaksandr Valialkin
820889cf53
docs/VictoriaLogs: document how to export logs from VictoriaLogs
While at it, mention vmctl-like tool for migrating logs from other systems to VictoriaLogs at docs/VictoriaLogs/Roadmap.md

Thanks to the question from Xavier Pestel at https://www.linkedin.com/feed/update/urn:li:activity:7212093021301927937?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A7212093021301927937%2C7212171550651731969%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287212171550651731969%2Curn%3Ali%3Aactivity%3A7212093021301927937%29
2024-06-28 02:06:12 +02:00
Aliaksandr Valialkin
0730f1324d
lib/logstorage: it is safe using | unroll pipe in live tailing
`| unroll` pipe can make multiple copies of rows from the input row.
This doesn't break live tailing, so allow `| unroll` pipe in live tailing.
2024-06-27 19:44:57 +02:00
Aliaksandr Valialkin
48780d9ec5
docs/VictoriaLogs/querying/README.md: clarify live tailing docs 2024-06-27 19:12:42 +02:00
Aliaksandr Valialkin
e26ab72705
docs/VictoriaLogs/Roadmap.md: remove Web UI from roadmap, since it is already implemented
See https://docs.victoriametrics.com/victorialogs/querying/#web-ui

While at it, add DataDog protocol for logs into 'data ingestion protocols' roadmap.
2024-06-27 18:59:46 +02:00
Aliaksandr Valialkin
107894f13c
docs/VictoriaLogs/querying: remove unused and outdated vmui.webp image
The image has been deleted from the docs/VictoriaLogs/querying/README.md file
in the commit 87f1c8bd6c
2024-06-27 15:16:06 +02:00
Aliaksandr Valialkin
7fcc0823f1
docs/VictoriaLogs/querying/README.md: add a link to live tailing docs from querying log docs 2024-06-27 15:12:49 +02:00
Aliaksandr Valialkin
7c8c040502
app/vlselect: properly return live tailing results 2024-06-27 15:05:57 +02:00
Aliaksandr Valialkin
87f1c8bd6c
lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
Aliaksandr Valialkin
96bdeb3f10
docs/goals.md: clarify that the usability of the existing functionality must be improved the first 2024-06-27 09:55:11 +02:00
Artem Navoiev
c9f496bdd0
dashboards: update statistic by tenant dashboard, fix billing disk usage pie panel (#6521)
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-06-27 09:29:25 +02:00
Arkadii Yakovets
8a982e88cd
github/workflows: run CodeQL analysis for both JavaScript and TypeScript (#6541)
### Describe Your Changes

Enables CodeQL TypeScript analysis.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>
2024-06-27 09:25:05 +02:00
Arkadii Yakovets
c3f8af5636
github/workflows: run CodeQL Go analysis for .go files only (#6540)
### Describe Your Changes

The current CodeQL setup runs Go analysis on irrelevant files (see [this
run](https://github.com/VictoriaMetrics/VictoriaMetrics/actions/runs/9380962792)
as an example that took around 3 minutes, [PR
link](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6419)).

The PR makes sure the Go CodeQL job is triggered by `**.go` path pattern
only.
It contains 2 commits -- first for a better diff, and second for
renaming the config file.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>
2024-06-27 09:23:00 +02:00