Commit graph

371 commits

Author SHA1 Message Date
Aliaksandr Valialkin
91f5417572
docs/VictoriaLogs: replace incorrect usafe of VictoriaMetrics with VictoriaLogs 2024-10-13 13:01:42 +02:00
Aliaksandr Valialkin
200d723b9a
docs/VictoriaLogs/Roadmap.md: remove OpenTelemetry, Fluentd and Telegraf from the list of data ingestion protocols, which are going to be supported by VictoriaLogs
These protocols are already supported:

- OpenTelemetry protocol - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4839 ,
  https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6218 and 01430a155c

- Fluentd protocol - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7098
  and 05a64a8c14

- Telegraf protocol - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5310 ,
  https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6443 and 1731c0eabf
2024-10-13 11:10:19 +02:00
Aliaksandr Valialkin
867f671cc4
lib/logstorage: make sure that bs.br is non-nil before checking br.bs.bsw.bh.rowsCount there
br.bs may be nil when br contains the block with additional filters applied during pipe calculations.
For example, `* | count() if (error) errors`.
2024-10-12 20:51:29 +02:00
Zakhar Bessarab
2239f5829f
docs/victoria-logs: fix typo (#7245)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-10-12 02:24:51 +04:00
Aliaksandr Valialkin
b9a8c1ff3a
docs/VictoriaLogs/querying/vlogscli.md: add the list of features of vlogscli 2024-10-10 11:49:10 +02:00
n4mine
f060b67da5
docs: fix typo in docs/VictoriaLogs/data-ingestion/Vector.md (#7222)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-10 09:45:30 +02:00
Aliaksandr Valialkin
baa87b5b36
deployment: update VictoriaLogs from v0.34.0-victorialogs to v0.35.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.35.0-victorialogs
2024-10-09 16:17:51 +02:00
Aliaksandr Valialkin
252aa792f7
docs/VictoriaLogs: cut v0.35.0 release 2024-10-09 15:55:20 +02:00
Aliaksandr Valialkin
9413b2de91
docs/VictoriaLogs: make vlogscli more visible 2024-10-09 15:54:25 +02:00
Aliaksandr Valialkin
ad5d8097da
app/vlogscli: add -accountID and -projectID command-line flags for querying the given tenants 2024-10-09 12:56:49 +02:00
Aliaksandr Valialkin
e31625e0b2
app/vlogscli: add support for live tailing 2024-10-09 12:30:17 +02:00
n4mine
344d61da79
docs: fix typo in docs/VictoriaLogs/data-ingestion/Vector.md (#7208)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-09 00:32:36 -07:00
Aliaksandr Valialkin
03862368b5
deployment: update VictoriaLogs image tag from v0.33.0-victorialogs to v0.34.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.34.0-victorialogs
2024-10-08 12:30:27 +02:00
Aliaksandr Valialkin
6878982c93
docs/VictoriaLogs/CHANGELOG.md: cut v0.34.0 release 2024-10-08 12:21:19 +02:00
Aliaksandr Valialkin
492190885d
app/vlogscli: add ability to display query results in logfmt, single-line and multi-line json modes 2024-10-07 12:20:06 +02:00
Aliaksandr Valialkin
daad96b3a5
app/vlogscli: return back sorting result fields by name
This simplifies locating the needed field when the number of fields per each returned result is big
2024-10-07 10:41:48 +02:00
Aliaksandr Valialkin
596e4de248
app/vlogscli: preserve the original order of fields in the displayed responses 2024-10-05 21:27:32 +02:00
Aliaksandr Valialkin
66645c3dff
docs/VictoriaLogs/LogsQL.md: remove redundant replace word in examples for conditional replace
Thanks to @p5i for the bugreport at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7168
2024-10-03 20:26:58 +02:00
Aliaksandr Valialkin
364f084b43
lib/logstorage: add len pipe for calculating byte length of log field values 2024-10-03 18:21:10 +02:00
Aliaksandr Valialkin
3634fefc64
docs/VictoriaLogs/querying/vlogscli.md: typo fixes 2024-10-01 21:10:05 +02:00
Aliaksandr Valialkin
496015aa0e
docs/VictoriaLogs/querying/vlogscli.md: provide instructions on how to download run vlogscli 2024-10-01 16:13:12 +02:00
Aliaksandr Valialkin
b05fbee63d
deployment: update VictoriaLogs docker image from v0.32.1-victorialogs to v0.33.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.33.0-victorialogs
2024-10-01 13:57:56 +02:00
Aliaksandr Valialkin
234c81754e
docs/VictoriaLogs/CHANGELOG.md: cut v0.33.0-victorialogs release 2024-10-01 13:42:18 +02:00
Aliaksandr Valialkin
a350be48b6
lib/logstorage: do not count dictionary values which have no matching logs in count_uniq stats function
Create blockResultColumn.forEachDictValue* helper functions for visiting matching
dictionary values. These helper functions should prevent from counting dictionary values
without matching logs in the future.

This is a follow-up for 0c0f013a60
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7152
2024-10-01 13:34:45 +02:00
Aliaksandr Valialkin
630211cfed
app/vlogscli: add interactive command-line tool for querying VictoriaLogs 2024-10-01 12:23:07 +02:00
Aliaksandr Valialkin
61ae077e43
docs/VictoriaLogs/FAQ.md: clarify answers a bit 2024-10-01 12:08:27 +02:00
Aliaksandr Valialkin
aafa9262c5
deployment/docker: update VictoriaLogs from v0.32.0-victorialogs to v0.32.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.1-victorialogs
2024-09-30 14:35:40 +02:00
Aliaksandr Valialkin
82482fca4b
docs/VictoriaLogs/CHANGELOG.md: cut v0.32.1-victorialogs release 2024-09-30 14:31:17 +02:00
Aliaksandr Valialkin
0c0f013a60
lib/logstorage: skip values with zero hits for 'uniq', 'top' and 'field_values' pipes
See https://github.com/VictoriaMetrics/victorialogs-datasource/issues/72#issuecomment-2352078483
2024-09-30 14:15:07 +02:00
Aliaksandr Valialkin
2047ad20ef
deployment: update VictoriaLogs docker image from v0.31.0-victorialogs to v0.32.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.0-victorialogs
2024-09-29 14:54:21 +02:00
Aliaksandr Valialkin
45cfb6b526
docs/VictoriaLogs/CHANGELOG.md: cut v0.32.0-victorialogs 2024-09-29 14:47:31 +02:00
Aliaksandr Valialkin
55eb321f77
lib/logstorage: clear hits slice obtained from encoding.GetUint64s() before updating it with hits for valueTypeDict column
encoding.GetUint64s() returns uninitialized slice, which may contain arbitrary values.
So values in this slice must be reset to zero before using it for counting hits in `uniq` and `top` pipes.
2024-09-29 10:29:13 +02:00
Aliaksandr Valialkin
0b91452ca4
lib/logstorage: add non-empty if (...) condition to automatically generated result names in stats pipe
This allows executing queries with `stats` pipe, which calculate multiple results with the same functions,
but with different `if (...)` conditions. For example:

  _time:5m | count(), count() if (error)

Previously such queries couldn't be executed becasue automatically generated name for the second result
didn't include `if (error)`, so names for both results were identical - `count(*)`.
2024-09-29 09:51:28 +02:00
Aliaksandr Valialkin
8772aea24b
lib/logstorage: support order alias for sort pipe
Now the following queries are equivalents:

    _time:5s | sort by (_time)

    _time:5s | order by (_time)

This is needed for convenience, since `order by` is commonly used in other query languages such as SQL.
2024-09-29 09:51:27 +02:00
Aliaksandr Valialkin
806bc2ac58
app/vlinsert: support unix timestamps in seconds and milliseconds in JSON stream data ingestion API 2024-09-28 21:56:50 +02:00
Aliaksandr Valialkin
7d7d7c03bc
app/vlinsert: accept unix timestamp in seconds additionally to milliseconds at ElasticSearch bulk API
Timestamps in seconds are sometimes used for data ingestion via ElasticSearch bulk API
2024-09-28 21:19:54 +02:00
Andrii Chubatiuk
05a64a8c14
victorialogs: marked fluentd support in roadmap, added syslog example (#7098)
### Describe Your Changes

Marked fluentd in victorialogs roadmap
Added fluentd syslog example setup

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-27 14:38:39 +02:00
Aliaksandr Valialkin
86c0eb816c
deployment: update VictoriaLogs docker image from v0.30.1-victorialogs to v0.31.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.31.0-victorialogs
2024-09-27 13:58:54 +02:00
Aliaksandr Valialkin
58c69386c7
docs/VictoriaLogs/CHANGELOG.md: cut v0.31.0-victorialogs release 2024-09-27 13:54:17 +02:00
Yury Molodov
8657d03433
vmui/logs: improve graph usability (#7025)
### Describe Your Changes

- Show the time range in the tooltip when hovering over staircase
graphs.
- Use bolder lines for staircase graphs.
- Increase the number of steps on the staircase graph to 100.
- Reduce the maximum width of the tooltip to 1/3 of the screen.
- Insert only the label name under the cursor into the query input field
when `Ctrl`-clicking the line legend.

See [this
comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545#issuecomment-2336805237).

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-09-27 13:19:46 +02:00
Aliaksandr Valialkin
09b309a82e
lib/logstorage: allow using - instead of ! as a shorthand for NOT operator in LogsQL 2024-09-27 13:14:47 +02:00
Aliaksandr Valialkin
76c1b0b8ea
lib/logstorage: support skipping _stream: prefix for stream filters
'_stream:{...}' can be written as '{...}'

This simplifies writing queries with stream filters, and makes them more familier to Loki users.
2024-09-27 13:14:46 +02:00
Yury Molodov
c896bf340d
vmui: add functionality to preserve selected columns (#7037)
### Describe Your Changes

1) Changed table settings from a popup to a modal window to simplify
future functionality additions.
2) Added functionality to save selected columns when data is modified or
the page is reloaded. See #7016.

<details>
  <summary>Example screenshots</summary>
  
<img alt="demo-1" width="600"
src="https://github.com/user-attachments/assets/a5d9a910-363c-4931-8b12-18ea8b3d97d8"/>
  
</details>


### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-27 11:52:01 +02:00
Aliaksandr Valialkin
b670b0e9ff
deployment: update VictoriaLogs docker image tag from v0.30.0-victorialogs to v0.30.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.30.1-victorialogs
2024-09-27 11:26:15 +02:00
Aliaksandr Valialkin
8077585303
docs/VictoriaLogs/CHANGELOG.md: cut v0.30.1-victorialogs release 2024-09-27 11:20:58 +02:00
Aliaksandr Valialkin
9367a9a6a2
lib/logstorage: consistently sort stream contexts belonging to different streams by the minimum time seen in the matching logs
This should simplify debugging of stream_context output, since it remains stable over repeated requests.
2024-09-27 11:19:26 +02:00
Aliaksandr Valialkin
13cc4a2618
docs/VictoriaLogs/LogsQL.md: use proper heading for blocks_count pipe docs
All the links in docs assume that the heading for pipe docs should end with `pipe` word.
This fixes broken links.

This is a follow-up for e9950f6307
2024-09-27 09:45:46 +02:00
Aliaksandr Valialkin
6f1fde24dc
deployment: update VictoriaLogs docker image from v0.29.0-victorialogs to v0.30.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.30.0-victorialogs
2024-09-27 09:34:09 +02:00
Aliaksandr Valialkin
31117c66d5
docs/VictoriaLogs/CHANGELOG.md: cut v0.30.0-victorialogs release 2024-09-27 09:18:31 +02:00
Aliaksandr Valialkin
4b1611267f
lib/logstorage: properly return surrounding logs outside the selected time range by stream_context pipe
Previously only logs inside the selected time range could be returned by stream_context pipe.
For example, the following query could return up to 10 surrounding logs only for the last 5 minutes,
while most users expect this query should return up to 10 surrounding logs without restrictions on the time range.

    _time:5m panic | stream_context before 10

This enables the ability to implement stream context feature at VictoriaLogs web UI: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7063 .

Reduce memory usage when returning stream context over big log streams with millions of entries.
The new logic scans over all the log messages for the selected log stream, while keeping in memory only
the given number of surrounding logs. Previously all the logs for the given log stream on the selected time range
were loaded in memory before selecting the needed surrounding logs.
This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6730 .

Reduce the scan performance for big log streams by fetching only the requested fields. For example, the following
query should be executed much faster than before if logs contain many fields other than _stream, _msg and _time:

    panic | stream_context after 30 | fields _stream, _msg, _time
2024-09-26 17:03:45 +02:00