Commit graph

359 commits

Author SHA1 Message Date
Yury Molodov
bc9cb69170
vmui/logs: add auto refresh (#7038)
### Describe Your Changes

Add auto refresh 
#7017


![image](https://github.com/user-attachments/assets/20ed1102-d5e4-4d3f-9c24-7d298d93400a)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-19 11:11:16 +02:00
Aliaksandr Valialkin
e86891b010
app/vlselect/logsql: call Query.Optimize() on the cloned query in order to replace * filter with filterNoop inside getLastNQueryResults()
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785
2024-09-18 18:24:17 +02:00
Dima Lazerka
8207879fa3
docs: fixes misspelled typos
Also tried to make it catch "Authorisation" in the future, fixed a lot
of other misspells along the way, but didn't make it catch
"Authorisation" anyway.

- Fix misspelled "Authorization" header name
- Fix misspelled "organization"
- Fix more misspells
2024-09-13 12:14:24 +02:00
Aliaksandr Valialkin
e92f347336
docs/VictoriaLogs/querying/README.md: typo fixes: use field_values instead field_names where needed 2024-09-11 09:39:22 +02:00
Artem Navoiev
277fed9990
victorialogs: add HA example for logstash and fluentbit (#6968)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 14:33:05 -07:00
Artem Navoiev
153cceb124
docs: adjust image size
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 21:30:54 +02:00
Artem Navoiev
ae5f98f46b
docs: fix links to docker
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 19:56:58 +02:00
Artem Navoiev
71df11dfbb
docs: victorialogs just mention telegraf as we support not only http but more
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 17:41:15 +02:00
Artem Navoiev
47cd7bb4c1
docs: update roadmap, mark telegraf and otel as done and link the docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 17:35:14 +02:00
Artem Navoiev
f9a8c09fe8
docs: Add VictoriaLogs HA example for single nodes (#6945)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 03:39:58 -07:00
Aliaksandr Valialkin
0ed835026f
deployment: update VictoriaLogs from v0.28.0-victorialogs to v0.29.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.29.0-victorialogs
2024-09-08 21:23:16 +02:00
Aliaksandr Valialkin
fb339d7574
docs/VictoriaLogs/CHANGELOG.md: cut v0.29.0 2024-09-08 21:10:17 +02:00
Aliaksandr Valialkin
eaee2d7db4
lib/logstorage: improve error logging for incorrect queries passed to /select/logsql/stats_query and /select/logsql/stats_query_range functions 2024-09-08 11:24:44 +02:00
Aliaksandr Valialkin
0a40064a6f
app/vlselect: add /select/logsql/stats_query_range endpoint for building time series panels in VictoriaLogs plugin for Grafana
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6943
Updates https://github.com/VictoriaMetrics/victorialogs-datasource/issues/61
2024-09-07 00:41:47 +02:00
Aliaksandr Valialkin
c9bb4ddeed
app/vlselect: add /select/logsql/stats_query endpoint, which is going to be used by vmalert
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6942
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6706
2024-09-06 23:06:43 +02:00
Aliaksandr Valialkin
00e7d5add3
lib/logstorage: substitute | operator with or operator at math pipe
This is needed for avoiding confusion between the `|` operator at `math` pipe and `|` pipe delimiter.
For example, the following query was parsed unexpectedly:

   * | math foo / bar | fields x

as

   * | math foo / (bar | fields) as x

Substituting `|` with `or` inside `math` pipe fixes this ambiguity.
2024-09-06 22:44:14 +02:00
Aliaksandr Valialkin
258ccfb953
lib/logstorage: pre-calculate hashes from tokens used in bloom filter search
Previously per-token hashes for per-block bloom filters were re-calculated on every scanned block.
This could be slow when the number of tokens is big or when the number of blocks to scan is big.
Pre-calculate hashes for bloom filters and then use them for searching in bloom filters.
This improves performance by 2.5x for in(...) filters with many values to search inside `in()`.
2024-09-05 19:44:17 +02:00
Aliaksandr Valialkin
2dd845fa53
lib/logstorage: properly fix incorrect extraction of common tokens for OR filters at distinct log fields
Previously (f1:foo OR f2:bar) was incorrectly returning `foo` token for `f1` and `bar` token for `f2`.
These tokens were used for checking against bloom filter for every data block, so the data block,
which didn't contain simultaneously `foo` token for `f1` field and `bar` token for `f2` field, was skipped.
This was incorrect, since such a block may contain logs matching the original OR filter.

The fix is to return common tokens from `OR`-delimted filters only if these tokens exist at EVERY such filter
for the given field name. If some `OR`-delimited filter misses the given field name, then `OR`-delimited filters
do not contain common tokens, which could be used for checking against bloom filter.

While at it, add more tests covering various edge cases for filters delimited by AND and OR.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6554
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6556
2024-09-05 14:29:50 +02:00
Artem Navoiev
25d433d75f
docs: remove old png file as we have webp now
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-05 14:20:22 +02:00
Github Actions
da9c41e6b0
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@edd3259 (#6946) 2024-09-05 05:19:03 -07:00
f41gh7
b8bbea8896
docs/changelog: moves victorialogs changes to proper file
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-04 15:36:33 +02:00
Andrii Chubatiuk
01430a155c
vlinsert: added opentelemetry logs support
Commit adds the following changes:

* Adds support of OpenTelemetry logs for Victoria Logs with protobuf encoded messages

*  json encoding is not supported for the following reasons:
   - It brings a lot of fragile code, which works inefficiently.
   - json encoding is impossible to use with language SDK.

* splits metrics and logs structures at lib/protoparser/opentelemetry/pb package.

* adds docs with examples for opentelemetry logs.

---
Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4839

Co-authored-by: AndrewChubatiuk <andrew.chubatiuk@gmail.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-09-03 20:12:05 +02:00
f41gh7
8b36529b32
follow-up after 1731c0eabf
* updates change log
* adds VL-Debug http header
* updates doc
* extracts only the first value of http headers for VL-Stream-Fields and VL-Ignore-Fields.
  It makes behaviour the same as Query string args. And allows to easily configure client applications.
  Since most of the client collectors don't support multi value headers.

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-03 19:16:10 +02:00
Andrii Chubatiuk
1731c0eabf
app/vlinsert: support getting _msg_field, _time_field, _stream_fields and _ignore_fields from headers
*  Many collectors don't support forwarding url query params to the remote system. It makes impossible to define stream fields for it. Workaround with proxy between VictoriaLogs and log shipper is too complicated solution.

* This commit adds the following changes:
 * Adds fallback to to headers params, if query param is empty for:
     _msg_field -> VL-Msg-Field
    _stream_fields -> VL-Stream-Fields
    _ignore_fields -> VL-Ignore-Fields
    _time_field -> VL-Time-Field
 * removes deprecations from victorialogs compose files, added more
output format examples for logstash, telegraf, fluent-bit

 related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5310
2024-09-03 17:43:26 +02:00
Github Actions
78067ff2c2
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@1f6f9ea (#6925) 2024-09-03 10:23:18 +02:00
jackyin
975ed27a76
lib/logstorage: and filter results in unexpected response (#6556)
fix #6554
andfilter shouldn't return orfilter field which result in bloomfilter
return false.

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-09-03 10:17:44 +02:00
hagen1778
00d09d67dc
docs/victorialogs/CHANGELOG.md: mention issue #4750 in the change
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-29 15:09:52 +02:00
Cuong Le
6154bc9466
VictoriaLogs/keyConcepts.md: fix broken anchor to #other-fields (#6896)
### Describe Your Changes

The anchor to "Other fields" section should be #other-fields (instead of
#other-field)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Cuong Le <cuongleqq@gmail.com>
2024-08-28 01:02:01 -07:00
Zhu Jiekun
52043796c9
docs: Add backup and restore doc for VictoriaLogs
This commit adds doc for VictoriaLogs backup and restore based on `rsync`.
2024-08-27 21:41:42 +02:00
Yury Molodov
e35237920a
vmui: add column search in table settings (#6804)
### Describe Your Changes

Add search functionality to the column display settings in the table
 #6668
 

![image](https://github.com/user-attachments/assets/e9bd52c3-6428-4d4f-8b7f-d83dd80b6912)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-22 16:57:26 +02:00
hagen1778
2dba4165e8
docs: move change from VM changelog to VL changelog
The change for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6764
was added to VM changelog by mistake

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-20 17:16:04 +02:00
Andrii Chubatiuk
36bc458e9e
docs: reuse hugo image for webp conversion (#6825)
Use same hugo docker image for webp conversion.
While there, remove unused *.png images.
2024-08-19 14:38:56 +02:00
jackyin
5f5bc46b3e
vlogs: add select/deselect all button to table settings in UI (#6680)
fix #6668, just add **select all** and "unselect all" func.


https://github.com/user-attachments/assets/0c31385b-def0-4618-aa9c-5ba4bb6f56c3

---------

Co-authored-by: Yury Molodov <yurymolodov@gmail.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-13 10:20:07 -04:00
Andrii Chubatiuk
35d77a3bed
docs: updated guides structure, removed deprecated sort option (#6767)
### Describe Your Changes

* `sort` param is unused by the current website engine, and was present only for compatibility
with previous website engine. It is time to remove it as it makes no effect
* re-structure guides content into folders to simplify assets management

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-07 16:48:08 +02:00
Yury Molodov
04c2232e45
vmui/logs: add display top streams in the hits graph (#6647)
### Describe Your Changes

- Adds support for displaying the top 5 log streams in the hits graph,
grouping the remaining streams into an "other" label.
   #6545

- Adds options to customize the graph display with bar, line, stepped
line, and points views.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-06 16:28:44 +02:00
Andrii Chubatiuk
2e16732fdb
docs: updated docs titles and links (#6741)
The changes are based on SEO report and supposed to improve
ranking and indexation by search engines by using prompt and unique titles
and by updating unreachable links.

It also updates links to have a simplified form and replaces relative links with absolute links
according to https://docs.victoriametrics.com/#documentation

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-08-06 15:54:52 +02:00
Zakhar Bessarab
58b6c54da2
app/vlinsert/elasticsearch: add fake response for logstash requests (#6742)
### Describe Your Changes

This is needed in order to support standard Elasticsearch output in
Logstash pipelines.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6660

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-08-06 15:43:33 +02:00
Tommy
f23650ccf9
docs: fix typo in VictoriaLogs FAQ.md (#6755)
Update VictoriaLogs FQA section to replace VictoriaMetrics with
VictoriaLogs.

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-06 15:41:04 +02:00
Zhu Jiekun
7551d799f7
docs: Added grafana version requirement for vm+vl datasorce (#6760)
### Describe Your Changes

https://victoriametrics.slack.com/archives/C05UNTPAEDN/p1722833182319299

Sometimes users may use the wrong (lower) version of Grafana when
setting up the VictoriaLogs datasource.

It would be good to document the requirements of Grafana versions to use
VictoriaMetrics/VictoriaLogs datasource.

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-05 16:19:17 +02:00
Yury Molodov
e06a19d85f
vmui/logs: improve UI functionality (#6688)
* add a toggle button to the "Group" tab that allows users to expand or collapse all groups at once
* introduce the ability to select a key for grouping logs within the "Group" tab
* display the number of entries within each log group.
* move the Markdown toggle to the general settings panel in the upper left corner.
2024-08-02 15:48:36 +02:00
Yury Molodov
a05317f61f
vmui/logs: add fields for tenant configuration (#6661)
Added fields for configuring AccountID and ProjectID
#6631
2024-08-02 09:57:39 +02:00
Andrii Chubatiuk
58e667c895
docs: grouped changelog docs, removed old make commands, replaced docs in root README with official docs links (#6727)
### Describe Your Changes

- replace docs in root README with a link to official documentation
- remove old make commands for documentation
- remove redundant "VictoriaMetrics" from document titles
- merge changelog docs into a section
- rm content of Single-server-VictoriaMetrics.md as it can be included from docs/README
- add basic information to README in the root folder, so it will be useful for github users
- rm `picture` tag from docs/README as it was needed for github only, we don't display VM logo at docs.victoriametrics.com
- update `## documentation` section in docs/README to reflect the changes
- rename DD pictures, as they now belong to docs/README

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-31 16:03:18 +02:00
Andrii Chubatiuk
88ea9c2fb3
docs: removed redundant 'VictoriaLogs' from title (#6715)
### Describe Your Changes

After breadcrumb was added to docs.victoriametrics.com there's no need
to specify parent page name in a title

<img width="1437" alt="Screenshot 2024-07-27 at 10 20 09"
src="https://github.com/user-attachments/assets/733f41f4-a727-4f52-a7c0-6019edf1b803">

Also added vmdocs to gitignore to avoid committing it

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-29 14:29:15 +02:00
Aliaksandr Valialkin
8cb1822b94
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)"
This reverts commit 2e9b1efeb9.

Reason for revert: relative links in docs are much harder to maintain in consistent state
comparing to absolute links:

- It is non-trivial to figure out the proper relative link path when creating and editing docs.
- Relative links break after moving the doc files to another paths, and it is non-trivial
  to figure which links are broken after that.
- The updated relative links do not work properly right now in the docs.
  For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code
  link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page.

This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 12:48:13 +02:00
Aliaksandr Valialkin
f357ee57ef
Revert "fixed victorialogs relative links (#6693)"
This reverts commit 683a69d53d.

Reason for revert: relative links in docs are much harder to maintain in consistent state
comparing to absolute links:

- It is non-trivial to figure out the proper relative link path when creating and editing docs.
- Relative links break after moving the doc files to another paths, and it is non-trivial
  to figure which links are broken after that.
- The updated relative links do not work properly right now in the docs.
  For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code
  link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page.

This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 12:48:12 +02:00
Github Actions
f05e17658a
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@d6bfe49 (#6697) 2024-07-25 02:00:34 -07:00
Artem Navoiev
0593310e89
docs: fix link in logsql doc
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-24 20:31:12 +02:00
Andrii Chubatiuk
683a69d53d
fixed victorialogs relative links (#6693)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-24 11:26:02 -07:00
Andrii Chubatiuk
2e9b1efeb9
removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)
### Describe Your Changes

Use relative markdown references, removed `{{< ref >}}` shortcodes

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-24 13:33:37 +03:00
Andrii Chubatiuk
ce4cc4cbb2
view documentation locally (#6677)
### Describe Your Changes

- moved files from root to VictoriaMetrics folder to be able to mount
operator docs and VictoriaMetrics docs independently
- added ability to run website locally

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-24 01:00:31 -07:00
Andrii Chubatiuk
332b7788fb
added _index.md for hugo, which point to README.md (#6686)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-23 17:30:49 +03:00
Aliaksandr Valialkin
e78a51bf39
docs/VictoriaLogs/querying/README.md: add link to Grafana datasource for VictoriaLogs to the top of the page 2024-07-17 18:34:41 +02:00
Aliaksandr Valialkin
1f96882367
docs: fix spellcheck errors found by make spellcheck
Thanks to the spellcheck Makefile rule added by @arkid15r at fabf0b928e
2024-07-17 18:14:20 +02:00
Github Actions
9c02f27ef9
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@3cbd5e9 (#6649) 2024-07-16 02:05:36 -07:00
Aliaksandr Valialkin
233e5f0a9e
lib/httpserver: skip basic auth check for additional request paths, which should call httpserver.CheckAuthFlag()
This is a follow-up for 61dce6f2a1

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6338
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6329
2024-07-16 01:00:45 +02:00
Artem Navoiev
8af9a3f0b8
docs/VictoriaLogs link Github Issues with Roadmap items. Add informat… (#6633)
…ion about Grafana Datasource in quering

### Describe Your Changes

Update Roadmap and Querying documentation for VictoriaLogs 

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-15 15:29:44 +02:00
Arkadii Yakovets
fabf0b928e
docs: add spellcheck command (#6562)
### Describe Your Changes

Implement spellcheck command:
  - add cspell configuration files
  - dockerize spellchecking process
  - add Makefile targets
 
This PR adds a standalone `make spellcheck` target to check `docs/*.md` files for spelling
errors. The target process is dockerized to be run in a separate npm environment.

Some `docs/` typo fixes also included.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-11 12:39:42 +02:00
Aliaksandr Valialkin
b173f24041
docs/VictoriaLogs/CHANGELOG.md: cut v0.28.0-victorialogs release 2024-07-10 03:08:16 +02:00
Aliaksandr Valialkin
00c666a6c3
app/vlselect/vmui: run make vmui-logs-update after 662e026279 2024-07-10 00:50:10 +02:00
Aliaksandr Valialkin
aa9bb99527
lib/logstorage: drop all the pipes from the query when calculating the number of matching logs at /select/logsql/hits API 2024-07-10 00:39:28 +02:00
Yury Molodov
662e026279
vmui/logs: add spinner to bar chart (#6577)
Add a spinner to the bar chart 

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6558

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-09 14:58:48 +02:00
Github Actions
5275b36380
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@0c2f8ef (#6592) 2024-07-05 15:09:37 +02:00
Github Actions
11989db586
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@4ed20bd (#6578) 2024-07-05 09:28:11 +02:00
Aliaksandr Valialkin
bbf1f33818
docs/VictoriaLogs/CHANGELOG.md: cut v0.27.1-victorialogs 2024-07-05 01:22:24 +02:00
Aliaksandr Valialkin
c0caa69939
lib/logstorage: use quicktemplate.AppendJSONString instead of strconv.AppendQuote for encoding JSON strings
The strconv.AppendQuote improperly encodes special chars such as \x1b . They must be encoded as \u001b .

See https://github.com/VictoriaMetrics/victorialogs-datasource/issues/24
2024-07-05 01:22:23 +02:00
Github Actions
27823278d1
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@a04844a (#6571) 2024-07-03 05:39:23 -07:00
Github Actions
c2beebc264
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@a53ccd7 (#6570) 2024-07-03 05:20:23 -07:00
Artem Navoiev
1738328e75
add file placeholder grafana logs datasource docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-03 14:07:02 +02:00
Aliaksandr Valialkin
d506e47c5a
docs/VictoriaLogs/CHANGELOG.md: cut v0.27.0-victorialogs 2024-07-02 01:43:46 +02:00
Aliaksandr Valialkin
e11f0aa9ec
app/vlinsert/insertutils: flush the ingested logs from in-memory buffer to storage every second
Previously the in-memory buffer could remain unflushed for long periods of time under low ingestion rate.
The ingested logs weren't visible for search during this time.
2024-07-02 01:38:19 +02:00
Aliaksandr Valialkin
ba6f82069f
app/vlinsert/syslog: add an ability to use log ingestion time as the _time field 2024-07-02 01:38:19 +02:00
Aliaksandr Valialkin
387b3b7fb7
docs/VictoriaLogs/CHANGELOG.md: use new url https://docs.victoriametrics.com/victorialogs/querying/ instead of old one https://docs.victoriametrics.com/VictoriaLogs/querying/ 2024-07-01 16:31:42 +02:00
Aliaksandr Valialkin
839596c00d
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.1-victorialogs 2024-07-01 02:32:51 +02:00
Aliaksandr Valialkin
6bb66cb3e9
lib/logstorage: properly search for the surrounding logs in stream_context pipe
The set of log fields in the found logs may differ from the set of log fields present in the log stream.
So compare only the log fields in the found logs when searching for the matching log entry in the log stream.

While at it, return _stream field in the delimiter log entry, since this field is used by VictoriaLogs Web UI
for grouping logs by log streams.
2024-07-01 02:29:50 +02:00
Aliaksandr Valialkin
e0ea83c466
docs/VictoriaLogs: remove "preview" warning - VictoriaLogs is ready for prod 2024-07-01 01:52:33 +02:00
Aliaksandr Valialkin
353e26cdfc
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.0-victorialogs 2024-07-01 01:49:49 +02:00
Aliaksandr Valialkin
bb0deb7ac4
lib/logstorage: add ability to store sorted log position into a separate field with sort ... rank <fieldName> syntax 2024-07-01 01:44:17 +02:00
Aliaksandr Valialkin
dc291d8980
lib/logstorage: add delimiter between log chunks returned from | stream_context pipe 2024-07-01 01:30:37 +02:00
Aliaksandr Valialkin
c9568d45dc
docs/VictoriaLogs: typo fixes 2024-06-28 19:26:24 +02:00
Aliaksandr Valialkin
9877c086b9
docs/VictoriaLogs/CHANGELOG.md: cut v0.25.0-victorialogs 2024-06-28 19:17:49 +02:00
Aliaksandr Valialkin
d4ca651547
lib/logstorage: add stream_context pipe, which allows selecting surrounding logs for the matching logs 2024-06-28 19:14:29 +02:00
Aliaksandr Valialkin
d7185f1b77
app/vlinsert/syslog: properly skip empty lines in Syslog protocol
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6548
2024-06-28 14:09:28 +02:00
Aliaksandr Valialkin
e8322147e9
app/vlselect/logsql: add optional fields_limit query arg to /select/logsql/hits HTTP endpoint
This query arg is needed for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545
in order to return top N groups with the biggest number of hits.
2024-06-28 03:08:40 +02:00
Aliaksandr Valialkin
59e639eed2
docs/VictoriaLogs: typo fixes 2024-06-28 02:21:19 +02:00
Aliaksandr Valialkin
820889cf53
docs/VictoriaLogs: document how to export logs from VictoriaLogs
While at it, mention vmctl-like tool for migrating logs from other systems to VictoriaLogs at docs/VictoriaLogs/Roadmap.md

Thanks to the question from Xavier Pestel at https://www.linkedin.com/feed/update/urn:li:activity:7212093021301927937?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A7212093021301927937%2C7212171550651731969%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287212171550651731969%2Curn%3Ali%3Aactivity%3A7212093021301927937%29
2024-06-28 02:06:12 +02:00
Aliaksandr Valialkin
48780d9ec5
docs/VictoriaLogs/querying/README.md: clarify live tailing docs 2024-06-27 19:12:42 +02:00
Aliaksandr Valialkin
e26ab72705
docs/VictoriaLogs/Roadmap.md: remove Web UI from roadmap, since it is already implemented
See https://docs.victoriametrics.com/victorialogs/querying/#web-ui

While at it, add DataDog protocol for logs into 'data ingestion protocols' roadmap.
2024-06-27 18:59:46 +02:00
Aliaksandr Valialkin
107894f13c
docs/VictoriaLogs/querying: remove unused and outdated vmui.webp image
The image has been deleted from the docs/VictoriaLogs/querying/README.md file
in the commit 87f1c8bd6c
2024-06-27 15:16:06 +02:00
Aliaksandr Valialkin
7fcc0823f1
docs/VictoriaLogs/querying/README.md: add a link to live tailing docs from querying log docs 2024-06-27 15:12:49 +02:00
Aliaksandr Valialkin
7c8c040502
app/vlselect: properly return live tailing results 2024-06-27 15:05:57 +02:00
Aliaksandr Valialkin
87f1c8bd6c
lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
Aliaksandr Valialkin
f9549533a9
docs/VictoriaLogs/README.md: document the relation between -retention.maxDiskSpaceUsageBytes and -retentionPeriod command-line flags 2024-06-26 12:02:34 +02:00
Yury Molodov
43342745ac
vmui/logs: fix the update of the relative time range (#6517)
### Describe Your Changes

- Fixed the update of the relative time range when `Execute Query` is
clicked
- Optimized server requests: now, if an error occurs in the `/query`
request, the `/hits` request will not be executed.

#6345 (duplicates: #6440, #6312)
2024-06-26 11:23:22 +02:00
Aliaksandr Valialkin
dff5008392
app/vlstorage: add -retention.maxDiskSpaceUsageBytes command-line flag for limiting the retention at VictoriaLogs by disk space usage 2024-06-25 17:30:33 +02:00
Aliaksandr Valialkin
7d026b4655
docs/VictoriaLogs/data-ingestion/README.md: document that Rsyslog and Syslog-ng can be used for ingesting logs into VictoriaLogs 2024-06-25 15:15:46 +02:00
Aliaksandr Valialkin
0c90eeb388
docs/VictoriaLogs/CHANGELOG.md: cut v0.23.0-victorialogs release 2024-06-25 14:59:15 +02:00
Aliaksandr Valialkin
3eacd43fff
lib/logstorage: parse syslog structured data into separate fields in order to simplify further querying of this data 2024-06-25 14:53:39 +02:00
Aliaksandr Valialkin
9e1c037249
lib/logstorage: properly parse timezone offset at TryParseTimestampRFC3339Nano()
The TryParseTimestampRFC3339Nano() must properly parse RFC3339 timestamps with timezone offsets.

While at it, make tryParseTimestampISO8601 function private in order to prevent
from improper usage of this function from outside the lib/logstorage package.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6508
2024-06-25 14:53:38 +02:00
Aliaksandr Valialkin
10390a7dfc
docs: return back spellcheck fixes from the commit 92b22581e6, which were accidentally reverted in 45cf83cd8f
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6470
2024-06-25 02:33:43 +02:00
Aliaksandr Valialkin
45cf83cd8f
Revert "docs: Add spellcheck workflow (#6470)"
This reverts commit 92b22581e6.

Reason for revert: too complex and slow approach for spellchecking task.

This approach may significantly slow down development pace. It also may take non-trivial
amounts of additional time and resources at CI/CD because of all this npm shit at cspell directory.

Note to @arkid15r : the idea with the ability to run spellchecker on all the VictoriaMetrics
codebase is great. But this shouldn't be mandatory pre-commit check. It is enough to have
a Makefile rule like `make spellcheck`, which could be run manually whenever spellcheck is needed
(e.g. once per month or once per quarter).
2024-06-25 02:23:48 +02:00