github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
8963 commits 150 branches 748 tags 347 MiB
Commit graph

289 commits

Author SHA1 Message Date
Aliaksandr Valialkin
ba6f82069f
app/vlinsert/syslog: add an ability to use log ingestion time as the _time field 2024-07-02 01:38:19 +02:00
Aliaksandr Valialkin
387b3b7fb7
docs/VictoriaLogs/CHANGELOG.md: use new url https://docs.victoriametrics.com/victorialogs/querying/ instead of old one https://docs.victoriametrics.com/VictoriaLogs/querying/ 2024-07-01 16:31:42 +02:00
Aliaksandr Valialkin
839596c00d
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.1-victorialogs 2024-07-01 02:32:51 +02:00
Aliaksandr Valialkin
6bb66cb3e9
lib/logstorage: properly search for the surrounding logs in stream_context pipe 
The set of log fields in the found logs may differ from the set of log fields present in the log stream.
So compare only the log fields in the found logs when searching for the matching log entry in the log stream.

While at it, return _stream field in the delimiter log entry, since this field is used by VictoriaLogs Web UI
for grouping logs by log streams.
 2024-07-01 02:29:50 +02:00
Aliaksandr Valialkin
e0ea83c466
docs/VictoriaLogs: remove "preview" warning - VictoriaLogs is ready for prod 2024-07-01 01:52:33 +02:00
Aliaksandr Valialkin
353e26cdfc
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.0-victorialogs 2024-07-01 01:49:49 +02:00
Aliaksandr Valialkin
bb0deb7ac4
lib/logstorage: add ability to store sorted log position into a separate field with sort ... rank <fieldName> syntax 2024-07-01 01:44:17 +02:00
Aliaksandr Valialkin
dc291d8980
lib/logstorage: add delimiter between log chunks returned from | stream_context pipe 2024-07-01 01:30:37 +02:00
Aliaksandr Valialkin
c9568d45dc
docs/VictoriaLogs: typo fixes 2024-06-28 19:26:24 +02:00
Aliaksandr Valialkin
9877c086b9
docs/VictoriaLogs/CHANGELOG.md: cut v0.25.0-victorialogs 2024-06-28 19:17:49 +02:00
Aliaksandr Valialkin
d4ca651547
lib/logstorage: add stream_context pipe, which allows selecting surrounding logs for the matching logs 2024-06-28 19:14:29 +02:00
Aliaksandr Valialkin
d7185f1b77
app/vlinsert/syslog: properly skip empty lines in Syslog protocol 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6548
 2024-06-28 14:09:28 +02:00
Aliaksandr Valialkin
e8322147e9
app/vlselect/logsql: add optional fields_limit query arg to /select/logsql/hits HTTP endpoint 
This query arg is needed for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545
in order to return top N groups with the biggest number of hits.
 2024-06-28 03:08:40 +02:00
Aliaksandr Valialkin
59e639eed2
docs/VictoriaLogs: typo fixes 2024-06-28 02:21:19 +02:00
Aliaksandr Valialkin
820889cf53
docs/VictoriaLogs: document how to export logs from VictoriaLogs 
While at it, mention vmctl-like tool for migrating logs from other systems to VictoriaLogs at docs/VictoriaLogs/Roadmap.md

Thanks to the question from Xavier Pestel at https://www.linkedin.com/feed/update/urn:li:activity:7212093021301927937?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A7212093021301927937%2C7212171550651731969%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287212171550651731969%2Curn%3Ali%3Aactivity%3A7212093021301927937%29
 2024-06-28 02:06:12 +02:00
Aliaksandr Valialkin
48780d9ec5
docs/VictoriaLogs/querying/README.md: clarify live tailing docs 2024-06-27 19:12:42 +02:00
Aliaksandr Valialkin
e26ab72705
docs/VictoriaLogs/Roadmap.md: remove Web UI from roadmap, since it is already implemented 
See https://docs.victoriametrics.com/victorialogs/querying/#web-ui

While at it, add DataDog protocol for logs into 'data ingestion protocols' roadmap.
 2024-06-27 18:59:46 +02:00
Aliaksandr Valialkin
107894f13c
docs/VictoriaLogs/querying: remove unused and outdated vmui.webp image 
The image has been deleted from the docs/VictoriaLogs/querying/README.md file
in the commit 87f1c8bd6c
 2024-06-27 15:16:06 +02:00
Aliaksandr Valialkin
7fcc0823f1
docs/VictoriaLogs/querying/README.md: add a link to live tailing docs from querying log docs 2024-06-27 15:12:49 +02:00
Aliaksandr Valialkin
7c8c040502
app/vlselect: properly return live tailing results 2024-06-27 15:05:57 +02:00
Aliaksandr Valialkin
87f1c8bd6c
lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
Aliaksandr Valialkin
f9549533a9
docs/VictoriaLogs/README.md: document the relation between -retention.maxDiskSpaceUsageBytes and -retentionPeriod command-line flags 2024-06-26 12:02:34 +02:00
Yury Molodov
43342745ac
vmui/logs: fix the update of the relative time range (#6517) 
### Describe Your Changes

- Fixed the update of the relative time range when `Execute Query` is
clicked
- Optimized server requests: now, if an error occurs in the `/query`
request, the `/hits` request will not be executed.

#6345 (duplicates: #6440, #6312)
 2024-06-26 11:23:22 +02:00
Aliaksandr Valialkin
dff5008392
app/vlstorage: add -retention.maxDiskSpaceUsageBytes command-line flag for limiting the retention at VictoriaLogs by disk space usage 2024-06-25 17:30:33 +02:00
Aliaksandr Valialkin
7d026b4655
docs/VictoriaLogs/data-ingestion/README.md: document that Rsyslog and Syslog-ng can be used for ingesting logs into VictoriaLogs 2024-06-25 15:15:46 +02:00
Aliaksandr Valialkin
0c90eeb388
docs/VictoriaLogs/CHANGELOG.md: cut v0.23.0-victorialogs release 2024-06-25 14:59:15 +02:00
Aliaksandr Valialkin
3eacd43fff
lib/logstorage: parse syslog structured data into separate fields in order to simplify further querying of this data 2024-06-25 14:53:39 +02:00
Aliaksandr Valialkin
9e1c037249
lib/logstorage: properly parse timezone offset at TryParseTimestampRFC3339Nano() 
The TryParseTimestampRFC3339Nano() must properly parse RFC3339 timestamps with timezone offsets.

While at it, make tryParseTimestampISO8601 function private in order to prevent
from improper usage of this function from outside the lib/logstorage package.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6508
 2024-06-25 14:53:38 +02:00
Aliaksandr Valialkin
10390a7dfc
docs: return back spellcheck fixes from the commit 92b22581e6, which were accidentally reverted in 45cf83cd8f 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6470
 2024-06-25 02:33:43 +02:00
Aliaksandr Valialkin
45cf83cd8f
Revert "docs: Add spellcheck workflow (#6470)" 
This reverts commit 92b22581e6.

Reason for revert: too complex and slow approach for spellchecking task.

This approach may significantly slow down development pace. It also may take non-trivial
amounts of additional time and resources at CI/CD because of all this npm shit at cspell directory.

Note to @arkid15r : the idea with the ability to run spellchecker on all the VictoriaMetrics
codebase is great. But this shouldn't be mandatory pre-commit check. It is enough to have
a Makefile rule like `make spellcheck`, which could be run manually whenever spellcheck is needed
(e.g. once per month or once per quarter).
 2024-06-25 02:23:48 +02:00
Aliaksandr Valialkin
de7450b7e0
lib/logstorage: work-in-progress 2024-06-24 23:27:12 +02:00
Arkadii Yakovets
92b22581e6
docs: Add spellcheck workflow (#6470) 
### Describe Your Changes

This is a follow-up PR containing workflow related part of the initial
[PR#6362](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6362).
It adds spell checking control based on
[cspell](https://github.com/streetsidesoftware/cspell). The related
configs are consolidated under `cspell` directory.

You can add VictoriaMetrics specific terms into `cspell/custom-dict.txt`
(it contains ~30 items atm). All other absent commonly used terms should
be added directly to respective
[cspell-dicts](https://github.com/streetsidesoftware/cspell-dicts/blob/main/CONTRIBUTING.md).

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>
 2024-06-21 11:56:20 -07:00
Aliaksandr Valialkin
7229dd8c33
lib/logstorage: work-in-progress 2024-06-20 03:10:08 +02:00
Yury Molodov
32fbffedd9
vmui/logs: add bar chart (#6461) 
- Added a bar chart displaying the number of log entries over a time
range.
   #6404
- When `_msg` is empty, all fields are displayed in a single line.
- Added double quotes when copying pairs: `key: "value"`.
- Minor style adjustments.
 2024-06-18 15:23:21 +02:00
Aliaksandr Valialkin
4e912c3c72
docs/VictoriaLogs/CHANGELOG.md: typo fix in the release date for v0.20.2-victorialogs 2024-06-18 00:28:40 +02:00
Aliaksandr Valialkin
3eda4617c0
app/vlinsert: properly parse timestamps with nanosecond precision at /insert/jsonline HTTP endpoint 
This has been broken in 2b6a634ec0
 2024-06-18 00:23:25 +02:00
Aliaksandr Valialkin
6f6fd9cb19
docs/VictoriaLogs/CHANGELOG.md: cut v0.20.1-victorialogs 2024-06-17 23:29:17 +02:00
Aliaksandr Valialkin
e498fa6960
app/vlinsert/syslog: allow accepting syslog messages with different configs at different ports 2024-06-17 23:16:34 +02:00
Aliaksandr Valialkin
478468e6cd
app/vlinsert: properly parse length-delimited syslog messages sent over TCP according to RFC5425 2024-06-17 22:28:26 +02:00
Aliaksandr Valialkin
b5a206cea1
docs/VictoriaLogs: mention which collectors support sending logs to VictoriaLogs in syslog format 2024-06-17 13:13:43 +02:00
Aliaksandr Valialkin
2b6a634ec0
lib/logstorage: work-in-progress 2024-06-17 12:13:18 +02:00
Artem Navoiev
d1baa41b6a
docs: add more redirects 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-06-14 12:32:12 +02:00
Aliaksandr Valialkin
0269c89e03
docs/VictoriaLogs/logsql-examples.md: added an example on how to search for a phrase with whitespace 2024-06-11 19:13:40 +02:00
Aliaksandr Valialkin
8f5dc966f6
lib/logstorage: work-in-progress 2024-06-11 17:50:32 +02:00
Aliaksandr Valialkin
0521e58a09
lib/logstorage: work-in-progress 2024-06-10 18:42:19 +02:00
Yury Molodov
84088e5a2d
vmui/logs: add markdown support (#6292) 
Add support for markdown format and emoji for the `_msg` field in the
"Group" view.
Add markdown rendering toggle. Disabled by default. Value is stored in
`localStorage`.
 2024-06-10 16:38:13 +02:00
Artem Navoiev
318e9e9de0
docs victorialogs: add alias for quickstart for VL to be consistent with other quickstarts 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-06-10 09:36:26 +02:00
Aliaksandr Valialkin
55d8379ae6
lib/logstorage: work-in-progress 2024-06-06 12:27:05 +02:00
Aliaksandr Valialkin
d54d5a17de
docs/VictoriaLogs/CHANGELOG.md: document the a68c2c0f17 2024-06-06 12:21:17 +02:00
Aliaksandr Valialkin
1d86ea49a0
docs/VictoriaLogs/LogsQL.md: typo fix: replace from _json with from _msg where appropriate 2024-06-05 10:15:46 +02:00
Aliaksandr Valialkin
c2cd47da07
docs/VictoriaLogs/LogsQL.md: substitute TAG with APP-NAME[PROCID] in rfc3164 syslog message format at unpack_syslog pipe docs 2024-06-05 10:13:41 +02:00
Aliaksandr Valialkin
80a7c65ab7
lib/logstorage: allow using eval keyword instead of math keyword in math pipe 2024-06-05 10:07:49 +02:00
Aliaksandr Valialkin
868c4c0253
docs/VictoriaLogs/LogsQL.md: typo fix: substitute "'" with "" in front of John` 2024-06-05 09:51:12 +02:00
Aliaksandr Valialkin
b9c0eb3dde
docs/VictoriaLogs/LogsQL.md: typo fix: its -> it 2024-06-05 09:45:55 +02:00
Aliaksandr Valialkin
43cf221681
lib/logstorage: work-in-progress 2024-06-05 03:18:12 +02:00
Aliaksandr Valialkin
cbf606979d
docs/VictoriaLogs/LogsQL.md: mention unpack_syslog pipe in the Transformations section 2024-06-04 03:17:07 +02:00
Aliaksandr Valialkin
68c07b0599
docs/VictoriaLogs/Roadmap.md: remove LogsQL transformations from roadmap, since they are already implemented 2024-06-04 02:21:00 +02:00
Aliaksandr Valialkin
49be53518e
docs/VictoriaLogs/logsql-examples.md: typo fix 2024-06-04 02:17:52 +02:00
Aliaksandr Valialkin
c71ad6d0e3
docs/VictoriaLogs/logsql-examples.md: typo fix 2024-06-04 02:10:48 +02:00
Aliaksandr Valialkin
539fce9227
lib/logstorage: work-in-progress 2024-06-04 01:49:02 +02:00
Arkadii Yakovets
c740a8042e
docs: fix docs/ and README.md spelling errors (#6362) 
Fixes `docs/` and `README.md` typos and errors.

Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>
 2024-06-03 10:04:13 +02:00
Aliaksandr Valialkin
55f9697520
docs/VictoriaLogs: typo fixes 2024-05-30 16:35:18 +02:00
Aliaksandr Valialkin
ceb2397381
deployment/docker: update VictoriaLogs Docker image from v0.14.0-victorialogs to v0.15.0-victorialogs 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.15.0-victorialogs
 2024-05-30 16:27:26 +02:00
Aliaksandr Valialkin
b30e80b071
lib/logstorage: work-in-progress 2024-05-30 16:19:23 +02:00
Aliaksandr Valialkin
1de187bcb7
lib/logstorage: work-in-progress 2024-05-29 01:52:13 +02:00
Aliaksandr Valialkin
bf33e7eda7
docs/VictoriaLogs: typo fixes 2024-05-28 22:48:47 +02:00
Aliaksandr Valialkin
89bd0dca0a
deployment/docker: update VictoriaLogs image from v0.12.1-victorialogs to v0.13.0-victorialogs 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.13.0-victorialogs
 2024-05-28 22:14:34 +02:00
Aliaksandr Valialkin
0aafca29be
lib/logstorage: work-in-progress 2024-05-28 19:29:41 +02:00
Aliaksandr Valialkin
31e23c6f6f
deployment: update VictoriaLogs image from v0.12.0-victorialogs to v0.12.1-victorialogs 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.12.1-victorialogs
 2024-05-26 02:08:24 +02:00
Aliaksandr Valialkin
1e203f35f7
lib/logstorage: work-in-progress 2024-05-26 01:55:21 +02:00
Aliaksandr Valialkin
7ac529c235
lib/logstorage: work-in-progress 2024-05-25 22:59:13 +02:00
Aliaksandr Valialkin
1882957585
docs/VictoriaLogs/CHANGELOG.md: cut v0.12.0-victorialogs 2024-05-25 21:40:10 +02:00
Aliaksandr Valialkin
dc55146752
lib/logstorage: work-in-progress 2024-05-25 21:36:16 +02:00
Aliaksandr Valialkin
e2590f0485
lib/logstorage: work-in-progress 2024-05-25 00:30:58 +02:00
Aliaksandr Valialkin
0915d6d841
deployment: update Docker image tag for VictoriaLogs from v0.8.0-victorialogs to v0.10.0-victorialogs 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.10.0-victorialogs
 2024-05-24 03:15:43 +02:00
Aliaksandr Valialkin
fa137bd6f1
docs/VictoriaLogs/CHANGELOG.md: cut v0.10.0-victorialogs 2024-05-24 03:08:17 +02:00
Aliaksandr Valialkin
4b458370c1
lib/logstorage: work-in-progress 2024-05-24 03:06:55 +02:00
Aliaksandr Valialkin
3fdd4dad82
app/vlselect: fix loading web UI 2024-05-22 23:24:31 +02:00
Aliaksandr Valialkin
ab855e10e7
docs/VictoriaLogs/LogsQL.md: mention that it is better from performance PoV parsing JSON and logfmt logs before the ingestion into VictoriaLogs 2024-05-22 23:04:40 +02:00
Aliaksandr Valialkin
7667abc6cc
docs/VictoriaLogs/LogsQL.md: typo fixes 2024-05-22 22:24:50 +02:00
Aliaksandr Valialkin
e2f62c5179
docs/VictoriaLogs/CHANGELOG.md: cut v0.8.0-victorialogs 2024-05-22 22:07:32 +02:00
Alexander Marshalov
7da541360e
[vmlogs] fixed time parsing with millisecond precision time (#6293) (#6295) 
fix for #6293

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-05-22 21:46:50 +02:00
Yury Molodov
75bd1831bb
vmui/logs: fix parsing long _msg values (#6310) 
This PR fixes an issue where parsing long `_msg` values caused errors,
resulting in some log records not being displayed.

The error occurred due to partial processing of strings. In some cases,
a long record could be split into multiple chunks, causing only part of
the record to be processed instead of the entire entry.

#6281

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-05-22 21:44:13 +02:00
Aliaksandr Valialkin
22107421eb
lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
Artem Navoiev
cee4bfebd7
docs: victorialogs mention that our bench suite can be run against Loki as well 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-05-21 10:39:58 +02:00
Yury Molodov
a6a599cbdc
vmui/logs: change time range to start and end query args (#6296) 
change time range limitation from `_time` in the expression to `start`
and `end` query args.
 2024-05-20 14:13:15 +02:00
Aliaksandr Valialkin
8f6af6df9f
docs/VictoriaLogs: typo fixes 2024-05-20 04:48:34 +02:00
Aliaksandr Valialkin
b031774cf5
docs/VictoriaLogs/LogsQL.md: typo fix 2024-05-20 04:27:34 +02:00
Aliaksandr Valialkin
b36a35d696
docs/VictoriaLogs/LogsQL.md: typo fix 2024-05-20 04:24:27 +02:00
Aliaksandr Valialkin
6ad1eb9a3d
deployment: update VictoriaLogs from v0.6.0-victorialogs to v0.8.0-victorialogs 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.8.0-victorialogs
 2024-05-20 04:17:37 +02:00
Aliaksandr Valialkin
ad505a7a9a
lib/logstorage: work-in-progress 2024-05-20 04:08:30 +02:00
Aliaksandr Valialkin
6fdba8599d
docs/VictoriaLogs/CHANGELOG.md: cut v0.7.0-victorialogs 2024-05-15 04:58:05 +02:00
Aliaksandr Valialkin
0aa19a2837
lib/logstorage: work-in-progress 2024-05-15 04:55:44 +02:00
Aliaksandr Valialkin
c90e6de13b
docs/VictoriaLogs/CHANGELOG.md: cut v0.6.1-victorialogs 2024-05-14 03:06:53 +02:00
Aliaksandr Valialkin
da3af090c6
lib/logstorage: work-in-progress 2024-05-14 03:05:03 +02:00
Aliaksandr Valialkin
cb35e62e04
lib/logstorage: work-in-progress 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6258
 2024-05-14 01:49:23 +02:00
Aliaksandr Valialkin
de98688489
deployment: update VictoriaLogs Docker image from v0.5.2-victorialogs to v0.6.0-victorialogs 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.6.0-victorialogs
 2024-05-12 23:22:50 +02:00
Aliaksandr Valialkin
8466ab109c
docs/VictoriaLogs/LogsQL.md: cross-reference uniq pipe with uniq_values stats function 2024-05-12 16:45:24 +02:00
Aliaksandr Valialkin
89c4dc1d8d
docs/VictoriaLogs/CHANGELOG.md: cut v0.6.0-victorialogs 2024-05-12 16:38:01 +02:00
Aliaksandr Valialkin
9dbd0f9085
lib/logstorage: initial implementation of pipes in LogsQL 
See https://docs.victoriametrics.com/victorialogs/logsql/#pipes
 2024-05-12 16:33:31 +02:00