Aliaksandr Valialkin
546bf7d579
lib/logstorage: properly skip filtered out dict values when calculating uniq_values, min, max, row_min and row_max stats functions
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7458
2024-11-08 23:21:21 +01:00
Aliaksandr Valialkin
2f1ce74d97
lib/logstorage: properly clone field values at values stats function
...
Previously field values weren't properly cloned, which could lead to garbage output for `values` stats function
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7458
2024-11-08 23:16:42 +01:00
Aliaksandr Valialkin
342f84c569
app/vlinsert/loki: show the original request body on parse errors
...
This should simplify debugging.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7490
2024-11-08 22:00:58 +01:00
Aliaksandr Valialkin
b7c290ea5e
docs/VictoriaLogs/CHANGELOG.md: refer to the issue related to adding fields to Syslog logs
...
This is a follow-up for cd60a4c589https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7354
2024-11-08 21:50:14 +01:00
Aliaksandr Valialkin
4f0bec6f03
app/vlinsert/syslog: allow changing the default set of log fields to use as stream fields during syslog data ingestion
...
Thanks to @AndrewChubatiuk for the initial implementation at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7488
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7480
See https://docs.victoriametrics.com/victorialogs/data-ingestion/syslog/#stream-fields
2024-11-08 21:21:08 +01:00
Aliaksandr Valialkin
cd60a4c589
app/vlinsert/syslog: add an ability to drop and add fields during data ingestion via Syslog protocol
...
See https://docs.victoriametrics.com/victorialogs/data-ingestion/syslog/#dropping-fields
and https://docs.victoriametrics.com/victorialogs/data-ingestion/syslog/#adding-extra-fields
2024-11-08 20:57:59 +01:00
Aliaksandr Valialkin
63c76b9b27
lib/logstorage: support for [label1=value1 ... labelN=valueN] syntax inside syslog messages for adding arbitrary labels (fields) to log entries
2024-11-08 19:57:22 +01:00
Aliaksandr Valialkin
f55791f20b
app/vlogscli: allow toggling wrapping long lines with \wrap_long_lines command
2024-11-08 17:08:50 +01:00
Aliaksandr Valialkin
62e6c9bd6f
docs/VictoriaLogs/README.md: add Security chapter
...
It is inspired by https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7428#discussion_r1831555716
2024-11-08 16:43:55 +01:00
Aliaksandr Valialkin
a44787372f
app/vlselect/logsql: add an ability to delay returning matching logs from live tailing via offset query arg
...
By default the delay equals to 1 second.
While at it, document refresh_interval query arg at /select/logsql/tail endpoint.
Thanks to @Fusl for the idea and the initial implementation at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7428
2024-11-08 16:43:55 +01:00
Aliaksandr Valialkin
e5537bc64d
lib/logstorage: properly take into account the end query arg when calculating time range for _time:duration filters
2024-11-08 16:43:54 +01:00
Aliaksandr Valialkin
a98fb495c6
lib/logstorage: allow specifying _time filter offset without time range
...
This is useful when builiding graphs on time ranges in the past.
2024-11-08 16:43:54 +01:00
Aliaksandr Valialkin
5a6531b329
lib/logstorage: add an ability to add prefix to resulting query field names in join pipe
...
See https://docs.victoriametrics.com/victorialogs/logsql/#join-pipe
2024-11-08 16:43:53 +01:00
Andrii Chubatiuk
30dd4cdc0d
docs: add make command for docs version update ( #7430 )
...
### Describe Your Changes
added make target, which updates `{{% available_from "#" %}}` shortcode
to `{{% available_from "$(PKG_TAG)" %}}` if PKG_TAG matches expression
`v.*`. `{{% available_from %}}` shortcode was introduced in
https://github.com/VictoriaMetrics/vmdocs/pull/89 to show a reference to
a version in a changelog since which a feature was introduced
related issue
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7376
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/ ).
2024-11-08 16:35:08 +01:00
Andrii Chubatiuk
b399f1c656
victorialogs: added missing changelog entry about DataDog support ( #7464 )
...
### Describe Your Changes
Added missing changelog entry for PR
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5536
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/ ).
---------
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-07 14:26:02 +01:00
Github Actions
39fee4fd98
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@0b86b62 ( #7477 )
2024-11-07 14:23:42 +01:00
Aliaksandr Valialkin
d1af84a49d
docs/VictoriaLogs/LogsQL.md: add missing reference to pipes docs from join pipe docs
...
This is needed for consistency with other pipe docs
2024-11-06 20:59:25 +01:00
Aliaksandr Valialkin
a549240852
deployment/docker: update VictoriaLogs from v0.40.0-victorialogs to v0.41.0-victorialogs
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.41.0-victorialogs
2024-11-06 20:26:26 +01:00
Aliaksandr Valialkin
a4ce80b4ea
docs/VictoriaLogs/CHANGELOG.md: cut v0.41.0-victorialogs
2024-11-06 20:10:21 +01:00
Yury Molodov
1e1952acf5
vmui/logs: add log sorting ( #7344 )
...
### Describe Your Changes
add sorting of logs by groups and within each group by time in desc
order. See #7184 and #7045
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/ ).
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-11-06 19:33:35 +01:00
Aliaksandr Valialkin
3d75c39ff4
app/vlinsert/loki: follow-up for 3aeb1b96a2
...
- Disallow more than 3 items in Loki line entry, since it must contain two mandatory entries: timestamp and message,
plus one optional entry - structured metadata. See https://grafana.com/docs/loki/latest/reference/loki-http-api/#ingest-logs
- Update references to structured metadata docs in Loki, in order to simplify further maintenance of the code
- Move the change from bugfix to feature at docs/VictoriaLogs/CHANGELOG.md, since VictoriaLogs never supported
structured metadata over JSON Loki protocol. The support for structured metadata in protobuf Loki protocol
has been added in ac06569c49https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7431
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7432
2024-11-06 19:23:38 +01:00
Aliaksandr Valialkin
42c9183281
docs/VictoriaLogs: properly sort log fields with floating-point numbers
2024-11-06 18:56:13 +01:00
Aliaksandr Valialkin
5ed54ebadf
lib/logstorage: add block_stats pipe for analyzing per-block storage stats
2024-11-06 18:55:45 +01:00
Aliaksandr Valialkin
f9e23bf8e3
lib/logstorage: add join pipe for joining multiple query results
2024-11-06 18:53:29 +01:00
Aliaksandr Valialkin
3a5f1019ba
app/vlselect: add start_offset query arg for /select/logsql/tail endpoint for returning historical logs before live tailing
2024-11-06 18:53:28 +01:00
Zhu Jiekun
3aeb1b96a2
app/vlinisert/loki: properly parse json logs with structured metadata
...
Loki protocol supports optional `metadata` object for each ingested line. It's added as 3rd field at the (ts,msg,metadata) tuple. Previously, loki request json parsers rejected log line if tuple size != 2.
This commit allows optional tuple field. It parses it as json object and adds it as log metadata fields to the log message stream.
related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7431
---------
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-11-06 17:25:05 +01:00
hagen1778
05f2e9548d
docs/victorialogs: recommend using separate installations of vmalert for vm and vl
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-06 14:58:11 +01:00
hagen1778
a5f1764171
docs/victorialogs: clarify usage of -rule.defaultRuleType=vlogs
...
User experience suggests that examples shouldn't have `-rule.defaultRuleType=vlogs` set,
as it may confuse users who run vmalert with their existing rules or only use
rules from examples for testing purposes.
This change is supposed to remove the confusion by removing `-rule.defaultRuleType=vlogs`
from default recommendations and explcitily specifying `type` on group level in examples.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-06 14:55:58 +01:00
Roman Khavronenko
0390d58a34
docs: fix typos in vlogs rules examples ( #7457 )
...
* fix typos in rules definition. Otherwise, they can't pass validation
* add code types for rendered examples
### Describe Your Changes
Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/ ).
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-06 13:55:08 +01:00
Andrii Chubatiuk
e0930687f1
vlinsert: support datadog logs
...
This commit adds the following changes:
- Added support to push datadog logs with examples of how to ingest data
using Vector and Fluentbit
- Updated VictoriaLogs examples directory structure to have single
container image for victorialogs, agent (fluentbit, vector, etc) but
multiple configurations for different protocols
Related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6632
2024-11-05 16:52:35 +01:00
Arie Heinrich
2e8f420d84
docs: spelling fixes ( #7420 )
...
### Describe Your Changes
Christmas is early and you get the first present in the shape of
spelling fixes.
Sorry for the big amount :)
### Checklist
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/ ).
2024-11-05 16:44:23 +01:00
Zakhar Bessarab
371e193279
docs/CHANGELOG.md: cut v1.106.0
...
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-11-04 11:13:39 -03:00
Aliaksandr Valialkin
5733e56e40
docs/VictoriaLogs/querying: refer to vlogscli from docs for /select/logsql/query and /select/logsql/tail HTTP endpoints
...
These endpoints are easier to query via vlogscli
2024-11-02 12:31:40 +01:00
Aliaksandr Valialkin
37a58677fa
docs/VictoriaLogs/data-ingestion: mention Grafana Agent and Grafana Alloy additionally to Promtail as log collectors for Grafana Loki
2024-11-02 11:36:22 +01:00
Aliaksandr Valialkin
c7e242b8ce
docs/VictoriaLogs/LogsQL.md: remove misleading information that min and max stats functions work only for numeric values
...
These functions work for both numeric and string values after ad505a7a9a
2024-11-01 20:25:33 +01:00
Aliaksandr Valialkin
a86df52adf
docs/VictoriaLogs/LogsQL.md: typo fix in the link to description for sum_len stats function: sum-len-stats -> sum_len-stats
...
This is a follow-up for 364f084b43
2024-11-01 20:13:47 +01:00
Aliaksandr Valialkin
4478e48eb6
app/vlinsert: implement the ability to add extra fields to the ingested logs
...
This can be done via extra_fields query arg or via VL-Extra-Fields HTTP header.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7354#issuecomment-2448671445
2024-11-01 20:06:17 +01:00
Aliaksandr Valialkin
b3e2db5647
docs/VictoriaLogs/keyConcepts.md: remove duplicate sentence about missing timezone information in the ingested _time field values
2024-11-01 17:20:28 +01:00
Github Actions
955f3660de
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@3ecaebd ( #7395 )
2024-10-31 14:08:35 +01:00
Aliaksandr Valialkin
24b6e117dd
deployment/docker: update VictoriaLogs from v0.39.0-victorialogs to v0.40.0-victorialogs
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.40.0-victorialogs
2024-10-30 23:43:20 +01:00
Aliaksandr Valialkin
1a7b55009b
docs/VictoriaLogs/CHANGELOG.md: cut v0.40.0-victorialogs release
2024-10-30 23:38:06 +01:00
Aliaksandr Valialkin
f88e2ae9fb
docs/VictoriaLogs/README.md: follow-up for 0f6b9e9490: consistently use canonical url for Grafana dashboards
...
See also 0a5ffb3bc1
2024-10-30 23:29:23 +01:00
Aliaksandr Valialkin
0f6b9e9490
docs/VictoriaLogs/README.md: recommend installing Grafana dashboard for VictoriaLogs at Monitoring chapter
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6886
2024-10-30 23:20:36 +01:00
Aliaksandr Valialkin
c5d08d317c
lib/logstorage: properly reset cached output fields for extract and extract_regexp pipes after the log entry matches if(...) condition
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7162
2024-10-30 22:29:52 +01:00
Aliaksandr Valialkin
2e635a42d8
lib/logstorage: properly cache replace() and replace_regexp() results for identical adjacent field values
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7162
2024-10-30 22:29:52 +01:00
Aliaksandr Valialkin
f62502a943
docs/VictoriaLogs/querying/README.md: typo fix after f2cd284cf4a7429848fb4be1522483a9b93a43e7: Alls -> All
2024-10-30 19:08:57 +01:00
Aliaksandr Valialkin
7603446850
app/vlselect: add support for extra_filters and extra_stream_filters query args across all the HTTP querying APIs
...
These query args are going to be used for quick filtering on field values at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7365
2024-10-30 17:59:49 +01:00
hagen1778
6494606924
deployment/alerts: consistently update path to alerting rules
...
Follow-up after 68bad22fd2
2024-10-30 16:06:57 +01:00
Aliaksandr Valialkin
25bca94d04
deployment/docker: update VictoriaLogs from v0.38.0-victorialogs to v0.39.0-victorialogs
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.39.0-victorialogs
2024-10-30 15:11:17 +01:00
Aliaksandr Valialkin
13822f6d11
docs/VictoriaLogs/CHANGELOG.md: cut v0.39.0-victorialogs release
2024-10-30 15:06:00 +01:00
