github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity

docs: [VictoriaLogs] FAQ add logs without msg field (#7487)

Browse source 
### Describe Your Changes

Add FAQs to VictoriaLogs:
- I want to ingest logs without message field, is that possible?
- What if my logs have multiple message fields candidates

Preview:
https://github.com/VictoriaMetrics/VictoriaMetrics/blob/docs/VL-FAQ-empty-msg/docs/VictoriaLogs/FAQ.md#i-want-to-ingest-logs-without-message-field-is-that-possible

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
This commit is contained in:
Zhu Jiekun 2024-11-09 06:32:56 +08:00 committed by GitHub
parent 0eb3a0a902
commit 4602752003
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 39 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

39
docs/VictoriaLogs/FAQ.md
View file

@ -131,3 +131,42 @@ Just send the query with the needed [filters](https://docs.victoriametrics.com/v
to [`/select/logsql/query`](https://docs.victoriametrics.com/victorialogs/querying/#querying-logs) - VictoriaLogs will return
the requested logs as a [stream of JSON lines](https://jsonlines.org/). It is recommended specifying [time filter](https://docs.victoriametrics.com/victorialogs/logsql/#time-filter)
for limiting the amounts of exported logs.
## I want to ingest logs without message field, is that possible?
Starting from version `v0.30.0`, VictoriaLogs started blocking the ingestion of logs **without a message field**, as it is a requirement of the [VictoriaLogs data model](https://docs.victoriametrics.com/victorialogs/keyconcepts/#message-field).
However, some logs do not have a message field and only contain other fields, such as logs in [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7056#issuecomment-2434189718) and [this slack thread](https://victoriametrics.slack.com/archives/C05UNTPAEDN/p1730982146818249). Therefore, starting from version `v0.39.0`, logs without a message field are **allowed to be ingested**,
and their message field will be recorded as:
```json
{"_msg": "missing _msg field; see https://docs.victoriametrics.com/victorialogs/keyconcepts/#message-field"}
```
The default message field value can be changed using the `-defaultMsgValue` flag, for example, `-defaultMsgValue=foo`.
Please note that the message field is **crucial** for VictoriaLogs, so it is important to fill it with meaningful content.
## What if my logs have multiple message fields candidates?
When ingesting with VictoriaLogs, the message fields is specified through `_msg_field` param, which can accept **multiple fields**, and the **first non-empty field** will be used as the message field.
Here is an example URL when pushing logs to VictoriaLogs with Promtail:
```yaml
clients:
- url: http://localhost:9428/insert/loki/api/v1/push?_stream_fields=instance,job,host,app&_msg=message,body
```
For the following log, its `_msg` will be `foo bar in message`:
```json
{
"message": "foo bar in message",
"body": "foo bar in body"
}
```
And for the following log, its `_msg` will be `foo bar in body`:
```json
{
"message": "",
"body": "foo bar in body"
}
```