From 46027520037c79ca77aa4c92b2d08d55ef72d13f Mon Sep 17 00:00:00 2001 From: Zhu Jiekun Date: Sat, 9 Nov 2024 06:32:56 +0800 Subject: [PATCH] docs: [VictoriaLogs] FAQ add logs without msg field (#7487) ### Describe Your Changes Add FAQs to VictoriaLogs: - I want to ingest logs without message field, is that possible? - What if my logs have multiple message fields candidates Preview: https://github.com/VictoriaMetrics/VictoriaMetrics/blob/docs/VL-FAQ-empty-msg/docs/VictoriaLogs/FAQ.md#i-want-to-ingest-logs-without-message-field-is-that-possible ### Checklist The following checks are **mandatory**: - [x] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --- docs/VictoriaLogs/FAQ.md | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/docs/VictoriaLogs/FAQ.md b/docs/VictoriaLogs/FAQ.md index fc1024ed5..2bc81bb2c 100644 --- a/docs/VictoriaLogs/FAQ.md +++ b/docs/VictoriaLogs/FAQ.md @@ -131,3 +131,42 @@ Just send the query with the needed [filters](https://docs.victoriametrics.com/v to [`/select/logsql/query`](https://docs.victoriametrics.com/victorialogs/querying/#querying-logs) - VictoriaLogs will return the requested logs as a [stream of JSON lines](https://jsonlines.org/). It is recommended specifying [time filter](https://docs.victoriametrics.com/victorialogs/logsql/#time-filter) for limiting the amounts of exported logs. + +## I want to ingest logs without message field, is that possible? + +Starting from version `v0.30.0`, VictoriaLogs started blocking the ingestion of logs **without a message field**, as it is a requirement of the [VictoriaLogs data model](https://docs.victoriametrics.com/victorialogs/keyconcepts/#message-field). + +However, some logs do not have a message field and only contain other fields, such as logs in [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7056#issuecomment-2434189718) and [this slack thread](https://victoriametrics.slack.com/archives/C05UNTPAEDN/p1730982146818249). Therefore, starting from version `v0.39.0`, logs without a message field are **allowed to be ingested**, +and their message field will be recorded as: +```json +{"_msg": "missing _msg field; see https://docs.victoriametrics.com/victorialogs/keyconcepts/#message-field"} +``` + +The default message field value can be changed using the `-defaultMsgValue` flag, for example, `-defaultMsgValue=foo`. + +Please note that the message field is **crucial** for VictoriaLogs, so it is important to fill it with meaningful content. + +## What if my logs have multiple message fields candidates? + +When ingesting with VictoriaLogs, the message fields is specified through `_msg_field` param, which can accept **multiple fields**, and the **first non-empty field** will be used as the message field. +Here is an example URL when pushing logs to VictoriaLogs with Promtail: +```yaml +clients: + - url: http://localhost:9428/insert/loki/api/v1/push?_stream_fields=instance,job,host,app&_msg=message,body +``` + +For the following log, its `_msg` will be `foo bar in message`: +```json +{ + "message": "foo bar in message", + "body": "foo bar in body" +} +``` + +And for the following log, its `_msg` will be `foo bar in body`: +```json +{ + "message": "", + "body": "foo bar in body" +} +``` \ No newline at end of file