VictoriaMetrics/docs/VictoriaLogs/data-ingestion/Fluentbit.md

Fluentbit setup

Fluentbit log collector supports HTTP output compatible with VictoriaMetrics JSON stream API.

Specify output section with Name http in the fluentbit.conf for sending the collected logs to VictoriaLogs:

[Output]
     Name http
     Match *
     host localhost
     port 9428
     uri /insert/jsonline/?_stream_fields=stream&_msg_field=log&_time_field=date
     format json_lines
     json_date_format iso8601

Substitute the address (localhost) and port (9428) inside Output section with the real TCP address of VictoriaLogs.

The _msg_field parameter must contain the field name with the log message generated by Fluentbit. This is usually message field. See these docs for details.

The _time_field parameter must contain the field name with the log timestamp generated by Fluentbit. This is usually @timestamp field. See these docs for details.

It is recommended specifying comma-separated list of field names, which uniquely identify every log stream collected by Fluentbit, in the _stream_fields parameter. See these docs for details.

If the Fluentbit sends logs to VictoriaLogs in another datacenter, then it may be useful enabling data compression via compress option. This usually allows saving network bandwidth and costs by up to 5 times:

[Output]
     Name http
     Match *
     host localhost
     port 9428
     uri /insert/jsonline/?_stream_fields=stream&_msg_field=log&_time_field=date
     format json_lines
     json_date_format iso8601
     compress gzip

By default, the ingested logs are stored in the (AccountID=0, ProjectID=0) tenant. If you need storing logs in other tenant, then specify the needed tenant via headers at output.elasticsearch section. For example, the following fluentbit.conf config instructs Filebeat to store the data to (AccountID=12, ProjectID=34) tenant:

[Output]
     Name http
     Match *
     host localhost
     port 9428
     uri /insert/jsonline/?_stream_fields=stream&_msg_field=log&_time_field=date
     format json_lines
     json_date_format iso8601
     header AccountID 12
     header ProjectID 23

More info about output tuning you can find in these docs.

Here is a demo for running Fluentbit with VictoriaLogs with docker-compose and collecting logs from docker-containers to VictoriaLogs.

The ingested log entries can be queried according to these docs.

See also data ingestion troubleshooting docs.