VictoriaMetrics/deployment/docker/victorialogs/fluentbit-docker
Zakhar Bessarab 4e5a68ed08
deployment/docker: add VictoriaLogs (#4929)
* deployment/docker: add VictoriaLogs configuration

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs: remove outdated comment

It was added in order to indicate that it is required to build VictoriaLogs manually before starting it at the time there was no public release available.
Currently, there is a public tag and it is not required to build it from sources.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/fluentbit: include log path in stream configuration

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker: add reference to monitoring setup for VictoriaLogs

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-01 10:45:43 +02:00
..
docker-compose.yml deployment/docker: add VictoriaLogs (#4929) 2023-09-01 10:45:43 +02:00
fluent-bit.conf deployment/docker: add VictoriaLogs (#4929) 2023-09-01 10:45:43 +02:00
README.md fix jsonline endpoint in docs 2023-06-22 08:48:11 +02:00

Docker compose Fluentbit integration with VictoriaLogs for docker

The folder contains the example of integration of fluentbit with Victorialogs

To spin-up environment run the following command:

docker compose up -d 

To shut down the docker-compose environment run the following command:

docker compose down
docker compose rm -f

The docker compose file contains the following components:

  • fluentbit - fluentbit is configured to collect logs from the docker, you can find configuration in the fluent-bit.conf. It writes data in VictoriaLogs
  • VictoriaLogs - the log database, it accepts the data from fluentbit by json line protocol

Querying the data

  • vmui - a web UI is accessible by http://localhost:9428/select/vmui
  • for querying the data via command-line please check these docs

the example of fluentbit configuration(filebeat.yml)

[INPUT]
    name              tail
    path              /var/lib/docker/containers/**/*.log
    multiline.parser  docker, cri
    Parser docker
    Docker_Mode  On

[INPUT]
    Name     syslog
    Listen   0.0.0.0
    Port     5140
    Parser   syslog-rfc3164
    Mode     tcp

[SERVICE]
    Flush        1
    Parsers_File parsers.conf

[Output]
    Name http
    Match *
    host victorialogs
    port 9428
    compress gzip
    uri /insert/jsonline?_stream_fields=stream&_msg_field=log&_time_field=date
    format json_lines
    json_date_format iso8601
    header AccountID 0
    header ProjectID 0

Please, note that _stream_fields parameter must follow recommended best practices to achieve better performance.