VictoriaMetrics/deployment/docker/victorialogs/vector-docker
2024-05-28 22:16:30 +02:00
..
docker-compose.yml deployment/docker: update VictoriaLogs image from v0.12.1-victorialogs to v0.13.0-victorialogs 2024-05-28 22:16:30 +02:00
README.md lib/logstorage: work-in-progress 2024-05-25 00:31:55 +02:00
scrape.yml Add docker compose examples: filebeat(docker, syslog), fluentbit(docker), logstash, vector(docker) 2023-07-06 21:25:31 -07:00
vector.toml deployment/vector: add example for JSON stream config 2024-04-17 09:40:30 +02:00

Docker compose Vector integration with VictoriaLogs for docker

The folder contains the example of integration of vector with Victorialogs

To spin-up environment run the following command:

docker compose up -d 

To shut down the docker-compose environment run the following command:

docker compose down
docker compose rm -f

The docker compose file contains the following components:

  • vector - vector is configured to collect logs from the docker, you can find configuration in the vector.toml. It writes data in VictoriaLogs. It pushes metrics to VictoriaMetrics.
  • VictoriaLogs - the log database, it accepts the data from vector by elastic protocol
  • VictoriaMetrics - collects metrics from VictoriaLogs and VictoriaMetrics

Querying the data

  • vmui - a web UI is accessible by http://localhost:9428/select/vmui
  • for querying the data via command-line please check these docs

the example of vector configuration(vector.toml)

[sources.docker]
  type = "docker_logs"

[transforms.msg_parser]
  type = "remap"
  inputs = ["docker"]
  source = '''
  .log = parse_json!(.message)
  del(.message)
  '''

[sinks.vlogs]
  type = "elasticsearch"
  inputs = [ "msg_parser" ]
  endpoints = [ "http://victorialogs:9428/insert/elasticsearch/" ]
  mode = "bulk"
  api_version = "v8"
  compression = "gzip"
  healthcheck.enabled = false

  [sinks.vlogs.query]
    _msg_field = "log.msg"
    _time_field = "timestamp"
    _stream_fields = "source_type,host,container_name"

  [sinks.vlogs.request.headers]
    AccountID = "0"
    ProjectID = "0"

Please, note that _stream_fields parameter must follow recommended best practices to achieve better performance.