mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2025-02-09 15:27:11 +00:00
deployment: restructure victorialogs examples (#6971)
### Describe Your Changes - Use common compose.yaml file for all victorialogs setups to set version in a single place and override it on demand for each agent and protocol - Replaced multiple victorialogs instances in HA setup with single setup with `deploy.replica` parameter set - Added fluentd setup ### Checklist The following checks are **mandatory**: - [ ] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/).
This commit is contained in:
Andrii Chubatiuk
GitHub
parent
612be0954c
commit
e75ae1b274
95 changed files with 705 additions and 1282 deletions
4
deployment/docker/victorialogs/compose-ha.yml
Normal file
4
deployment/docker/victorialogs/compose-ha.yml
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
services:
|
||||||
|
victorialogs-2:
|
||||||
|
deploy:
|
||||||
|
replicas: 1
|
||||||
|
|
@ -1,49 +1,47 @@
|
||||||
services:
|
services:
|
||||||
telegraf:
|
# meta service will be ignored by compose
|
||||||
image: bitnami/telegraf:1.31.0
|
.victorialogs:
|
||||||
restart: on-failure
|
|
||||||
volumes:
|
|
||||||
- type: bind
|
|
||||||
source: /var/run/docker.sock
|
|
||||||
target: /var/run/docker.sock
|
|
||||||
- type: bind
|
|
||||||
source: /var/lib/docker
|
|
||||||
target: /var/lib/docker
|
|
||||||
- ./telegraf.conf:/etc/telegraf/telegraf.conf:ro
|
|
||||||
command: --config /etc/telegraf/telegraf.conf
|
|
||||||
depends_on:
|
|
||||||
victorialogs:
|
|
||||||
condition: service_healthy
|
|
||||||
victoriametrics:
|
|
||||||
condition: service_healthy
|
|
||||||
|
|
||||||
victorialogs:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vl:/vlogs
|
|
||||||
ports:
|
|
||||||
- '9428:9428'
|
|
||||||
command:
|
command:
|
||||||
- -storageDataPath=/vlogs
|
- -storageDataPath=/vlogs
|
||||||
- -loggerFormat=json
|
- -loggerFormat=json
|
||||||
- -syslog.listenAddr.tcp=0.0.0.0:8094
|
- -syslog.listenAddr.tcp=0.0.0.0:8094
|
||||||
|
deploy:
|
||||||
|
replicas: 0
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9428/health"]
|
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9428/health"]
|
||||||
interval: 1s
|
interval: 1s
|
||||||
timeout: 1s
|
timeout: 1s
|
||||||
retries: 10
|
retries: 10
|
||||||
|
|
||||||
|
victorialogs:
|
||||||
|
extends: .victorialogs
|
||||||
|
ports:
|
||||||
|
- '9428:9428'
|
||||||
|
volumes:
|
||||||
|
- victorialogs:/vlogs
|
||||||
|
deploy:
|
||||||
|
replicas: 1
|
||||||
|
|
||||||
|
# second replica is needed for HA setup and its replica count is set to 1 in compose-ha.yaml file
|
||||||
|
victorialogs-2:
|
||||||
|
extends: .victorialogs
|
||||||
|
ports:
|
||||||
|
- '9429:9428'
|
||||||
|
volumes:
|
||||||
|
- victorialogs-2:/vlogs
|
||||||
|
deploy:
|
||||||
|
replicas: 0
|
||||||
|
|
||||||
victoriametrics:
|
victoriametrics:
|
||||||
image: victoriametrics/victoria-metrics:latest
|
image: victoriametrics/victoria-metrics:latest
|
||||||
ports:
|
ports:
|
||||||
- '8428:8428'
|
- '8428:8428'
|
||||||
command:
|
command:
|
||||||
- -storageDataPath=/vmsingle
|
- -storageDataPath=/vmsingle
|
||||||
- -promscrape.config=/promscrape.yml
|
|
||||||
- -loggerFormat=json
|
- -loggerFormat=json
|
||||||
volumes:
|
volumes:
|
||||||
- victorialogs-vector-docker-vm:/vmsingle
|
- victoriametrics:/vmsingle
|
||||||
- ./scrape.yml:/promscrape.yml
|
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8428/health"]
|
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8428/health"]
|
||||||
interval: 1s
|
interval: 1s
|
||||||
|
|
@ -51,5 +49,6 @@ services:
|
||||||
retries: 10
|
retries: 10
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
victorialogs-vector-docker-vl:
|
victorialogs:
|
||||||
victorialogs-vector-docker-vm:
|
victorialogs-2:
|
||||||
|
victoriametrics:
|
||||||
|
|
@ -1,45 +0,0 @@
|
||||||
# Docker compose Filebeat integration with VictoriaLogs for syslog
|
|
||||||
|
|
||||||
The folder contains the example of integration of [filebeat](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-overview.html) with Victorialogs
|
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
|
||||||
```
|
|
||||||
docker compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
To shut down the docker-compose environment run the following command:
|
|
||||||
```
|
|
||||||
docker compose down
|
|
||||||
docker compose rm -f
|
|
||||||
```
|
|
||||||
|
|
||||||
The docker compose file contains the following components:
|
|
||||||
|
|
||||||
* filebeat - fileabeat is configured to accept `syslog` logs in `rfc3164` format on `5140` port, you can find configuration in the `filebeat.yml`. It writes data in VictoriaLogs
|
|
||||||
* VictoriaLogs - the log database, it accepts the data from `filebeat` by elastic protocol
|
|
||||||
|
|
||||||
Querying the data
|
|
||||||
|
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
|
||||||
|
|
||||||
the example of filebeat configuration(`filebeat.yml`)
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
filebeat.inputs:
|
|
||||||
- type: syslog
|
|
||||||
format: rfc3164
|
|
||||||
protocol.tcp:
|
|
||||||
host: "0.0.0.0:5140"
|
|
||||||
|
|
||||||
output.elasticsearch:
|
|
||||||
hosts: [ "http://victorialogs:9428/insert/elasticsearch/" ]
|
|
||||||
worker: 5
|
|
||||||
bulk_max_size: 1000
|
|
||||||
parameters:
|
|
||||||
_msg_field: "message"
|
|
||||||
_time_field: "@timestamp"
|
|
||||||
_stream_fields: "host.name,process.program,process.pid,container.name"
|
|
||||||
```
|
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
|
||||||
|
|
@ -1,23 +0,0 @@
|
||||||
services:
|
|
||||||
filebeat-vlogs:
|
|
||||||
image: docker.elastic.co/beats/filebeat:8.8.1
|
|
||||||
user: root
|
|
||||||
command:
|
|
||||||
- "--strict.perms=false"
|
|
||||||
volumes:
|
|
||||||
- ./filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
|
|
||||||
depends_on: [victorialogs]
|
|
||||||
ports:
|
|
||||||
- "5140:5140"
|
|
||||||
|
|
||||||
victorialogs:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-filebeat-syslog-vl:/vlogs
|
|
||||||
ports:
|
|
||||||
- "9428:9428"
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-filebeat-syslog-vl:
|
|
||||||
|
|
@ -1,8 +1,11 @@
|
||||||
# Docker compose Filebeat integration with VictoriaLogs for docker
|
# Docker compose Filebeat integration with VictoriaLogs using listed below protocols:
|
||||||
|
|
||||||
|
* [syslog](./syslog)
|
||||||
|
* [elasticsearch](./elasticsearch)
|
||||||
|
|
||||||
The folder contains the example of integration of [filebeat](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-overview.html) with Victorialogs
|
The folder contains the example of integration of [filebeat](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-overview.html) with Victorialogs
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
To spin-up environment `cd` to any of listed above directories run the following command:
|
||||||
```
|
```
|
||||||
docker compose up -d
|
docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
@ -16,39 +19,16 @@ docker compose rm -f
|
||||||
The docker compose file contains the following components:
|
The docker compose file contains the following components:
|
||||||
|
|
||||||
* filebeat - fileabeat is configured to collect logs from the `docker`, you can find configuration in the `filebeat.yml`. It writes data in VictoriaLogs
|
* filebeat - fileabeat is configured to collect logs from the `docker`, you can find configuration in the `filebeat.yml`. It writes data in VictoriaLogs
|
||||||
* filebeat-exporter - it export metrics about the filebeat
|
|
||||||
* VictoriaLogs - the log database, it accepts the data from `filebeat` by elastic protocol
|
* VictoriaLogs - the log database, it accepts the data from `filebeat` by elastic protocol
|
||||||
* VictoriaMetrics - collects metrics from `filebeat` via `filebeat-exporter`, `VictoriaLogs` and `VictoriaMetrics`
|
* VictoriaMetrics - collects metrics from `filebeat` via `filebeat-exporter`, `VictoriaLogs` and `VictoriaMetrics`
|
||||||
* grafana - it comes with two predefined dashboards for `VictoriaLogs` and `VictoriaMetrics`
|
|
||||||
|
|
||||||
Querying the data
|
Querying the data
|
||||||
|
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
||||||
|
|
||||||
the example of filebeat configuration(`filebeat.yml`)
|
Filebeat configuration example can be found below:
|
||||||
|
- [syslog](./syslog/filebeat.yml)
|
||||||
```yaml
|
- [elasticsearch](./elasticsearch/filebeat.yml)
|
||||||
filebeat.autodiscover:
|
|
||||||
providers:
|
|
||||||
- type: docker
|
|
||||||
hints.enabled: true
|
|
||||||
|
|
||||||
processors:
|
|
||||||
- add_docker_metadata: ~
|
|
||||||
|
|
||||||
output.elasticsearch:
|
|
||||||
hosts: [ "http://victorialogs:9428/insert/elasticsearch/" ]
|
|
||||||
worker: 5
|
|
||||||
parameters:
|
|
||||||
_msg_field: "message"
|
|
||||||
_time_field: "@timestamp"
|
|
||||||
_stream_fields: "container.name"
|
|
||||||
|
|
||||||
http:
|
|
||||||
enabled: true
|
|
||||||
host: 0.0.0.0
|
|
||||||
port: 5066
|
|
||||||
```
|
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,8 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
services:
|
services:
|
||||||
filebeat-victorialogs:
|
filebeat-victorialogs:
|
||||||
image: docker.elastic.co/beats/filebeat:8.8.1
|
image: docker.elastic.co/beats/filebeat:8.15.0
|
||||||
restart: on-failure
|
restart: on-failure
|
||||||
volumes:
|
volumes:
|
||||||
- type: bind
|
- type: bind
|
||||||
|
|
@ -9,50 +11,8 @@ services:
|
||||||
- type: bind
|
- type: bind
|
||||||
source: /var/lib/docker/containers/
|
source: /var/lib/docker/containers/
|
||||||
target: /var/lib/docker/containers/
|
target: /var/lib/docker/containers/
|
||||||
- ./filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
|
- ${PWD}/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
|
||||||
user: root
|
user: root
|
||||||
command:
|
command:
|
||||||
- "--strict.perms=false"
|
- "--strict.perms=false"
|
||||||
depends_on: [ victorialogs ]
|
depends_on: [victorialogs]
|
||||||
beat-exporter-victorialogs:
|
|
||||||
image: trustpilot/beat-exporter:0.4.0
|
|
||||||
command:
|
|
||||||
- -beat.uri=http://filebeat-victorialogs:5066
|
|
||||||
|
|
||||||
victorialogs:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-filebeat-docker-vl:/vlogs
|
|
||||||
ports:
|
|
||||||
- '9428:9428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
- -loggerFormat=json
|
|
||||||
grafana:
|
|
||||||
container_name: grafana
|
|
||||||
image: grafana/grafana:9.2.7
|
|
||||||
depends_on:
|
|
||||||
- "victoriametrics"
|
|
||||||
ports:
|
|
||||||
- 3000:3000
|
|
||||||
volumes:
|
|
||||||
- victorialogs-filebeat-docker-grafana:/var/lib/grafana
|
|
||||||
- ./../../provisioning/:/etc/grafana/provisioning/
|
|
||||||
- ./../../../../dashboards/victoriametrics.json:/var/lib/grafana/dashboards/vm.json
|
|
||||||
- ./../../../../dashboards/victorialogs.json:/var/lib/grafana/dashboards/vl.json
|
|
||||||
victoriametrics:
|
|
||||||
image: victoriametrics/victoria-metrics:latest
|
|
||||||
ports:
|
|
||||||
- '8428:8428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vmsingle
|
|
||||||
- -promscrape.config=/promscrape.yml
|
|
||||||
- -loggerFormat=json
|
|
||||||
volumes:
|
|
||||||
- victorialogs-filebeat-docker-vm:/vmsingle
|
|
||||||
- ./scrape.yml:/promscrape.yml
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-filebeat-docker-vl:
|
|
||||||
victorialogs-filebeat-docker-vm:
|
|
||||||
victorialogs-filebeat-docker-grafana:
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: filebeat-elasticsearch
|
||||||
|
|
@ -1,16 +0,0 @@
|
||||||
scrape_configs:
|
|
||||||
- job_name: "filebeat"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- beat-exporter-victorialogs:9479
|
|
||||||
- job_name: "victoriametrics"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victoriametrics:8428
|
|
||||||
- job_name: "victorialogs"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victorialogs:9428
|
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: filebeat-syslog
|
||||||
|
|
@ -1,88 +0,0 @@
|
||||||
# Docker compose Fluentbit integration with VictoriaLogs for docker. High-Availability example
|
|
||||||
|
|
||||||
The folder contains the example of integration of [fluentbit](https://docs.fluentbit.io/manual) with VictoriaLogs Single-Nodes(s) and [vmauth](https://docs.victoriametrics.com/vmauth/) for achieving High Availability.
|
|
||||||
|
|
||||||
Check [this documentation](https://docs.victoriametrics.com/victorialogs/#high-availability) with a description of the architecture and components.
|
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
|
||||||
|
|
||||||
```shell
|
|
||||||
docker compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
To shut down the docker-compose environment run the following command:
|
|
||||||
|
|
||||||
```shell
|
|
||||||
docker compose down
|
|
||||||
docker compose rm -f
|
|
||||||
```
|
|
||||||
|
|
||||||
The docker compose file contains the following components:
|
|
||||||
|
|
||||||
* fluentbit - fluentbit is configured to collect logs from the `docker`, you can find configuration in the `fluent-bit.conf`. It writes data in VictoriaLogs
|
|
||||||
* VictoriaLogs - the two instances of log database, they accept the data from `fluentbit` by json line protocol
|
|
||||||
* vmauth - load balancer for proxying requests to one of VictoriaLogs
|
|
||||||
|
|
||||||
Querying the data
|
|
||||||
|
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:8427/select/vmui/`
|
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
|
||||||
|
|
||||||
|
|
||||||
the example of fluentbit configuration(`fluent-bit.conf`)
|
|
||||||
|
|
||||||
```text
|
|
||||||
[INPUT]
|
|
||||||
name tail
|
|
||||||
path /var/lib/docker/containers/**/*.log
|
|
||||||
path_key path
|
|
||||||
multiline.parser docker, cri
|
|
||||||
Parser docker
|
|
||||||
Docker_Mode On
|
|
||||||
|
|
||||||
[INPUT]
|
|
||||||
Name syslog
|
|
||||||
Listen 0.0.0.0
|
|
||||||
Port 5140
|
|
||||||
Parser syslog-rfc3164
|
|
||||||
Mode tcp
|
|
||||||
|
|
||||||
[SERVICE]
|
|
||||||
Flush 1
|
|
||||||
Parsers_File parsers.conf
|
|
||||||
|
|
||||||
[OUTPUT]
|
|
||||||
Name http
|
|
||||||
Match *
|
|
||||||
host victorialogs-2
|
|
||||||
port 9428
|
|
||||||
compress gzip
|
|
||||||
uri /insert/jsonline?_stream_fields=stream,path&_msg_field=log&_time_field=date
|
|
||||||
format json_lines
|
|
||||||
json_date_format iso8601
|
|
||||||
header AccountID 0
|
|
||||||
header ProjectID 0
|
|
||||||
|
|
||||||
[OUTPUT]
|
|
||||||
Name http
|
|
||||||
Match *
|
|
||||||
host victorialogs-1
|
|
||||||
port 9428
|
|
||||||
compress gzip
|
|
||||||
uri /insert/jsonline?_stream_fields=stream,path&_msg_field=log&_time_field=date
|
|
||||||
format json_lines
|
|
||||||
json_date_format iso8601
|
|
||||||
header AccountID 0
|
|
||||||
header ProjectID 0
|
|
||||||
```
|
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
|
||||||
|
|
||||||
The example of vmauth configuration (`auth.yml`)
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
unauthorized_user:
|
|
||||||
url_prefix:
|
|
||||||
- http://victorialogs-1:9428
|
|
||||||
- http://victorialogs-2:9428
|
|
||||||
```
|
|
||||||
|
|
@ -1,6 +0,0 @@
|
||||||
# balance load among victorialogs instances
|
|
||||||
# see https://docs.victoriametrics.com/vmauth/#load-balancing
|
|
||||||
unauthorized_user:
|
|
||||||
url_prefix:
|
|
||||||
- http://victorialogs-1:9428
|
|
||||||
- http://victorialogs-2:9428
|
|
||||||
|
|
@ -1,38 +0,0 @@
|
||||||
services:
|
|
||||||
fluentbit:
|
|
||||||
image: cr.fluentbit.io/fluent/fluent-bit:3.0.7
|
|
||||||
volumes:
|
|
||||||
- /var/lib/docker/containers:/var/lib/docker/containers:ro
|
|
||||||
- ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf
|
|
||||||
depends_on: [victorialogs-1,victorialogs-2]
|
|
||||||
ports:
|
|
||||||
- "5140:5140"
|
|
||||||
|
|
||||||
victorialogs-1:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-fluentbit-vl-ha-single-1:/vlogs
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
victorialogs-2:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-fluentbit-vl-ha-single-2:/vlogs
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
vmauth:
|
|
||||||
container_name: vmauth
|
|
||||||
image: victoriametrics/vmauth:v1.103.0
|
|
||||||
depends_on:
|
|
||||||
- "victorialogs-1"
|
|
||||||
- "victorialogs-2"
|
|
||||||
volumes:
|
|
||||||
- ./auth.yml:/etc/auth.yml
|
|
||||||
command:
|
|
||||||
- '--auth.config=/etc/auth.yml'
|
|
||||||
ports:
|
|
||||||
- 8427:8427
|
|
||||||
restart: always
|
|
||||||
volumes:
|
|
||||||
victorialogs-fluentbit-vl-ha-single-1:
|
|
||||||
victorialogs-fluentbit-vl-ha-single-2:
|
|
||||||
|
|
@ -1,8 +1,12 @@
|
||||||
# Docker compose Fluentbit integration with VictoriaLogs for docker
|
# Docker compose Fluentbit integration with VictoriaLogs using given below protocols:
|
||||||
|
|
||||||
|
* [loki](./loki)
|
||||||
|
* [jsonline single node](./jsonline)
|
||||||
|
* [jsonline HA setup](./jsonline-ha)
|
||||||
|
|
||||||
The folder contains the example of integration of [fluentbit](https://docs.fluentbit.io/manual) with Victorialogs
|
The folder contains the example of integration of [fluentbit](https://docs.fluentbit.io/manual) with Victorialogs
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
To spin-up environment `cd` to any of listed above directories run the following command:
|
||||||
```
|
```
|
||||||
docker compose up -d
|
docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
@ -23,39 +27,9 @@ Querying the data
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
||||||
|
|
||||||
|
FluentBit configuration example can be found below:
|
||||||
the example of fluentbit configuration(`fluent-bit.conf`)
|
* [loki](./loki/fluent-bit.conf)
|
||||||
|
* [jsonline single node](./jsonline/fluent-bit.conf)
|
||||||
```shell
|
* [jsonline HA setup](./jsonline-ha/fluent-bit.conf)
|
||||||
[INPUT]
|
|
||||||
name tail
|
|
||||||
path /var/lib/docker/containers/**/*.log
|
|
||||||
multiline.parser docker, cri
|
|
||||||
Parser docker
|
|
||||||
Docker_Mode On
|
|
||||||
|
|
||||||
[INPUT]
|
|
||||||
Name syslog
|
|
||||||
Listen 0.0.0.0
|
|
||||||
Port 5140
|
|
||||||
Parser syslog-rfc3164
|
|
||||||
Mode tcp
|
|
||||||
|
|
||||||
[SERVICE]
|
|
||||||
Flush 1
|
|
||||||
Parsers_File parsers.conf
|
|
||||||
|
|
||||||
[Output]
|
|
||||||
Name http
|
|
||||||
Match *
|
|
||||||
host victorialogs
|
|
||||||
port 9428
|
|
||||||
compress gzip
|
|
||||||
uri /insert/jsonline?_stream_fields=stream&_msg_field=log&_time_field=date
|
|
||||||
format json_lines
|
|
||||||
json_date_format iso8601
|
|
||||||
header AccountID 0
|
|
||||||
header ProjectID 0
|
|
||||||
```
|
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
||||||
|
|
|
||||||
|
|
@ -1,21 +1,11 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
services:
|
services:
|
||||||
fluentbit:
|
fluentbit:
|
||||||
image: cr.fluentbit.io/fluent/fluent-bit:3.0.7
|
image: cr.fluentbit.io/fluent/fluent-bit:3.1.7
|
||||||
volumes:
|
volumes:
|
||||||
- /var/lib/docker/containers:/var/lib/docker/containers:ro
|
- /var/lib/docker/containers:/var/lib/docker/containers:ro
|
||||||
- ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf
|
- ${PWD}/fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf
|
||||||
depends_on: [victorialogs]
|
depends_on: [victorialogs]
|
||||||
ports:
|
ports:
|
||||||
- "5140:5140"
|
- "5140:5140"
|
||||||
|
|
||||||
victorialogs:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-fluentbit-vl:/vlogs
|
|
||||||
ports:
|
|
||||||
- "9428:9428"
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-fluentbit-vl:
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,5 @@
|
||||||
|
include:
|
||||||
|
- path:
|
||||||
|
- ../compose.yml
|
||||||
|
- ../../compose-ha.yml
|
||||||
|
name: fluentbit-jsonline-ha
|
||||||
|
|
@ -1,10 +1,10 @@
|
||||||
[INPUT]
|
[INPUT]
|
||||||
name tail
|
name tail
|
||||||
path /var/lib/docker/containers/**/*.log
|
path /var/lib/docker/containers/**/*.log
|
||||||
path_key path
|
path_key path
|
||||||
multiline.parser docker, cri
|
multiline.parser docker, cri
|
||||||
Parser docker
|
Parser docker
|
||||||
Docker_Mode On
|
Docker_Mode On
|
||||||
|
|
||||||
[INPUT]
|
[INPUT]
|
||||||
Name syslog
|
Name syslog
|
||||||
|
|
@ -17,7 +17,19 @@
|
||||||
Flush 1
|
Flush 1
|
||||||
Parsers_File parsers.conf
|
Parsers_File parsers.conf
|
||||||
|
|
||||||
[OUTPUT]
|
[Output]
|
||||||
|
Name http
|
||||||
|
Match *
|
||||||
|
host victorialogs
|
||||||
|
port 9428
|
||||||
|
compress gzip
|
||||||
|
uri /insert/jsonline?_stream_fields=stream,path&_msg_field=log&_time_field=date
|
||||||
|
format json_lines
|
||||||
|
json_date_format iso8601
|
||||||
|
header AccountID 0
|
||||||
|
header ProjectID 0
|
||||||
|
|
||||||
|
[Output]
|
||||||
Name http
|
Name http
|
||||||
Match *
|
Match *
|
||||||
host victorialogs-2
|
host victorialogs-2
|
||||||
|
|
@ -28,15 +40,3 @@
|
||||||
json_date_format iso8601
|
json_date_format iso8601
|
||||||
header AccountID 0
|
header AccountID 0
|
||||||
header ProjectID 0
|
header ProjectID 0
|
||||||
|
|
||||||
[OUTPUT]
|
|
||||||
Name http
|
|
||||||
Match *
|
|
||||||
host victorialogs-1
|
|
||||||
port 9428
|
|
||||||
compress gzip
|
|
||||||
uri /insert/jsonline?_stream_fields=stream,path&_msg_field=log&_time_field=date
|
|
||||||
format json_lines
|
|
||||||
json_date_format iso8601
|
|
||||||
header AccountID 0
|
|
||||||
header ProjectID 0
|
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: fluentbit-jsonline
|
||||||
|
|
@ -0,0 +1,30 @@
|
||||||
|
[INPUT]
|
||||||
|
name tail
|
||||||
|
path /var/lib/docker/containers/**/*.log
|
||||||
|
path_key path
|
||||||
|
multiline.parser docker, cri
|
||||||
|
Parser docker
|
||||||
|
Docker_Mode On
|
||||||
|
|
||||||
|
[INPUT]
|
||||||
|
Name syslog
|
||||||
|
Listen 0.0.0.0
|
||||||
|
Port 5140
|
||||||
|
Parser syslog-rfc3164
|
||||||
|
Mode tcp
|
||||||
|
|
||||||
|
[SERVICE]
|
||||||
|
Flush 1
|
||||||
|
Parsers_File parsers.conf
|
||||||
|
|
||||||
|
[Output]
|
||||||
|
Name http
|
||||||
|
Match *
|
||||||
|
host victorialogs
|
||||||
|
port 9428
|
||||||
|
compress gzip
|
||||||
|
uri /insert/jsonline?_stream_fields=stream,path&_msg_field=log&_time_field=date
|
||||||
|
format json_lines
|
||||||
|
json_date_format iso8601
|
||||||
|
header AccountID 0
|
||||||
|
header ProjectID 0
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: fluentbit-loki
|
||||||
|
|
@ -17,31 +17,6 @@
|
||||||
Flush 1
|
Flush 1
|
||||||
Parsers_File parsers.conf
|
Parsers_File parsers.conf
|
||||||
|
|
||||||
[OUTPUT]
|
|
||||||
Name http
|
|
||||||
Match *
|
|
||||||
host victorialogs
|
|
||||||
port 9428
|
|
||||||
compress gzip
|
|
||||||
uri /insert/jsonline?_stream_fields=stream,path&_msg_field=log&_time_field=date
|
|
||||||
format json_lines
|
|
||||||
json_date_format iso8601
|
|
||||||
header AccountID 0
|
|
||||||
header ProjectID 0
|
|
||||||
|
|
||||||
[OUTPUT]
|
|
||||||
Name es
|
|
||||||
Match *
|
|
||||||
host victorialogs
|
|
||||||
port 9428
|
|
||||||
compress gzip
|
|
||||||
path /insert/elasticsearch
|
|
||||||
header AccountID 0
|
|
||||||
header ProjectID 0
|
|
||||||
header VL-Stream-Fields path
|
|
||||||
header VL-Msg-Field log
|
|
||||||
header VL-Time-Field @timestamp
|
|
||||||
|
|
||||||
[OUTPUT]
|
[OUTPUT]
|
||||||
name loki
|
name loki
|
||||||
match *
|
match *
|
||||||
8
deployment/docker/victorialogs/fluentd/Dockerfile
Normal file
8
deployment/docker/victorialogs/fluentd/Dockerfile
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
FROM fluent/fluentd:v1.17
|
||||||
|
USER root
|
||||||
|
RUN \
|
||||||
|
gem install \
|
||||||
|
fluent-plugin-datadog \
|
||||||
|
fluent-plugin-grafana-loki \
|
||||||
|
fluent-plugin-elasticsearch
|
||||||
|
USER fluent
|
||||||
35
deployment/docker/victorialogs/fluentd/README.md
Normal file
35
deployment/docker/victorialogs/fluentd/README.md
Normal file
|
|
@ -0,0 +1,35 @@
|
||||||
|
# Docker compose Fluentd integration with VictoriaLogs using given below protocols:
|
||||||
|
|
||||||
|
* [loki](./loki)
|
||||||
|
* [jsonline](./jsonline)
|
||||||
|
* [elasticsearch](./elasticsearch)
|
||||||
|
|
||||||
|
The folder contains the example of integration of [fluentd](https://www.fluentd.org/) with Victorialogs
|
||||||
|
|
||||||
|
To spin-up environment `cd` to any of listed above directories run the following command:
|
||||||
|
```
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
To shut down the docker-compose environment run the following command:
|
||||||
|
```
|
||||||
|
docker compose down
|
||||||
|
docker compose rm -f
|
||||||
|
```
|
||||||
|
|
||||||
|
The docker compose file contains the following components:
|
||||||
|
|
||||||
|
* fluentd - fluentd is configured to collect logs from the `docker`, you can find configuration in the `fluent-bit.conf`. It writes data in VictoriaLogs
|
||||||
|
* VictoriaLogs - the log database, it accepts the data from `fluentd` by json line protocol
|
||||||
|
|
||||||
|
Querying the data
|
||||||
|
|
||||||
|
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
||||||
|
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
||||||
|
|
||||||
|
Fluentd configuration example can be found below:
|
||||||
|
* [loki](./loki/fluent.conf)
|
||||||
|
* [jsonline](./jsonline/fluent.conf)
|
||||||
|
* [elasticsearch](./elasticsearch/fluent.conf)
|
||||||
|
|
||||||
|
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
||||||
11
deployment/docker/victorialogs/fluentd/compose.yml
Normal file
11
deployment/docker/victorialogs/fluentd/compose.yml
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
services:
|
||||||
|
fluentd:
|
||||||
|
build: .
|
||||||
|
volumes:
|
||||||
|
- /var/lib/docker/containers:/var/lib/docker/containers:ro
|
||||||
|
- ${PWD}/fluent.conf:/fluentd/etc/fluent.conf
|
||||||
|
depends_on: [victorialogs]
|
||||||
|
ports:
|
||||||
|
- "5140:5140"
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: fluentd-elasticsearch
|
||||||
|
|
@ -0,0 +1,13 @@
|
||||||
|
<source>
|
||||||
|
@type tail
|
||||||
|
format none
|
||||||
|
tag docker.testlog
|
||||||
|
path /var/lib/docker/containers/**/*.log
|
||||||
|
</source>
|
||||||
|
|
||||||
|
<match **>
|
||||||
|
@type elasticsearch
|
||||||
|
host victorialogs
|
||||||
|
path /insert/elasticsearch
|
||||||
|
port 9428
|
||||||
|
</match>
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: fluentd-jsonline
|
||||||
12
deployment/docker/victorialogs/fluentd/jsonline/fluent.conf
Normal file
12
deployment/docker/victorialogs/fluentd/jsonline/fluent.conf
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
<source>
|
||||||
|
@type tail
|
||||||
|
format none
|
||||||
|
tag docker.testlog
|
||||||
|
path /var/lib/docker/containers/**/*.log
|
||||||
|
</source>
|
||||||
|
|
||||||
|
<match **>
|
||||||
|
@type http
|
||||||
|
endpoint "http://victorialogs:9428/insert/jsonline"
|
||||||
|
headers {"VL-Msg-Field": "log", "VL-Time-Field": "time", "VL-Stream-Fields": "path"}
|
||||||
|
</match>
|
||||||
3
deployment/docker/victorialogs/fluentd/loki/compose.yml
Normal file
3
deployment/docker/victorialogs/fluentd/loki/compose.yml
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: fluentbit-loki
|
||||||
17
deployment/docker/victorialogs/fluentd/loki/fluent.conf
Normal file
17
deployment/docker/victorialogs/fluentd/loki/fluent.conf
Normal file
|
|
@ -0,0 +1,17 @@
|
||||||
|
<source>
|
||||||
|
@type tail
|
||||||
|
format none
|
||||||
|
tag docker.testlog
|
||||||
|
path /var/lib/docker/containers/**/*.log
|
||||||
|
</source>
|
||||||
|
|
||||||
|
<match **>
|
||||||
|
@type loki
|
||||||
|
url "http://victorialogs:9428/insert"
|
||||||
|
<buffer>
|
||||||
|
flush_interval 10s
|
||||||
|
flush_at_shutdown true
|
||||||
|
</buffer>
|
||||||
|
custom_headers {"VL-Msg-Field": "log", "VL-Time-Field": "time", "VL-Stream-Fields": "path"}
|
||||||
|
buffer_chunk_limit 1m
|
||||||
|
</match>
|
||||||
|
|
@ -1,71 +0,0 @@
|
||||||
# Docker compose Logstash integration with VictoriaLogs for docker. High-Availability example
|
|
||||||
|
|
||||||
The folder contains the example of integration of [logstash](https://www.elastic.co/logstash) with VictoriaLogs Single-Node(s) and [vmauth](https://docs.victoriametrics.com/vmauth/) for achieving High Availability.
|
|
||||||
|
|
||||||
Check [this documentation](https://docs.victoriametrics.com/victorialogs/#high-availability) with a description of the architecture and components.
|
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
|
||||||
|
|
||||||
```shell
|
|
||||||
docker compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
To shut down the docker-compose environment run the following command:
|
|
||||||
|
|
||||||
```shell
|
|
||||||
docker compose down
|
|
||||||
docker compose rm -f
|
|
||||||
```
|
|
||||||
|
|
||||||
The docker compose file contains the following components:
|
|
||||||
|
|
||||||
* logstash - logstash is configured to read docker log files, you can find configuration in the `pipeline.conf`. It writes data in two instances of VictoriaLogs
|
|
||||||
* VictoriaLogs - the two instances of log database, they accept the data from `fluentbit` by json line protocol
|
|
||||||
* vmauth - load balancer for proxying requests to one of VictoriaLogs
|
|
||||||
|
|
||||||
Querying the data
|
|
||||||
|
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:8427/select/vmui/`
|
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
|
||||||
|
|
||||||
|
|
||||||
Here is an example of logstash configuration(`pipeline.conf`):
|
|
||||||
|
|
||||||
```text
|
|
||||||
input {
|
|
||||||
file {
|
|
||||||
path => "/var/lib/docker/containers/*/*.log"
|
|
||||||
start_position => "beginning"
|
|
||||||
type => "docker"
|
|
||||||
sincedb_path => "/dev/null"
|
|
||||||
codec => "json"
|
|
||||||
add_field => {
|
|
||||||
"path" => "%{[@metadata][path]}"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
output {
|
|
||||||
http {
|
|
||||||
url => "http://victorialogs-1:9428/insert/jsonline?_stream_fields=host.name,stream&_msg_field=log&_time_field=time"
|
|
||||||
format => "json"
|
|
||||||
http_method => "post"
|
|
||||||
}
|
|
||||||
http {
|
|
||||||
url => "http://victorialogs-2:9428/insert/jsonline?_stream_fields=host.name,stream&_msg_field=log&_time_field=time"
|
|
||||||
format => "json"
|
|
||||||
http_method => "post"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
|
||||||
|
|
||||||
The example of vmauth configuration (`auth.yml`)
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
unauthorized_user:
|
|
||||||
url_prefix:
|
|
||||||
- http://victorialogs-1:9428
|
|
||||||
- http://victorialogs-2:9428
|
|
||||||
```
|
|
||||||
|
|
@ -1,6 +0,0 @@
|
||||||
# balance load among victorialogs instances
|
|
||||||
# see https://docs.victoriametrics.com/vmauth/#load-balancing
|
|
||||||
unauthorized_user:
|
|
||||||
url_prefix:
|
|
||||||
- http://victorialogs-1:9428
|
|
||||||
- http://victorialogs-2:9428
|
|
||||||
|
|
@ -1,41 +0,0 @@
|
||||||
services:
|
|
||||||
logstash:
|
|
||||||
image: docker.elastic.co/logstash/logstash:8.8.1
|
|
||||||
user: root
|
|
||||||
volumes:
|
|
||||||
- ./pipeline.conf:/usr/share/logstash/pipeline/logstash.conf:ro
|
|
||||||
- ./logstash.yml:/usr/share/logstash/config/logstash.yml:ro
|
|
||||||
- /var/lib/docker/containers:/var/lib/docker/containers:ro
|
|
||||||
depends_on: [victorialogs-1,victorialogs-2]
|
|
||||||
ports:
|
|
||||||
- "5140:5140"
|
|
||||||
|
|
||||||
victorialogs-1:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-logstash-vl-ha-single-1:/vlogs
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
victorialogs-2:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-logstash-vl-ha-single-2:/vlogs
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
vmauth:
|
|
||||||
container_name: vmauth
|
|
||||||
image: victoriametrics/vmauth:v1.103.0
|
|
||||||
depends_on:
|
|
||||||
- "victorialogs-1"
|
|
||||||
- "victorialogs-2"
|
|
||||||
volumes:
|
|
||||||
- ./auth.yml:/etc/auth.yml
|
|
||||||
command:
|
|
||||||
- '--auth.config=/etc/auth.yml'
|
|
||||||
ports:
|
|
||||||
- 8427:8427
|
|
||||||
restart: always
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-logstash-vl-ha-single-1:
|
|
||||||
victorialogs-logstash-vl-ha-single-2:
|
|
||||||
|
|
@ -1,2 +0,0 @@
|
||||||
http.host: 0.0.0.0
|
|
||||||
xpack.monitoring.enabled: false
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
FROM docker.elastic.co/logstash/logstash:8.8.1
|
FROM docker.elastic.co/logstash/logstash:8.15.0
|
||||||
|
|
||||||
RUN bin/logstash-plugin install \
|
RUN bin/logstash-plugin install \
|
||||||
logstash-output-opensearch \
|
logstash-output-opensearch \
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,9 @@
|
||||||
# Docker compose Logstash integration with VictoriaLogs for syslog
|
# Docker compose Logstash integration with VictoriaLogs for given below protocols:
|
||||||
|
|
||||||
|
* [loki](./loki)
|
||||||
|
* [jsonline single node](./jsonline)
|
||||||
|
* [jsonline HA setup](./jsonline-ha)
|
||||||
|
* [elasticsearch](./elasticsearch)
|
||||||
|
|
||||||
It is required to use [OpenSearch plugin](https://github.com/opensearch-project/logstash-output-opensearch) for output configuration.
|
It is required to use [OpenSearch plugin](https://github.com/opensearch-project/logstash-output-opensearch) for output configuration.
|
||||||
Plugin can be installed by using the following command:
|
Plugin can be installed by using the following command:
|
||||||
|
|
@ -7,7 +12,7 @@ bin/logstash-plugin install logstash-output-opensearch
|
||||||
```
|
```
|
||||||
OpenSearch plugin is required because elasticsearch output plugin performs various checks for Elasticsearch version and license which are not applicable for VictoriaLogs.
|
OpenSearch plugin is required because elasticsearch output plugin performs various checks for Elasticsearch version and license which are not applicable for VictoriaLogs.
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
To spin-up environment `cd` to any of listed above directories run the following command:
|
||||||
```
|
```
|
||||||
docker compose up -d
|
docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
@ -28,29 +33,10 @@ Querying the data
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
||||||
|
|
||||||
|
Logstash configuration example can be found below:
|
||||||
Here is an example of logstash configuration(`pipeline.conf`):
|
* [loki](./loki/pipeline.conf)
|
||||||
|
* [jsonline single node](./jsonline/pipeline.conf)
|
||||||
```
|
* [jsonline HA setup](./jsonline-ha/pipeline.conf)
|
||||||
input {
|
* [elasticsearch](./elasticsearch/pipeline.conf)
|
||||||
syslog {
|
|
||||||
port => 5140
|
|
||||||
}
|
|
||||||
}
|
|
||||||
output {
|
|
||||||
opensearch {
|
|
||||||
hosts => ["http://victorialogs:9428/insert/elasticsearch"]
|
|
||||||
custom_headers => {
|
|
||||||
"AccountID" => "0"
|
|
||||||
"ProjectID" => "0"
|
|
||||||
}
|
|
||||||
parameters => {
|
|
||||||
"_stream_fields" => "host.ip,process.name"
|
|
||||||
"_msg_field" => "message"
|
|
||||||
"_time_field" => "@timestamp"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
||||||
|
|
|
||||||
|
|
@ -1,24 +1,15 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
services:
|
services:
|
||||||
logstash:
|
logstash:
|
||||||
build:
|
build:
|
||||||
context: .
|
context: .
|
||||||
dockerfile: Dockerfile
|
user: root
|
||||||
restart: on-failure
|
restart: on-failure
|
||||||
volumes:
|
volumes:
|
||||||
- ./pipeline.conf:/usr/share/logstash/pipeline/logstash.conf:ro
|
- /var/lib/docker/containers:/var/lib/docker/containers
|
||||||
|
- ${PWD}/pipeline.conf:/usr/share/logstash/pipeline/logstash.conf:ro
|
||||||
- ./logstash.yml:/usr/share/logstash/config/logstash.yml:ro
|
- ./logstash.yml:/usr/share/logstash/config/logstash.yml:ro
|
||||||
depends_on: [victorialogs]
|
depends_on: [victorialogs]
|
||||||
ports:
|
ports:
|
||||||
- "5140:5140"
|
- "5140:5140"
|
||||||
|
|
||||||
victorialogs:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-logstash-vl:/vlogs
|
|
||||||
ports:
|
|
||||||
- "9428:9428"
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-logstash-vl:
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: logstash-elasticsearch
|
||||||
|
|
@ -17,12 +17,4 @@ output {
|
||||||
"_time_field" => "@timestamp"
|
"_time_field" => "@timestamp"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
http {
|
|
||||||
url => "http://victorialogs:9428/insert/jsonline?_stream_fields=host.ip,process.name&_msg_field=message&_time_field=@timestamp"
|
|
||||||
format => "json"
|
|
||||||
http_method => "post"
|
|
||||||
}
|
|
||||||
loki {
|
|
||||||
url => "http://victorialogs:9428/insert/loki/api/v1/push?_stream_fields=host.ip,process.name&_msg_field=message&_time_field=@timestamp"
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
@ -0,0 +1,5 @@
|
||||||
|
include:
|
||||||
|
- path:
|
||||||
|
- ../compose.yml
|
||||||
|
- ../../compose-ha.yml
|
||||||
|
name: logstash-jsonline-ha
|
||||||
|
|
@ -13,7 +13,7 @@ input {
|
||||||
|
|
||||||
output {
|
output {
|
||||||
http {
|
http {
|
||||||
url => "http://victorialogs-1:9428/insert/jsonline?_stream_fields=host.name,stream&_msg_field=log&_time_field=time"
|
url => "http://victorialogs:9428/insert/jsonline?_stream_fields=host.name,stream&_msg_field=log&_time_field=time"
|
||||||
format => "json"
|
format => "json"
|
||||||
http_method => "post"
|
http_method => "post"
|
||||||
}
|
}
|
||||||
|
|
@ -22,4 +22,4 @@ output {
|
||||||
format => "json"
|
format => "json"
|
||||||
http_method => "post"
|
http_method => "post"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: logstash-jsonline
|
||||||
|
|
@ -0,0 +1,13 @@
|
||||||
|
input {
|
||||||
|
syslog {
|
||||||
|
port => 5140
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
output {
|
||||||
|
http {
|
||||||
|
url => "http://victorialogs:9428/insert/jsonline?_stream_fields=host.ip,process.name&_msg_field=message&_time_field=@timestamp"
|
||||||
|
format => "json"
|
||||||
|
http_method => "post"
|
||||||
|
}
|
||||||
|
}
|
||||||
3
deployment/docker/victorialogs/logstash/loki/compose.yml
Normal file
3
deployment/docker/victorialogs/logstash/loki/compose.yml
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: logstash-loki
|
||||||
11
deployment/docker/victorialogs/logstash/loki/pipeline.conf
Normal file
11
deployment/docker/victorialogs/logstash/loki/pipeline.conf
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
input {
|
||||||
|
syslog {
|
||||||
|
port => 5140
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
output {
|
||||||
|
loki {
|
||||||
|
url => "http://victorialogs:9428/insert/loki/api/v1/push?_stream_fields=host.ip,process.name&_msg_field=message&_time_field=@timestamp"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,8 +1,14 @@
|
||||||
# Docker compose OpenTelemetry Elasticsearch integration with VictoriaLogs for docker
|
# Docker compose OpenTelemetry integration with VictoriaLogs using protocols:
|
||||||
|
|
||||||
|
* [loki](./loki)
|
||||||
|
* [otlp](./otlp)
|
||||||
|
* [syslog](./syslog)
|
||||||
|
* [elasticsearch single node](./elasticsearch)
|
||||||
|
* [elasticsearch HA mode](./elasticsearch-ha/)
|
||||||
|
|
||||||
The folder contains the example of integration of [OpenTelemetry collector](https://opentelemetry.io/docs/collector/) with Victorialogs
|
The folder contains the example of integration of [OpenTelemetry collector](https://opentelemetry.io/docs/collector/) with Victorialogs
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
To spin-up environment `cd` to any of listed above directories run the following command:
|
||||||
```
|
```
|
||||||
docker compose up -d
|
docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
@ -24,4 +30,11 @@ Querying the data
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
||||||
|
|
||||||
|
OpenTelemetry collector configuration example can be found below:
|
||||||
|
* [loki](./loki/config.yaml)
|
||||||
|
* [otlp](./otlp/config.yaml)
|
||||||
|
* [syslog](./syslog/config.yaml)
|
||||||
|
* [elasticsearch single node](./elasticsearch/config.yaml)
|
||||||
|
* [elasticsearch HA mode](./elasticsearch-ha/config.yaml)
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
||||||
|
|
@ -0,0 +1,15 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
services:
|
||||||
|
collector:
|
||||||
|
image: docker.io/otel/opentelemetry-collector-contrib:0.108.0
|
||||||
|
restart: on-failure
|
||||||
|
user: 0:0
|
||||||
|
volumes:
|
||||||
|
- ${PWD}/config.yaml:/etc/otelcol-contrib/config.yaml
|
||||||
|
- /var/lib/docker/containers:/var/lib/docker/containers
|
||||||
|
depends_on:
|
||||||
|
victorialogs:
|
||||||
|
condition: service_healthy
|
||||||
|
victoriametrics:
|
||||||
|
condition: service_healthy
|
||||||
|
|
@ -0,0 +1,5 @@
|
||||||
|
include:
|
||||||
|
- path:
|
||||||
|
- ../compose.yml
|
||||||
|
- ../../compose-ha.yml
|
||||||
|
name: collector-elasticsearch-ha
|
||||||
|
|
@ -0,0 +1,15 @@
|
||||||
|
exporters:
|
||||||
|
elasticsearch:
|
||||||
|
endpoints:
|
||||||
|
- http://victorialogs:9428/insert/elasticsearch
|
||||||
|
- http://victorialogs-2:9428/insert/elasticsearch
|
||||||
|
receivers:
|
||||||
|
filelog:
|
||||||
|
include: [/var/lib/docker/containers/**/*.log]
|
||||||
|
resource:
|
||||||
|
region: us-east-1
|
||||||
|
service:
|
||||||
|
pipelines:
|
||||||
|
logs:
|
||||||
|
receivers: [filelog]
|
||||||
|
exporters: [elasticsearch]
|
||||||
|
|
@ -1,48 +1,3 @@
|
||||||
services:
|
include:
|
||||||
collector:
|
- ../compose.yml
|
||||||
image: docker.io/otel/opentelemetry-collector-contrib:0.102.1
|
name: collector-elasticsearch
|
||||||
restart: on-failure
|
|
||||||
volumes:
|
|
||||||
- $PWD/logs:/tmp/logs
|
|
||||||
- $PWD/config.yaml:/etc/otelcol-contrib/config.yaml
|
|
||||||
depends_on:
|
|
||||||
victorialogs:
|
|
||||||
condition: service_healthy
|
|
||||||
victoriametrics:
|
|
||||||
condition: service_healthy
|
|
||||||
|
|
||||||
victorialogs:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vl:/vlogs
|
|
||||||
ports:
|
|
||||||
- '9428:9428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
- -loggerFormat=json
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9428/health"]
|
|
||||||
interval: 1s
|
|
||||||
timeout: 1s
|
|
||||||
retries: 10
|
|
||||||
|
|
||||||
victoriametrics:
|
|
||||||
image: victoriametrics/victoria-metrics:latest
|
|
||||||
ports:
|
|
||||||
- '8428:8428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vmsingle
|
|
||||||
- -promscrape.config=/promscrape.yml
|
|
||||||
- -loggerFormat=json
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vm:/vmsingle
|
|
||||||
- ./scrape.yml:/promscrape.yml
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8428/health"]
|
|
||||||
interval: 1s
|
|
||||||
timeout: 1s
|
|
||||||
retries: 10
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-vector-docker-vl:
|
|
||||||
victorialogs-vector-docker-vm:
|
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@ exporters:
|
||||||
- http://victorialogs:9428/insert/elasticsearch
|
- http://victorialogs:9428/insert/elasticsearch
|
||||||
receivers:
|
receivers:
|
||||||
filelog:
|
filelog:
|
||||||
include: [/tmp/logs/*.log]
|
include: [/var/lib/docker/containers/**/*.log]
|
||||||
resource:
|
resource:
|
||||||
region: us-east-1
|
region: us-east-1
|
||||||
service:
|
service:
|
||||||
|
|
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
scrape_configs:
|
|
||||||
- job_name: "victoriametrics"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victoriametrics:8428
|
|
||||||
- job_name: "victorialogs"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victorialogs:9428
|
|
||||||
|
|
@ -1,27 +0,0 @@
|
||||||
# Docker compose OpenTelemetry Loki integration with VictoriaLogs for docker
|
|
||||||
|
|
||||||
The folder contains the example of integration of [OpenTelemetry collector](https://opentelemetry.io/docs/collector/) with Victorialogs
|
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
|
||||||
```
|
|
||||||
docker compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
To shut down the docker-compose environment run the following command:
|
|
||||||
```
|
|
||||||
docker compose down
|
|
||||||
docker compose rm -f
|
|
||||||
```
|
|
||||||
|
|
||||||
The docker compose file contains the following components:
|
|
||||||
|
|
||||||
* collector - vector is configured to collect logs from the `docker`, you can find configuration in the `config.yaml`. It writes data in VictoriaLogs. It pushes metrics to VictoriaMetrics.
|
|
||||||
* VictoriaLogs - the log database, it accepts the data from `collector` by Loki protocol
|
|
||||||
* VictoriaMetrics - collects metrics from `VictoriaLogs` and `VictoriaMetrics`
|
|
||||||
|
|
||||||
Querying the data
|
|
||||||
|
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
|
||||||
|
|
@ -1,48 +1,3 @@
|
||||||
services:
|
include:
|
||||||
collector:
|
- ../compose.yml
|
||||||
image: docker.io/otel/opentelemetry-collector-contrib:0.102.1
|
name: collector-loki
|
||||||
restart: on-failure
|
|
||||||
volumes:
|
|
||||||
- $PWD/logs:/tmp/logs
|
|
||||||
- $PWD/config.yaml:/etc/otelcol-contrib/config.yaml
|
|
||||||
depends_on:
|
|
||||||
victorialogs:
|
|
||||||
condition: service_healthy
|
|
||||||
victoriametrics:
|
|
||||||
condition: service_healthy
|
|
||||||
|
|
||||||
victorialogs:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vl:/loki
|
|
||||||
ports:
|
|
||||||
- '9428:9428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/loki
|
|
||||||
- -loggerFormat=json
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9428/health"]
|
|
||||||
interval: 1s
|
|
||||||
timeout: 1s
|
|
||||||
retries: 10
|
|
||||||
|
|
||||||
victoriametrics:
|
|
||||||
image: victoriametrics/victoria-metrics:latest
|
|
||||||
ports:
|
|
||||||
- '8428:8428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vmsingle
|
|
||||||
- -promscrape.config=/promscrape.yml
|
|
||||||
- -loggerFormat=json
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vm:/vmsingle
|
|
||||||
- ./scrape.yml:/promscrape.yml
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8428/health"]
|
|
||||||
interval: 1s
|
|
||||||
timeout: 1s
|
|
||||||
retries: 10
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-vector-docker-vl:
|
|
||||||
victorialogs-vector-docker-vm:
|
|
||||||
|
|
|
||||||
|
|
@ -3,7 +3,7 @@ exporters:
|
||||||
endpoint: http://victorialogs:9428/insert/loki/api/v1/push
|
endpoint: http://victorialogs:9428/insert/loki/api/v1/push
|
||||||
receivers:
|
receivers:
|
||||||
filelog:
|
filelog:
|
||||||
include: [/tmp/logs/*.log]
|
include: [/var/lib/docker/containers/**/*.log]
|
||||||
resource:
|
resource:
|
||||||
region: us-east-1
|
region: us-east-1
|
||||||
service:
|
service:
|
||||||
|
|
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
scrape_configs:
|
|
||||||
- job_name: "victoriametrics"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victoriametrics:8428
|
|
||||||
- job_name: "victorialogs"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victorialogs:9428
|
|
||||||
|
|
@ -1,27 +0,0 @@
|
||||||
# Docker compose OpenTelemetry OTLP integration with VictoriaLogs for docker
|
|
||||||
|
|
||||||
The folder contains the example of integration of [OpenTelemetry collector](https://opentelemetry.io/docs/collector/) with Victorialogs
|
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
|
||||||
```
|
|
||||||
docker compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
To shut down the docker-compose environment run the following command:
|
|
||||||
```
|
|
||||||
docker compose down
|
|
||||||
docker compose rm -f
|
|
||||||
```
|
|
||||||
|
|
||||||
The docker compose file contains the following components:
|
|
||||||
|
|
||||||
* collector - vector is configured to collect logs from the `docker`, you can find configuration in the `config.yaml`. It writes data in VictoriaLogs. It pushes metrics to VictoriaMetrics.
|
|
||||||
* VictoriaLogs - the log database, it accepts the data from `collector` by otlp protocol
|
|
||||||
* VictoriaMetrics - collects metrics from `VictoriaLogs` and `VictoriaMetrics`
|
|
||||||
|
|
||||||
Querying the data
|
|
||||||
|
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
|
||||||
|
|
@ -1,48 +1,3 @@
|
||||||
services:
|
include:
|
||||||
collector:
|
- ../compose.yml
|
||||||
image: docker.io/otel/opentelemetry-collector-contrib:0.102.1
|
name: collector-otlp
|
||||||
restart: on-failure
|
|
||||||
volumes:
|
|
||||||
- $PWD/logs:/tmp/logs
|
|
||||||
- $PWD/config.yaml:/etc/otelcol-contrib/config.yaml
|
|
||||||
depends_on:
|
|
||||||
victorialogs:
|
|
||||||
condition: service_healthy
|
|
||||||
victoriametrics:
|
|
||||||
condition: service_healthy
|
|
||||||
|
|
||||||
victorialogs:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vl:/otlp
|
|
||||||
ports:
|
|
||||||
- '9428:9428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/otlp
|
|
||||||
- -loggerFormat=json
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9428/health"]
|
|
||||||
interval: 1s
|
|
||||||
timeout: 1s
|
|
||||||
retries: 10
|
|
||||||
|
|
||||||
victoriametrics:
|
|
||||||
image: victoriametrics/victoria-metrics:latest
|
|
||||||
ports:
|
|
||||||
- '8428:8428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vmsingle
|
|
||||||
- -promscrape.config=/promscrape.yml
|
|
||||||
- -loggerFormat=json
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vm:/vmsingle
|
|
||||||
- ./scrape.yml:/promscrape.yml
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8428/health"]
|
|
||||||
interval: 1s
|
|
||||||
timeout: 1s
|
|
||||||
retries: 10
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-vector-docker-vl:
|
|
||||||
victorialogs-vector-docker-vm:
|
|
||||||
|
|
|
||||||
|
|
@ -5,7 +5,7 @@ exporters:
|
||||||
verbosity: detailed
|
verbosity: detailed
|
||||||
receivers:
|
receivers:
|
||||||
filelog:
|
filelog:
|
||||||
include: [/tmp/logs/*.log]
|
include: [/var/lib/docker/containers/**/*.log]
|
||||||
resource:
|
resource:
|
||||||
region: us-east-1
|
region: us-east-1
|
||||||
service:
|
service:
|
||||||
|
|
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
scrape_configs:
|
|
||||||
- job_name: "victoriametrics"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victoriametrics:8428
|
|
||||||
- job_name: "victorialogs"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victorialogs:9428
|
|
||||||
|
|
@ -1,27 +0,0 @@
|
||||||
# Docker compose OpenTelemetry Syslog integration with VictoriaLogs for docker
|
|
||||||
|
|
||||||
The folder contains the example of integration of [OpenTelemetry collector](https://opentelemetry.io/docs/collector/) with Victorialogs
|
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
|
||||||
```
|
|
||||||
docker compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
To shut down the docker-compose environment run the following command:
|
|
||||||
```
|
|
||||||
docker compose down
|
|
||||||
docker compose rm -f
|
|
||||||
```
|
|
||||||
|
|
||||||
The docker compose file contains the following components:
|
|
||||||
|
|
||||||
* collector - vector is configured to collect logs from the `docker`, you can find configuration in the `config.yaml`. It writes data in VictoriaLogs. It pushes metrics to VictoriaMetrics.
|
|
||||||
* VictoriaLogs - the log database, it accepts the data from `collector` by syslog protocol
|
|
||||||
* VictoriaMetrics - collects metrics from `VictoriaLogs` and `VictoriaMetrics`
|
|
||||||
|
|
||||||
Querying the data
|
|
||||||
|
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
|
||||||
|
|
@ -1,49 +1,3 @@
|
||||||
services:
|
include:
|
||||||
collector:
|
- ../compose.yml
|
||||||
image: docker.io/otel/opentelemetry-collector-contrib:0.107.0
|
name: collector-syslog
|
||||||
restart: on-failure
|
|
||||||
volumes:
|
|
||||||
- $PWD/logs:/tmp/logs
|
|
||||||
- $PWD/config.yaml:/etc/otelcol-contrib/config.yaml
|
|
||||||
depends_on:
|
|
||||||
victorialogs:
|
|
||||||
condition: service_healthy
|
|
||||||
victoriametrics:
|
|
||||||
condition: service_healthy
|
|
||||||
|
|
||||||
victorialogs:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vl:/syslog
|
|
||||||
ports:
|
|
||||||
- '9428:9428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/syslog
|
|
||||||
- -syslog.listenAddr.tcp=:5410
|
|
||||||
- -syslog.useLocalTimestamp.tcp
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9428/health"]
|
|
||||||
interval: 1s
|
|
||||||
timeout: 1s
|
|
||||||
retries: 10
|
|
||||||
|
|
||||||
victoriametrics:
|
|
||||||
image: victoriametrics/victoria-metrics:latest
|
|
||||||
ports:
|
|
||||||
- '8428:8428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vmsingle
|
|
||||||
- -promscrape.config=/promscrape.yml
|
|
||||||
- -loggerFormat=json
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vm:/vmsingle
|
|
||||||
- ./scrape.yml:/promscrape.yml
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8428/health"]
|
|
||||||
interval: 1s
|
|
||||||
timeout: 1s
|
|
||||||
retries: 10
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-vector-docker-vl:
|
|
||||||
victorialogs-vector-docker-vm:
|
|
||||||
|
|
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
scrape_configs:
|
|
||||||
- job_name: "victoriametrics"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victoriametrics:8428
|
|
||||||
- job_name: "victorialogs"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victorialogs:9428
|
|
||||||
|
|
@ -1,22 +1,12 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
services:
|
services:
|
||||||
promtail:
|
promtail:
|
||||||
image: grafana/promtail:2.8.2
|
image: grafana/promtail:3.0.1
|
||||||
volumes:
|
volumes:
|
||||||
- /var/lib/docker/containers:/var/lib/docker/containers:ro
|
- /var/lib/docker/containers:/var/lib/docker/containers:ro
|
||||||
- /var/log:/var/log:ro
|
- /var/log:/var/log:ro
|
||||||
- ./config.yml:/etc/promtail/docker-config.yml:ro
|
- ${PWD}/config.yml:/etc/promtail/docker-config.yml:ro
|
||||||
command: -config.file=/etc/promtail/docker-config.yml
|
command: -config.file=/etc/promtail/docker-config.yml
|
||||||
ports:
|
ports:
|
||||||
- "5140:5140"
|
- "5140:5140"
|
||||||
|
|
||||||
vlogs:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-promtail-docker:/vlogs
|
|
||||||
ports:
|
|
||||||
- '9428:9428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-promtail-docker:
|
|
||||||
|
|
|
||||||
3
deployment/docker/victorialogs/promtail/loki/compose.yml
Normal file
3
deployment/docker/victorialogs/promtail/loki/compose.yml
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: promtail-elasticsearch
|
||||||
|
|
@ -6,7 +6,7 @@ positions:
|
||||||
filename: /tmp/positions.yaml
|
filename: /tmp/positions.yaml
|
||||||
|
|
||||||
clients:
|
clients:
|
||||||
- url: http://vlogs:9428/insert/loki/api/v1/push?_stream_fields=instance,job,host,app
|
- url: http://victorialogs:9428/insert/loki/api/v1/push?_stream_fields=instance,job,host,app
|
||||||
tenant_id: "0:0"
|
tenant_id: "0:0"
|
||||||
|
|
||||||
scrape_configs:
|
scrape_configs:
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
scrape_configs:
|
|
||||||
- job_name: "victoriametrics"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victoriametrics:8428
|
|
||||||
- job_name: "victorialogs"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victorialogs:9428
|
|
||||||
|
|
@ -1,8 +1,13 @@
|
||||||
# Docker compose Telegraf integration with VictoriaLogs for docker
|
# Docker compose Telegraf integration with VictoriaLogs for docker
|
||||||
|
|
||||||
The folder contains the example of integration of [telegraf](https://www.influxdata.com/time-series-platform/telegraf/) with VictoriaLogs
|
The folder contains the examples of integration of [telegraf](https://www.influxdata.com/time-series-platform/telegraf/) with VictoriaLogs using:
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
* [elasticsearch](./elasticsearch)
|
||||||
|
* [loki](./loki)
|
||||||
|
* [jsonline](./jsonline)
|
||||||
|
* [syslog](./syslog)
|
||||||
|
|
||||||
|
To spin-up environment `cd` to any of listed above directories run the following command:
|
||||||
```
|
```
|
||||||
docker compose up -d
|
docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
@ -23,3 +28,11 @@ Querying the data
|
||||||
|
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
||||||
|
|
||||||
|
Telegraf configuration example can be found below:
|
||||||
|
* [elasticsearch](./elasticsearch/telegraf.conf)
|
||||||
|
* [loki](./loki/telegraf.conf)
|
||||||
|
* [jsonline](./jsonline/telegraf.conf)
|
||||||
|
* [syslog](./syslog/telegraf.conf)
|
||||||
|
|
||||||
|
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
||||||
20
deployment/docker/victorialogs/telegraf/compose.yml
Normal file
20
deployment/docker/victorialogs/telegraf/compose.yml
Normal file
|
|
@ -0,0 +1,20 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
services:
|
||||||
|
telegraf:
|
||||||
|
image: bitnami/telegraf:1.31.3
|
||||||
|
restart: on-failure
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: /var/run/docker.sock
|
||||||
|
target: /var/run/docker.sock
|
||||||
|
- type: bind
|
||||||
|
source: /var/lib/docker
|
||||||
|
target: /var/lib/docker
|
||||||
|
- ${PWD}/telegraf.conf:/etc/telegraf/telegraf.conf:ro
|
||||||
|
command: --config /etc/telegraf/telegraf.conf
|
||||||
|
depends_on:
|
||||||
|
victorialogs:
|
||||||
|
condition: service_healthy
|
||||||
|
victoriametrics:
|
||||||
|
condition: service_healthy
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: telegraf-elasticsearch
|
||||||
|
|
@ -16,23 +16,6 @@
|
||||||
|
|
||||||
[[inputs.cpu]]
|
[[inputs.cpu]]
|
||||||
|
|
||||||
[[outputs.http]]
|
|
||||||
url = "http://victorialogs:9428/insert/jsonline?_msg_field=fields.msg&_stream_fields=tags.log_source,tags.metric_type"
|
|
||||||
data_format = "json"
|
|
||||||
namepass = ["docker_log"]
|
|
||||||
use_batch_format = false
|
|
||||||
|
|
||||||
[[outputs.loki]]
|
|
||||||
domain = "http://victorialogs:9428"
|
|
||||||
endpoint = "/insert/loki/api/v1/push?_msg_field=docker_log.msg&_time_field=@timestamp&_stream_fields=log_source,metric_type"
|
|
||||||
namepass = ["docker_log"]
|
|
||||||
gzip_request = true
|
|
||||||
sanitize_label_names = true
|
|
||||||
|
|
||||||
[[outputs.syslog]]
|
|
||||||
address = "tcp://victorialogs:8094"
|
|
||||||
namepass = ["docker_log"]
|
|
||||||
|
|
||||||
[[outputs.elasticsearch]]
|
[[outputs.elasticsearch]]
|
||||||
urls = ["http://victorialogs:9428/insert/elasticsearch"]
|
urls = ["http://victorialogs:9428/insert/elasticsearch"]
|
||||||
timeout = "1m"
|
timeout = "1m"
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: telegraf-jsonline
|
||||||
|
|
@ -0,0 +1,43 @@
|
||||||
|
[agent]
|
||||||
|
interval = "10s"
|
||||||
|
round_interval = true
|
||||||
|
metric_batch_size = 1000
|
||||||
|
metric_buffer_limit = 100000
|
||||||
|
collection_jitter = "0s"
|
||||||
|
flush_interval = "10s"
|
||||||
|
flush_jitter = "0s"
|
||||||
|
precision = ""
|
||||||
|
debug = false
|
||||||
|
quiet = false
|
||||||
|
logtarget = "file"
|
||||||
|
logfile = "/dev/null"
|
||||||
|
hostname = "pop-os"
|
||||||
|
omit_hostname = false
|
||||||
|
|
||||||
|
[[inputs.cpu]]
|
||||||
|
|
||||||
|
[[outputs.http]]
|
||||||
|
url = "http://victorialogs:9428/insert/jsonline?_msg_field=fields.msg&_stream_fields=tags.log_source,tags.metric_type"
|
||||||
|
data_format = "json"
|
||||||
|
namepass = ["docker_log"]
|
||||||
|
use_batch_format = false
|
||||||
|
|
||||||
|
[[outputs.http]]
|
||||||
|
url = "http://victoriametrics:8428/api/v1/write"
|
||||||
|
data_format = "prometheusremotewrite"
|
||||||
|
namepass = ["cpu"]
|
||||||
|
[outputs.http.headers]
|
||||||
|
Content-Type = "application/x-protobuf"
|
||||||
|
Content-Encoding = "snappy"
|
||||||
|
X-Prometheus-Remote-Write-Version = "0.1.0"
|
||||||
|
|
||||||
|
[[inputs.docker_log]]
|
||||||
|
[inputs.docker_log.tags]
|
||||||
|
metric_type = "logs"
|
||||||
|
log_source = "telegraf"
|
||||||
|
|
||||||
|
[[processors.rename]]
|
||||||
|
namepass = ["docker_log"]
|
||||||
|
[[processors.rename.replace]]
|
||||||
|
field = "message"
|
||||||
|
dest = "msg"
|
||||||
3
deployment/docker/victorialogs/telegraf/loki/compose.yml
Normal file
3
deployment/docker/victorialogs/telegraf/loki/compose.yml
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: telegraf-loki
|
||||||
44
deployment/docker/victorialogs/telegraf/loki/telegraf.conf
Normal file
44
deployment/docker/victorialogs/telegraf/loki/telegraf.conf
Normal file
|
|
@ -0,0 +1,44 @@
|
||||||
|
[agent]
|
||||||
|
interval = "10s"
|
||||||
|
round_interval = true
|
||||||
|
metric_batch_size = 1000
|
||||||
|
metric_buffer_limit = 100000
|
||||||
|
collection_jitter = "0s"
|
||||||
|
flush_interval = "10s"
|
||||||
|
flush_jitter = "0s"
|
||||||
|
precision = ""
|
||||||
|
debug = false
|
||||||
|
quiet = false
|
||||||
|
logtarget = "file"
|
||||||
|
logfile = "/dev/null"
|
||||||
|
hostname = "pop-os"
|
||||||
|
omit_hostname = false
|
||||||
|
|
||||||
|
[[inputs.cpu]]
|
||||||
|
|
||||||
|
[[outputs.loki]]
|
||||||
|
domain = "http://victorialogs:9428"
|
||||||
|
endpoint = "/insert/loki/api/v1/push?_msg_field=docker_log.msg&_time_field=@timestamp&_stream_fields=log_source,metric_type"
|
||||||
|
namepass = ["docker_log"]
|
||||||
|
gzip_request = true
|
||||||
|
sanitize_label_names = true
|
||||||
|
|
||||||
|
[[outputs.http]]
|
||||||
|
url = "http://victoriametrics:8428/api/v1/write"
|
||||||
|
data_format = "prometheusremotewrite"
|
||||||
|
namepass = ["cpu"]
|
||||||
|
[outputs.http.headers]
|
||||||
|
Content-Type = "application/x-protobuf"
|
||||||
|
Content-Encoding = "snappy"
|
||||||
|
X-Prometheus-Remote-Write-Version = "0.1.0"
|
||||||
|
|
||||||
|
[[inputs.docker_log]]
|
||||||
|
[inputs.docker_log.tags]
|
||||||
|
metric_type = "logs"
|
||||||
|
log_source = "telegraf"
|
||||||
|
|
||||||
|
[[processors.rename]]
|
||||||
|
namepass = ["docker_log"]
|
||||||
|
[[processors.rename.replace]]
|
||||||
|
field = "message"
|
||||||
|
dest = "msg"
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: telegraf-syslog
|
||||||
41
deployment/docker/victorialogs/telegraf/syslog/telegraf.conf
Normal file
41
deployment/docker/victorialogs/telegraf/syslog/telegraf.conf
Normal file
|
|
@ -0,0 +1,41 @@
|
||||||
|
[agent]
|
||||||
|
interval = "10s"
|
||||||
|
round_interval = true
|
||||||
|
metric_batch_size = 1000
|
||||||
|
metric_buffer_limit = 100000
|
||||||
|
collection_jitter = "0s"
|
||||||
|
flush_interval = "10s"
|
||||||
|
flush_jitter = "0s"
|
||||||
|
precision = ""
|
||||||
|
debug = false
|
||||||
|
quiet = false
|
||||||
|
logtarget = "file"
|
||||||
|
logfile = "/dev/null"
|
||||||
|
hostname = "pop-os"
|
||||||
|
omit_hostname = false
|
||||||
|
|
||||||
|
[[inputs.cpu]]
|
||||||
|
|
||||||
|
[[outputs.syslog]]
|
||||||
|
address = "tcp://victorialogs:8094"
|
||||||
|
namepass = ["docker_log"]
|
||||||
|
|
||||||
|
[[outputs.http]]
|
||||||
|
url = "http://victoriametrics:8428/api/v1/write"
|
||||||
|
data_format = "prometheusremotewrite"
|
||||||
|
namepass = ["cpu"]
|
||||||
|
[outputs.http.headers]
|
||||||
|
Content-Type = "application/x-protobuf"
|
||||||
|
Content-Encoding = "snappy"
|
||||||
|
X-Prometheus-Remote-Write-Version = "0.1.0"
|
||||||
|
|
||||||
|
[[inputs.docker_log]]
|
||||||
|
[inputs.docker_log.tags]
|
||||||
|
metric_type = "logs"
|
||||||
|
log_source = "telegraf"
|
||||||
|
|
||||||
|
[[processors.rename]]
|
||||||
|
namepass = ["docker_log"]
|
||||||
|
[[processors.rename.replace]]
|
||||||
|
field = "message"
|
||||||
|
dest = "msg"
|
||||||
|
|
@ -1,104 +0,0 @@
|
||||||
# Docker compose Vector integration with VictoriaLogs for docker. High-Availability example
|
|
||||||
|
|
||||||
The folder contains the example of integration of [vector](https://vector.dev/docs/) with VictoriaLogs Single-Node(s) and [vmauth](https://docs.victoriametrics.com/vmauth/) for achieving High Availability.
|
|
||||||
|
|
||||||
Check [this documentation](https://docs.victoriametrics.com/victorialogs/#high-availability) with a description of the architecture and components.
|
|
||||||
|
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
|
||||||
|
|
||||||
```shell
|
|
||||||
docker compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
To shut down the docker-compose environment run the following command:
|
|
||||||
|
|
||||||
```shell
|
|
||||||
docker compose down
|
|
||||||
docker compose rm -f
|
|
||||||
```
|
|
||||||
|
|
||||||
The docker compose file contains the following components:
|
|
||||||
|
|
||||||
* vector - vector is configured to collect logs from the `docker`, you can find configuration in the `vector.yaml`. It writes data in two instances of VictoriaLogs
|
|
||||||
* VictoriaLogs - the two instances of log database, they accept the data from `vector` by json line protocol
|
|
||||||
* vmauth - load balancer for proxying requests to one of VictoriaLogs
|
|
||||||
|
|
||||||
Querying the data
|
|
||||||
|
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:8427/select/vmui/`
|
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
|
||||||
|
|
||||||
|
|
||||||
the example of vector configuration(`vector.yaml`)
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
api:
|
|
||||||
enabled: true
|
|
||||||
address: 0.0.0.0:8686
|
|
||||||
sources:
|
|
||||||
docker:
|
|
||||||
type: docker_logs
|
|
||||||
transforms:
|
|
||||||
msg_parser:
|
|
||||||
type: remap
|
|
||||||
inputs:
|
|
||||||
- docker
|
|
||||||
source: |
|
|
||||||
if exists(.message) {
|
|
||||||
.log, err = parse_json(.message)
|
|
||||||
if err == null {
|
|
||||||
del(.message)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
sinks:
|
|
||||||
console_out:
|
|
||||||
type: console
|
|
||||||
inputs:
|
|
||||||
- msg_parser
|
|
||||||
encoding:
|
|
||||||
codec: json
|
|
||||||
vlogs_http_1:
|
|
||||||
type: http
|
|
||||||
inputs:
|
|
||||||
- msg_parser
|
|
||||||
uri: http://victorialogs-1:9428/insert/jsonline?_stream_fields=source_type,host,container_name&_msg_field=log.msg&_time_field=timestamp
|
|
||||||
encoding:
|
|
||||||
codec: json
|
|
||||||
framing:
|
|
||||||
method: newline_delimited
|
|
||||||
compression: gzip
|
|
||||||
healthcheck:
|
|
||||||
enabled: false
|
|
||||||
request:
|
|
||||||
headers:
|
|
||||||
AccountID: '0'
|
|
||||||
ProjectID: '0'
|
|
||||||
vlogs_http_2:
|
|
||||||
type: http
|
|
||||||
inputs:
|
|
||||||
- msg_parser
|
|
||||||
uri: http://victorialogs-2:9428/insert/jsonline?_stream_fields=source_type,host,container_name&_msg_field=log.msg&_time_field=timestamp
|
|
||||||
encoding:
|
|
||||||
codec: json
|
|
||||||
framing:
|
|
||||||
method: newline_delimited
|
|
||||||
compression: gzip
|
|
||||||
healthcheck:
|
|
||||||
enabled: false
|
|
||||||
request:
|
|
||||||
headers:
|
|
||||||
AccountID: '0'
|
|
||||||
ProjectID: '0'
|
|
||||||
```
|
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
|
||||||
|
|
||||||
The example of vmauth configuration (`auth.yml`)
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
unauthorized_user:
|
|
||||||
url_prefix:
|
|
||||||
- http://victorialogs-1:9428
|
|
||||||
- http://victorialogs-2:9428
|
|
||||||
```
|
|
||||||
|
|
@ -1,6 +0,0 @@
|
||||||
# balance load among victorialogs instances
|
|
||||||
# see https://docs.victoriametrics.com/vmauth/#load-balancing
|
|
||||||
unauthorized_user:
|
|
||||||
url_prefix:
|
|
||||||
- http://victorialogs-1:9428
|
|
||||||
- http://victorialogs-2:9428
|
|
||||||
|
|
@ -1,48 +0,0 @@
|
||||||
services:
|
|
||||||
vector:
|
|
||||||
image: docker.io/timberio/vector:0.40.1-distroless-static
|
|
||||||
restart: on-failure
|
|
||||||
volumes:
|
|
||||||
- type: bind
|
|
||||||
source: /var/run/docker.sock
|
|
||||||
target: /var/run/docker.sock
|
|
||||||
- type: bind
|
|
||||||
source: /var/lib/docker
|
|
||||||
target: /var/lib/docker
|
|
||||||
- ./vector.yaml:/etc/vector/vector.yaml:ro
|
|
||||||
user: root
|
|
||||||
ports:
|
|
||||||
- '8686:8686'
|
|
||||||
depends_on: [victorialogs-1,victorialogs-2]
|
|
||||||
|
|
||||||
victorialogs-1:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vl-ha-single-1:/vlogs
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
- -loggerFormat=json
|
|
||||||
victorialogs-2:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vl-ha-single-2:/vlogs
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
- -loggerFormat=json
|
|
||||||
vmauth:
|
|
||||||
container_name: vmauth
|
|
||||||
image: victoriametrics/vmauth:v1.103.0
|
|
||||||
depends_on:
|
|
||||||
- "victorialogs-1"
|
|
||||||
- "victorialogs-2"
|
|
||||||
volumes:
|
|
||||||
- ./auth.yml:/etc/auth.yml
|
|
||||||
command:
|
|
||||||
- '--auth.config=/etc/auth.yml'
|
|
||||||
ports:
|
|
||||||
- 8427:8427
|
|
||||||
restart: always
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-vector-docker-vl-ha-single-1:
|
|
||||||
victorialogs-vector-docker-vl-ha-single-2:
|
|
||||||
|
|
@ -1,58 +0,0 @@
|
||||||
api:
|
|
||||||
enabled: true
|
|
||||||
address: 0.0.0.0:8686
|
|
||||||
sources:
|
|
||||||
docker:
|
|
||||||
type: docker_logs
|
|
||||||
transforms:
|
|
||||||
msg_parser:
|
|
||||||
type: remap
|
|
||||||
inputs:
|
|
||||||
- docker
|
|
||||||
source: |
|
|
||||||
if exists(.message) {
|
|
||||||
.log, err = parse_json(.message)
|
|
||||||
if err == null {
|
|
||||||
del(.message)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
sinks:
|
|
||||||
console_out:
|
|
||||||
type: console
|
|
||||||
inputs:
|
|
||||||
- msg_parser
|
|
||||||
encoding:
|
|
||||||
codec: json
|
|
||||||
vlogs_http_1:
|
|
||||||
type: http
|
|
||||||
inputs:
|
|
||||||
- msg_parser
|
|
||||||
uri: http://victorialogs-1:9428/insert/jsonline?_stream_fields=source_type,host,container_name&_msg_field=log.msg&_time_field=timestamp
|
|
||||||
encoding:
|
|
||||||
codec: json
|
|
||||||
framing:
|
|
||||||
method: newline_delimited
|
|
||||||
compression: gzip
|
|
||||||
healthcheck:
|
|
||||||
enabled: false
|
|
||||||
request:
|
|
||||||
headers:
|
|
||||||
AccountID: '0'
|
|
||||||
ProjectID: '0'
|
|
||||||
vlogs_http_2:
|
|
||||||
type: http
|
|
||||||
inputs:
|
|
||||||
- msg_parser
|
|
||||||
uri: http://victorialogs-2:9428/insert/jsonline?_stream_fields=source_type,host,container_name&_msg_field=log.msg&_time_field=timestamp
|
|
||||||
encoding:
|
|
||||||
codec: json
|
|
||||||
framing:
|
|
||||||
method: newline_delimited
|
|
||||||
compression: gzip
|
|
||||||
healthcheck:
|
|
||||||
enabled: false
|
|
||||||
request:
|
|
||||||
headers:
|
|
||||||
AccountID: '0'
|
|
||||||
ProjectID: '0'
|
|
||||||
|
|
||||||
|
|
@ -1,8 +1,13 @@
|
||||||
# Docker compose Vector integration with VictoriaLogs for docker
|
# Docker compose Vector integration with VictoriaLogs using given below protocols:
|
||||||
|
|
||||||
|
* [elasticsearch](./elasticsearch)
|
||||||
|
* [loki](./loki)
|
||||||
|
* [jsonline single node](./jsonline)
|
||||||
|
* [jsonline HA setup](./jsonline-ha)
|
||||||
|
|
||||||
The folder contains the example of integration of [vector](https://vector.dev/docs/) with Victorialogs
|
The folder contains the example of integration of [vector](https://vector.dev/docs/) with Victorialogs
|
||||||
|
|
||||||
To spin-up environment run the following command:
|
To spin-up environment `cd` to any of listed above directories run the following command:
|
||||||
```
|
```
|
||||||
docker compose up -d
|
docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
@ -16,7 +21,7 @@ docker compose rm -f
|
||||||
The docker compose file contains the following components:
|
The docker compose file contains the following components:
|
||||||
|
|
||||||
* vector - vector is configured to collect logs from the `docker`, you can find configuration in the `vector.yaml`. It writes data in VictoriaLogs. It pushes metrics to VictoriaMetrics.
|
* vector - vector is configured to collect logs from the `docker`, you can find configuration in the `vector.yaml`. It writes data in VictoriaLogs. It pushes metrics to VictoriaMetrics.
|
||||||
* VictoriaLogs - the log database, it accepts the data from `vector` by elastic protocol
|
* VictoriaLogs - the log database, it accepts the data from `vector` by DataDog protocol
|
||||||
* VictoriaMetrics - collects metrics from `VictoriaLogs` and `VictoriaMetrics`
|
* VictoriaMetrics - collects metrics from `VictoriaLogs` and `VictoriaMetrics`
|
||||||
|
|
||||||
Querying the data
|
Querying the data
|
||||||
|
|
@ -24,40 +29,10 @@ Querying the data
|
||||||
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
* [vmui](https://docs.victoriametrics.com/victorialogs/querying/#vmui) - a web UI is accessible by `http://localhost:9428/select/vmui`
|
||||||
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
* for querying the data via command-line please check [these docs](https://docs.victoriametrics.com/victorialogs/querying/#command-line)
|
||||||
|
|
||||||
|
Vector configuration example can be found below:
|
||||||
the example of vector configuration(`vector.yaml`)
|
* [elasticsearch](./elasticsearch/vector.yaml)
|
||||||
|
* [loki](./loki/vector.yaml)
|
||||||
```
|
* [jsonline single node](./jsonline/vector.yaml)
|
||||||
sources:
|
* [jsonline HA setup](./jsonline-ha/vector.yaml)
|
||||||
docker:
|
|
||||||
type: docker_logs
|
|
||||||
transforms:
|
|
||||||
msg_parser:
|
|
||||||
type: remap
|
|
||||||
inputs:
|
|
||||||
- docker
|
|
||||||
source: |
|
|
||||||
.log = parse_json!(.message)
|
|
||||||
del(.message)
|
|
||||||
sinks:
|
|
||||||
vlogs_es:
|
|
||||||
type: elasticsearch
|
|
||||||
inputs:
|
|
||||||
- msg_parser
|
|
||||||
endpoints:
|
|
||||||
- http://victorialogs:9428/insert/elasticsearch/
|
|
||||||
mode: bulk
|
|
||||||
api_version: v8
|
|
||||||
compression: gzip
|
|
||||||
healthcheck.enabled: false
|
|
||||||
query:
|
|
||||||
_msg_field: log.msg
|
|
||||||
_time_field: timestamp
|
|
||||||
_stream_fields: source_type,host,container_name
|
|
||||||
request:
|
|
||||||
headers:
|
|
||||||
AccountID: "0"
|
|
||||||
ProjectID: "0"
|
|
||||||
```
|
|
||||||
|
|
||||||
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
Please, note that `_stream_fields` parameter must follow recommended [best practices](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) to achieve better performance.
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,8 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
services:
|
services:
|
||||||
vector:
|
vector:
|
||||||
image: docker.io/timberio/vector:0.38.0-distroless-static
|
image: docker.io/timberio/vector:0.40.0-distroless-static
|
||||||
restart: on-failure
|
restart: on-failure
|
||||||
volumes:
|
volumes:
|
||||||
- type: bind
|
- type: bind
|
||||||
|
|
@ -9,7 +11,7 @@ services:
|
||||||
- type: bind
|
- type: bind
|
||||||
source: /var/lib/docker
|
source: /var/lib/docker
|
||||||
target: /var/lib/docker
|
target: /var/lib/docker
|
||||||
- ./vector.yaml:/etc/vector/vector.yaml:ro
|
- ${PWD}/vector.yaml:/etc/vector/vector.yaml:ro
|
||||||
user: root
|
user: root
|
||||||
ports:
|
ports:
|
||||||
- '8686:8686'
|
- '8686:8686'
|
||||||
|
|
@ -18,39 +20,3 @@ services:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
victoriametrics:
|
victoriametrics:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
|
|
||||||
victorialogs:
|
|
||||||
image: docker.io/victoriametrics/victoria-logs:v0.29.0-victorialogs
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vl:/vlogs
|
|
||||||
ports:
|
|
||||||
- '9428:9428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vlogs
|
|
||||||
- -loggerFormat=json
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:9428/health"]
|
|
||||||
interval: 1s
|
|
||||||
timeout: 1s
|
|
||||||
retries: 10
|
|
||||||
|
|
||||||
victoriametrics:
|
|
||||||
image: victoriametrics/victoria-metrics:latest
|
|
||||||
ports:
|
|
||||||
- '8428:8428'
|
|
||||||
command:
|
|
||||||
- -storageDataPath=/vmsingle
|
|
||||||
- -promscrape.config=/promscrape.yml
|
|
||||||
- -loggerFormat=json
|
|
||||||
volumes:
|
|
||||||
- victorialogs-vector-docker-vm:/vmsingle
|
|
||||||
- ./scrape.yml:/promscrape.yml
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8428/health"]
|
|
||||||
interval: 1s
|
|
||||||
timeout: 1s
|
|
||||||
retries: 10
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
victorialogs-vector-docker-vl:
|
|
||||||
victorialogs-vector-docker-vm:
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: vector-elasticsearch
|
||||||
|
|
@ -0,0 +1,43 @@
|
||||||
|
api:
|
||||||
|
enabled: true
|
||||||
|
address: 0.0.0.0:8686
|
||||||
|
sources:
|
||||||
|
docker:
|
||||||
|
type: docker_logs
|
||||||
|
demo:
|
||||||
|
type: demo_logs
|
||||||
|
format: apache_common
|
||||||
|
interval: 10
|
||||||
|
vector_metrics:
|
||||||
|
type: internal_metrics
|
||||||
|
transforms:
|
||||||
|
msg_parser:
|
||||||
|
type: remap
|
||||||
|
inputs: [docker]
|
||||||
|
source: |
|
||||||
|
.log = parse_json!(.message)
|
||||||
|
del(.message)
|
||||||
|
sinks:
|
||||||
|
elasticsearch:
|
||||||
|
type: elasticsearch
|
||||||
|
inputs: [demo]
|
||||||
|
endpoints: [http://victorialogs:9428/insert/elasticsearch/]
|
||||||
|
mode: bulk
|
||||||
|
api_version: v8
|
||||||
|
compression: gzip
|
||||||
|
healthcheck:
|
||||||
|
enabled: false
|
||||||
|
query:
|
||||||
|
_msg_field: message
|
||||||
|
_time_field: timestamp
|
||||||
|
_stream_fields: source_type
|
||||||
|
request:
|
||||||
|
headers:
|
||||||
|
AccountID: "0"
|
||||||
|
ProjectID: "0"
|
||||||
|
victoriametrics:
|
||||||
|
type: prometheus_remote_write
|
||||||
|
endpoint: http://victoriametrics:8428/api/v1/write
|
||||||
|
inputs: [vector_metrics]
|
||||||
|
healthcheck:
|
||||||
|
enabled: false
|
||||||
|
|
@ -0,0 +1,5 @@
|
||||||
|
include:
|
||||||
|
- path:
|
||||||
|
- ../compose.yml
|
||||||
|
- ../../compose-ha.yml
|
||||||
|
name: vector-jsonline-ha
|
||||||
|
|
@ -18,7 +18,7 @@ transforms:
|
||||||
.log = parse_json!(.message)
|
.log = parse_json!(.message)
|
||||||
del(.message)
|
del(.message)
|
||||||
sinks:
|
sinks:
|
||||||
vlogs_http:
|
vlogs-1:
|
||||||
type: http
|
type: http
|
||||||
inputs:
|
inputs:
|
||||||
- msg_parser
|
- msg_parser
|
||||||
|
|
@ -34,40 +34,26 @@ sinks:
|
||||||
headers:
|
headers:
|
||||||
AccountID: '0'
|
AccountID: '0'
|
||||||
ProjectID: '0'
|
ProjectID: '0'
|
||||||
vlogs_loki:
|
vlogs-2:
|
||||||
type: loki
|
type: http
|
||||||
inputs:
|
inputs:
|
||||||
- demo
|
- msg_parser
|
||||||
endpoint: http://victorialogs:9428/insert/loki/
|
uri: http://victorialogs-2:9428/insert/jsonline?_stream_fields=source_type,host,container_name&_msg_field=log.msg&_time_field=timestamp
|
||||||
compression: gzip
|
|
||||||
path: /api/v1/push?_msg_field=message.message&_time_field=timestamp&_stream_fields=source
|
|
||||||
encoding:
|
encoding:
|
||||||
codec: json
|
codec: json
|
||||||
labels:
|
framing:
|
||||||
source: vector
|
method: newline_delimited
|
||||||
victoriametrics:
|
|
||||||
type: prometheus_remote_write
|
|
||||||
endpoint: http://victoriametrics:8428/api/v1/write
|
|
||||||
inputs:
|
|
||||||
- metrics
|
|
||||||
healthcheck:
|
|
||||||
enabled: false
|
|
||||||
vlogs_es:
|
|
||||||
type: elasticsearch
|
|
||||||
inputs:
|
|
||||||
- demo
|
|
||||||
endpoints:
|
|
||||||
- http://victorialogs:9428/insert/elasticsearch/
|
|
||||||
mode: bulk
|
|
||||||
api_version: v8
|
|
||||||
compression: gzip
|
compression: gzip
|
||||||
healthcheck:
|
healthcheck:
|
||||||
enabled: false
|
enabled: false
|
||||||
query:
|
|
||||||
_msg_field: message
|
|
||||||
_time_field: timestamp
|
|
||||||
_stream_fields: source_type
|
|
||||||
request:
|
request:
|
||||||
headers:
|
headers:
|
||||||
AccountID: '0'
|
AccountID: '0'
|
||||||
ProjectID: '0'
|
ProjectID: '0'
|
||||||
|
victoriametrics:
|
||||||
|
type: prometheus_remote_write
|
||||||
|
endpoint: http://victoriametrics:8428/api/v1/write
|
||||||
|
inputs:
|
||||||
|
- metrics
|
||||||
|
healthcheck:
|
||||||
|
enabled: false
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: vector-jsonline
|
||||||
43
deployment/docker/victorialogs/vector/jsonline/vector.yaml
Normal file
43
deployment/docker/victorialogs/vector/jsonline/vector.yaml
Normal file
|
|
@ -0,0 +1,43 @@
|
||||||
|
api:
|
||||||
|
enabled: true
|
||||||
|
address: 0.0.0.0:8686
|
||||||
|
sources:
|
||||||
|
docker:
|
||||||
|
type: docker_logs
|
||||||
|
demo:
|
||||||
|
type: demo_logs
|
||||||
|
format: json
|
||||||
|
metrics:
|
||||||
|
type: internal_metrics
|
||||||
|
transforms:
|
||||||
|
msg_parser:
|
||||||
|
type: remap
|
||||||
|
inputs:
|
||||||
|
- docker
|
||||||
|
source: |
|
||||||
|
.log = parse_json!(.message)
|
||||||
|
del(.message)
|
||||||
|
sinks:
|
||||||
|
http:
|
||||||
|
type: http
|
||||||
|
inputs:
|
||||||
|
- msg_parser
|
||||||
|
uri: http://victorialogs:9428/insert/jsonline?_stream_fields=source_type,host,container_name&_msg_field=log.msg&_time_field=timestamp
|
||||||
|
encoding:
|
||||||
|
codec: json
|
||||||
|
framing:
|
||||||
|
method: newline_delimited
|
||||||
|
compression: gzip
|
||||||
|
healthcheck:
|
||||||
|
enabled: false
|
||||||
|
request:
|
||||||
|
headers:
|
||||||
|
AccountID: '0'
|
||||||
|
ProjectID: '0'
|
||||||
|
victoriametrics:
|
||||||
|
type: prometheus_remote_write
|
||||||
|
endpoint: http://victoriametrics:8428/api/v1/write
|
||||||
|
inputs:
|
||||||
|
- metrics
|
||||||
|
healthcheck:
|
||||||
|
enabled: false
|
||||||
3
deployment/docker/victorialogs/vector/loki/compose.yml
Normal file
3
deployment/docker/victorialogs/vector/loki/compose.yml
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
include:
|
||||||
|
- ../compose.yml
|
||||||
|
name: vector-loki
|
||||||
38
deployment/docker/victorialogs/vector/loki/vector.yaml
Normal file
38
deployment/docker/victorialogs/vector/loki/vector.yaml
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
api:
|
||||||
|
enabled: true
|
||||||
|
address: 0.0.0.0:8686
|
||||||
|
sources:
|
||||||
|
docker:
|
||||||
|
type: docker_logs
|
||||||
|
demo:
|
||||||
|
type: demo_logs
|
||||||
|
format: json
|
||||||
|
metrics:
|
||||||
|
type: internal_metrics
|
||||||
|
transforms:
|
||||||
|
msg_parser:
|
||||||
|
type: remap
|
||||||
|
inputs:
|
||||||
|
- docker
|
||||||
|
source: |
|
||||||
|
.log = parse_json!(.message)
|
||||||
|
del(.message)
|
||||||
|
sinks:
|
||||||
|
loki:
|
||||||
|
type: loki
|
||||||
|
inputs:
|
||||||
|
- demo
|
||||||
|
endpoint: http://victorialogs:9428/insert/loki/
|
||||||
|
compression: gzip
|
||||||
|
path: /api/v1/push?_msg_field=message.message&_time_field=timestamp&_stream_fields=source
|
||||||
|
encoding:
|
||||||
|
codec: json
|
||||||
|
labels:
|
||||||
|
source: vector
|
||||||
|
victoriametrics:
|
||||||
|
type: prometheus_remote_write
|
||||||
|
endpoint: http://victoriametrics:8428/api/v1/write
|
||||||
|
inputs:
|
||||||
|
- metrics
|
||||||
|
healthcheck:
|
||||||
|
enabled: false
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
scrape_configs:
|
|
||||||
- job_name: "victoriametrics"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victoriametrics:8428
|
|
||||||
- job_name: "victorialogs"
|
|
||||||
scrape_interval: 30s
|
|
||||||
static_configs:
|
|
||||||
- targets:
|
|
||||||
- victorialogs:9428
|
|
||||||
|
|
@ -120,4 +120,4 @@ See also:
|
||||||
- [Data ingestion troubleshooting](https://docs.victoriametrics.com/victorialogs/data-ingestion/#troubleshooting).
|
- [Data ingestion troubleshooting](https://docs.victoriametrics.com/victorialogs/data-ingestion/#troubleshooting).
|
||||||
- [How to query VictoriaLogs](https://docs.victoriametrics.com/victorialogs/querying/).
|
- [How to query VictoriaLogs](https://docs.victoriametrics.com/victorialogs/querying/).
|
||||||
- [Elasticsearch output docs for Telegraf](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/elasticsearch).
|
- [Elasticsearch output docs for Telegraf](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/elasticsearch).
|
||||||
- [Docker-compose demo for Telegraf integration with VictoriaLogs](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/victorialogs/telegraf-docker).
|
- [Docker-compose demo for Telegraf integration with VictoriaLogs](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/victorialogs/telegraf).
|
||||||
|
|
|
||||||
|
|
@ -213,4 +213,4 @@ See also:
|
||||||
- [Data ingestion troubleshooting](https://docs.victoriametrics.com/victorialogs/data-ingestion/#troubleshooting).
|
- [Data ingestion troubleshooting](https://docs.victoriametrics.com/victorialogs/data-ingestion/#troubleshooting).
|
||||||
- [How to query VictoriaLogs](https://docs.victoriametrics.com/victorialogs/querying/).
|
- [How to query VictoriaLogs](https://docs.victoriametrics.com/victorialogs/querying/).
|
||||||
- [Elasticsearch output docs for Vector](https://vector.dev/docs/reference/configuration/sinks/elasticsearch/).
|
- [Elasticsearch output docs for Vector](https://vector.dev/docs/reference/configuration/sinks/elasticsearch/).
|
||||||
- [Docker-compose demo for Filebeat integration with VictoriaLogs](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/victorialogs/vector).
|
- [Docker-compose demo for Vector integration with VictoriaLogs](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/victorialogs/vector).
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue