mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-12-01 14:47:38 +00:00
277fed9990
### Describe Your Changes Please provide a brief description of the changes you made. Be as specific as possible to help others understand the purpose and impact of your modifications. ### Checklist The following checks are **mandatory**: - [ ] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --------- Signed-off-by: Artem Navoiev <tenmozes@gmail.com> |
||
---|---|---|
.. | ||
auth.yml | ||
compose.yml | ||
logstash.yml | ||
pipeline.conf | ||
README.md |
Docker compose Logstash integration with VictoriaLogs for docker. High-Availability example
The folder contains the example of integration of logstash with VictoriaLogs Single-Node(s) and vmauth for achieving High Availability.
Check this documentation with a description of the architecture and components.
To spin-up environment run the following command:
docker compose up -d
To shut down the docker-compose environment run the following command:
docker compose down
docker compose rm -f
The docker compose file contains the following components:
- logstash - logstash is configured to read docker log files, you can find configuration in the
pipeline.conf
. It writes data in two instances of VictoriaLogs - VictoriaLogs - the two instances of log database, they accept the data from
fluentbit
by json line protocol - vmauth - load balancer for proxying requests to one of VictoriaLogs
Querying the data
- vmui - a web UI is accessible by
http://localhost:8427/select/vmui/
- for querying the data via command-line please check these docs
Here is an example of logstash configuration(pipeline.conf
):
input {
file {
path => "/var/lib/docker/containers/*/*.log"
start_position => "beginning"
type => "docker"
sincedb_path => "/dev/null"
codec => "json"
add_field => {
"path" => "%{[@metadata][path]}"
}
}
}
output {
http {
url => "http://victorialogs-1:9428/insert/jsonline?_stream_fields=host.name,stream&_msg_field=log&_time_field=time"
format => "json"
http_method => "post"
}
http {
url => "http://victorialogs-2:9428/insert/jsonline?_stream_fields=host.name,stream&_msg_field=log&_time_field=time"
format => "json"
http_method => "post"
}
}
Please, note that _stream_fields
parameter must follow recommended best practices to achieve better performance.
The example of vmauth configuration (auth.yml
)
unauthorized_user:
url_prefix:
- http://victorialogs-1:9428
- http://victorialogs-2:9428