github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-11 14:53:49 +00:00
Code Issues Projects Releases Packages Wiki Activity
VictoriaMetrics/deployment/docker/victorialogs/logstash
History
Andrii Chubatiuk 73afeaf510
deployment: restructure victorialogs examples (#6971) 
### Describe Your Changes

- Use common compose.yaml file for all victorialogs setups to set
version in a single place and override it on demand for each agent and
protocol
- Replaced multiple victorialogs instances in HA setup with single setup
with `deploy.replica` parameter set
- Added fluentd setup

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit e75ae1b274)
2024-09-25 18:39:43 +02:00
..
elasticsearch deployment: restructure victorialogs examples (#6971) 2024-09-25 18:39:43 +02:00
jsonline deployment: restructure victorialogs examples (#6971) 2024-09-25 18:39:43 +02:00
jsonline-ha deployment: restructure victorialogs examples (#6971) 2024-09-25 18:39:43 +02:00
loki deployment: restructure victorialogs examples (#6971) 2024-09-25 18:39:43 +02:00
compose.yml deployment: restructure victorialogs examples (#6971) 2024-09-25 18:39:43 +02:00
Dockerfile deployment: restructure victorialogs examples (#6971) 2024-09-25 18:39:43 +02:00
logstash.yml Add docker compose examples: filebeat(docker, syslog), fluentbit(docker), logstash, vector(docker) 2023-07-06 21:25:31 -07:00
README.md deployment: restructure victorialogs examples (#6971) 2024-09-25 18:39:43 +02:00

README.md

Docker compose Logstash integration with VictoriaLogs for given below protocols:

It is required to use OpenSearch plugin for output configuration. Plugin can be installed by using the following command:

bin/logstash-plugin install logstash-output-opensearch

OpenSearch plugin is required because elasticsearch output plugin performs various checks for Elasticsearch version and license which are not applicable for VictoriaLogs.

To spin-up environment cd to any of listed above directories run the following command:

docker compose up -d

To shut down the docker-compose environment run the following command:

docker compose down
docker compose rm -f

The docker compose file contains the following components:

  • logstash - logstash is configured to accept syslog on 5140 port, you can find configuration in the pipeline.conf. It writes data in VictoriaLogs
  • VictoriaLogs - the log database, it accepts the data from logstash by elastic protocol

Querying the data

  • vmui - a web UI is accessible by http://localhost:9428/select/vmui
  • for querying the data via command-line please check these docs

Logstash configuration example can be found below:

Please, note that _stream_fields parameter must follow recommended best practices to achieve better performance.