mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-12-01 14:47:38 +00:00
d5fe4566e5
* Many collectors don't support forwarding url query params to the remote system. It makes impossible to define stream fields for it. Workaround with proxy between VictoriaLogs and log shipper is too complicated solution. * This commit adds the following changes: * Adds fallback to to headers params, if query param is empty for: _msg_field -> VL-Msg-Field _stream_fields -> VL-Stream-Fields _ignore_fields -> VL-Ignore-Fields _time_field -> VL-Time-Field * removes deprecations from victorialogs compose files, added more output format examples for logstash, telegraf, fluent-bit related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5310 |
||
---|---|---|
.. | ||
compose.yml | ||
README.md | ||
scrape.yml | ||
vector.yaml |
Docker compose Vector integration with VictoriaLogs for docker
The folder contains the example of integration of vector with Victorialogs
To spin-up environment run the following command:
docker compose up -d
To shut down the docker-compose environment run the following command:
docker compose down
docker compose rm -f
The docker compose file contains the following components:
- vector - vector is configured to collect logs from the
docker
, you can find configuration in thevector.yaml
. It writes data in VictoriaLogs. It pushes metrics to VictoriaMetrics. - VictoriaLogs - the log database, it accepts the data from
vector
by elastic protocol - VictoriaMetrics - collects metrics from
VictoriaLogs
andVictoriaMetrics
Querying the data
- vmui - a web UI is accessible by
http://localhost:9428/select/vmui
- for querying the data via command-line please check these docs
the example of vector configuration(vector.yaml
)
sources:
docker:
type: docker_logs
transforms:
msg_parser:
type: remap
inputs:
- docker
source: |
.log = parse_json!(.message)
del(.message)
sinks:
vlogs_es:
type: elasticsearch
inputs:
- msg_parser
endpoints:
- http://victorialogs:9428/insert/elasticsearch/
mode: bulk
api_version: v8
compression: gzip
healthcheck.enabled: false
query:
_msg_field: log.msg
_time_field: timestamp
_stream_fields: source_type,host,container_name
request:
headers:
AccountID: "0"
ProjectID: "0"
Please, note that _stream_fields
parameter must follow recommended best practices to achieve better performance.