VictoriaMetrics/app/vlinsert/loki
Zakhar Bessarab 09df5b66fd
app/vlinsert: add support of loki push protocol (#4482)
* app/vlinsert: add support of loki push protocol

- implemented loki push protocol for both Protobuf and JSON formats
- added examples in documentation
- added example docker-compose

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert: move protobuf metric into its own file

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: update reference to docker image

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: make volume name unique

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: add license reference

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: fix volume name

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/VictoriaLogs/data-ingestion: add stream fields for loki JSON ingestion example

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: move entities to places where those are used

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: refactor to use common components

- use CommonParameters from insertutils
- stop ingestion after first error similar to elasticsearch and jsonline

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: address review feedback

- add missing logstorage.PutLogRows calls
- refactor tenant ID parsing to use common function
- reduce number of allocations for parsing by reusing  logfields slices
- add tests and benchmarks for requests processing funcs

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-20 10:10:55 +02:00
..
loki.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
loki_json.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
loki_json_test.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
loki_json_timing_test.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
loki_protobuf.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
loki_protobuf_test.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
loki_protobuf_timing_test.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
push_request.pb.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
push_request.proto app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
timestamp.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
types.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00