VictoriaMetrics/app/vminsert
Nikolay 090cb2c9de
app/vmagent: allow to disabled on-disk persistence (#5088)
* app/vmagent: allow to disabled on-disk queue
Previously, it wasn't possible to build data processing pipeline with a
chain of vmagents. In case when remoteWrite for the last vmagent in the
chain wasn't accessible, it persisted data only when it has enough disk
capacity. If disk queue is full, it started to silently drop ingested
metrics.

New flags allows to disable on-disk persistent and immediatly return an
error if remoteWrite is not accessible anymore. It blocks any writes and
notify client, that data ingestion isn't possible.

Main use case for this feature - use external queue such as kafka for
data persistence.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2110

* adds test, updates readme

* apply review suggestions

* update docs for vmagent

* makes linter happy

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-11-24 13:42:11 +01:00
..
common vmalert-tool: implement unittest (#4789) 2023-10-13 13:54:33 +02:00
csvimport lib/protoparser/csvimport: extract stream parsing code into a separate stream package 2023-02-13 10:25:46 -08:00
datadog lib/protoparser/datadog: extract stream parsing code into a separate stream package 2023-02-13 09:51:47 -08:00
graphite lib/protoparser/graphite: extract stream parsing code into a separate stream package 2023-02-13 10:32:36 -08:00
influx fixed typos in documentation and commandline flags descriptions (#4275) 2023-05-10 09:50:41 +02:00
native lib/protoparser/native: extract stream parsing code into a separate stream package 2023-02-13 10:43:05 -08:00
newrelic app/{vmagent,vminsert}: follow-up for NewRelic data ingestion protocol support 2023-10-16 00:25:25 +02:00
opentelemetry opentelemetry: return human readable error for json encoding. (#4822) 2023-08-12 05:05:16 -07:00
opentsdb lib/protoparser/opentsdb: extract stream parsing code into a separate stream package 2023-02-13 10:03:16 -08:00
opentsdbhttp lib/protoparser/opentsdbhttp: extract stream parsing code into a separate stream package 2023-02-13 10:16:03 -08:00
prometheusimport lib/promscrape: make concurrency control optional (#5073) 2023-10-02 21:32:11 +02:00
prompush app,lib: fix typos in comments (#3804) 2023-02-13 13:27:13 +01:00
promremotewrite app/vmagent: add support for VictoriaMetrics remote write protocol, which allows saving up to 10x on network bandwidth costs under high load 2023-02-20 19:11:30 -08:00
relabel lib/promrelabel: properly replace : char with _ in metric names when -usePromCompatibleNaming command-line flag is set 2023-08-14 16:14:42 +02:00
static/css all: follow-up after 8edb390e21 2022-06-07 00:57:09 +03:00
vmimport lib/protoparser/vmimport: extract stream parsing code into a separate stream package 2023-02-13 10:20:19 -08:00
main.go app/vmagent: allow to disabled on-disk persistence (#5088) 2023-11-24 13:42:11 +01:00
README.md all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00

vminsert routes the ingested data to vmstorage.