github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2025-01-10 15:14:09 +00:00
Code Issues Projects Releases Packages Wiki Activity

docs/vmagent.md: remove the claim that VictoriaMetrics remote write protocol reduces the network bandwidth usage by up to 10x comparing to Prometheus remote write protocol

Browse source 
The 10x savings are reproduced only on artificial data.
The savings on production data are usually in the range 2x-4x.
This commit is contained in:
Aliaksandr Valialkin 2023-02-20 19:09:20 -08:00
parent a678cbe62e
commit f4f1f2f976
No known key found for this signature in database
GPG key ID: A72BEC6CD3D0DED1
3 changed files with 5 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
app/vmagent/README.md
View file

@ -26,7 +26,7 @@ additionally to [discovering Prometheus-compatible targets and scraping metrics
* Can accept data via all the ingestion protocols supported by VictoriaMetrics - see [these docs](#how-to-push-data-to-vmagent).
* Can aggregate incoming samples by time and by labels before sending them to remote storage - see [these docs](https://docs.victoriametrics.com/stream-aggregation.html).
* Can replicate collected metrics simultaneously to multiple Prometheus-compatible remote storage systems - see [these docs](#replication-and-high-availability).
* Can save egress network bandwidth usage costs by up to 10x when [VictoriaMetrics remote write protocol](#victoriametrics-remote-write-protocol)
* Can save egress network bandwidth usage costs when [VictoriaMetrics remote write protocol](#victoriametrics-remote-write-protocol)
is used for sending the data to VictoriaMetrics.
* Works smoothly in environments with unstable connections to remote storage. If the remote storage is unavailable, the collected metrics
are buffered at `-remoteWrite.tmpDataPath`. The buffered metrics are sent to remote storage as soon as the connection
@ -188,7 +188,7 @@ The Prometheus remote_write protocol may require big amounts of network bandwidt
This may result in high network egress costs when the configured remote storage is located in remote datacenter or availability zone.
This also may result in the increased disk IO at `vmagent` when it writes to disk the pending data, which must be sent to remote storage.
In this case the `vmagent` can be instructed to use VictoriaMetrics remote write protocol.
This allows reducing egress network bandwidth costs by up to 10x while reducing disk read/write IO at `vmagent` side under high load.
This allows reducing egress network bandwidth costs while reducing disk read/write IO at `vmagent` side under high load.
The `-remoteWrite.useVMProto=true` command-line flag instructs `vmagent` to send the data to the corresponding `-remoteWrite.url`
via VictoriaMetrics remote write protocol.

2
docs/CHANGELOG.md
View file

@ -18,7 +18,7 @@ The following tip changes can be tested by building VictoriaMetrics components f
* SECURITY: upgrade base docker image (alpine) from 3.17.1 to 3.17.2. See [alpine 3.17.2 release notes](https://alpinelinux.org/posts/Alpine-3.17.2-released.html).
* SECURITY: upgrade Go builder from Go1.20.0 to Go1.20.1. See [the list of issues addressed in Go1.20.1](https://github.com/golang/go/issues?q=milestone%3AGo1.20.1+label%3ACherryPickApproved).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add support for [VictoriaMetrics remote write protocol](https://docs.victoriametrics.com/vmagent.html). This protocol allows saving egress network bandwidth costs by up to 10x when sending data from `vmagent` to VictoriaMetrics located in another datacenter or availability zone. This also allows reducing disk IO under high load when `vmagent` starts queuing the collected data to disk when the remote storage is temporarily unavailable or cannot keep up with the data ingestion rate. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225).
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add support for [VictoriaMetrics remote write protocol](https://docs.victoriametrics.com/vmagent.html). This protocol allows saving egress network bandwidth costs when sending data from `vmagent` to VictoriaMetrics located in another datacenter or availability zone. This also allows reducing disk IO under high load when `vmagent` starts queuing the collected data to disk when the remote storage is temporarily unavailable or cannot keep up with the data ingestion rate. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225).
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth.html): add the ability to limit the number of concurrent requests on a per-user basis via `-maxConcurrentPerUserRequests` command-line flag and via `max_concurrent_requests` config option. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3346) and [these docs](https://docs.victoriametrics.com/vmauth.html#concurrency-limiting).
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth.html): automatically retry failing `GET` requests on all [the configured backends](https://docs.victoriametrics.com/vmauth.html#load-balancing). Previously the backend error has been immediately returned to the client without retrying the request on the remaining backends.
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth.html): choose the backend with the minimum number of concurrently executed requests [among the configured backends](https://docs.victoriametrics.com/vmauth.html#load-balancing) in a round-robin manner for serving the incoming requests. This allows spreading the load among backends more evenly, while improving the response time.

4
docs/vmagent.md
View file

@ -30,7 +30,7 @@ additionally to [discovering Prometheus-compatible targets and scraping metrics
* Can accept data via all the ingestion protocols supported by VictoriaMetrics - see [these docs](#how-to-push-data-to-vmagent).
* Can aggregate incoming samples by time and by labels before sending them to remote storage - see [these docs](https://docs.victoriametrics.com/stream-aggregation.html).
* Can replicate collected metrics simultaneously to multiple Prometheus-compatible remote storage systems - see [these docs](#replication-and-high-availability).
* Can save egress network bandwidth usage costs by up to 10x when [VictoriaMetrics remote write protocol](#victoriametrics-remote-write-protocol)
* Can save egress network bandwidth usage costs when [VictoriaMetrics remote write protocol](#victoriametrics-remote-write-protocol)
is used for sending the data to VictoriaMetrics.
* Works smoothly in environments with unstable connections to remote storage. If the remote storage is unavailable, the collected metrics
are buffered at `-remoteWrite.tmpDataPath`. The buffered metrics are sent to remote storage as soon as the connection
@ -192,7 +192,7 @@ The Prometheus remote_write protocol may require big amounts of network bandwidt
This may result in high network egress costs when the configured remote storage is located in remote datacenter or availability zone.
This also may result in the increased disk IO at `vmagent` when it writes to disk the pending data, which must be sent to remote storage.
In this case the `vmagent` can be instructed to use VictoriaMetrics remote write protocol.
This allows reducing egress network bandwidth costs by up to 10x while reducing disk read/write IO at `vmagent` side under high load.
This allows reducing egress network bandwidth costs while reducing disk read/write IO at `vmagent` side under high load.
The `-remoteWrite.useVMProto=true` command-line flag instructs `vmagent` to send the data to the corresponding `-remoteWrite.url`
via VictoriaMetrics remote write protocol.