mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2025-03-11 15:34:56 +00:00
RemoteWriteConnectionIsSaturated alert: add another saturation cause to the alert description (#8195)
### Describe Your Changes
Currently the alert descrption considers only one end of the connection
(vmagent). While saturation can also be caused by slowness of the
receiving components (vminsert, vmstorage). Update the alert description
with a brief suggestion to also check the dashboards of these
components.
### Checklist
The following checks are **mandatory**:
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
---------
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 30af662d84
)
This commit is contained in:
parent
e803b9b68b
commit
1ea2586856
1 changed files with 3 additions and 2 deletions
|
@ -80,8 +80,9 @@ groups:
|
|||
summary: "Remote write connection from \"{{ $labels.job }}\" (instance {{ $labels.instance }}) to {{ $labels.url }} is saturated"
|
||||
description: "The remote write connection between vmagent \"{{ $labels.job }}\" (instance {{ $labels.instance }}) and destination \"{{ $labels.url }}\"
|
||||
is saturated by more than 90% and vmagent won't be able to keep up.\n
|
||||
This usually means that `-remoteWrite.queues` command-line flag must be increased in order to increase
|
||||
the number of connections per each remote storage."
|
||||
There could be the following reasons for this:\n
|
||||
* vmagent can't send data fast enough through the existing network connections. Increase `-remoteWrite.queues` cmd-line flag value to establish more connections per destination.\n
|
||||
* remote destination can't accept data fast enough. Check if remote destination has enough resources for processing."
|
||||
|
||||
- alert: PersistentQueueForWritesIsSaturated
|
||||
expr: rate(vm_persistentqueue_write_duration_seconds_total[5m]) > 0.9
|
||||
|
|
Loading…
Reference in a new issue