alerts: update TooHighMemoryUsage threshold (#4256)

It appears that 90% usage for anonymous mem usage
is already concerning. So we lowering the threshold to 80%.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
This commit is contained in:
Roman Khavronenko 2023-05-07 22:18:56 +02:00 committed by Aliaksandr Valialkin
parent 8c9dc837b9
commit 3049754575
No known key found for this signature in database
GPG key ID: A72BEC6CD3D0DED1
2 changed files with 3 additions and 2 deletions

View file

@ -34,12 +34,12 @@ groups:
Consider to increase the limit as fast as possible." Consider to increase the limit as fast as possible."
- alert: TooHighMemoryUsage - alert: TooHighMemoryUsage
expr: (process_resident_memory_anon_bytes / vm_available_memory_bytes) > 0.9 expr: (process_resident_memory_anon_bytes / vm_available_memory_bytes) > 0.8
for: 5m for: 5m
labels: labels:
severity: critical severity: critical
annotations: annotations:
summary: "It is more than 90% of memory used by \"{{ $labels.job }}\"(\"{{ $labels.instance }}\") during the last 5m" summary: "It is more than 80% of memory used by \"{{ $labels.job }}\"(\"{{ $labels.instance }}\") during the last 5m"
description: "Too high memory usage may result into multiple issues such as OOMs or degraded performance. description: "Too high memory usage may result into multiple issues such as OOMs or degraded performance.
Consider to either increase available memory or decrease the load on the process." Consider to either increase available memory or decrease the load on the process."

View file

@ -57,6 +57,7 @@ The following tip changes can be tested by building VictoriaMetrics components f
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent.html): add `__meta_kubernetes_endpoints_name` label for all ports discovered from endpoint. Previously, ports not matched by `Service` did not have this label. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4154) for details. Thanks to @thunderbird86 for discovering and [fixing](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4253) the issue. * BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent.html): add `__meta_kubernetes_endpoints_name` label for all ports discovered from endpoint. Previously, ports not matched by `Service` did not have this label. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4154) for details. Thanks to @thunderbird86 for discovering and [fixing](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4253) the issue.
* BUGFIX: fix indexdb rotation getting in infinite loop when using `retentionTimezoneOffset` and local timezone is not UTC. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4207) for details. Thanks to @faceair for the fix. * BUGFIX: fix indexdb rotation getting in infinite loop when using `retentionTimezoneOffset` and local timezone is not UTC. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4207) for details. Thanks to @faceair for the fix.
* BUGFIX: max value for `memory.allowedPercent` changed from 200 to 100. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4171). * BUGFIX: max value for `memory.allowedPercent` changed from 200 to 100. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4171).
* BUGFIX: [alerts-health](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/alerts-health.yml): update threshold for TooHighMemoryUsage from 90% to 80%.
## [v1.90.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.90.0) ## [v1.90.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.90.0)