Commit graph

211 commits

Author SHA1 Message Date
Roman Khavronenko
bec42ac975
security: bump go version to 1.20.2 (#3935)
upgrade Go builder from Go1.20.1 to Go1.20.2
See the list of issues addressed in Go1.20.2 here (https://github.com/golang/go/issues?q=milestone%3AGo1.20.2+label%3ACherryPickApproved).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-03-12 01:42:35 -08:00
Aliaksandr Valialkin
a9e23cdf80
all: update Go builder from Go1.20.0 to Go1.20.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.20.1+label%3ACherryPickApproved
2023-02-14 23:11:04 -08:00
Aliaksandr Valialkin
0e1d5a72e4
all: update alpine base docker image from 1.17.1 to 1.17.2
See https://alpinelinux.org/posts/Alpine-3.17.2-released.html
2023-02-14 22:57:01 -08:00
Aliaksandr Valialkin
b4022c7648
deployment/docker: update Go builder from v1.19.4 to v1.19.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.5+label%3ACherryPickApproved
2023-01-10 20:04:52 -08:00
Aliaksandr Valialkin
52b380e0bf
deployment/docker: update Alpine base image from v3.17.0 to v3.17.1
See https://alpinelinux.org/posts/Alpine-3.17.1-released.html
2023-01-10 20:03:52 -08:00
Aliaksandr Valialkin
d7d34515c5
deployment/docker: do not publish lts releases with the latest tag at DockerHub
This should prevent from overwriting the real `latest` images published
from the latest releases of VictoriaMetrics
2022-12-20 12:34:03 -08:00
Aliaksandr Valialkin
a88bcdee24
all: update Go builder from v1.19.3 to v1.19.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.4+label%3ACherryPickApproved
2022-12-11 00:23:32 -08:00
Aliaksandr Valialkin
df8ec22662
app/vmalert/templates: properly escape all the special chars in quotesEscape function
Previously the `quotesEscape` function was escaping only double quotes.
This wasn't enough, since the input string could contain other special chars,
which must be escaped when put inside JSON string. For example, carriage return and line feed chars (\n\r),
backslash char, etc. This led to the following issues, which were improperly fixed:

- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890 - this issue
  was "fixed" by introducing the `crlfEscape` function, which led to unnecessary
  complications in user templates, while not fixing various corner cases
  such as backslash chars in the input string.
  See 1de15ad490

- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3139 - this issue
  was "fixed" by urlencoding the whole string passed to -external.alert.source
  command-line flag. This led to invalid urls, which couldn't be parsed by Grafana.
  See 00c838353d
  and 4bd0244599

This commit properly encodes the input string passed to `quotesEscape`, so it can be safely embedded inside JSON strings.

This commit deprecates crlfEscape template function and adds the following new template functions:

- strvalue and stripDomain - these functions are supported by Prometheus, so they were added
  for compatibility purposes.
- jsonEscape and htmlEscape for converting the input string to valid quoted JSON string
  and for html-escaping the input string, so it could be safely embedded as a plaintext
  into html.

This commit also documents all supported template functions at https://docs.victoriametrics.com/vmalert.html#template-functions
The deprecated crlfEscape function isn't documented on purpose, since its usefulness is negative in general case.
2022-11-07 14:41:44 +02:00
Aliaksandr Valialkin
5e17c6de1c
deployment/docker: update Go builder from v1.19.2 to v1.19.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.3+label%3ACherryPickApproved
2022-11-07 14:26:31 +02:00
Aliaksandr Valialkin
d5d5288858
deployment/docker: update Go builder from v1.19.0 to v1.19.2
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.2+label%3ACherryPickApproved
and https://github.com/golang/go/issues?q=milestone%3AGo1.19.1+label%3ACherryPickApproved
2022-10-07 01:26:07 +03:00
Dmytro Kozlov
0672061d35
vmagent: expose metric vmagent_remotewrite_queues (#2871) (#3087)
* vmagent: expose metric `vmagent_remotewrite_queues` (#2871)

The new metric `vmagent_remotewrite_queues` exports a static value of
number of configured remote write queus. This metric is useful to
calculate total saturation per each configured URL with given number
of queues. See corresponding changes to vmagent alerts and dashboard.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update dashboards/vmagent.json

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-08 21:04:12 +03:00
Denys Holius
e775634cf8
deployment/docker/Makefile: bump version of Alpine linux to latest 3.16.2 to fix CVE-2022-37434 (#3035)
see https://alpinelinux.org/posts/Alpine-3.13.12-3.14.8-3.15.6-3.16.2-released.html
2022-08-30 11:28:20 +03:00
Aliaksandr Valialkin
aea2bcd495
deployment/docker: update Go builder from v1.18.4 to v1.18.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.5+label%3ACherryPickApproved
2022-08-03 10:53:37 +03:00
Aliaksandr Valialkin
8b4726ed74
deployment/docker: update alpine base image from 3.16.0 to 3.16.1
See https://alpinelinux.org/posts/Alpine-3.16.1-released.html
2022-08-01 15:27:51 +08:00
Aliaksandr Valialkin
83221bc480
deployment/docker: update Go builder from go1.18.3 to go1.18.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.4+label%3ACherryPickApproved
2022-07-13 20:31:54 +03:00
Aliaksandr Valialkin
da6c85a2f6
all: follow-up for d99ba3481b 2022-07-13 17:17:08 +03:00
Dmytro Kozlov
4e4def9df8
Rename release packages (#2810)
* makefile: add os to each release file

* makefile: update vmutils arm64

* makefile: update victoria-metrics release process

* makefile: update publish with os

* makefile: update publish with os

* makefile: change tar library

* update release logic

* copy all releases

* sort command by GOOS

* rollback commands

* rollback OSARCH

* fix commands

* cleanup

* fix windows build

* sort build by GOOS, update README.md
2022-07-13 17:11:01 +03:00
Denys Holius
083af28cf8
deployment/docker/docker-compose.yml: update Grafana from v8.5.1 to v9.0.2 (#2859)
See https://grafana.com/blog/2022/06/14/grafana-9.0-release-oss-and-cloud-features
2022-07-12 20:03:28 +03:00
Roman Khavronenko
a42063909f
alerts: correct expression for DiskRunsOutOfSpaceIn3Days (#2856)
The negative value for ETA can happen when deduplication is enabled
and `rate` over `vm_deduplicated_samples_total` becomes bigger
than actual ingestion rate.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-12 14:14:47 +02:00
Yurii Kravets
14397ba23e
Changed the level type in alerts.yml for TooManyLogs alert (#2759)
alerts: filter out non error log messages for `TooManyLogs`

Info and Warn error levels aren't always a result of malfunctioning
or faulty state. So we filter them out.
2022-06-20 16:45:52 +02:00
Aliaksandr Valialkin
079fdd3158
all: update Go builder for production builds from 1.18.2 to 1.18.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.3+label%3ACherryPickApproved
2022-06-20 14:30:48 +03:00
Aliaksandr Valialkin
f133756f02
deployment/docker: update base image from alpine:3.15.4 to alpine:3.16.0 2022-06-01 02:54:53 +03:00
Aliaksandr Valialkin
fe631850ee
deployment/docker: update Go builder from 1.18.1 to 1.18.2
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.2+label%3ACherryPickApproved
2022-05-23 10:58:38 +03:00
Aliaksandr Valialkin
0a420c4708
deployment/docker: pass -buildvs=false to go build for production builds
This should resolve the `error obtaining VCS status: exit status 128` error
when the environment contains incorrect version of git or has incorrect access rights
to the directory with VictoriaMetrics source code.

See the following links for additional info:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2508#issuecomment-1117126702 ,
- https://github.com/google/ko/issues/672
- https://github.com/golang/go/issues/49004
2022-05-11 13:57:59 +03:00
Dmytro Kozlov
028c28b84a
deployment/docker: added vmalert.proxyURL flag (#2549) 2022-05-06 17:34:09 +03:00
Aliaksandr Valialkin
5f41c48e4f
deployment/docker/docker-compose.yml: update Grafana version from v8.3.5 to v8.5.1 2022-04-29 12:02:32 +03:00
Aliaksandr Valialkin
4fc679c9cb
deployment/docker: update Go builder from go1.18.0 to go1.18.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.1+label%3ACherryPickApproved
2022-04-13 14:14:25 +03:00
Aliaksandr Valialkin
302464d27d
deployment/docker: update base Docker image from Alpine 3.15.3 to Alpine 3.15.4
See https://alpinelinux.org/posts/Alpine-3.12.12-3.13.10-3.14.6-3.15.4-released.html
2022-04-12 13:01:56 +03:00
Denys Holius
26cc40ab00
Update alpine linux base image to the latest v3.15.3 (#2384)
Updated alpine linux base image to the latest v3.15.3 which has fix for [CVE-2018-25032](https://security.alpinelinux.org/vuln/CVE-2018-25032).
See https://alpinelinux.org/posts/Alpine-3.12.11-3.13.9-3.14.5-3.15.3-released.html
2022-04-01 12:03:41 +03:00
Denys Holius
fb0fb41ea9
Update alpine linux base image to the latest v3.15.2
Update alpine linux base image to the latest v3.15.2 which has fix for CVE-2022-0778.
See https://alpinelinux.org/posts/Alpine-3.15.2-released.html
2022-03-26 11:30:36 +02:00
Aliaksandr Valialkin
0e8bd3b4cf
deployment/docker: upgrade Go builder for production builds from v1.17.7 to v1.18.0
See https://tip.golang.org/doc/go1.18
2022-03-16 14:08:52 +02:00
Roman Khavronenko
3458a3d593
Monitoring cluster (#2191)
* dashboards: add `CPU percentage` panel for cluster dashboards

The new panel `CPU percentage` was added instead if adding a limit
to the existing `CPU` panel because dasbhoard may display big number
of components each with own limits. The separate panel should provide
a clear display of CPU load.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards: sync vmagent and vmalert changes from single version

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docker: remove unsupported param from vmagent config

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* alerts: add `TooHighCPUUsage` alert for all VM components

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-02-15 11:57:58 +02:00
Nikolay
48a9e068be
adds release build for macos darwin amd64 and arm64 (#2185)
* adds release build for macos darwin amd64 and arm64

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1896
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1851

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-14 17:42:33 +02:00
Aliaksandr Valialkin
fc57035343
deployment/docker/docker-compose.yml: update Grafana from v8.3.4 to v8.3.5
See https://grafana.com/blog/2022/02/08/grafana-7.5.15-and-8.3.5-released-with-moderate-severity-security-fixes
2022-02-14 13:22:06 +02:00
Aliaksandr Valialkin
70cfb87c0e
deployment/docker: update Go builder from v1.17.6 to v1.17.7
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.7+label%3ACherryPickApproved
2022-02-12 01:14:07 +02:00
Nikolay
feefbbab48
adds CGO build for arm64 (#2102)
* adds CGO build for arm64
it must improve performance for arm64 based deployments of vmstorage and
vmsingle for 15-20%

it depends on gozstd package update for correct musl gozstd vendoring

* typo fixes

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-08 16:26:55 +02:00
Aliaksandr Valialkin
8dde20091f
deployment/docker: update Grafana from v8.3.2 to v8.3.4 2022-01-18 22:42:32 +02:00
Aliaksandr Valialkin
bc3718f719
deployment/docker: update Go builder from v1.17.5 to v1.17.6
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.6+label%3ACherryPickApproved
2022-01-07 13:34:15 +02:00
Roman Khavronenko
ada18cd963
Dashboards vmagent updates (#1973)
* dashboards/vmagent: shuffle panels for better visibility

More important error/dropped panels were moved higher on the main row.
Network usage panel moved to Resource usage row.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmagent: add Troubleshooting row to show top 5 instances/jobs by churn rate

New panels are supposed to show top 5 jobs or targets which generate the most
of the churn rate. They were placed into a new row "Troubleshooting".

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmagent: add panels for showing persistent queue saturation

New panels were added to Torubleshooting row to show the persistent queue
saturation. The corresponding alerts were added and linked to these
panels as well.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmagent: add alert "RejectedRemoteWriteDataBlocksAreDropped"

New alert suppose to send a notification when vmagent starts to drop
data blocks rejected by configured remote write destiantion.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-20 12:19:17 +02:00
Aliaksandr Valialkin
b73dc2b3df
deployment/docker/docker-compose.yml: update Grafana version from 8.2.2 to 8.3.2
See https://grafana.com/blog/2021/12/10/grafana-8.3.2-and-7.5.12-released-with-moderate-severity-security-fix/
2021-12-14 15:09:37 +02:00
Aliaksandr Valialkin
945135620a
deployment/docker: update Go builder from v1.17.4 to v1.17.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.5+label%3ACherryPickApproved
2021-12-12 18:18:52 +02:00
Aliaksandr Valialkin
3366124341
deployment/docker: update Go builder from v1.17.3 to v1.17.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.4+label%3ACherryPickApproved
2021-12-09 18:51:50 +02:00
Denis Golius
efdbe0adfe
Bumped Alpine linux version to 3.15.0 2021-11-29 12:18:04 +02:00
Denys Holius
64af0de0fc
Bumped Alpine linux version to the latest (#1811)
See this https://alpinelinux.org/posts/Alpine-3.14.3-released.html
2021-11-14 13:56:07 +02:00
Aliaksandr Valialkin
8cb7074030
deployment/docker: allow using / chars in ROOT_IMAGE when running make package-*
This fixes the following command:

ROOT_IMAGE=gcr.io/distroless/static make package-victoria-metrics
2021-11-14 12:22:22 +02:00
Aliaksandr Valialkin
9806e4793f
deployment/dm: update Go builder from Go1.17.2 to Go1.17.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.3+label%3ACherryPickApproved
2021-11-05 12:50:34 +02:00
Aliaksandr Valialkin
e18aa6e07e
deployment/docker: update Grafana from v8.2.0 to v8.2.2 2021-10-22 19:30:06 +03:00
Yurii Kravets
f7cb850f8d
Updated docker-compose.yml (#1730)
* Updated docker-compose.yml

Added one more Vmstorage to compose and pointed vmselect and vminsert to it.

* Update prometheus.yml

According to requested changes for new vmstorage

* Update docker-compose

Added port mapping

* Updated docker-compose

Set same ports for both storages

* Update prometheus.yml

Changed target for vmstorage1

* Updated docker compose

Changed the flags for vmstorage1

* Update docker compose

Changed flags for vmisert and vmselect to point to vmstorage1

* Update docker compose

Made the vmstorage names uniform

* Update prometheus.ymk

* Update docker-compose.yml

Removed additional flags as they are using default values.

* Update prometheus.yml

Put vmstorage targets under the same job

* Update prometheus.yml

Changed targets format

* Update docker-compose.yml

Set two different volumes for every vmstorage node

* Update deployment/docker/prometheus.yml

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-22 19:26:02 +03:00
Aliaksandr Valialkin
e4ebcebc8a
deployment/docker/alerts.yml: formatting fixes after 865a60f13e 2021-10-19 09:00:05 +03:00
Yurii Kravets
34f52de3a5
Update alerts.yml
Added Series Limit day\hour alerts
2021-10-19 09:00:05 +03:00