Commit graph

166 commits

Author SHA1 Message Date
Aliaksandr Valialkin
58fba96943
all: update Go builder from v1.19.3 to v1.19.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.4+label%3ACherryPickApproved
2022-12-11 00:23:55 -08:00
Aliaksandr Valialkin
ae64be4f2c
app/vmalert/templates: properly escape all the special chars in quotesEscape function
Previously the `quotesEscape` function was escaping only double quotes.
This wasn't enough, since the input string could contain other special chars,
which must be escaped when put inside JSON string. For example, carriage return and line feed chars (\n\r),
backslash char, etc. This led to the following issues, which were improperly fixed:

- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890 - this issue
  was "fixed" by introducing the `crlfEscape` function, which led to unnecessary
  complications in user templates, while not fixing various corner cases
  such as backslash chars in the input string.
  See 1de15ad490

- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3139 - this issue
  was "fixed" by urlencoding the whole string passed to -external.alert.source
  command-line flag. This led to invalid urls, which couldn't be parsed by Grafana.
  See 00c838353d
  and 4bd0244599

This commit properly encodes the input string passed to `quotesEscape`, so it can be safely embedded inside JSON strings.

This commit deprecates crlfEscape template function and adds the following new template functions:

- strvalue and stripDomain - these functions are supported by Prometheus, so they were added
  for compatibility purposes.
- jsonEscape and htmlEscape for converting the input string to valid quoted JSON string
  and for html-escaping the input string, so it could be safely embedded as a plaintext
  into html.

This commit also documents all supported template functions at https://docs.victoriametrics.com/vmalert.html#template-functions
The deprecated crlfEscape function isn't documented on purpose, since its usefulness is negative in general case.
2022-11-07 14:40:24 +02:00
Aliaksandr Valialkin
3ddab2d789
deployment/docker: update Go builder from v1.19.2 to v1.19.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.3+label%3ACherryPickApproved
2022-11-07 14:26:04 +02:00
Aliaksandr Valialkin
7d4101931c
deployment/docker: update Go builder from v1.19.0 to v1.19.2
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.2+label%3ACherryPickApproved
and https://github.com/golang/go/issues?q=milestone%3AGo1.19.1+label%3ACherryPickApproved
2022-10-07 01:25:19 +03:00
Dmytro Kozlov
e7119de7f7
vmagent: expose metric vmagent_remotewrite_queues (#2871) (#3087)
* vmagent: expose metric `vmagent_remotewrite_queues` (#2871)

The new metric `vmagent_remotewrite_queues` exports a static value of
number of configured remote write queus. This metric is useful to
calculate total saturation per each configured URL with given number
of queues. See corresponding changes to vmagent alerts and dashboard.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update dashboards/vmagent.json

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-08 20:53:51 +03:00
Denys Holius
0aa41430dd
deployment/docker/Makefile: bump version of Alpine linux to latest 3.16.2 to fix CVE-2022-37434 (#3035)
see https://alpinelinux.org/posts/Alpine-3.13.12-3.14.8-3.15.6-3.16.2-released.html
2022-08-30 11:27:10 +03:00
Aliaksandr Valialkin
8afcc01582
deployment/docker: update Go builder from v1.18.4 to v1.18.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.5+label%3ACherryPickApproved
2022-08-03 10:53:21 +03:00
Aliaksandr Valialkin
5df6790daf
deployment/docker: update alpine base image from 3.16.0 to 3.16.1
See https://alpinelinux.org/posts/Alpine-3.16.1-released.html
2022-08-01 14:44:36 +08:00
Aliaksandr Valialkin
2d9dbaf75d
deployment/docker: update Go builder from go1.18.3 to go1.18.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.4+label%3ACherryPickApproved
2022-07-13 18:46:34 +03:00
Aliaksandr Valialkin
ed93330e66
all: follow-up for d99ba3481b 2022-07-13 16:44:39 +03:00
Dmytro Kozlov
d99ba3481b
Rename release packages (#2810)
* makefile: add os to each release file

* makefile: update vmutils arm64

* makefile: update victoria-metrics release process

* makefile: update publish with os

* makefile: update publish with os

* makefile: change tar library

* update release logic

* copy all releases

* sort command by GOOS

* rollback commands

* rollback OSARCH

* fix commands

* cleanup

* fix windows build

* sort build by GOOS, update README.md
2022-07-13 15:42:48 +03:00
Denys Holius
37a0f5705e
deployment/docker/docker-compose.yml: update Grafana from v8.5.1 to v9.0.2 (#2858)
See https://grafana.com/blog/2022/06/14/grafana-9.0-release-oss-and-cloud-features
2022-07-12 20:03:02 +03:00
Aliaksandr Valialkin
8a6fb5ef2b
deployment/docker/alerts.yml: backport a42063909f 2022-07-12 19:53:06 +03:00
Yurii Kravets
aeeaf877ac
Changed the level type in alerts.yml for TooManyLogs alert (#2760)
alerts: filter out non error log messages for `TooManyLogs`

Info and Warn error levels aren't always a result of malfunctioning
or faulty state. So we filter them out.
2022-06-20 16:44:47 +02:00
Aliaksandr Valialkin
81d1497b4c
all: update Go builder for production builds from 1.18.2 to 1.18.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.3+label%3ACherryPickApproved
2022-06-20 14:26:41 +03:00
Aliaksandr Valialkin
afc26c57cc
all: replace bash with console blocks in all the *.md files
This is a follow-up for 954a7a6fc6
2022-06-19 23:00:39 +03:00
Aliaksandr Valialkin
b97ad42b6e
deployment/docker: update base image from alpine:3.15.4 to alpine:3.16.0 2022-06-01 02:55:21 +03:00
Aliaksandr Valialkin
1fe0657828
deployment/docker: update Go builder from 1.18.1 to 1.18.2
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.2+label%3ACherryPickApproved
2022-05-23 10:57:16 +03:00
Aliaksandr Valialkin
991688ea65
deployment/docker: pass -buildvs=false to go build for production builds
This should resolve the `error obtaining VCS status: exit status 128` error
when the environment contains incorrect version of git or has incorrect access rights
to the directory with VictoriaMetrics source code.

See the following links for additional info:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2508#issuecomment-1117126702 ,
- https://github.com/google/ko/issues/672
- https://github.com/golang/go/issues/49004
2022-05-11 14:16:14 +03:00
Dmytro Kozlov
1235e754a3
deployment/docker: added vmalert.proxyURL flag (#2549) 2022-05-06 17:30:41 +03:00
Aliaksandr Valialkin
e33232bce3
deployment/docker/docker-compose.yml: update Grafana version from v8.3.5 to v8.5.1 2022-04-29 12:01:53 +03:00
Aliaksandr Valialkin
28f21f5f17
deployment/docker: update Go builder from go1.18.0 to go1.18.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.1+label%3ACherryPickApproved
2022-04-13 14:06:49 +03:00
Aliaksandr Valialkin
ae3017d3a6
deployment/docker: update base Docker image from Alpine 3.15.3 to Alpine 3.15.4
See https://alpinelinux.org/posts/Alpine-3.12.12-3.13.10-3.14.6-3.15.4-released.html
2022-04-12 13:03:42 +03:00
Denys Holius
0123295d50
Update alpine linux base image to the latest v3.15.3 (#2384)
Updated alpine linux base image to the latest v3.15.3 which has fix for [CVE-2018-25032](https://security.alpinelinux.org/vuln/CVE-2018-25032).
See https://alpinelinux.org/posts/Alpine-3.12.11-3.13.9-3.14.5-3.15.3-released.html
2022-03-29 12:48:11 +02:00
Denys Holius
a462b97859 Update alpine linux base image to the latest v3.15.2
Update alpine linux base image to the latest v3.15.2 which has fix for CVE-2022-0778.
See https://alpinelinux.org/posts/Alpine-3.15.2-released.html
2022-03-25 17:05:55 +01:00
Arash Hatami
a8de1ab000
A good change for MD files (#2353)
* Lint YAML

* Remove extra comment

* Fix command problem

* Format MD files

* Format & fix problem of MD files for docs

* Another fix for MD files
2022-03-22 13:40:55 +02:00
Aliaksandr Valialkin
7bfb5efaef
deployment/docker: upgrade Go builder for production builds from v1.17.7 to v1.18.0
See https://tip.golang.org/doc/go1.18
2022-03-16 14:07:43 +02:00
Denys Holius
e93e168bdc
Added missed runbook for udpating k8s VM Cluster in DO (#2219)
* added missed runbook for udpating k8s VM Cluster in DO

* Update deployment/marketplace/digitialocean/one-click-droplet/RELEASE_GUIDE.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-18 16:29:04 +02:00
Roman Khavronenko
7cd371f08f
alerts: lower the threshold for TooHighSlowInsertsRate (#2210)
Lowering threshold from 50% to 5% will be more sufficient
for discovering un-healthy system state. It also goes in
sync with alert definition in cluster branch.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-02-18 13:42:24 +02:00
Roman Khavronenko
e29b2b8444
Monitoring single (#2190)
* dashboards: plot cpu limits for vmagent, vmalert and vm-single dashboards

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* alerts: add `TooHighCPUUsage` alert for all VM components

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards: bump components version requirements

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-02-15 11:54:28 +02:00
Nikolay
75e84144c7
adds release build for macos darwin amd64 and arm64 (#2185)
* adds release build for macos darwin amd64 and arm64

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1896
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1851

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-14 17:28:56 +02:00
Aliaksandr Valialkin
93c2db5546
deployment/docker/docker-compose.yml: update Grafana from v8.3.4 to v8.3.5
See https://grafana.com/blog/2022/02/08/grafana-7.5.15-and-8.3.5-released-with-moderate-severity-security-fixes
2022-02-14 13:22:25 +02:00
Aliaksandr Valialkin
e08b74fcd6
deployment/docker: update Go builder from v1.17.6 to v1.17.7
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.7+label%3ACherryPickApproved
2022-02-12 01:13:05 +02:00
Nikolay
a8acad7453
adds CGO build for arm64 (#2102)
* adds CGO build for arm64
it must improve performance for arm64 based deployments of vmstorage and
vmsingle for 15-20%

it depends on gozstd package update for correct musl gozstd vendoring

* typo fixes

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-08 16:25:59 +02:00
Aliaksandr Valialkin
bce7d7ac60
deployment/docker: update Grafana from v8.3.2 to v8.3.4 2022-01-18 22:42:15 +02:00
Aliaksandr Valialkin
e47385d34a
deployment/docker: update Go builder from v1.17.5 to v1.17.6
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.6+label%3ACherryPickApproved
2022-01-07 13:34:32 +02:00
Denis Golius
dd1b789c15 removed not needed directory 2022-01-06 12:17:53 +03:00
Denys Holius
d44cc14c6b
added packer build for DigitalOcean Droplets (#1917)
* added packer build for DigitalOcean Droplets

* fixed typo

* added packer RELEASE_GUIDE.md, Makefile

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* added corrections amd improvements

* added packer link & templating for sed version

* fixed typo

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2021-12-21 12:09:14 +02:00
Roman Khavronenko
bc79bdf68a
Dashboards vmagent updates (#1973)
* dashboards/vmagent: shuffle panels for better visibility

More important error/dropped panels were moved higher on the main row.
Network usage panel moved to Resource usage row.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmagent: add Troubleshooting row to show top 5 instances/jobs by churn rate

New panels are supposed to show top 5 jobs or targets which generate the most
of the churn rate. They were placed into a new row "Troubleshooting".

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmagent: add panels for showing persistent queue saturation

New panels were added to Torubleshooting row to show the persistent queue
saturation. The corresponding alerts were added and linked to these
panels as well.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmagent: add alert "RejectedRemoteWriteDataBlocksAreDropped"

New alert suppose to send a notification when vmagent starts to drop
data blocks rejected by configured remote write destiantion.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-20 12:16:53 +02:00
Aliaksandr Valialkin
496b6e4d3d
deployment/docker/docker-compose.yml: update Grafana version from 8.2.2 to 8.3.2
See https://grafana.com/blog/2021/12/10/grafana-8.3.2-and-7.5.12-released-with-moderate-severity-security-fix/
2021-12-14 15:09:49 +02:00
Aliaksandr Valialkin
a8ad870bd0
deployment/docker: update Go builder from v1.17.4 to v1.17.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.5+label%3ACherryPickApproved
2021-12-12 18:17:49 +02:00
Aliaksandr Valialkin
a2e0275f14
deployment/docker: update Go builder from v1.17.3 to v1.17.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.4+label%3ACherryPickApproved
2021-12-09 18:51:58 +02:00
Thomas Danielsson
77e19b3f87
Fix vmsingle dashboard link (#1894) 2021-12-02 14:43:30 +02:00
Denis Golius
37faf1f426 Bumped Alpine linux version to 3.15.0 2021-11-28 20:53:48 +02:00
Aliaksandr Valialkin
129b0d2b22
deployment/docker: allow using / chars in ROOT_IMAGE when running make package-*
This fixes the following command:

ROOT_IMAGE=gcr.io/distroless/static make package-victoria-metrics
2021-11-14 13:55:05 +02:00
Denys Holius
49ee952e9a
Bumped Alpine linux version to the latest (#1811)
See this https://alpinelinux.org/posts/Alpine-3.14.3-released.html
2021-11-14 12:59:27 +03:00
Aliaksandr Valialkin
3db1f2d550
deployment/dm: update Go builder from Go1.17.2 to Go1.17.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.17.3+label%3ACherryPickApproved
2021-11-05 11:51:38 +02:00
Aliaksandr Valialkin
b76db7c772
deployment/docker: update Grafana from v8.2.0 to v8.2.2 2021-10-22 19:33:22 +03:00
Aliaksandr Valialkin
ec40affb59
deployment/docker/alerts.yml: formatting fixes after 865a60f13e 2021-10-19 08:53:03 +03:00
Yurii Kravets
865a60f13e
Update alerts.yml
Added Series Limit day\hour alerts
2021-10-18 18:14:49 +03:00