vendor: update github.com/VictoriaMetrics/metrics from v1.14.0 to v1.15.0

The v1.15.0 exports the following additional metrics:

    process_io_read_bytes_total - the number of bytes read via io syscalls such as read and pread
    process_io_written_bytes_total - the number of bytes written via io syscalls such as write and pwrite
    process_io_read_syscalls_total - the number of read syscalls such as read and pread
    process_io_write_syscalls_total - the number of write syscalls such as write and pwrite
    process_io_storage_read_bytes_total - the number of bytes read from storage layer
    process_io_storage_written_bytes_total - the number of bytes written to storage layer

These metrics can be used for monitoring process io
This commit is contained in:
Aliaksandr Valialkin 2021-02-21 22:50:03 +02:00
parent 636c55b526
commit 901e12024d
6 changed files with 64 additions and 8 deletions

View file

@ -3,6 +3,14 @@
# tip # tip
* FEATURE: vmagent: add `scrape_align_interval` config option, which can be used for aligning scrapes to the beginning of the configured interval. See [these docs](https://victoriametrics.github.io/vmagent.html#troubleshooting) for details. * FEATURE: vmagent: add `scrape_align_interval` config option, which can be used for aligning scrapes to the beginning of the configured interval. See [these docs](https://victoriametrics.github.io/vmagent.html#troubleshooting) for details.
* FEATURE: expose io-related metrics at `/metrics` page for every VictoriaMetrics component:
* `process_io_read_bytes_total` - the number of bytes read via io syscalls such as read and pread
* `process_io_written_bytes_total` - the number of bytes written via io syscalls such as write and pwrite
* `process_io_read_syscalls_total` - the number of read syscalls such as read and pread
* `process_io_write_syscalls_total` - the number of write syscalls such as write and pwrite
* `process_io_storage_read_bytes_total` - the number of bytes read from storage layer
* `process_io_storage_written_bytes_total` - the number of bytes written to storage layer
* BUGFIX: vmagent: properly perform graceful shutdown on `SIGINT` and `SIGTERM` signals. The graceful shutdown has been broken in `v1.54.0`. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065 * BUGFIX: vmagent: properly perform graceful shutdown on `SIGINT` and `SIGTERM` signals. The graceful shutdown has been broken in `v1.54.0`. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
* BUGFIX: reduce the probability of `duplicate time series` errors when querying Kubernetes metrics. * BUGFIX: reduce the probability of `duplicate time series` errors when querying Kubernetes metrics.

2
go.mod
View file

@ -8,7 +8,7 @@ require (
// Do not use the original github.com/valyala/fasthttp because of issues // Do not use the original github.com/valyala/fasthttp because of issues
// like https://github.com/valyala/fasthttp/commit/996610f021ff45fdc98c2ce7884d5fa4e7f9199b // like https://github.com/valyala/fasthttp/commit/996610f021ff45fdc98c2ce7884d5fa4e7f9199b
github.com/VictoriaMetrics/fasthttp v1.0.12 github.com/VictoriaMetrics/fasthttp v1.0.12
github.com/VictoriaMetrics/metrics v1.14.0 github.com/VictoriaMetrics/metrics v1.15.0
github.com/VictoriaMetrics/metricsql v0.10.1 github.com/VictoriaMetrics/metricsql v0.10.1
github.com/aws/aws-sdk-go v1.37.12 github.com/aws/aws-sdk-go v1.37.12
github.com/cespare/xxhash/v2 v2.1.1 github.com/cespare/xxhash/v2 v2.1.1

4
go.sum
View file

@ -85,8 +85,8 @@ github.com/VictoriaMetrics/fastcache v1.5.7/go.mod h1:ptDBkNMQI4RtmVo8VS/XwRY6Ro
github.com/VictoriaMetrics/fasthttp v1.0.12 h1:Ag0E119yrH4BTxVyjKD9TeiSImtG9bUcg/stItLJhSE= github.com/VictoriaMetrics/fasthttp v1.0.12 h1:Ag0E119yrH4BTxVyjKD9TeiSImtG9bUcg/stItLJhSE=
github.com/VictoriaMetrics/fasthttp v1.0.12/go.mod h1:3SeUL4zwB/p/a9aEeRc6gdlbrtNHXBJR6N376EgiSHU= github.com/VictoriaMetrics/fasthttp v1.0.12/go.mod h1:3SeUL4zwB/p/a9aEeRc6gdlbrtNHXBJR6N376EgiSHU=
github.com/VictoriaMetrics/metrics v1.12.2/go.mod h1:Z1tSfPfngDn12bTfZSCqArT3OPY3u88J12hSoOhuiRE= github.com/VictoriaMetrics/metrics v1.12.2/go.mod h1:Z1tSfPfngDn12bTfZSCqArT3OPY3u88J12hSoOhuiRE=
github.com/VictoriaMetrics/metrics v1.14.0 h1:yvyEVo7cPN2Hv+Hrm1zPTA1f/squmEZTq6xtPH/8F64= github.com/VictoriaMetrics/metrics v1.15.0 h1:HGmGaILioC4vNk6UhkcwLIaDlg5y4MVganq1verl5js=
github.com/VictoriaMetrics/metrics v1.14.0/go.mod h1:Z1tSfPfngDn12bTfZSCqArT3OPY3u88J12hSoOhuiRE= github.com/VictoriaMetrics/metrics v1.15.0/go.mod h1:Z1tSfPfngDn12bTfZSCqArT3OPY3u88J12hSoOhuiRE=
github.com/VictoriaMetrics/metricsql v0.10.1 h1:wLl/YbMmBGFPyLKMfqNLC333iygibosSM5iSvlH2B4A= github.com/VictoriaMetrics/metricsql v0.10.1 h1:wLl/YbMmBGFPyLKMfqNLC333iygibosSM5iSvlH2B4A=
github.com/VictoriaMetrics/metricsql v0.10.1/go.mod h1:ylO7YITho/Iw6P71oEaGyHbO94bGoGtzWfLGqFhMIg8= github.com/VictoriaMetrics/metricsql v0.10.1/go.mod h1:ylO7YITho/Iw6P71oEaGyHbO94bGoGtzWfLGqFhMIg8=
github.com/VividCortex/ewma v1.1.1 h1:MnEK4VOv6n0RSY4vtRe3h11qjxL3+t0B8yOL8iMXdcM= github.com/VividCortex/ewma v1.1.1 h1:MnEK4VOv6n0RSY4vtRe3h11qjxL3+t0B8yOL8iMXdcM=

View file

@ -93,7 +93,7 @@ exposed from your application.
#### How to implement [CounterVec](https://godoc.org/github.com/prometheus/client_golang/prometheus#CounterVec) in `metrics`? #### How to implement [CounterVec](https://godoc.org/github.com/prometheus/client_golang/prometheus#CounterVec) in `metrics`?
Just use [GetOrCreateCounter](http://godoc.org/github.com/VictoriaMetrics/metrics#GetOrCreateCounter) Just use [GetOrCreateCounter](http://godoc.org/github.com/VictoriaMetrics/metrics#GetOrCreateCounter)
instead of `CounterVec.With`. See [this example](https://godoc.org/github.com/VictoriaMetrics/metrics#example-Counter--Vec) for details. instead of `CounterVec.With`. See [this example](https://pkg.go.dev/github.com/VictoriaMetrics/metrics#example-Counter-Vec) for details.
#### Why [Histogram](http://godoc.org/github.com/VictoriaMetrics/metrics#Histogram) buckets contain `vmrange` labels instead of `le` labels like in Prometheus histograms? #### Why [Histogram](http://godoc.org/github.com/VictoriaMetrics/metrics#Histogram) buckets contain `vmrange` labels instead of `le` labels like in Prometheus histograms?

View file

@ -12,8 +12,6 @@ import (
"time" "time"
) )
const statFilepath = "/proc/self/stat"
// See https://github.com/prometheus/procfs/blob/a4ac0826abceb44c40fc71daed2b301db498b93e/proc_stat.go#L40 . // See https://github.com/prometheus/procfs/blob/a4ac0826abceb44c40fc71daed2b301db498b93e/proc_stat.go#L40 .
const userHZ = 100 const userHZ = 100
@ -44,6 +42,7 @@ type procStat struct {
} }
func writeProcessMetrics(w io.Writer) { func writeProcessMetrics(w io.Writer) {
statFilepath := "/proc/self/stat"
data, err := ioutil.ReadFile(statFilepath) data, err := ioutil.ReadFile(statFilepath)
if err != nil { if err != nil {
log.Printf("ERROR: cannot open %s: %s", statFilepath, err) log.Printf("ERROR: cannot open %s: %s", statFilepath, err)
@ -68,7 +67,8 @@ func writeProcessMetrics(w io.Writer) {
} }
// It is expensive obtaining `process_open_fds` when big number of file descriptors is opened, // It is expensive obtaining `process_open_fds` when big number of file descriptors is opened,
// don't do it here. // so don't do it here.
// See writeFDMetrics instead.
utime := float64(p.Utime) / userHZ utime := float64(p.Utime) / userHZ
stime := float64(p.Stime) / userHZ stime := float64(p.Stime) / userHZ
@ -81,6 +81,54 @@ func writeProcessMetrics(w io.Writer) {
fmt.Fprintf(w, "process_resident_memory_bytes %d\n", p.Rss*4096) fmt.Fprintf(w, "process_resident_memory_bytes %d\n", p.Rss*4096)
fmt.Fprintf(w, "process_start_time_seconds %d\n", startTimeSeconds) fmt.Fprintf(w, "process_start_time_seconds %d\n", startTimeSeconds)
fmt.Fprintf(w, "process_virtual_memory_bytes %d\n", p.Vsize) fmt.Fprintf(w, "process_virtual_memory_bytes %d\n", p.Vsize)
writeIOMetrics(w)
}
func writeIOMetrics(w io.Writer) {
ioFilepath := "/proc/self/io"
data, err := ioutil.ReadFile(ioFilepath)
if err != nil {
log.Printf("ERROR: cannot open %q: %s", ioFilepath, err)
}
getInt := func(s string) int64 {
n := strings.IndexByte(s, ' ')
if n < 0 {
log.Printf("ERROR: cannot find whitespace in %q at %q", s, ioFilepath)
return 0
}
v, err := strconv.ParseInt(s[n+1:], 10, 64)
if err != nil {
log.Printf("ERROR: cannot parse %q at %q: %s", s, ioFilepath, err)
return 0
}
return v
}
var rchar, wchar, syscr, syscw, readBytes, writeBytes int64
lines := strings.Split(string(data), "\n")
for _, s := range lines {
s = strings.TrimSpace(s)
switch {
case strings.HasPrefix(s, "rchar: "):
rchar = getInt(s)
case strings.HasPrefix(s, "wchar: "):
wchar = getInt(s)
case strings.HasPrefix(s, "syscr: "):
syscr = getInt(s)
case strings.HasPrefix(s, "syscw: "):
syscw = getInt(s)
case strings.HasPrefix(s, "read_bytes: "):
readBytes = getInt(s)
case strings.HasPrefix(s, "write_bytes: "):
writeBytes = getInt(s)
}
}
fmt.Fprintf(w, "process_io_read_bytes_total %d\n", rchar)
fmt.Fprintf(w, "process_io_written_bytes_total %d\n", wchar)
fmt.Fprintf(w, "process_io_read_syscalls_total %d\n", syscr)
fmt.Fprintf(w, "process_io_write_syscalls_total %d\n", syscw)
fmt.Fprintf(w, "process_io_storage_read_bytes_total %d\n", readBytes)
fmt.Fprintf(w, "process_io_storage_written_bytes_total %d\n", writeBytes)
} }
var startTimeSeconds = time.Now().Unix() var startTimeSeconds = time.Now().Unix()

2
vendor/modules.txt vendored
View file

@ -14,7 +14,7 @@ github.com/VictoriaMetrics/fastcache
github.com/VictoriaMetrics/fasthttp github.com/VictoriaMetrics/fasthttp
github.com/VictoriaMetrics/fasthttp/fasthttputil github.com/VictoriaMetrics/fasthttp/fasthttputil
github.com/VictoriaMetrics/fasthttp/stackless github.com/VictoriaMetrics/fasthttp/stackless
# github.com/VictoriaMetrics/metrics v1.14.0 # github.com/VictoriaMetrics/metrics v1.15.0
github.com/VictoriaMetrics/metrics github.com/VictoriaMetrics/metrics
# github.com/VictoriaMetrics/metricsql v0.10.1 # github.com/VictoriaMetrics/metricsql v0.10.1
github.com/VictoriaMetrics/metricsql github.com/VictoriaMetrics/metricsql