github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-01 14:47:38 +00:00
Code Issues Projects Releases Packages Wiki Activity

Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files

Browse source
This commit is contained in:
Aliaksandr Valialkin 2021-03-30 15:41:16 +03:00
commit 80ba07dc95
18 changed files with 65 additions and 41 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
Makefile
View file

@ -61,6 +61,14 @@ vmutils-arm64: \
vmrestore-arm64 \
vmctl-arm64
vmutils-arm: \
vmagent-arm \
vmalert-arm \
vmauth-arm \
vmbackup-arm \
vmrestore-arm \
vmctl-arm
vmutils-windows-amd64: \
vmagent-windows-amd64 \
vmalert-windows-amd64 \
@ -77,11 +85,15 @@ release: \
release-victoria-metrics: \
release-victoria-metrics-amd64 \
release-victoria-metrics-arm \
release-victoria-metrics-arm64
release-victoria-metrics-amd64:
GOARCH=amd64 $(MAKE) release-victoria-metrics-generic
release-victoria-metrics-arm:
GOARCH=arm $(MAKE) release-victoria-metrics-generic
release-victoria-metrics-arm64:
GOARCH=arm64 $(MAKE) release-victoria-metrics-generic
@ -96,6 +108,7 @@ release-victoria-metrics-generic: victoria-metrics-$(GOARCH)-prod
release-vmutils: \
release-vmutils-amd64 \
release-vmutils-arm64 \
release-vmutils-arm \
release-vmutils-windows-amd64
release-vmutils-amd64:
@ -104,6 +117,9 @@ release-vmutils-amd64:
release-vmutils-arm64:
GOARCH=arm64 $(MAKE) release-vmutils-generic
release-vmutils-arm:
GOARCH=arm $(MAKE) release-vmutils-generic
release-vmutils-windows-amd64:
GOARCH=amd64 $(MAKE) release-vmutils-windows-generic

6
README.md
View file

@ -658,7 +658,7 @@ to your needs or when testing bugfixes.
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make victoria-metrics` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `victoria-metrics` binary and puts it into the `bin` folder.
@ -674,7 +674,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
### Development ARM build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make victoria-metrics-arm` or `make victoria-metrics-arm64` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `victoria-metrics-arm` or `victoria-metrics-arm64` binary respectively and puts it into the `bin` folder.
@ -688,7 +688,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
`Pure Go` mode builds only Go code without [cgo](https://golang.org/cmd/cgo/) dependencies.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make victoria-metrics-pure` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `victoria-metrics-pure` binary and puts it into the `bin` folder.

4
app/vmagent/README.md
View file

@ -415,7 +415,7 @@ We recommend using [binary releases](https://github.com/VictoriaMetrics/Victoria
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmagent` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds the `vmagent` binary and puts it into the `bin` folder.
@ -444,7 +444,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
### Development ARM build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmagent-arm` or `make vmagent-arm64` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics)
It builds `vmagent-arm` or `vmagent-arm64` binary respectively and puts it into the `bin` folder.

4
app/vmalert/README.md
View file

@ -390,7 +390,7 @@ It is recommended using
#### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmalert` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmalert` binary and puts it into the `bin` folder.
@ -407,7 +407,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
#### Development ARM build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmalert-arm` or `make vmalert-arm64` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmalert-arm` or `vmalert-arm64` binary respectively and puts it into the `bin` folder.

2
app/vmauth/README.md
View file

@ -110,7 +110,7 @@ It is recommended using [binary releases](https://github.com/VictoriaMetrics/Vic
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmauth` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmauth` binary and puts it into the `bin` folder.

2
app/vmbackup/README.md
View file

@ -235,7 +235,7 @@ It is recommended using [binary releases](https://github.com/VictoriaMetrics/Vic
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmbackup` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmbackup` binary and puts it into the `bin` folder.

4
app/vmctl/README.md
View file

@ -48,7 +48,7 @@ It is recommended using [binary releases](https://github.com/VictoriaMetrics/Vic
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmctl` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmctl` binary and puts it into the `bin` folder.
@ -77,7 +77,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
#### Development ARM build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmctl-arm` or `make vmctl-arm64` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmctl-arm` or `vmctl-arm64` binary respectively and puts it into the `bin` folder.

2
app/vmrestore/README.md
View file

@ -131,7 +131,7 @@ It is recommended using [binary releases](https://github.com/VictoriaMetrics/Vic
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmrestore` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmrestore` binary and puts it into the `bin` folder.

31
app/vmselect/netstorage/netstorage.go
View file

@ -83,7 +83,7 @@ func (rss *Results) mustClose() {
var timeseriesWorkCh = make(chan *timeseriesWork, gomaxprocs*16)
type timeseriesWork struct {
mustStop uint64
mustStop *uint32
rss *Results
pts *packedTimeseries
f func(rs *Result, workerID uint) error
@ -93,7 +93,7 @@ type timeseriesWork struct {
}
func (tsw *timeseriesWork) reset() {
tsw.mustStop = 0
tsw.mustStop = nil
tsw.rss = nil
tsw.pts = nil
tsw.f = nil
@ -130,21 +130,24 @@ func timeseriesWorker(workerID uint) {
var rs Result
var rsLastResetTime uint64
for tsw := range timeseriesWorkCh {
rss := tsw.rss
if rss.deadline.Exceeded() {
tsw.doneCh <- fmt.Errorf("timeout exceeded during query execution: %s", rss.deadline.String())
continue
}
if atomic.LoadUint64(&tsw.mustStop) != 0 {
if atomic.LoadUint32(tsw.mustStop) != 0 {
tsw.doneCh <- nil
continue
}
rss := tsw.rss
if rss.deadline.Exceeded() {
atomic.StoreUint32(tsw.mustStop, 1)
tsw.doneCh <- fmt.Errorf("timeout exceeded during query execution: %s", rss.deadline.String())
continue
}
if err := tsw.pts.Unpack(&rs, rss.tbf, rss.tr, rss.fetchData); err != nil {
atomic.StoreUint32(tsw.mustStop, 1)
tsw.doneCh <- fmt.Errorf("error during time series unpacking: %w", err)
continue
}
if len(rs.Timestamps) > 0 || !rss.fetchData {
if err := tsw.f(&rs, workerID); err != nil {
atomic.StoreUint32(tsw.mustStop, 1)
tsw.doneCh <- err
continue
}
@ -172,11 +175,13 @@ func (rss *Results) RunParallel(f func(rs *Result, workerID uint) error) error {
// Feed workers with work.
tsws := make([]*timeseriesWork, len(rss.packedTimeseries))
var mustStop uint32
for i := range rss.packedTimeseries {
tsw := getTimeseriesWork()
tsw.rss = rss
tsw.pts = &rss.packedTimeseries[i]
tsw.f = f
tsw.mustStop = &mustStop
timeseriesWorkCh <- tsw
tsws[i] = tsw
}
@ -187,15 +192,9 @@ func (rss *Results) RunParallel(f func(rs *Result, workerID uint) error) error {
var firstErr error
rowsProcessedTotal := 0
for _, tsw := range tsws {
err := <-tsw.doneCh
if err != nil && firstErr == nil {
// Return just the first error, since other errors
// are likely duplicate the first error.
if err := <-tsw.doneCh; err != nil && firstErr == nil {
// Return just the first error, since other errors are likely duplicate the first error.
firstErr = err
// Notify all the the tsws that they shouldn't be executed.
for _, tsw := range tsws {
atomic.StoreUint64(&tsw.mustStop, 1)
}
}
rowsProcessedTotal += tsw.rowsProcessed
putTimeseriesWork(tsw)

9
docs/CHANGELOG.md
View file

@ -3,6 +3,15 @@
# tip
# [v1.57.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.57.1)
* FEATURE: publish vmutils for `GOOS=arm` on [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases).
* BUGFIX: prevent from possible incomplete query results after timed out query.
* BUGFIX: vmselect: remove `-search.storageTimeout` command-line flag, since it has the same meaning as `-search.maxQueryDuration`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711#issuecomment-808884995).
* BUGFIX: vminsert: return back `type` label to per-tenant metric `vm_tenant_inserted_rows_total`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/932).
# [v1.57.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.57.0)
* FEATURE: optimize query performance by up to 10x on systems with many CPU cores. See [this tweet](https://twitter.com/MetricsVictoria/status/1375064484860067840).

2
docs/Cluster-VictoriaMetrics.md
View file

@ -96,7 +96,7 @@ vmstorage-prod
### Development Builds
1. [Install go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make` from [the repository root](https://github.com/VictoriaMetrics/VictoriaMetrics). It should build `vmstorage`, `vmselect`
and `vminsert` binaries and put them into the `bin` folder.

6
docs/Single-server-VictoriaMetrics.md
View file

@ -658,7 +658,7 @@ to your needs or when testing bugfixes.
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make victoria-metrics` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `victoria-metrics` binary and puts it into the `bin` folder.
@ -674,7 +674,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
### Development ARM build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make victoria-metrics-arm` or `make victoria-metrics-arm64` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `victoria-metrics-arm` or `victoria-metrics-arm64` binary respectively and puts it into the `bin` folder.
@ -688,7 +688,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
`Pure Go` mode builds only Go code without [cgo](https://golang.org/cmd/cgo/) dependencies.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make victoria-metrics-pure` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `victoria-metrics-pure` binary and puts it into the `bin` folder.

4
docs/vmagent.md
View file

@ -415,7 +415,7 @@ We recommend using [binary releases](https://github.com/VictoriaMetrics/Victoria
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmagent` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds the `vmagent` binary and puts it into the `bin` folder.
@ -444,7 +444,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
### Development ARM build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmagent-arm` or `make vmagent-arm64` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics)
It builds `vmagent-arm` or `vmagent-arm64` binary respectively and puts it into the `bin` folder.

4
docs/vmalert.md
View file

@ -390,7 +390,7 @@ It is recommended using
#### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmalert` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmalert` binary and puts it into the `bin` folder.
@ -407,7 +407,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
#### Development ARM build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmalert-arm` or `make vmalert-arm64` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmalert-arm` or `vmalert-arm64` binary respectively and puts it into the `bin` folder.

2
docs/vmauth.md
View file

@ -110,7 +110,7 @@ It is recommended using [binary releases](https://github.com/VictoriaMetrics/Vic
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmauth` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmauth` binary and puts it into the `bin` folder.

2
docs/vmbackup.md
View file

@ -235,7 +235,7 @@ It is recommended using [binary releases](https://github.com/VictoriaMetrics/Vic
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmbackup` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmbackup` binary and puts it into the `bin` folder.

4
docs/vmctl.md
View file

@ -48,7 +48,7 @@ It is recommended using [binary releases](https://github.com/VictoriaMetrics/Vic
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmctl` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmctl` binary and puts it into the `bin` folder.
@ -77,7 +77,7 @@ ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://b
#### Development ARM build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmctl-arm` or `make vmctl-arm64` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmctl-arm` or `vmctl-arm64` binary respectively and puts it into the `bin` folder.

2
docs/vmrestore.md
View file

@ -131,7 +131,7 @@ It is recommended using [binary releases](https://github.com/VictoriaMetrics/Vic
### Development build
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.14.
1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.15.
2. Run `make vmrestore` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
It builds `vmrestore` binary and puts it into the `bin` folder.