mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2025-02-19 15:30:17 +00:00
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files
This commit is contained in:
Aliaksandr Valialkin
commit
31a3672982
289 changed files with 12911 additions and 8680 deletions
53
README.md
53
README.md
|
|
@ -110,6 +110,7 @@ Case studies:
|
|||
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
|
||||
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
|
||||
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
|
||||
* [Criteo](https://docs.victoriametrics.com/CaseStudies.html#criteo)
|
||||
* [Dig Security](https://docs.victoriametrics.com/CaseStudies.html#dig-security)
|
||||
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
|
||||
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
|
||||
|
|
@ -364,6 +365,8 @@ See the [example VMUI at VictoriaMetrics playground](https://play.victoriametric
|
|||
* queries with the biggest average execution duration;
|
||||
* queries that took the most summary time for execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/top_queries` HTTP endpoint.
|
||||
|
||||
## Active queries
|
||||
|
||||
[VMUI](#vmui) provides `active queries` tab, which shows currently execute queries.
|
||||
|
|
@ -373,6 +376,8 @@ It provides the following information per each query:
|
|||
- The duration of the query execution.
|
||||
- The client address, who initiated the query execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/active_queries` HTTP endpoint.
|
||||
|
||||
## Metrics explorer
|
||||
|
||||
[VMUI](#vmui) provides an ability to explore metrics exported by a particular `job` / `instance` in the following way:
|
||||
|
|
@ -404,14 +409,16 @@ matching the specified [series selector](https://prometheus.io/docs/prometheus/l
|
|||
|
||||
Cardinality explorer is built on top of [/api/v1/status/tsdb](#tsdb-stats).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## Cardinality explorer statistic inaccuracy
|
||||
|
||||
In [cluster version of VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html) each vmstorage tracks the stored time series individually.
|
||||
vmselect requests stats via [/api/v1/status/tsdb](#tsdb-stats) API from each vmstorage node and merges the results by summing per-series stats.
|
||||
This may lead to inflated values when samples for the same time series are spread across multiple vmstorage nodes
|
||||
due to [replication](#replication) or [rerouting](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html?highlight=re-routes#cluster-availability).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## How to apply new config to VictoriaMetrics
|
||||
|
||||
VictoriaMetrics is configured via command-line flags, so it must be restarted when new command-line flags should be applied:
|
||||
|
|
@ -616,6 +623,28 @@ Some plugins for Telegraf such as [fluentd](https://github.com/fangli/fluent-plu
|
|||
or [Juniper/jitmon](https://github.com/Juniper/jtimon) send `SHOW DATABASES` query to `/query` and expect a particular database name in the response.
|
||||
Comma-separated list of expected databases can be passed to VictoriaMetrics via `-influx.databaseNames` command-line flag.
|
||||
|
||||
### How to send data in InfluxDB v2 format
|
||||
|
||||
VictoriaMetrics exposes endpoint for InfluxDB v2 HTTP API at `/influx/api/v2/write` and `/api/v2/write`.
|
||||
|
||||
|
||||
In order to write data with InfluxDB line protocol to local VictoriaMetrics using `curl`:
|
||||
|
||||
<div class="with-copy" markdown="1">
|
||||
|
||||
```console
|
||||
curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/api/v2/write'
|
||||
```
|
||||
|
||||
</div>
|
||||
|
||||
The `/api/v1/export` endpoint should return the following response:
|
||||
|
||||
```json
|
||||
{"metric":{"__name__":"measurement_field1","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1695902762311]}
|
||||
{"metric":{"__name__":"measurement_field2","tag1":"value1","tag2":"value2"},"values":[1.23],"timestamps":[1695902762311]}
|
||||
```
|
||||
|
||||
## How to send data from Graphite-compatible agents such as [StatsD](https://github.com/etsy/statsd)
|
||||
|
||||
Enable Graphite receiver in VictoriaMetrics by setting `-graphiteListenAddr` command line flag. For instance,
|
||||
|
|
@ -830,7 +859,7 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
* `/api/v1/series/count` - returns the total number of time series in the database. Some notes:
|
||||
* the handler scans all the inverted index, so it can be slow if the database contains tens of millions of time series;
|
||||
* the handler may count [deleted time series](#how-to-delete-time-series) additionally to normal time series due to internal implementation restrictions;
|
||||
* `/api/v1/status/active_queries` - returns a list of currently running queries.
|
||||
* `/api/v1/status/active_queries` - returns the list of currently running queries. This list is also available at [`active queries` page at VMUI](#active-queries).
|
||||
* `/api/v1/status/top_queries` - returns the following query lists:
|
||||
* the most frequently executed queries - `topByCount`
|
||||
* queries with the biggest average execution duration - `topByAvgDuration`
|
||||
|
|
@ -840,6 +869,8 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
For example, request to `/api/v1/status/top_queries?topN=5&maxLifetime=30s` would return up to 5 queries per list, which were executed during the last 30 seconds.
|
||||
VictoriaMetrics tracks the last `-search.queryStats.lastQueriesCount` queries with durations at least `-search.queryStats.minQueryDuration`.
|
||||
|
||||
See also [`top queries` page at VMUI](#top-queries).
|
||||
|
||||
### Timestamp formats
|
||||
|
||||
VictoriaMetrics accepts the following formats for `time`, `start` and `end` query args
|
||||
|
|
@ -1654,9 +1685,10 @@ See [these docs](https://docs.victoriametrics.com/guides/guide-vmcluster-multipl
|
|||
which allow configuring multiple retentions for distinct sets of time series matching the configured [series filters](https://docs.victoriametrics.com/keyConcepts.html#filtering)
|
||||
via `-retentionFilter` command-line flag. This flag accepts `filter:duration` options, where `filter` must be
|
||||
a valid [series filter](https://docs.victoriametrics.com/keyConcepts.html#filtering), while the `duration`
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`. If series doesn't match
|
||||
any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention) command-line flag is applied to it.
|
||||
If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`.
|
||||
The `duration` of the `-retentionFilter` must be lower or equal to [-retentionPeriod](#retention) flag value.
|
||||
If series doesn't match any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention)
|
||||
command-line flag is applied to it. If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
|
||||
For example, the following config sets 3 days retention for time series with `team="juniors"` label,
|
||||
30 days retention for time series with `env="dev"` or `env="staging"` label and 1 year retention for the remaining time series:
|
||||
|
|
@ -1790,9 +1822,9 @@ Graphs on the dashboards contain useful hints - hover the `i` icon in the top le
|
|||
We recommend setting up [alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#alerts)
|
||||
via [vmalert](https://docs.victoriametrics.com/vmalert.html) or via Prometheus.
|
||||
|
||||
VictoriaMetrics exposes currently running queries and their execution times at `/api/v1/status/active_queries` page.
|
||||
VictoriaMetrics exposes currently running queries and their execution times at [`active queries` page](#active-queries).
|
||||
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at `/api/v1/status/top_queries` page.
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at [`top queries` page](#top-queries).
|
||||
|
||||
See also [VictoriaMetrics Monitoring](https://victoriametrics.com/blog/victoriametrics-monitoring/)
|
||||
and [troubleshooting docs](https://docs.victoriametrics.com/Troubleshooting.html).
|
||||
|
|
@ -1937,9 +1969,6 @@ and [cardinality explorer docs](#cardinality-explorer).
|
|||
has at least 20% of free space. The remaining amount of free space
|
||||
can be [monitored](#monitoring) via `vm_free_disk_space_bytes` metric. The total size of data
|
||||
stored on the disk can be monitored via sum of `vm_data_size_bytes` metrics.
|
||||
See also `vm_merge_need_free_disk_space` metrics, which are set to values higher than 0
|
||||
if background merge cannot be initiated due to free disk space shortage. The value shows the number of per-month partitions,
|
||||
which would start background merge if they had more free disk space.
|
||||
|
||||
* VictoriaMetrics buffers incoming data in memory for up to a few seconds before flushing it to persistent storage.
|
||||
This may lead to the following "issues":
|
||||
|
|
|
|||
|
|
@ -12,6 +12,8 @@ import (
|
|||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vlinsert/insertutils"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vlstorage"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/bufferedwriter"
|
||||
|
|
@ -22,7 +24,6 @@ import (
|
|||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logstorage"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/common"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/writeconcurrencylimiter"
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
)
|
||||
|
||||
var (
|
||||
|
|
@ -93,16 +94,20 @@ func RequestHandler(path string, w http.ResponseWriter, r *http.Request) bool {
|
|||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
if err := vlstorage.CanWriteData(); err != nil {
|
||||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
lr := logstorage.GetLogRows(cp.StreamFields, cp.IgnoreFields)
|
||||
processLogMessage := cp.GetProcessLogMessageFunc(lr)
|
||||
isGzip := r.Header.Get("Content-Encoding") == "gzip"
|
||||
n, err := readBulkRequest(r.Body, isGzip, cp.TimeField, cp.MsgField, processLogMessage)
|
||||
vlstorage.MustAddRows(lr)
|
||||
logstorage.PutLogRows(lr)
|
||||
if err != nil {
|
||||
logger.Warnf("cannot decode log message #%d in /_bulk request: %s", n, err)
|
||||
return true
|
||||
}
|
||||
vlstorage.MustAddRows(lr)
|
||||
logstorage.PutLogRows(lr)
|
||||
|
||||
tookMs := time.Since(startTime).Milliseconds()
|
||||
bw := bufferedwriter.Get(w)
|
||||
|
|
@ -220,6 +225,7 @@ func readBulkLine(sc *bufio.Scanner, timeField, msgField string,
|
|||
p.RenameField(msgField, "_msg")
|
||||
processLogMessage(ts, p.Fields)
|
||||
logjson.PutParser(p)
|
||||
|
||||
return true, nil
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -36,6 +36,10 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
|
|||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
if err := vlstorage.CanWriteData(); err != nil {
|
||||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
lr := logstorage.GetLogRows(cp.StreamFields, cp.IgnoreFields)
|
||||
processLogMessage := cp.GetProcessLogMessageFunc(lr)
|
||||
|
||||
|
|
@ -115,6 +119,7 @@ func readLine(sc *bufio.Scanner, timeField, msgField string, processLogMessage f
|
|||
p.RenameField(msgField, "_msg")
|
||||
processLogMessage(ts, p.Fields)
|
||||
logjson.PutParser(p)
|
||||
|
||||
return true, nil
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -47,15 +47,20 @@ func handleJSON(r *http.Request, w http.ResponseWriter) bool {
|
|||
httpserver.Errorf(w, r, "cannot parse common params from request: %s", err)
|
||||
return true
|
||||
}
|
||||
if err := vlstorage.CanWriteData(); err != nil {
|
||||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
lr := logstorage.GetLogRows(cp.StreamFields, cp.IgnoreFields)
|
||||
processLogMessage := cp.GetProcessLogMessageFunc(lr)
|
||||
n, err := parseJSONRequest(data, processLogMessage)
|
||||
vlstorage.MustAddRows(lr)
|
||||
logstorage.PutLogRows(lr)
|
||||
if err != nil {
|
||||
httpserver.Errorf(w, r, "cannot parse Loki request: %s", err)
|
||||
httpserver.Errorf(w, r, "cannot parse Loki json request: %s", err)
|
||||
return true
|
||||
}
|
||||
|
||||
rowsIngestedJSONTotal.Add(n)
|
||||
|
||||
// update lokiRequestJSONDuration only for successfully parsed requests
|
||||
|
|
@ -166,7 +171,6 @@ func parseJSONRequest(data []byte, processLogMessage func(timestamp int64, field
|
|||
Value: bytesutil.ToUnsafeString(msg),
|
||||
})
|
||||
processLogMessage(ts, fields)
|
||||
|
||||
}
|
||||
rowsIngested += len(lines)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -39,13 +39,17 @@ func handleProtobuf(r *http.Request, w http.ResponseWriter) bool {
|
|||
httpserver.Errorf(w, r, "cannot parse common params from request: %s", err)
|
||||
return true
|
||||
}
|
||||
if err := vlstorage.CanWriteData(); err != nil {
|
||||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
lr := logstorage.GetLogRows(cp.StreamFields, cp.IgnoreFields)
|
||||
processLogMessage := cp.GetProcessLogMessageFunc(lr)
|
||||
n, err := parseProtobufRequest(data, processLogMessage)
|
||||
vlstorage.MustAddRows(lr)
|
||||
logstorage.PutLogRows(lr)
|
||||
if err != nil {
|
||||
httpserver.Errorf(w, r, "cannot parse loki request: %s", err)
|
||||
httpserver.Errorf(w, r, "cannot parse Loki protobuf request: %s", err)
|
||||
return true
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -6,8 +6,9 @@ import (
|
|||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logstorage"
|
||||
"github.com/golang/snappy"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logstorage"
|
||||
)
|
||||
|
||||
func BenchmarkParseProtobufRequest(b *testing.B) {
|
||||
|
|
|
|||
|
|
@ -1,14 +1,14 @@
|
|||
{
|
||||
"files": {
|
||||
"main.css": "./static/css/main.17914339.css",
|
||||
"main.js": "./static/js/main.b6509627.js",
|
||||
"main.css": "./static/css/main.3258b6c4.css",
|
||||
"main.js": "./static/js/main.41b816cc.js",
|
||||
"static/js/522.b5ae4365.chunk.js": "./static/js/522.b5ae4365.chunk.js",
|
||||
"static/media/Lato-Regular.ttf": "./static/media/Lato-Regular.d714fec1633b69a9c2e9.ttf",
|
||||
"static/media/Lato-Bold.ttf": "./static/media/Lato-Bold.32360ba4b57802daa4d6.ttf",
|
||||
"index.html": "./index.html"
|
||||
},
|
||||
"entrypoints": [
|
||||
"static/css/main.17914339.css",
|
||||
"static/js/main.b6509627.js"
|
||||
"static/css/main.3258b6c4.css",
|
||||
"static/js/main.41b816cc.js"
|
||||
]
|
||||
}
|
||||
|
|
@ -1 +1 @@
|
|||
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.b6509627.js"></script><link href="./static/css/main.17914339.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
|
||||
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.41b816cc.js"></script><link href="./static/css/main.3258b6c4.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
|
||||
File diff suppressed because one or more lines are too long
1
app/vlselect/vmui/static/css/main.3258b6c4.css
Normal file
1
app/vlselect/vmui/static/css/main.3258b6c4.css
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -3,14 +3,17 @@ package vlstorage
|
|||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logstorage"
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
)
|
||||
|
||||
var (
|
||||
|
|
@ -29,6 +32,8 @@ var (
|
|||
"see https://docs.victoriametrics.com/VictoriaLogs/keyConcepts.html#stream-fields ; see also -logIngestedRows")
|
||||
logIngestedRows = flag.Bool("logIngestedRows", false, "Whether to log all the ingested log entries; this can be useful for debugging of data ingestion; "+
|
||||
"see https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/ ; see also -logNewStreams")
|
||||
minFreeDiskSpaceBytes = flagutil.NewBytes("storage.minFreeDiskSpaceBytes", 10e6, "The minimum free disk space at -storageDataPath after which "+
|
||||
"the storage stops accepting new data")
|
||||
)
|
||||
|
||||
// Init initializes vlstorage.
|
||||
|
|
@ -43,11 +48,12 @@ func Init() {
|
|||
logger.Fatalf("-retentionPeriod cannot be smaller than a day; got %s", retentionPeriod)
|
||||
}
|
||||
cfg := &logstorage.StorageConfig{
|
||||
Retention: retentionPeriod.Duration(),
|
||||
FlushInterval: *inmemoryDataFlushInterval,
|
||||
FutureRetention: futureRetention.Duration(),
|
||||
LogNewStreams: *logNewStreams,
|
||||
LogIngestedRows: *logIngestedRows,
|
||||
Retention: retentionPeriod.Duration(),
|
||||
FlushInterval: *inmemoryDataFlushInterval,
|
||||
FutureRetention: futureRetention.Duration(),
|
||||
LogNewStreams: *logNewStreams,
|
||||
LogIngestedRows: *logIngestedRows,
|
||||
MinFreeDiskSpaceBytes: minFreeDiskSpaceBytes.N,
|
||||
}
|
||||
logger.Infof("opening storage at -storageDataPath=%s", *storageDataPath)
|
||||
startTime := time.Now()
|
||||
|
|
@ -74,7 +80,21 @@ func Stop() {
|
|||
var strg *logstorage.Storage
|
||||
var storageMetrics *metrics.Set
|
||||
|
||||
// CanWriteData returns non-nil error if it cannot write data to vlstorage.
|
||||
func CanWriteData() error {
|
||||
if strg.IsReadOnly() {
|
||||
return &httpserver.ErrorWithStatusCode{
|
||||
Err: fmt.Errorf("cannot add rows into storage in read-only mode; the storage can be in read-only mode "+
|
||||
"because of lack of free disk space at -storageDataPath=%s", *storageDataPath),
|
||||
StatusCode: http.StatusTooManyRequests,
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// MustAddRows adds lr to vlstorage
|
||||
//
|
||||
// It is advised to call CanWriteData() before calling MustAddRows()
|
||||
func MustAddRows(lr *logstorage.LogRows) {
|
||||
strg.MustAddRows(lr)
|
||||
}
|
||||
|
|
@ -107,6 +127,12 @@ func initStorageMetrics(strg *logstorage.Storage) *metrics.Set {
|
|||
ms.NewGauge(fmt.Sprintf(`vl_free_disk_space_bytes{path=%q}`, *storageDataPath), func() float64 {
|
||||
return float64(fs.MustGetFreeSpace(*storageDataPath))
|
||||
})
|
||||
ms.NewGauge(fmt.Sprintf(`vl_storage_is_read_only{path=%q}`, *storageDataPath), func() float64 {
|
||||
if m().IsReadOnly {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
})
|
||||
|
||||
ms.NewGauge(`vl_active_merges{type="inmemory"}`, func() float64 {
|
||||
return float64(m().InmemoryActiveMerges)
|
||||
|
|
|
|||
|
|
@ -3,7 +3,8 @@
|
|||
`vmagent` is a tiny agent which helps you collect metrics from various sources,
|
||||
[relabel and filter the collected metrics](#relabeling)
|
||||
and store them in [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics)
|
||||
or any other storage systems via Prometheus `remote_write` protocol.
|
||||
or any other storage systems via Prometheus `remote_write` protocol
|
||||
or via [VictoriaMetrics `remote_write` protocol](#victoriametrics-remote-write-protocol).
|
||||
|
||||
See [Quick Start](#quick-start) for details.
|
||||
|
||||
|
|
|
|||
|
|
@ -11,6 +11,8 @@ import (
|
|||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmagent/csvimport"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmagent/datadog"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmagent/graphite"
|
||||
|
|
@ -39,7 +41,6 @@ import (
|
|||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promscrape"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/common"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/pushmetrics"
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
)
|
||||
|
||||
var (
|
||||
|
|
@ -208,7 +209,7 @@ func getAuthTokenFromPath(path string) (*auth.Token, error) {
|
|||
if p.Suffix != "opentsdb/api/put" {
|
||||
return nil, fmt.Errorf("unsupported path requested: %q; expecting 'opentsdb/api/put'", p.Suffix)
|
||||
}
|
||||
return auth.NewToken(p.AuthToken)
|
||||
return auth.NewTokenPossibleMultitenant(p.AuthToken)
|
||||
}
|
||||
|
||||
func requestHandler(w http.ResponseWriter, r *http.Request) bool {
|
||||
|
|
@ -251,7 +252,7 @@ func requestHandler(w http.ResponseWriter, r *http.Request) bool {
|
|||
w.WriteHeader(statusCode)
|
||||
return true
|
||||
}
|
||||
if strings.HasPrefix(path, "datadog/") {
|
||||
if strings.HasPrefix(path, "/datadog/") {
|
||||
// Trim suffix from paths starting from /datadog/ in order to support legacy DataDog agent.
|
||||
// See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2670
|
||||
path = strings.TrimSuffix(path, "/")
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ func InsertHandler(at *auth.Token, req *http.Request) error {
|
|||
return err
|
||||
}
|
||||
isGzipped := req.Header.Get("Content-Encoding") == "gzip"
|
||||
return stream.Parse(req.Body, defaultTimestamp, isGzipped, func(rows []parser.Row) error {
|
||||
return stream.Parse(req.Body, defaultTimestamp, isGzipped, true, func(rows []parser.Row) error {
|
||||
return insertRows(at, rows, extraLabels)
|
||||
}, func(s string) {
|
||||
httpserver.LogError(req, s)
|
||||
|
|
|
|||
|
|
@ -526,7 +526,7 @@ Alertmanagers.
|
|||
|
||||
To avoid recording rules results and alerts state duplication in VictoriaMetrics server
|
||||
don't forget to configure [deduplication](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#deduplication).
|
||||
The recommended value for `-dedup.minScrapeInterval` must be multiple of vmalert's `evaluation_interval`.
|
||||
The recommended value for `-dedup.minScrapeInterval` must be multiple of vmalert's `-evaluationInterval`.
|
||||
If you observe inconsistent or "jumping" values in series produced by vmalert, try disabling `-datasource.queryTimeAlignment`
|
||||
command line flag. Because of alignment, two or more vmalert HA pairs will produce results with the same timestamps.
|
||||
But due of backfilling (data delivered to the datasource with some delay) values of such results may differ,
|
||||
|
|
@ -778,7 +778,7 @@ may get empty response from the datasource and produce empty recording rules or
|
|||
|
||||
Try the following recommendations to reduce the chance of hitting the data delay issue:
|
||||
|
||||
* Always configure group's `evaluationInterval` to be bigger or at least equal to
|
||||
* Always configure group's `-evaluationInterval` to be bigger or at least equal to
|
||||
[time series resolution](https://docs.victoriametrics.com/keyConcepts.html#time-series-resolution);
|
||||
* Ensure that `[duration]` value is at least twice bigger than
|
||||
[time series resolution](https://docs.victoriametrics.com/keyConcepts.html#time-series-resolution). For example,
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ The auth config can be reloaded via the following ways:
|
|||
and apply new changes every 5 seconds.
|
||||
|
||||
Docker images for `vmauth` are available [here](https://hub.docker.com/r/victoriametrics/vmauth/tags).
|
||||
See how `vmauth` used in [docker-compose env](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/README.md#victoriametrics-cluster).
|
||||
|
||||
Pass `-help` to `vmauth` in order to see all the supported command-line flags with their descriptions.
|
||||
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ func InsertHandler(req *http.Request) error {
|
|||
return err
|
||||
}
|
||||
isGzipped := req.Header.Get("Content-Encoding") == "gzip"
|
||||
return stream.Parse(req.Body, defaultTimestamp, isGzipped, func(rows []parser.Row) error {
|
||||
return stream.Parse(req.Body, defaultTimestamp, isGzipped, true, func(rows []parser.Row) error {
|
||||
return insertRows(rows, extraLabels)
|
||||
}, func(s string) {
|
||||
httpserver.LogError(req, s)
|
||||
|
|
|
|||
|
|
@ -111,6 +111,12 @@ func maySortResults(e metricsql.Expr) bool {
|
|||
"bottomk_max", "bottomk_min", "bottomk_avg", "bottomk_median", "bottomk_last":
|
||||
return false
|
||||
}
|
||||
case *metricsql.BinaryOpExpr:
|
||||
if strings.ToLower(v.Op) == "or" {
|
||||
// Do not sort results for `a or b` in the same way as Prometheus does.
|
||||
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
|
|
|||
|
|
@ -5,10 +5,11 @@ import (
|
|||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/netstorage"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/searchutils"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
)
|
||||
|
||||
func TestEscapeDots(t *testing.T) {
|
||||
|
|
@ -245,6 +246,27 @@ func TestExecSuccess(t *testing.T) {
|
|||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("bitmap_and(NaN, 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_and(NaN, 1)`
|
||||
f(q, nil)
|
||||
})
|
||||
t.Run("bitmap_and(1, NaN)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_and(1, NaN)`
|
||||
f(q, nil)
|
||||
})
|
||||
t.Run("bitmap_and(round(rand(1) > 0.5, 1), 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_and(round(rand(1) > 0.5, 1), 1)`
|
||||
r := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{1, 1, 1, nan, nan, 1},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("bitmap_or(0xA2, 0x11)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_or(0xA2, 0x11)`
|
||||
|
|
@ -267,6 +289,22 @@ func TestExecSuccess(t *testing.T) {
|
|||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("bitmap_or(NaN, 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_or(NaN, 1)`
|
||||
f(q, nil)
|
||||
})
|
||||
t.Run("bitmap_or(round(rand(1) > 0.5, 1), 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_or(round(rand(1) > 0.5, 1), 1)`
|
||||
r := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{1, 1, 1, nan, nan, 1},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("bitmap_xor(0xB3, 0x11)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_xor(0xB3, 0x11)`
|
||||
|
|
@ -289,6 +327,22 @@ func TestExecSuccess(t *testing.T) {
|
|||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("bitmap_xor(NaN, 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_xor(NaN, 1)`
|
||||
f(q, nil)
|
||||
})
|
||||
t.Run("bitmap_xor(round(rand(1) > 0.5, 1), 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_xor(round(rand(1) > 0.5, 1), 1)`
|
||||
r := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{0, 0, 0, nan, nan, 0},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("timezone_offset(UTC)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `timezone_offset("UTC")`
|
||||
|
|
@ -7669,7 +7723,7 @@ func TestExecSuccess(t *testing.T) {
|
|||
})
|
||||
t.Run(`aggr_over_time(multi-func)`, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `sort(aggr_over_time(("min_over_time", "count_over_time", "max_over_time"), round(rand(0),0.1)[:10s]))`
|
||||
q := `sort(aggr_over_time(("min_over_time", "median_over_time", "max_over_time"), round(rand(0),0.1)[:10s]))`
|
||||
r1 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{0, 0, 0, 0, 0, 0},
|
||||
|
|
@ -7681,21 +7735,21 @@ func TestExecSuccess(t *testing.T) {
|
|||
}}
|
||||
r2 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{0.8, 0.9, 1, 0.9, 1, 0.9},
|
||||
Values: []float64{0.4, 0.5, 0.5, 0.75, 0.6, 0.45},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
r2.MetricName.Tags = []storage.Tag{{
|
||||
Key: []byte("rollup"),
|
||||
Value: []byte("max_over_time"),
|
||||
Value: []byte("median_over_time"),
|
||||
}}
|
||||
r3 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{20, 20, 20, 20, 20, 20},
|
||||
Values: []float64{0.8, 0.9, 1, 0.9, 1, 0.9},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
r3.MetricName.Tags = []storage.Tag{{
|
||||
Key: []byte("rollup"),
|
||||
Value: []byte("count_over_time"),
|
||||
Value: []byte("max_over_time"),
|
||||
}}
|
||||
resultExpected := []netstorage.Result{r1, r2, r3}
|
||||
f(q, resultExpected)
|
||||
|
|
@ -8479,11 +8533,11 @@ func TestExecSuccess(t *testing.T) {
|
|||
})
|
||||
t.Run(`result sorting`, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `label_set(1, "instance", "localhost:1001", "type", "free")
|
||||
or label_set(1, "instance", "localhost:1001", "type", "buffers")
|
||||
or label_set(1, "instance", "localhost:1000", "type", "buffers")
|
||||
or label_set(1, "instance", "localhost:1000", "type", "free")
|
||||
`
|
||||
q := `(label_set(1, "instance", "localhost:1001", "type", "free"),
|
||||
label_set(1, "instance", "localhost:1001", "type", "buffers"),
|
||||
label_set(1, "instance", "localhost:1000", "type", "buffers"),
|
||||
label_set(1, "instance", "localhost:1000", "type", "free"),
|
||||
)`
|
||||
r1 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{1, 1, 1, 1, 1, 1},
|
||||
|
|
@ -8515,6 +8569,34 @@ func TestExecSuccess(t *testing.T) {
|
|||
resultExpected := []netstorage.Result{r1, r2, r3, r4}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run(`no_sorting_for_or`, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `label_set(2, "foo", "bar") or label_set(1, "foo", "baz")`
|
||||
r1 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{2, 2, 2, 2, 2, 2},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
r1.MetricName.Tags = []storage.Tag{
|
||||
{
|
||||
Key: []byte("foo"),
|
||||
Value: []byte("bar"),
|
||||
},
|
||||
}
|
||||
r2 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{1, 1, 1, 1, 1, 1},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
r2.MetricName.Tags = []storage.Tag{
|
||||
{
|
||||
Key: []byte("foo"),
|
||||
Value: []byte("baz"),
|
||||
},
|
||||
}
|
||||
resultExpected := []netstorage.Result{r1, r2}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run(`sort_by_label_numeric(multiple_labels_only_string)`, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `sort_by_label_numeric((
|
||||
|
|
|
|||
|
|
@ -7,11 +7,12 @@ import (
|
|||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
)
|
||||
|
||||
var minStalenessInterval = flag.Duration("search.minStalenessInterval", 0, "The minimum interval for staleness calculations. "+
|
||||
|
|
@ -58,6 +59,7 @@ var rollupFuncs = map[string]newRollupFunc{
|
|||
"lifetime": newRollupFuncOneArg(rollupLifetime),
|
||||
"mad_over_time": newRollupFuncOneArg(rollupMAD),
|
||||
"max_over_time": newRollupFuncOneArg(rollupMax),
|
||||
"median_over_time": newRollupFuncOneArg(rollupMedian),
|
||||
"min_over_time": newRollupFuncOneArg(rollupMin),
|
||||
"mode_over_time": newRollupFuncOneArg(rollupModeOverTime),
|
||||
"predict_linear": newRollupPredictLinear,
|
||||
|
|
@ -125,6 +127,7 @@ var rollupAggrFuncs = map[string]rollupFunc{
|
|||
"lifetime": rollupLifetime,
|
||||
"mad_over_time": rollupMAD,
|
||||
"max_over_time": rollupMax,
|
||||
"median_over_time": rollupMedian,
|
||||
"min_over_time": rollupMin,
|
||||
"mode_over_time": rollupModeOverTime,
|
||||
"present_over_time": rollupPresent,
|
||||
|
|
@ -224,6 +227,7 @@ var rollupFuncsKeepMetricName = map[string]bool{
|
|||
"holt_winters": true,
|
||||
"last_over_time": true,
|
||||
"max_over_time": true,
|
||||
"median_over_time": true,
|
||||
"min_over_time": true,
|
||||
"mode_over_time": true,
|
||||
"predict_linear": true,
|
||||
|
|
@ -1396,6 +1400,10 @@ func rollupMax(rfa *rollupFuncArg) float64 {
|
|||
return maxValue
|
||||
}
|
||||
|
||||
func rollupMedian(rfa *rollupFuncArg) float64 {
|
||||
return quantile(0.5, rfa.values)
|
||||
}
|
||||
|
||||
func rollupTmin(rfa *rollupFuncArg) float64 {
|
||||
// There is no need in handling NaNs here, since they must be cleaned up
|
||||
// before calling rollup funcs.
|
||||
|
|
|
|||
|
|
@ -11,12 +11,13 @@ import (
|
|||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/searchutils"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
)
|
||||
|
||||
var transformFuncs = map[string]transformFunc{
|
||||
|
|
@ -2589,7 +2590,12 @@ func newTransformBitmap(bitmapFunc func(a, b uint64) uint64) func(tfa *transform
|
|||
}
|
||||
tf := func(values []float64) {
|
||||
for i, v := range values {
|
||||
values[i] = float64(bitmapFunc(uint64(v), uint64(ns[i])))
|
||||
w := ns[i]
|
||||
result := nan
|
||||
if !math.IsNaN(v) && !math.IsNaN(w) {
|
||||
result = float64(bitmapFunc(uint64(v), uint64(w)))
|
||||
}
|
||||
values[i] = result
|
||||
}
|
||||
}
|
||||
return doTransformValues(args[0], tf, tfa.fe)
|
||||
|
|
|
|||
|
|
@ -1,14 +1,14 @@
|
|||
{
|
||||
"files": {
|
||||
"main.css": "./static/css/main.e95426eb.css",
|
||||
"main.js": "./static/js/main.8d3e794d.js",
|
||||
"main.css": "./static/css/main.4ab6595e.css",
|
||||
"main.js": "./static/js/main.dbf8fb4f.js",
|
||||
"static/js/522.b5ae4365.chunk.js": "./static/js/522.b5ae4365.chunk.js",
|
||||
"static/media/Lato-Regular.ttf": "./static/media/Lato-Regular.d714fec1633b69a9c2e9.ttf",
|
||||
"static/media/Lato-Bold.ttf": "./static/media/Lato-Bold.32360ba4b57802daa4d6.ttf",
|
||||
"index.html": "./index.html"
|
||||
},
|
||||
"entrypoints": [
|
||||
"static/css/main.e95426eb.css",
|
||||
"static/js/main.8d3e794d.js"
|
||||
"static/css/main.4ab6595e.css",
|
||||
"static/js/main.dbf8fb4f.js"
|
||||
]
|
||||
}
|
||||
|
|
@ -1 +1 @@
|
|||
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.8d3e794d.js"></script><link href="./static/css/main.e95426eb.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
|
||||
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.dbf8fb4f.js"></script><link href="./static/css/main.4ab6595e.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
|
||||
1
app/vmselect/vmui/static/css/main.4ab6595e.css
Normal file
1
app/vmselect/vmui/static/css/main.4ab6595e.css
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
2
app/vmselect/vmui/static/js/main.dbf8fb4f.js
Normal file
2
app/vmselect/vmui/static/js/main.dbf8fb4f.js
Normal file
File diff suppressed because one or more lines are too long
|
|
@ -589,11 +589,6 @@ func registerStorageMetrics(strg *storage.Storage) {
|
|||
return float64(idbm().ItemsAddedSizeBytes)
|
||||
})
|
||||
|
||||
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686
|
||||
metrics.NewGauge(`vm_merge_need_free_disk_space`, func() float64 {
|
||||
return float64(tm().MergeNeedFreeDiskSpace)
|
||||
})
|
||||
|
||||
metrics.NewGauge(`vm_pending_rows{type="storage"}`, func() float64 {
|
||||
return float64(tm().PendingRows)
|
||||
})
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ interface Hyperlink {
|
|||
children?: ReactNode;
|
||||
colored?: boolean;
|
||||
underlined?: boolean;
|
||||
withIcon?: boolean;
|
||||
}
|
||||
|
||||
const Hyperlink: FC<Hyperlink> = ({
|
||||
|
|
@ -15,14 +16,16 @@ const Hyperlink: FC<Hyperlink> = ({
|
|||
href,
|
||||
children,
|
||||
colored = true,
|
||||
underlined = false
|
||||
underlined = false,
|
||||
withIcon = false,
|
||||
}) => (
|
||||
<a
|
||||
href={href}
|
||||
className={classNames({
|
||||
"vm-link": true,
|
||||
"vm-link_colored": colored,
|
||||
"vm-link_underlined": underlined
|
||||
"vm-link_underlined": underlined,
|
||||
"vm-link_with-icon": withIcon,
|
||||
})}
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
|
|
|
|||
|
|
@ -430,3 +430,23 @@ export const ListIcon = () => (
|
|||
<path d="M3 14h4v-4H3v4zm0 5h4v-4H3v4zM3 9h4V5H3v4zm5 5h13v-4H8v4zm0 5h13v-4H8v4zM8 5v4h13V5H8z"></path>
|
||||
</svg>
|
||||
);
|
||||
|
||||
export const StarBorderIcon = () => (
|
||||
<svg
|
||||
viewBox="0 0 24 24"
|
||||
fill="currentColor"
|
||||
>
|
||||
<path
|
||||
d="m22 9.24-7.19-.62L12 2 9.19 8.63 2 9.24l5.46 4.73L5.82 21 12 17.27 18.18 21l-1.63-7.03L22 9.24zM12 15.4l-3.76 2.27 1-4.28-3.32-2.88 4.38-.38L12 6.1l1.71 4.04 4.38.38-3.32 2.88 1 4.28L12 15.4z"
|
||||
></path>
|
||||
</svg>
|
||||
);
|
||||
|
||||
export const StarIcon = () => (
|
||||
<svg
|
||||
viewBox="0 0 24 24"
|
||||
fill="currentColor"
|
||||
>
|
||||
<path d="M12 17.27 18.18 21l-1.64-7.03L22 9.24l-7.19-.61L12 2 9.19 8.63 2 9.24l5.46 4.73L5.82 21z"></path>
|
||||
</svg>
|
||||
);
|
||||
|
|
|
|||
|
|
@ -32,6 +32,10 @@ $padding-modal: $padding-medium;
|
|||
&-header {
|
||||
padding: $padding-small $padding-small $padding-small $padding-global;
|
||||
margin-bottom: $padding-global;
|
||||
|
||||
&__title {
|
||||
max-width: 80vw;
|
||||
}
|
||||
}
|
||||
|
||||
&-body {
|
||||
|
|
@ -69,6 +73,10 @@ $padding-modal: $padding-medium;
|
|||
&__title {
|
||||
font-weight: bold;
|
||||
user-select: none;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
max-width: 50vw;
|
||||
}
|
||||
|
||||
&__close {
|
||||
|
|
|
|||
|
|
@ -8,8 +8,10 @@ import classNames from "classnames";
|
|||
import { useAppState } from "../../../state/common/StateContext";
|
||||
import useDeviceDetect from "../../../hooks/useDeviceDetect";
|
||||
import Button from "../../Main/Button/Button";
|
||||
import { humanizeSeconds } from "../../../utils/time";
|
||||
|
||||
interface RecursiveProps {
|
||||
isRoot?: boolean;
|
||||
trace: Trace;
|
||||
totalMsec: number;
|
||||
}
|
||||
|
|
@ -18,7 +20,7 @@ interface OpenLevels {
|
|||
[x: number]: boolean
|
||||
}
|
||||
|
||||
const NestedNav: FC<RecursiveProps> = ({ trace, totalMsec }) => {
|
||||
const NestedNav: FC<RecursiveProps> = ({ isRoot, trace, totalMsec }) => {
|
||||
const { isDarkTheme } = useAppState();
|
||||
const { isMobile } = useDeviceDetect();
|
||||
const [openLevels, setOpenLevels] = useState({} as OpenLevels);
|
||||
|
|
@ -27,6 +29,8 @@ const NestedNav: FC<RecursiveProps> = ({ trace, totalMsec }) => {
|
|||
const [isExpanded, setIsExpanded] = useState(false);
|
||||
const [showFullMessage, setShowFullMessage] = useState(false);
|
||||
|
||||
const duration = humanizeSeconds(trace.duration / 1000) || `${trace.duration}ms`;
|
||||
|
||||
useEffect(() => {
|
||||
if (!messageRef.current) return;
|
||||
const contentElement = messageRef.current;
|
||||
|
|
@ -40,24 +44,29 @@ const NestedNav: FC<RecursiveProps> = ({ trace, totalMsec }) => {
|
|||
setShowFullMessage(prev => !prev);
|
||||
};
|
||||
|
||||
const hasChildren = trace.children && !!trace.children.length;
|
||||
const progress = trace.duration / totalMsec * 100;
|
||||
const handleListClick = (level: number) => () => {
|
||||
if (!hasChildren) return;
|
||||
setOpenLevels((prevState:OpenLevels) => {
|
||||
return { ...prevState, [level]: !prevState[level] };
|
||||
});
|
||||
};
|
||||
const hasChildren = trace.children && !!trace.children.length;
|
||||
const progress = trace.duration / totalMsec * 100;
|
||||
|
||||
return (
|
||||
<div
|
||||
className={classNames({
|
||||
"vm-nested-nav": true,
|
||||
"vm-nested-nav_root": isRoot,
|
||||
"vm-nested-nav_dark": isDarkTheme,
|
||||
"vm-nested-nav_mobile": isMobile,
|
||||
})}
|
||||
>
|
||||
<div
|
||||
className="vm-nested-nav-header"
|
||||
className={classNames({
|
||||
"vm-nested-nav-header": true,
|
||||
"vm-nested-nav-header_open": openLevels[trace.idValue],
|
||||
})}
|
||||
onClick={handleListClick(trace.idValue)}
|
||||
>
|
||||
{hasChildren && (
|
||||
|
|
@ -84,7 +93,7 @@ const NestedNav: FC<RecursiveProps> = ({ trace, totalMsec }) => {
|
|||
</div>
|
||||
<div className="vm-nested-nav-header-bottom">
|
||||
<div className="vm-nested-nav-header-bottom__duration">
|
||||
{`duration: ${trace.duration} ms`}
|
||||
{`duration: ${duration}`}
|
||||
</div>
|
||||
{(isExpanded || showFullMessage) && (
|
||||
<Button
|
||||
|
|
@ -97,15 +106,17 @@ const NestedNav: FC<RecursiveProps> = ({ trace, totalMsec }) => {
|
|||
)}
|
||||
</div>
|
||||
</div>
|
||||
{openLevels[trace.idValue] && <div>
|
||||
{hasChildren && trace.children.map((trace) => (
|
||||
<NestedNav
|
||||
key={trace.duration}
|
||||
trace={trace}
|
||||
totalMsec={totalMsec}
|
||||
/>
|
||||
))}
|
||||
</div>}
|
||||
{openLevels[trace.idValue] && (
|
||||
<div className="vm-nested-nav__childrens">
|
||||
{hasChildren && trace.children.map((trace) => (
|
||||
<NestedNav
|
||||
key={trace.duration}
|
||||
trace={trace}
|
||||
totalMsec={totalMsec}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,29 +1,71 @@
|
|||
@use "src/styles/variables" as *;
|
||||
|
||||
$color-base-nested-nav: $color-tropical-blue;
|
||||
$color-base-nested-nav-dark: $color-background-body;
|
||||
|
||||
.vm-nested-nav {
|
||||
margin-left: $padding-medium;
|
||||
position: relative;
|
||||
margin-left: $padding-global;
|
||||
border-radius: $border-radius-small;
|
||||
background-color: rgba($color-tropical-blue, 0.4);
|
||||
|
||||
&_dark &-header {
|
||||
background-color: $color-base-nested-nav-dark;
|
||||
|
||||
&:after, &:before {
|
||||
background-color: $color-base-nested-nav-dark;
|
||||
}
|
||||
|
||||
&:hover {
|
||||
box-shadow: rgba($color-white, 0.08) 0 0 0 1px;
|
||||
}
|
||||
}
|
||||
|
||||
&_mobile {
|
||||
margin-left: $padding-small;
|
||||
}
|
||||
|
||||
&_dark {
|
||||
background-color: rgba($color-black, 0.1);
|
||||
&_root > &-header {
|
||||
&:before,
|
||||
&:after {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
|
||||
&-header {
|
||||
position: relative;
|
||||
display: grid;
|
||||
grid-template-columns: auto 1fr;
|
||||
gap: $padding-small;
|
||||
padding: $padding-small;
|
||||
border-radius: $border-radius-small;
|
||||
transition: background-color 200ms ease-in-out;
|
||||
transition: box-shadow 200ms ease-in-out;
|
||||
cursor: pointer;
|
||||
background-color: rgba($color-base-nested-nav, 0.4);
|
||||
margin-bottom: $padding-small;
|
||||
z-index: 2;
|
||||
|
||||
&:after {
|
||||
content: "";
|
||||
position: absolute;
|
||||
top: calc(50% - 1px);
|
||||
height: 2px;
|
||||
width: $padding-small;
|
||||
background-color: $color-base-nested-nav;
|
||||
left: calc(-1 * $padding-small);
|
||||
}
|
||||
|
||||
&:before {
|
||||
content: "";
|
||||
position: absolute;
|
||||
bottom: 50%;
|
||||
left: calc($padding-global / -2);
|
||||
height: calc(50% + $padding-small);
|
||||
width: 2px;
|
||||
background-color: $color-base-nested-nav;
|
||||
}
|
||||
|
||||
&:hover {
|
||||
background-color: $color-hover-black;
|
||||
box-shadow: rgba($color-black, 0.08) 0 0 0 1px;
|
||||
}
|
||||
|
||||
&__icon {
|
||||
|
|
@ -32,9 +74,11 @@
|
|||
justify-content: center;
|
||||
width: 20px;
|
||||
transition: transform 200ms ease-in-out;
|
||||
transform: rotate(-90deg);
|
||||
color: $color-text-secondary;
|
||||
|
||||
&_open {
|
||||
transform: rotate(180deg);
|
||||
transform: rotate(0);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -72,4 +116,22 @@
|
|||
}
|
||||
}
|
||||
}
|
||||
|
||||
&__childrens > .vm-nested-nav:not(:last-child) {
|
||||
&:before {
|
||||
content: "";
|
||||
position: absolute;
|
||||
top: 0;
|
||||
left: calc($padding-global / -2);
|
||||
height: 100%;
|
||||
width: 2px;
|
||||
background-color: $color-base-nested-nav;
|
||||
}
|
||||
}
|
||||
|
||||
&__childrens > .vm-nested-nav_dark:not(:last-child) {
|
||||
&:before {
|
||||
background-color: $color-base-nested-nav-dark;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -89,6 +89,7 @@ const TracingsView: FC<TraceViewProps> = ({ traces, jsonEditor = false, onDelete
|
|||
})}
|
||||
>
|
||||
<NestedNav
|
||||
isRoot
|
||||
trace={trace}
|
||||
totalMsec={trace.duration}
|
||||
/>
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import { GraphSize, SeriesItemStats } from "../types";
|
||||
|
||||
export const MAX_QUERY_FIELDS = 4;
|
||||
export const MAX_QUERIES_HISTORY = 25;
|
||||
export const DEFAULT_MAX_SERIES = {
|
||||
table: 100,
|
||||
chart: 20,
|
||||
|
|
|
|||
|
|
@ -125,7 +125,7 @@ export const useFetchQuery = ({
|
|||
}
|
||||
|
||||
isHistogramResult = isDisplayChart && isHistogramData(resp.data.result);
|
||||
seriesLimit = isHistogramResult ? Infinity : Math.max(totalLength, defaultLimit);
|
||||
seriesLimit = isHistogramResult ? Infinity : defaultLimit;
|
||||
const freeTempSize = seriesLimit - tempData.length;
|
||||
resp.data.result.slice(0, freeTempSize).forEach((d: MetricBase) => {
|
||||
d.group = counter;
|
||||
|
|
@ -140,7 +140,7 @@ export const useFetchQuery = ({
|
|||
counter++;
|
||||
}
|
||||
|
||||
const limitText = `Showing ${seriesLimit} series out of ${totalLength} series due to performance reasons. Please narrow down the query, so it returns less series`;
|
||||
const limitText = `Showing ${tempData.length} series out of ${totalLength} series due to performance reasons. Please narrow down the query, so it returns less series`;
|
||||
setWarning(totalLength > seriesLimit ? limitText : "");
|
||||
isDisplayChart ? setGraphData(tempData as MetricResult[]) : setLiveData(tempData as InstantMetricResult[]);
|
||||
setTraces(tempTraces);
|
||||
|
|
|
|||
|
|
@ -11,8 +11,9 @@ import { useSearchParams } from "react-router-dom";
|
|||
import CardinalityTotals, { CardinalityTotalsProps } from "../CardinalityTotals/CardinalityTotals";
|
||||
import useSearchParamsFromObject from "../../../hooks/useSearchParamsFromObject";
|
||||
import useStateSearchParams from "../../../hooks/useStateSearchParams";
|
||||
import Hyperlink from "../../../components/Main/Hyperlink/Hyperlink";
|
||||
|
||||
const CardinalityConfigurator: FC<CardinalityTotalsProps> = ({ isPrometheus, ...props }) => {
|
||||
const CardinalityConfigurator: FC<CardinalityTotalsProps> = ({ isPrometheus, isCluster, ...props }) => {
|
||||
const { isMobile } = useDeviceDetect();
|
||||
const [searchParams] = useSearchParams();
|
||||
const { setSearchParamsFromKeys } = useSearchParamsFromObject();
|
||||
|
|
@ -105,19 +106,29 @@ const CardinalityConfigurator: FC<CardinalityTotalsProps> = ({ isPrometheus, ...
|
|||
<div className="vm-cardinality-configurator-bottom">
|
||||
<CardinalityTotals
|
||||
isPrometheus={isPrometheus}
|
||||
isCluster={isCluster}
|
||||
{...props}
|
||||
/>
|
||||
{isCluster &&
|
||||
<div className="vm-cardinality-configurator-bottom-helpful">
|
||||
<Hyperlink
|
||||
href="https://docs.victoriametrics.com/#cardinality-explorer-statistic-inaccurancy"
|
||||
withIcon={true}
|
||||
>
|
||||
<WikiIcon/>
|
||||
Statistic inaccuracy explanation
|
||||
</Hyperlink>
|
||||
</div>
|
||||
}
|
||||
|
||||
<div className="vm-cardinality-configurator-bottom-helpful">
|
||||
<a
|
||||
className="vm-link vm-link_with-icon"
|
||||
target="_blank"
|
||||
<Hyperlink
|
||||
href="https://docs.victoriametrics.com/#cardinality-explorer"
|
||||
rel="help noreferrer"
|
||||
withIcon={true}
|
||||
>
|
||||
<WikiIcon/>
|
||||
Documentation
|
||||
</a>
|
||||
</Hyperlink>
|
||||
</div>
|
||||
|
||||
<div className="vm-cardinality-configurator-bottom__execute">
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ export interface CardinalityTotalsProps {
|
|||
totalLabelValuePairs: number;
|
||||
seriesCountByMetricName: TopHeapEntry[];
|
||||
isPrometheus?: boolean;
|
||||
isCluster: boolean;
|
||||
}
|
||||
|
||||
const CardinalityTotals: FC<CardinalityTotalsProps> = ({
|
||||
|
|
@ -21,7 +22,7 @@ const CardinalityTotals: FC<CardinalityTotalsProps> = ({
|
|||
totalSeriesPrev = 0,
|
||||
totalSeriesAll = 0,
|
||||
seriesCountByMetricName = [],
|
||||
isPrometheus
|
||||
isPrometheus,
|
||||
}) => {
|
||||
const { isMobile } = useDeviceDetect();
|
||||
|
||||
|
|
@ -50,7 +51,7 @@ const CardinalityTotals: FC<CardinalityTotalsProps> = ({
|
|||
value: isNaN(progress) ? "-" : `${progress.toFixed(2)}%`,
|
||||
display: isMetric,
|
||||
info: "The share of these series in the total number of time series."
|
||||
}
|
||||
},
|
||||
].filter(t => t.display);
|
||||
|
||||
if (!totals.length) {
|
||||
|
|
|
|||
|
|
@ -7,12 +7,14 @@ import AppConfigurator from "../appConfigurator";
|
|||
import { useSearchParams } from "react-router-dom";
|
||||
import dayjs from "dayjs";
|
||||
import { DATE_FORMAT } from "../../../constants/date";
|
||||
import { getTenantIdFromUrl } from "../../../utils/tenants";
|
||||
|
||||
export const useFetchQuery = (): {
|
||||
fetchUrl?: string[],
|
||||
isLoading: boolean,
|
||||
error?: ErrorTypes | string
|
||||
appConfigurator: AppConfigurator,
|
||||
isCluster: boolean,
|
||||
} => {
|
||||
const appConfigurator = new AppConfigurator();
|
||||
|
||||
|
|
@ -26,6 +28,7 @@ export const useFetchQuery = (): {
|
|||
const [isLoading, setIsLoading] = useState(false);
|
||||
const [error, setError] = useState<ErrorTypes | string>();
|
||||
const [tsdbStatus, setTSDBStatus] = useState<TSDBStatus>(appConfigurator.defaultTSDBStatus);
|
||||
const [isCluster, setIsCluster] = useState<boolean>(false);
|
||||
|
||||
const getResponseJson = async (url: string) => {
|
||||
const response = await fetch(url);
|
||||
|
|
@ -115,6 +118,12 @@ export const useFetchQuery = (): {
|
|||
}
|
||||
}, [error]);
|
||||
|
||||
useEffect(() => {
|
||||
const id = getTenantIdFromUrl(serverUrl);
|
||||
setIsCluster(!!id);
|
||||
}, [serverUrl]);
|
||||
|
||||
|
||||
appConfigurator.tsdbStatusData = tsdbStatus;
|
||||
return { isLoading, appConfigurator: appConfigurator, error };
|
||||
return { isLoading, appConfigurator: appConfigurator, error, isCluster };
|
||||
};
|
||||
|
|
|
|||
|
|
@ -31,7 +31,7 @@ const CardinalityPanel: FC = () => {
|
|||
const match = searchParams.get("match") || "";
|
||||
const focusLabel = searchParams.get("focusLabel") || "";
|
||||
|
||||
const { isLoading, appConfigurator, error } = useFetchQuery();
|
||||
const { isLoading, appConfigurator, error, isCluster } = useFetchQuery();
|
||||
const { tsdbStatusData, getDefaultState, tablesHeaders, sectionsTips } = appConfigurator;
|
||||
const defaultState = getDefaultState(match, focusLabel);
|
||||
|
||||
|
|
@ -62,6 +62,7 @@ const CardinalityPanel: FC = () => {
|
|||
totalSeriesAll={tsdbStatusData.totalSeriesByAll}
|
||||
totalLabelValuePairs={tsdbStatusData.totalLabelValuePairs}
|
||||
seriesCountByMetricName={tsdbStatusData.seriesCountByMetricName}
|
||||
isCluster={isCluster}
|
||||
/>
|
||||
|
||||
{showTips && (
|
||||
|
|
@ -69,7 +70,7 @@ const CardinalityPanel: FC = () => {
|
|||
{!match && !focusLabel && <TipHighNumberOfSeries/>}
|
||||
{match && !focusLabel && <TipCardinalityOfSingle/>}
|
||||
{!match && !focusLabel && <TipHighNumberOfValues/>}
|
||||
{focusLabel && <TipCardinalityOfLabel/>}
|
||||
{focusLabel && <TipCardinalityOfLabel />}
|
||||
</div>
|
||||
)}
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import React, { FC, StateUpdater, useEffect, useState } from "preact/compat";
|
|||
import QueryEditor from "../../../components/Configurators/QueryEditor/QueryEditor";
|
||||
import AdditionalSettings from "../../../components/Configurators/AdditionalSettings/AdditionalSettings";
|
||||
import usePrevious from "../../../hooks/usePrevious";
|
||||
import { MAX_QUERY_FIELDS } from "../../../constants/graph";
|
||||
import { MAX_QUERIES_HISTORY, MAX_QUERY_FIELDS } from "../../../constants/graph";
|
||||
import { useQueryDispatch, useQueryState } from "../../../state/query/QueryStateContext";
|
||||
import { useTimeDispatch } from "../../../state/time/TimeStateContext";
|
||||
import {
|
||||
|
|
@ -22,7 +22,7 @@ import { arrayEquals } from "../../../utils/array";
|
|||
import useDeviceDetect from "../../../hooks/useDeviceDetect";
|
||||
import { QueryStats } from "../../../api/types";
|
||||
import { usePrettifyQuery } from "./hooks/usePrettifyQuery";
|
||||
import QueryHistoryList from "../QueryHistory/QueryHistoryList";
|
||||
import QueryHistory from "../QueryHistory/QueryHistory";
|
||||
|
||||
export interface QueryConfiguratorProps {
|
||||
queryErrors: string[];
|
||||
|
|
@ -66,7 +66,7 @@ const QueryConfigurator: FC<QueryConfiguratorProps> = ({
|
|||
const newValues = !queryEqual && q ? [...h.values, q] : h.values;
|
||||
|
||||
// limit the history
|
||||
if (newValues.length > 25) newValues.shift();
|
||||
if (newValues.length > MAX_QUERIES_HISTORY) newValues.shift();
|
||||
|
||||
return {
|
||||
index: h.values.length - Number(queryEqual),
|
||||
|
|
@ -243,10 +243,7 @@ const QueryConfigurator: FC<QueryConfiguratorProps> = ({
|
|||
<div className="vm-query-configurator-settings">
|
||||
<AdditionalSettings/>
|
||||
<div className="vm-query-configurator-settings__buttons">
|
||||
<QueryHistoryList
|
||||
history={queryHistory}
|
||||
handleSelectQuery={handleSelectHistory}
|
||||
/>
|
||||
<QueryHistory handleSelectQuery={handleSelectHistory}/>
|
||||
{stateQuery.length < MAX_QUERY_FIELDS && (
|
||||
<Button
|
||||
variant="outlined"
|
||||
|
|
|
|||
|
|
@ -0,0 +1,188 @@
|
|||
import React, { FC, useEffect, useMemo, useState } from "preact/compat";
|
||||
import Button from "../../../components/Main/Button/Button";
|
||||
import { ClockIcon, DeleteIcon } from "../../../components/Main/Icons";
|
||||
import Tooltip from "../../../components/Main/Tooltip/Tooltip";
|
||||
import useBoolean from "../../../hooks/useBoolean";
|
||||
import Modal from "../../../components/Main/Modal/Modal";
|
||||
import Tabs from "../../../components/Main/Tabs/Tabs";
|
||||
import useDeviceDetect from "../../../hooks/useDeviceDetect";
|
||||
import useEventListener from "../../../hooks/useEventListener";
|
||||
import { useQueryState } from "../../../state/query/QueryStateContext";
|
||||
import { getQueriesFromStorage } from "./utils";
|
||||
import QueryHistoryItem from "./QueryHistoryItem";
|
||||
import classNames from "classnames";
|
||||
import "./style.scss";
|
||||
import { saveToStorage } from "../../../utils/storage";
|
||||
import { arrayEquals } from "../../../utils/array";
|
||||
|
||||
interface Props {
|
||||
handleSelectQuery: (query: string, index: number) => void
|
||||
}
|
||||
|
||||
export const HistoryTabTypes = {
|
||||
session: "session",
|
||||
storage: "saved",
|
||||
favorite: "favorite",
|
||||
};
|
||||
|
||||
export const historyTabs = [
|
||||
{ label: "Session history", value: HistoryTabTypes.session },
|
||||
{ label: "Saved history", value: HistoryTabTypes.storage },
|
||||
{ label: "Favorite queries", value: HistoryTabTypes.favorite },
|
||||
];
|
||||
|
||||
const QueryHistory: FC<Props> = ({ handleSelectQuery }) => {
|
||||
const { queryHistory: historyState } = useQueryState();
|
||||
const { isMobile } = useDeviceDetect();
|
||||
|
||||
const {
|
||||
value: openModal,
|
||||
setTrue: handleOpenModal,
|
||||
setFalse: handleCloseModal,
|
||||
} = useBoolean(false);
|
||||
|
||||
const [activeTab, setActiveTab] = useState(historyTabs[0].value);
|
||||
const [historyStorage, setHistoryStorage] = useState(getQueriesFromStorage("QUERY_HISTORY"));
|
||||
const [historyFavorites, setHistoryFavorites] = useState(getQueriesFromStorage("QUERY_FAVORITES"));
|
||||
|
||||
const historySession = useMemo(() => {
|
||||
return historyState.map((h) => h.values.filter(q => q).reverse());
|
||||
}, [historyState]);
|
||||
|
||||
const list = useMemo(() => {
|
||||
switch (activeTab) {
|
||||
case HistoryTabTypes.favorite:
|
||||
return historyFavorites;
|
||||
case HistoryTabTypes.storage:
|
||||
return historyStorage;
|
||||
default:
|
||||
return historySession;
|
||||
}
|
||||
}, [activeTab, historyFavorites, historyStorage, historySession]);
|
||||
|
||||
const isNoData = list?.every(s => !s.length);
|
||||
|
||||
const noDataText = useMemo(() => {
|
||||
switch (activeTab) {
|
||||
case HistoryTabTypes.favorite:
|
||||
return "Favorites queries are empty.\nTo see your favorites, mark a query as a favorite.";
|
||||
default:
|
||||
return "Query history is empty.\nTo see the history, please make a query.";
|
||||
}
|
||||
}, [activeTab]);
|
||||
|
||||
const handleRunQuery = (group: number) => (value: string) => {
|
||||
handleSelectQuery(value, group);
|
||||
handleCloseModal();
|
||||
};
|
||||
|
||||
const handleToggleFavorite = (value: string, isFavorite: boolean) => {
|
||||
setHistoryFavorites((prev) => {
|
||||
const values = prev[0] || [];
|
||||
if (isFavorite) return [values.filter(v => v !== value)];
|
||||
if (!isFavorite && !values.includes(value)) return [[...values, value]];
|
||||
return prev;
|
||||
});
|
||||
};
|
||||
|
||||
const updateStageHistory = () => {
|
||||
setHistoryStorage(getQueriesFromStorage("QUERY_HISTORY"));
|
||||
setHistoryFavorites(getQueriesFromStorage("QUERY_FAVORITES"));
|
||||
};
|
||||
|
||||
const handleClearStorage = () => {
|
||||
saveToStorage("QUERY_HISTORY", "");
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
const nextValue = historyFavorites[0] || [];
|
||||
const prevValue = getQueriesFromStorage("QUERY_FAVORITES")[0] || [];
|
||||
const isEqual = arrayEquals(nextValue, prevValue);
|
||||
if (isEqual) return;
|
||||
saveToStorage("QUERY_FAVORITES", JSON.stringify(historyFavorites));
|
||||
}, [historyFavorites]);
|
||||
|
||||
useEventListener("storage", updateStageHistory);
|
||||
|
||||
return (
|
||||
<>
|
||||
<Tooltip title={"Show history"}>
|
||||
<Button
|
||||
color="primary"
|
||||
variant="text"
|
||||
onClick={handleOpenModal}
|
||||
startIcon={<ClockIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
|
||||
{openModal && (
|
||||
<Modal
|
||||
title={"Query history"}
|
||||
onClose={handleCloseModal}
|
||||
>
|
||||
<div
|
||||
className={classNames({
|
||||
"vm-query-history": true,
|
||||
"vm-query-history_mobile": isMobile,
|
||||
})}
|
||||
>
|
||||
<div
|
||||
className={classNames({
|
||||
"vm-query-history__tabs": true,
|
||||
"vm-section-header__tabs": true,
|
||||
"vm-query-history__tabs_mobile": isMobile,
|
||||
})}
|
||||
>
|
||||
<Tabs
|
||||
activeItem={activeTab}
|
||||
items={historyTabs}
|
||||
onChange={setActiveTab}
|
||||
/>
|
||||
</div>
|
||||
<div className="vm-query-history-list">
|
||||
{isNoData && <div className="vm-query-history-list__no-data">{noDataText}</div>}
|
||||
{list.map((queries, group) => (
|
||||
<div key={group}>
|
||||
{list.length > 1 && (
|
||||
<div
|
||||
className={classNames({
|
||||
"vm-query-history-list__group-title": true,
|
||||
"vm-query-history-list__group-title_first": group === 0,
|
||||
})}
|
||||
>
|
||||
Query {group + 1}
|
||||
</div>
|
||||
)}
|
||||
{queries.map((query, index) => (
|
||||
<QueryHistoryItem
|
||||
key={index}
|
||||
query={query}
|
||||
favorites={historyFavorites.flat()}
|
||||
onRun={handleRunQuery(group)}
|
||||
onToggleFavorite={handleToggleFavorite}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
))}
|
||||
{(activeTab === HistoryTabTypes.storage) && !isNoData && (
|
||||
<div className="vm-query-history-footer">
|
||||
<Button
|
||||
color="error"
|
||||
variant="outlined"
|
||||
size="small"
|
||||
startIcon={<DeleteIcon/>}
|
||||
onClick={handleClearStorage}
|
||||
>
|
||||
clear history
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</Modal>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
export default QueryHistory;
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
import React, { FC, useMemo } from "preact/compat";
|
||||
import Button from "../../../components/Main/Button/Button";
|
||||
import { CopyIcon, PlayCircleOutlineIcon, StarBorderIcon, StarIcon } from "../../../components/Main/Icons";
|
||||
import Tooltip from "../../../components/Main/Tooltip/Tooltip";
|
||||
import useCopyToClipboard from "../../../hooks/useCopyToClipboard";
|
||||
import "./style.scss";
|
||||
|
||||
interface Props {
|
||||
query: string;
|
||||
favorites: string[];
|
||||
onRun: (query: string) => void;
|
||||
onToggleFavorite: (query: string, isFavorite: boolean) => void;
|
||||
}
|
||||
|
||||
const QueryHistoryItem: FC<Props> = ({ query, favorites, onRun, onToggleFavorite }) => {
|
||||
const copyToClipboard = useCopyToClipboard();
|
||||
const isFavorite = useMemo(() => favorites.includes(query), [query, favorites]);
|
||||
|
||||
const handleCopyQuery = async () => {
|
||||
await copyToClipboard(query, "Query has been copied");
|
||||
};
|
||||
|
||||
const handleRunQuery = () => {
|
||||
onRun(query);
|
||||
};
|
||||
|
||||
const handleToggleFavorite = () => {
|
||||
onToggleFavorite(query, isFavorite);
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="vm-query-history-item">
|
||||
<span className="vm-query-history-item__value">{query}</span>
|
||||
<div className="vm-query-history-item__buttons">
|
||||
<Tooltip title={"Execute query"}>
|
||||
<Button
|
||||
size="small"
|
||||
variant="text"
|
||||
onClick={handleRunQuery}
|
||||
startIcon={<PlayCircleOutlineIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip title={"Copy query"}>
|
||||
<Button
|
||||
size="small"
|
||||
variant="text"
|
||||
onClick={handleCopyQuery}
|
||||
startIcon={<CopyIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip title={isFavorite ? "Remove Favorite" : "Add to Favorites"}>
|
||||
<Button
|
||||
size="small"
|
||||
variant="text"
|
||||
color={isFavorite ? "warning" : "primary"}
|
||||
onClick={handleToggleFavorite}
|
||||
startIcon={isFavorite ? <StarIcon/> : <StarBorderIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
export default QueryHistoryItem;
|
||||
|
|
@ -1,114 +0,0 @@
|
|||
import React, { FC, useMemo } from "preact/compat";
|
||||
import Button from "../../../components/Main/Button/Button";
|
||||
import { ClockIcon, CopyIcon, PlayCircleOutlineIcon } from "../../../components/Main/Icons";
|
||||
import Tooltip from "../../../components/Main/Tooltip/Tooltip";
|
||||
import { QueryHistory } from "../../../state/query/reducer";
|
||||
import useBoolean from "../../../hooks/useBoolean";
|
||||
import Modal from "../../../components/Main/Modal/Modal";
|
||||
import "./style.scss";
|
||||
import Tabs from "../../../components/Main/Tabs/Tabs";
|
||||
import { useState } from "react";
|
||||
import useCopyToClipboard from "../../../hooks/useCopyToClipboard";
|
||||
import useDeviceDetect from "../../../hooks/useDeviceDetect";
|
||||
import classNames from "classnames";
|
||||
|
||||
interface QueryHistoryProps {
|
||||
history: QueryHistory[];
|
||||
handleSelectQuery: (query: string, index: number) => void
|
||||
}
|
||||
|
||||
const QueryHistoryList: FC<QueryHistoryProps> = ({ history, handleSelectQuery }) => {
|
||||
const { isMobile } = useDeviceDetect();
|
||||
const copyToClipboard = useCopyToClipboard();
|
||||
const {
|
||||
value: openModal,
|
||||
setTrue: handleOpenModal,
|
||||
setFalse: handleCloseModal,
|
||||
} = useBoolean(false);
|
||||
|
||||
const [activeTab, setActiveTab] = useState("0");
|
||||
const tabs = useMemo(() => history.map((item, i) => ({
|
||||
value: `${i}`,
|
||||
label: `Query ${i+1}`,
|
||||
})), [history]);
|
||||
|
||||
const queries = useMemo(() => {
|
||||
const historyItem = history[+activeTab];
|
||||
return historyItem ? historyItem.values.filter(q => q).reverse() : [];
|
||||
}, [activeTab, history]);
|
||||
|
||||
const handleCopyQuery = (value: string) => async () => {
|
||||
await copyToClipboard(value, "Query has been copied");
|
||||
};
|
||||
|
||||
const handleRunQuery = (value: string, index: number) => () => {
|
||||
handleSelectQuery(value, index);
|
||||
handleCloseModal();
|
||||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<Tooltip title={"Show history"}>
|
||||
<Button
|
||||
color="primary"
|
||||
variant="text"
|
||||
onClick={handleOpenModal}
|
||||
startIcon={<ClockIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
|
||||
{openModal && (
|
||||
<Modal
|
||||
title={"Query history"}
|
||||
onClose={handleCloseModal}
|
||||
>
|
||||
<div className="vm-query-history">
|
||||
<div
|
||||
className={classNames({
|
||||
"vm-query-history__tabs": true,
|
||||
"vm-section-header__tabs": true,
|
||||
"vm-query-history__tabs_mobile": isMobile,
|
||||
})}
|
||||
>
|
||||
<Tabs
|
||||
activeItem={activeTab}
|
||||
items={tabs}
|
||||
onChange={setActiveTab}
|
||||
/>
|
||||
</div>
|
||||
<div className="vm-query-history-list">
|
||||
{queries.map((query, index) => (
|
||||
<div
|
||||
className="vm-query-history-list-item"
|
||||
key={index}
|
||||
>
|
||||
<span className="vm-query-history-list-item__value">{query}</span>
|
||||
<div className="vm-query-history-list-item__buttons">
|
||||
<Tooltip title={"Execute query"}>
|
||||
<Button
|
||||
size="small"
|
||||
variant="text"
|
||||
onClick={handleRunQuery(query, +activeTab)}
|
||||
startIcon={<PlayCircleOutlineIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip title={"Copy query"}>
|
||||
<Button
|
||||
size="small"
|
||||
variant="text"
|
||||
onClick={handleCopyQuery(query)}
|
||||
startIcon={<CopyIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</Modal>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
export default QueryHistoryList;
|
||||
|
|
@ -2,11 +2,17 @@
|
|||
|
||||
.vm-query-history {
|
||||
max-width: 80vw;
|
||||
min-width: 40vw;
|
||||
min-width: 500px;
|
||||
|
||||
&_mobile {
|
||||
max-width: 100vw;
|
||||
min-width: 100vw;
|
||||
}
|
||||
|
||||
&__tabs {
|
||||
margin: (-$padding-medium) (-$padding-medium) 0;
|
||||
padding: 0 $padding-medium;
|
||||
padding: 0 $padding-small;
|
||||
border-bottom: $border-divider;
|
||||
|
||||
&_mobile {
|
||||
margin: (-$padding-global) (-$padding-medium) 0;
|
||||
|
|
@ -17,29 +23,51 @@
|
|||
display: grid;
|
||||
align-items: flex-start;
|
||||
|
||||
&-item {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr auto;
|
||||
gap: $padding-small;
|
||||
align-items: center;
|
||||
&__group-title {
|
||||
font-weight: bold;
|
||||
margin: 0 (-$padding-medium) 0;
|
||||
padding: $padding-small calc($padding-medium + $padding-small);
|
||||
border-bottom: $border-divider;
|
||||
padding: $padding-medium $padding-global $padding-small;
|
||||
|
||||
&:first-child {
|
||||
border-top: $border-divider;
|
||||
}
|
||||
|
||||
&__value {
|
||||
white-space: pre-wrap;
|
||||
overflow-wrap: anywhere;
|
||||
font-family: $font-family-monospace;
|
||||
}
|
||||
|
||||
&__buttons {
|
||||
display: flex;
|
||||
gap: $padding-small;
|
||||
&_first {
|
||||
padding-top: $padding-global;
|
||||
}
|
||||
}
|
||||
|
||||
&__no-data {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: $padding-large $padding-global;
|
||||
color: $color-text-secondary;
|
||||
text-align: center;
|
||||
line-height: $font-size-large;
|
||||
white-space: pre-line;
|
||||
}
|
||||
}
|
||||
|
||||
&-item {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr auto;
|
||||
gap: $padding-small;
|
||||
align-items: center;
|
||||
margin: 0 (-$padding-medium) 0;
|
||||
padding: $padding-small $padding-medium;
|
||||
border-bottom: $border-divider;
|
||||
|
||||
&__value {
|
||||
white-space: pre-wrap;
|
||||
overflow-wrap: anywhere;
|
||||
font-family: $font-family-monospace;
|
||||
}
|
||||
|
||||
&__buttons {
|
||||
display: flex;
|
||||
}
|
||||
}
|
||||
|
||||
&-footer {
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
padding-top: $padding-medium;
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,27 @@
|
|||
import { getFromStorage, saveToStorage, StorageKeys } from "../../../utils/storage";
|
||||
import { QueryHistoryType } from "../../../state/query/reducer";
|
||||
import { MAX_QUERIES_HISTORY, MAX_QUERY_FIELDS } from "../../../constants/graph";
|
||||
|
||||
export const getQueriesFromStorage = (key: StorageKeys) => {
|
||||
const list = getFromStorage(key) as string;
|
||||
return list ? JSON.parse(list) as string[][] : [];
|
||||
};
|
||||
|
||||
export const setQueriesToStorage = (history: QueryHistoryType[]) => {
|
||||
// For localStorage, avoid splitting into query fields because when working from multiple tabs can cause confusion.
|
||||
// For convenience, we maintain the original structure of `string[][]`
|
||||
const lastValues = history.map(h => h.values[h.index]);
|
||||
const storageValues = getQueriesFromStorage("QUERY_HISTORY");
|
||||
if (!storageValues[0]) storageValues[0] = [];
|
||||
|
||||
const values = storageValues[0];
|
||||
const TOTAL_LIMIT = MAX_QUERIES_HISTORY * MAX_QUERY_FIELDS;
|
||||
|
||||
lastValues.forEach((v) => {
|
||||
const already = values.includes(v);
|
||||
if (!already && v) values.unshift(v);
|
||||
if (values.length > TOTAL_LIMIT) values.shift();
|
||||
});
|
||||
|
||||
saveToStorage("QUERY_HISTORY", JSON.stringify(storageValues));
|
||||
};
|
||||
|
|
@ -27,7 +27,7 @@ export const initialCustomPanelState: CustomPanelState = {
|
|||
displayType: (displayType?.value || "chart") as DisplayType,
|
||||
nocache: false,
|
||||
isTracingEnabled: false,
|
||||
seriesLimits: limitsStorage ? JSON.parse(getFromStorage("SERIES_LIMITS") as string) : DEFAULT_MAX_SERIES,
|
||||
seriesLimits: limitsStorage ? JSON.parse(limitsStorage) : DEFAULT_MAX_SERIES,
|
||||
tableCompact: getFromStorage("TABLE_COMPACT") as boolean || false,
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -1,22 +1,23 @@
|
|||
import { getFromStorage, saveToStorage } from "../../utils/storage";
|
||||
import { getQueryArray } from "../../utils/query-string";
|
||||
import { setQueriesToStorage } from "../../pages/CustomPanel/QueryHistory/utils";
|
||||
|
||||
export interface QueryHistory {
|
||||
export interface QueryHistoryType {
|
||||
index: number;
|
||||
values: string[];
|
||||
}
|
||||
|
||||
export interface QueryState {
|
||||
query: string[];
|
||||
queryHistory: QueryHistory[];
|
||||
queryHistory: QueryHistoryType[];
|
||||
autocomplete: boolean;
|
||||
|
||||
}
|
||||
|
||||
export type QueryAction =
|
||||
| { type: "SET_QUERY", payload: string[] }
|
||||
| { type: "SET_QUERY_HISTORY_BY_INDEX", payload: {value: QueryHistory, queryNumber: number} }
|
||||
| { type: "SET_QUERY_HISTORY", payload: QueryHistory[] }
|
||||
| { type: "SET_QUERY_HISTORY_BY_INDEX", payload: {value: QueryHistoryType, queryNumber: number} }
|
||||
| { type: "SET_QUERY_HISTORY", payload: QueryHistoryType[] }
|
||||
| { type: "TOGGLE_AUTOCOMPLETE"}
|
||||
|
||||
const query = getQueryArray();
|
||||
|
|
@ -34,6 +35,7 @@ export function reducer(state: QueryState, action: QueryAction): QueryState {
|
|||
query: action.payload.map(q => q)
|
||||
};
|
||||
case "SET_QUERY_HISTORY":
|
||||
setQueriesToStorage(action.payload);
|
||||
return {
|
||||
...state,
|
||||
queryHistory: action.payload
|
||||
|
|
|
|||
|
|
@ -1,7 +1,4 @@
|
|||
export type StorageKeys = "BASIC_AUTH_DATA"
|
||||
| "BEARER_AUTH_DATA"
|
||||
| "AUTH_TYPE"
|
||||
| "AUTOCOMPLETE"
|
||||
export type StorageKeys = "AUTOCOMPLETE"
|
||||
| "NO_CACHE"
|
||||
| "QUERY_TRACING"
|
||||
| "SERIES_LIMITS"
|
||||
|
|
@ -10,6 +7,8 @@ export type StorageKeys = "BASIC_AUTH_DATA"
|
|||
| "THEME"
|
||||
| "LOGS_LIMIT"
|
||||
| "EXPLORE_METRICS_TIPS"
|
||||
| "QUERY_HISTORY"
|
||||
| "QUERY_FAVORITES"
|
||||
|
||||
export const saveToStorage = (key: StorageKeys, value: string | boolean | Record<string, unknown>): void => {
|
||||
if (value) {
|
||||
|
|
@ -22,7 +21,7 @@ export const saveToStorage = (key: StorageKeys, value: string | boolean | Record
|
|||
};
|
||||
|
||||
// TODO: make this aware of data type that is stored
|
||||
export const getFromStorage = (key: StorageKeys): undefined | boolean | string | Record<string, unknown> => {
|
||||
export const getFromStorage = (key: StorageKeys): undefined | boolean | string | Record<string, unknown> => {
|
||||
const valueObj = window.localStorage.getItem(key);
|
||||
if (valueObj === null) {
|
||||
return undefined;
|
||||
|
|
@ -36,6 +35,3 @@ export const getFromStorage = (key: StorageKeys): undefined | boolean | string |
|
|||
};
|
||||
|
||||
export const removeFromStorage = (keys: StorageKeys[]): void => keys.forEach(k => window.localStorage.removeItem(k));
|
||||
|
||||
export const authKeys: StorageKeys[] = ["BASIC_AUTH_DATA", "BEARER_AUTH_DATA"];
|
||||
|
||||
|
|
|
|||
|
|
@ -30,6 +30,10 @@ const shortDurations = supportedDurations.map(d => d.short);
|
|||
|
||||
export const roundToMilliseconds = (num: number): number => Math.round(num*1000)/1000;
|
||||
|
||||
export const humanizeSeconds = (num: number): string => {
|
||||
return getDurationFromMilliseconds(dayjs.duration(num, "seconds").asMilliseconds());
|
||||
};
|
||||
|
||||
export const roundStep = (step: number) => {
|
||||
let result = roundToMilliseconds(step);
|
||||
const integerStep = Math.round(step);
|
||||
|
|
@ -46,8 +50,7 @@ export const roundStep = (step: number) => {
|
|||
if (step < 1 && step > 0.01) {
|
||||
result = Math.round(step * 40) / 40; // float to thousandths multiple of 5
|
||||
}
|
||||
|
||||
const humanize = getDurationFromMilliseconds(dayjs.duration(result || 0.001, "seconds").asMilliseconds());
|
||||
const humanize = humanizeSeconds(result || 0.001);
|
||||
return humanize.replace(/\s/g, "");
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -2,8 +2,8 @@
|
|||
|
||||
DOCKER_NAMESPACE ?= victoriametrics
|
||||
|
||||
ROOT_IMAGE ?= alpine:3.18.3
|
||||
CERTS_IMAGE := alpine:3.18.3
|
||||
ROOT_IMAGE ?= alpine:3.18.4
|
||||
CERTS_IMAGE := alpine:3.18.4
|
||||
|
||||
GO_BUILDER_IMAGE := golang:1.21.1-alpine
|
||||
BUILDER_IMAGE := local/builder:2.0.0-$(shell echo $(GO_BUILDER_IMAGE) | tr :/ __)-1
|
||||
|
|
|
|||
|
|
@ -42,30 +42,36 @@ The communication scheme between components is the following:
|
|||
and recording rules back to it;
|
||||
* [alertmanager](#alertmanager) is configured to receive notifications from `vmalert`.
|
||||
|
||||
To access `vmalert` use link [http://localhost:8428/vmalert](http://localhost:8428/vmalert/).
|
||||
To access Grafana use link [http://localhost:3000](http://localhost:3000).
|
||||
|
||||
To access [vmui](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui)
|
||||
use link [http://localhost:8428/vmui](http://localhost:8428/vmui).
|
||||
|
||||
To access `vmalert` use link [http://localhost:8428/vmalert](http://localhost:8428/vmalert/).
|
||||
|
||||
|
||||
## VictoriaMetrics cluster
|
||||
|
||||
VictoriaMetrics cluster environment consists of `vminsert`, `vmstorage` and `vmselect` components.
|
||||
`vmselect` has exposed port `:8481`, `vminsert` has exposed port `:8480` and the rest of components
|
||||
are available only inside the environment.
|
||||
`vminsert` has exposed port `:8480`, access to `vmselect` components goes through `vmauth` on port `:8427`,
|
||||
and the rest of components are available only inside the environment.
|
||||
|
||||
The communication scheme between components is the following:
|
||||
* [vmagent](#vmagent) sends scraped metrics to `vminsert`;
|
||||
* `vminsert` forwards data to `vmstorage`;
|
||||
* `vmselect` is connected to `vmstorage` for querying data;
|
||||
* [grafana](#grafana) is configured with datasource pointing to `vmselect`;
|
||||
* [vmalert](#vmalert) is configured to query `vmselect` and send alerts state
|
||||
* `vmselect`s are connected to `vmstorage` for querying data;
|
||||
* [vmauth](#vmauth) balances incoming read requests among `vmselect`s;
|
||||
* [grafana](#grafana) is configured with datasource pointing to `vmauth`;
|
||||
* [vmalert](#vmalert) is configured to query `vmselect`s via `vmauth` and send alerts state
|
||||
and recording rules to `vminsert`;
|
||||
* [alertmanager](#alertmanager) is configured to receive notifications from `vmalert`.
|
||||
|
||||
To access `vmalert` use link [http://localhost:8481/select/0/prometheus/vmalert](http://localhost:8481/select/0/prometheus/vmalert/).
|
||||
To access Grafana use link [http://localhost:3000](http://localhost:3000).
|
||||
|
||||
To access [vmui](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui)
|
||||
use link [http://localhost:8481/select/0/prometheus/vmui](http://localhost:8481/select/0/prometheus/vmui).
|
||||
To access [vmui](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui)
|
||||
use link [http://localhost:8427/select/0/prometheus/vmui/](http://localhost:8427/select/0/prometheus/vmui/).
|
||||
|
||||
To access `vmalert` use link [http://localhost:8427/select/0/prometheus/vmalert/](http://localhost:8427/select/0/prometheus/vmalert/).
|
||||
|
||||
## vmagent
|
||||
|
||||
|
|
@ -75,6 +81,13 @@ with listed targets for scraping.
|
|||
|
||||
[Web interface link](http://localhost:8429/).
|
||||
|
||||
## vmauth
|
||||
|
||||
[vmauth](https://docs.victoriametrics.com/vmauth.html) acts as a [balancer](https://docs.victoriametrics.com/vmauth.html#load-balancing)
|
||||
to spread the load across `vmselect`'s. [Grafana](#grafana) and [vmalert](#vmalert) use vmauth for read queries.
|
||||
vmauth config is available [here](ttps://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/auth-cluster.yml)
|
||||
|
||||
|
||||
## vmalert
|
||||
|
||||
vmalert evaluates alerting rules [alerts.yml](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/alerts.yml)
|
||||
|
|
|
|||
6
deployment/docker/auth-cluster.yml
Normal file
6
deployment/docker/auth-cluster.yml
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
# balance load among vmselects
|
||||
# see https://docs.victoriametrics.com/vmauth.html#load-balancing
|
||||
unauthorized_user:
|
||||
url_prefix:
|
||||
- http://vmselect-1:8481
|
||||
- http://vmselect-2:8481
|
||||
|
|
@ -2,7 +2,7 @@ version: '3.5'
|
|||
services:
|
||||
vmagent:
|
||||
container_name: vmagent
|
||||
image: victoriametrics/vmagent:v1.93.4
|
||||
image: victoriametrics/vmagent:v1.93.5
|
||||
depends_on:
|
||||
- "vminsert"
|
||||
ports:
|
||||
|
|
@ -19,7 +19,7 @@ services:
|
|||
container_name: grafana
|
||||
image: grafana/grafana:9.2.7
|
||||
depends_on:
|
||||
- "vmselect"
|
||||
- "vmauth"
|
||||
ports:
|
||||
- 3000:3000
|
||||
restart: always
|
||||
|
|
@ -32,7 +32,7 @@ services:
|
|||
|
||||
vmstorage-1:
|
||||
container_name: vmstorage-1
|
||||
image: victoriametrics/vmstorage:v1.93.4-cluster
|
||||
image: victoriametrics/vmstorage:v1.93.5-cluster
|
||||
ports:
|
||||
- 8482
|
||||
- 8400
|
||||
|
|
@ -44,7 +44,7 @@ services:
|
|||
restart: always
|
||||
vmstorage-2:
|
||||
container_name: vmstorage-2
|
||||
image: victoriametrics/vmstorage:v1.93.4-cluster
|
||||
image: victoriametrics/vmstorage:v1.93.5-cluster
|
||||
ports:
|
||||
- 8482
|
||||
- 8400
|
||||
|
|
@ -54,9 +54,10 @@ services:
|
|||
command:
|
||||
- '--storageDataPath=/storage'
|
||||
restart: always
|
||||
|
||||
vminsert:
|
||||
container_name: vminsert
|
||||
image: victoriametrics/vminsert:v1.93.4-cluster
|
||||
image: victoriametrics/vminsert:v1.93.5-cluster
|
||||
depends_on:
|
||||
- "vmstorage-1"
|
||||
- "vmstorage-2"
|
||||
|
|
@ -66,9 +67,10 @@ services:
|
|||
ports:
|
||||
- 8480:8480
|
||||
restart: always
|
||||
vmselect:
|
||||
container_name: vmselect
|
||||
image: victoriametrics/vmselect:v1.93.4-cluster
|
||||
|
||||
vmselect-1:
|
||||
container_name: vmselect-1
|
||||
image: victoriametrics/vmselect:v1.93.5-cluster
|
||||
depends_on:
|
||||
- "vmstorage-1"
|
||||
- "vmstorage-2"
|
||||
|
|
@ -77,14 +79,43 @@ services:
|
|||
- '--storageNode=vmstorage-2:8401'
|
||||
- '--vmalert.proxyURL=http://vmalert:8880'
|
||||
ports:
|
||||
- 8481:8481
|
||||
- 8481
|
||||
restart: always
|
||||
|
||||
vmselect-2:
|
||||
container_name: vmselect-2
|
||||
image: victoriametrics/vmselect:v1.93.5-cluster
|
||||
depends_on:
|
||||
- "vmstorage-1"
|
||||
- "vmstorage-2"
|
||||
command:
|
||||
- '--storageNode=vmstorage-1:8401'
|
||||
- '--storageNode=vmstorage-2:8401'
|
||||
- '--vmalert.proxyURL=http://vmalert:8880'
|
||||
ports:
|
||||
- 8481
|
||||
restart: always
|
||||
|
||||
vmauth:
|
||||
container_name: vmauth
|
||||
image: victoriametrics/vmauth:v1.93.5
|
||||
depends_on:
|
||||
- "vmselect-1"
|
||||
- "vmselect-2"
|
||||
volumes:
|
||||
- ./auth-cluster.yml:/etc/auth.yml
|
||||
# - /var/run/docker.sock:/var/run/docker.sock
|
||||
command:
|
||||
- '--auth.config=/etc/auth.yml'
|
||||
ports:
|
||||
- 8427:8427
|
||||
restart: always
|
||||
|
||||
vmalert:
|
||||
container_name: vmalert
|
||||
image: victoriametrics/vmalert:v1.93.4
|
||||
image: victoriametrics/vmalert:v1.93.5
|
||||
depends_on:
|
||||
- "vmselect"
|
||||
- "vmauth"
|
||||
ports:
|
||||
- 8880:8880
|
||||
volumes:
|
||||
|
|
@ -93,8 +124,8 @@ services:
|
|||
- ./alerts-vmagent.yml:/etc/alerts/alerts-vmagent.yml
|
||||
- ./alerts-vmalert.yml:/etc/alerts/alerts-vmalert.yml
|
||||
command:
|
||||
- '--datasource.url=http://vmselect:8481/select/0/prometheus'
|
||||
- '--remoteRead.url=http://vmselect:8481/select/0/prometheus'
|
||||
- '--datasource.url=http://vmauth:8427/select/0/prometheus'
|
||||
- '--remoteRead.url=http://vmauth:8427/select/0/prometheus'
|
||||
- '--remoteWrite.url=http://vminsert:8480/insert/0/prometheus'
|
||||
- '--notifier.url=http://alertmanager:9093/'
|
||||
- '--rule=/etc/alerts/*.yml'
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ version: "3.5"
|
|||
services:
|
||||
vmagent:
|
||||
container_name: vmagent
|
||||
image: victoriametrics/vmagent:v1.93.4
|
||||
image: victoriametrics/vmagent:v1.93.5
|
||||
depends_on:
|
||||
- "victoriametrics"
|
||||
ports:
|
||||
|
|
@ -18,7 +18,7 @@ services:
|
|||
restart: always
|
||||
victoriametrics:
|
||||
container_name: victoriametrics
|
||||
image: victoriametrics/victoria-metrics:v1.93.4
|
||||
image: victoriametrics/victoria-metrics:v1.93.5
|
||||
ports:
|
||||
- 8428:8428
|
||||
- 8089:8089
|
||||
|
|
@ -57,7 +57,7 @@ services:
|
|||
restart: always
|
||||
vmalert:
|
||||
container_name: vmalert
|
||||
image: victoriametrics/vmalert:v1.93.4
|
||||
image: victoriametrics/vmalert:v1.93.5
|
||||
depends_on:
|
||||
- "victoriametrics"
|
||||
- "alertmanager"
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ scrape_configs:
|
|||
- targets: ['vminsert:8480']
|
||||
- job_name: 'vmselect'
|
||||
static_configs:
|
||||
- targets: ['vmselect:8481']
|
||||
- targets: ['vmselect-1:8481', 'vmselect-2:8481']
|
||||
- job_name: 'vmstorage'
|
||||
static_configs:
|
||||
- targets: ['vmstorage-1:8482', 'vmstorage-2:8482']
|
||||
|
|
@ -10,5 +10,5 @@ datasources:
|
|||
- name: VictoriaMetrics - cluster
|
||||
type: prometheus
|
||||
access: proxy
|
||||
url: http://vmselect:8481/select/0/prometheus
|
||||
url: http://vmauth:8427/select/0/prometheus
|
||||
isDefault: false
|
||||
|
|
@ -105,7 +105,7 @@ services:
|
|||
- '--config=/config.yml'
|
||||
|
||||
vmsingle:
|
||||
image: victoriametrics/victoria-metrics:v1.93.4
|
||||
image: victoriametrics/victoria-metrics:v1.93.5
|
||||
ports:
|
||||
- '8428:8428'
|
||||
command:
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
4. Set variables `DIGITALOCEAN_API_TOKEN` with `VM_VERSION` for `packer` environment and run make from example below:
|
||||
|
||||
```console
|
||||
make release-victoria-metrics-digitalocean-oneclick-droplet DIGITALOCEAN_API_TOKEN="dop_v23_2e46f4759ceeeba0d0248" VM_VERSION="1.93.4"
|
||||
make release-victoria-metrics-digitalocean-oneclick-droplet DIGITALOCEAN_API_TOKEN="dop_v23_2e46f4759ceeeba0d0248" VM_VERSION="1.93.5"
|
||||
```
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -19,8 +19,8 @@ On the server:
|
|||
* VictoriaMetrics is running on ports: 8428, 8089, 4242, 2003 and they are bound to the local interface.
|
||||
|
||||
********************************************************************************
|
||||
# This image includes 1.93.4 version of VictoriaMetrics.
|
||||
# See Release notes https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.4
|
||||
# This image includes 1.93.5 version of VictoriaMetrics.
|
||||
# See Release notes https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.5
|
||||
|
||||
# Welcome to VictoriaMetrics droplet!
|
||||
|
||||
|
|
|
|||
|
|
@ -137,3 +137,5 @@ See also [case studies](https://docs.victoriametrics.com/CaseStudies.html).
|
|||
* [VictoriaMetrics Meetup December 2022](https://www.youtube.com/watch?v=Mesc6JBFNhQ). See also [slides for "VictoriaMetrics 2022: new features" talk](https://docs.google.com/presentation/d/1jI7XZoodmuzLymdu4MToG9onAKQjzCNwMO2NDupyUkQ/edit?usp=sharing).
|
||||
* [Comparing Thanos to VictoriaMetrics cluster](https://faun.pub/comparing-thanos-to-victoriametrics-cluster-b193bea1683)
|
||||
* [Evaluation performance and correctness: VictoriaMetrics response](https://valyala.medium.com/evaluating-performance-and-correctness-victoriametrics-response-e27315627e87)
|
||||
* [How to reduce expenses on monitoring slides](https://www.slideshare.net/RomanKhavronenko/how-to-reduce-expenses-on-monitoringpdf)
|
||||
* [Writing a TSDB from Scratch: Performance Optimization](https://www.youtube.com/watch?v=NdjuW98ep_w)
|
||||
|
|
|
|||
|
|
@ -21,9 +21,17 @@ The following `tip` changes can be tested by building VictoriaMetrics components
|
|||
* [How to build vmauth](https://docs.victoriametrics.com/vmauth.html#how-to-build-from-sources)
|
||||
* [How to build vmctl](https://docs.victoriametrics.com/vmctl.html#how-to-build)
|
||||
|
||||
Metrics of the latest version of VictoriaMetrics cluster are available for viewing at our
|
||||
[sandbox](https://play-grafana.victoriametrics.com/d/oS7Bi_0Wz_vm/victoriametrics-cluster-vm).
|
||||
The sandbox cluster installation is running under the constant load generated by
|
||||
[prometheus-benchmark](https://github.com/VictoriaMetrics/prometheus-benchmark) and used for testing latest releases.
|
||||
|
||||
## tip
|
||||
|
||||
## [v1.94.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.94.0)
|
||||
|
||||
Released at 2023-10-02
|
||||
|
||||
* FEATURE: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): add support for numbers with underscore delimiters such as `1_234_567_890` and `1.234_567_890`. These numbers are easier to read than `1234567890` and `1.234567890`.
|
||||
* FEATURE: [vmbackup](https://docs.victoriametrics.com/vmbackup.html): add support for server-side copy of existing backups. See [these docs](https://docs.victoriametrics.com/vmbackup.html#server-side-copy-of-the-existing-backup) for details.
|
||||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add the option to see the latest 25 queries. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4718).
|
||||
|
|
@ -37,6 +45,8 @@ The following `tip` changes can be tested by building VictoriaMetrics components
|
|||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add button for auto-formatting PromQL/MetricsQL queries. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4681). Thanks to @aramattamara for the [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4694).
|
||||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): improve accessibility score to 100 according to [Google's Lighthouse](https://developer.chrome.com/docs/lighthouse/accessibility/) tests.
|
||||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): organize `min`, `max`, `median` values on the chart legend and tooltips for better visibility.
|
||||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add explanation about [cardinality explorer](https://docs.victoriametrics.com/#cardinality-explorer) statistic inaccuracy in VictoriaMetrics cluster. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3070).
|
||||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add storage of query history in `localStorage`. See [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5022).
|
||||
* FEATURE: dashboards: provide copies of Grafana dashboards alternated with VictoriaMetrics datasource at [dashboards/vm](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/dashboards/vm).
|
||||
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth.html): added ability to set, override and clear request and response headers on a per-user and per-path basis. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4825) and [these docs](https://docs.victoriametrics.com/vmauth.html#auth-config) for details.
|
||||
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth.html): add ability to retry requests to the [remaining backends](https://docs.victoriametrics.com/vmauth.html#load-balancing) if they return response status codes specified in the `retry_status_codes` list. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893).
|
||||
|
|
@ -45,11 +55,20 @@ The following `tip` changes can be tested by building VictoriaMetrics components
|
|||
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): add `eval_offset` attribute for [Groups](https://docs.victoriametrics.com/vmalert.html#groups). If specified, Group will be evaluated at the exact time offset on the range of [0...evaluationInterval]. The setting might be useful for cron-like rules which must be evaluated at specific moments of time. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3409) for details.
|
||||
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): validate [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html) function names in alerting and recording rules when `vmalert` runs with `-dryRun` command-line flag. Previously it was allowed to use unknown (aka invalid) MetricsQL function names there. For example, `foo()` was counted as a valid query. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4933).
|
||||
* FEATURE: limit the length of string params in log messages to 500 chars. Longer string params are replaced with the `first_250_chars..last_250_chars`. This prevents from too long log lines, which can be emitted by VictoriaMetrics components.
|
||||
* FEATURE: [docker compose environment](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker): add `vmauth` component to cluster's docker-compose example for balancing load among multiple `vmselect` components.
|
||||
* FEATURE: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): make sure that `q2` series are returned after `q1` series in the results of `q1 or q2` query, in the same way as Prometheus does. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763).
|
||||
* FEATURE: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): return empty result from [`bitmap_and(a, b)`](https://docs.victoriametrics.com/MetricsQL.html#bitmap_and), [`bitmap_or(a, b)`](https://docs.victoriametrics.com/MetricsQL.html#bitmap_or) and [`bitmap_xor(a, b)`](https://docs.victoriametrics.com/MetricsQL.html#bitmap_xor) if `a` or `b` have no value at the particular timestamp. Previously `0` was returned in this case. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4996).
|
||||
* FEATURE: stop exposing `vm_merge_need_free_disk_space` metric, since it has been appeared that it confuses users while doesn't bring any useful information. See [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686#issuecomment-1733844128).
|
||||
|
||||
* BUGFIX: [Official Grafana dashboards for VictoriaMetrics](https://grafana.com/orgs/victoriametrics): fix display of ingested rows rate for `Samples ingested/s` and `Samples rate` panels for vmagent's dasbhoard. Previously, not all ingested protocols were accounted in these panels. An extra panel `Rows rate` was added to `Ingestion` section to display the split for rows ingested rate by protocol.
|
||||
* BUGFIX: [vmui](https://docs.victoriametrics.com/#vmui): fix the bug causing render looping when switching to heatmap.
|
||||
* BUGFIX: [VictoriaMetrics enterprise](https://docs.victoriametrics.com/enterprise.html) validate `-dedup.minScrapeInterval` value and `-downsampling.period` intervals are multiples of each other. See [these docs](https://docs.victoriametrics.com/#downsampling).
|
||||
* BUGFIX: [vmbackup](https://docs.victoriametrics.com/vmbackup.html): properly copy `appliedRetention.txt` files inside `<-storageDataPath>/{data}` folders during [incremental backups](https://docs.victoriametrics.com/vmbackup.html#incremental-backups). Previously the new `appliedRetention.txt` could be skipped during incremental backups, which could lead to increased load on storage after restoring from backup. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005).
|
||||
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent.html): suppress `context canceled` error messages in logs when `vmagent` is reloading service discovery config. This error could appear starting from [v1.93.5](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.5). See [this PR](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5048).
|
||||
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent.html): remove concurrency limit during parsing of scraped metrics, which was mistakenly applied to it. With this change cmd-line flag `-maxConcurrentInserts` won't have effect on scraping anymore.
|
||||
* BUGFIX: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): allow passing [median_over_time](https://docs.victoriametrics.com/MetricsQL.html#median_over_time) to [aggr_over_time](https://docs.victoriametrics.com/MetricsQL.html#aggr_over_time). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5034).
|
||||
* BUGFIX: [vminsert](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html): fix ingestion via [multitenant url](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multitenancy-via-labels) for opentsdbhttp. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5061). The bug has been introduced in [v1.93.2](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.2).
|
||||
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent.html): fix support of legacy DataDog agent, which adds trailing slashes to urls. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5078). Thanks to @maxb for spotting the issue.
|
||||
|
||||
## [v1.93.5](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.5)
|
||||
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ where you can chat with VictoriaMetrics users to get additional references, revi
|
|||
- [Brandwatch](#brandwatch)
|
||||
- [CERN](#cern)
|
||||
- [COLOPL](#colopl)
|
||||
- [Criteo](#criteo)
|
||||
- [Dig Security](#dig-security)
|
||||
- [Fly.io](#flyio)
|
||||
- [German Research Center for Artificial Intelligence](#german-research-center-for-artificial-intelligence)
|
||||
|
|
@ -242,6 +243,13 @@ after evaulating the following remote storage solutions for Prometheus:
|
|||
See [slides](https://speakerdeck.com/inletorder/monitoring-platform-with-victoria-metrics) and [video](https://www.youtube.com/watch?v=hUpHIluxw80)
|
||||
from `Large-scale, super-load system monitoring platform built with VictoriaMetrics` talk at [Prometheus Meetup Tokyo #3](https://prometheus.connpass.com/event/157721/).
|
||||
|
||||
## Criteo
|
||||
|
||||
[Criteo](https://www.criteo.com/) is a global technology company that helps marketers and media owners reach their goals through the world’s leading Commerce Media Platform.
|
||||
|
||||
See [this blog post](https://medium.com/criteo-engineering/victoriametrics-a-prometheus-remote-storage-solution-57081a3d8e61) on how Criteo started using VictoriaMetrics
|
||||
and why they prefer VictoriaMetrics over competing solutions.
|
||||
|
||||
## Dig Security
|
||||
|
||||
[Dig Security](https://www.dig.security) is a cloud data security startup with 50+ employees that provides real-time visibility, control, and protection of data assets.
|
||||
|
|
|
|||
|
|
@ -140,7 +140,7 @@ vmstorage-prod
|
|||
|
||||
### Development Builds
|
||||
|
||||
1. [Install go](https://golang.org/doc/install). The minimum supported version is Go 1.18.
|
||||
1. [Install go](https://golang.org/doc/install). The minimum supported version is Go 1.20.
|
||||
1. Run `make` from [the repository root](https://github.com/VictoriaMetrics/VictoriaMetrics). It should build `vmstorage`, `vmselect`
|
||||
and `vminsert` binaries and put them into the `bin` folder.
|
||||
|
||||
|
|
@ -821,15 +821,15 @@ Below is the output for `/path/to/vminsert -help`:
|
|||
-cacheExpireDuration duration
|
||||
Items are removed from in-memory caches after they aren't accessed for this duration. Lower values may reduce memory usage at the cost of higher CPU usage. See also -prevCacheRemovalPercent (default 30m0s)
|
||||
-cluster.tls
|
||||
Whether to use TLS for connections to -storageNode. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Whether to use TLS for connections to -storageNode. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsCAFile string
|
||||
Path to TLS CA file to use for verifying certificates provided by -storageNode if -cluster.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to TLS CA file to use for verifying certificates provided by -storageNode if -cluster.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsCertFile string
|
||||
Path to client-side TLS certificate file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to client-side TLS certificate file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsInsecureSkipVerify
|
||||
Whether to skip verification of TLS certificates provided by -storageNode nodes if -cluster.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in enterprise version of VictoriaMetrics
|
||||
Whether to skip verification of TLS certificates provided by -storageNode nodes if -cluster.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsKeyFile string
|
||||
Path to client-side TLS key file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to client-side TLS key file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternativeListenAddr string
|
||||
TCP address to listen for data from other vminsert nodes in multi-level cluster setup. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multi-level-cluster-setup . Usually :8400 should be set to match default vmstorage port for vminsert. Disabled work if empty
|
||||
-csvTrimTimestamp duration
|
||||
|
|
@ -937,7 +937,7 @@ Below is the output for `/path/to/vminsert -help`:
|
|||
-loggerWarnsPerSecondLimit int
|
||||
Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
|
||||
-maxConcurrentInserts int
|
||||
The maximum number of concurrent insert requests. The default value should work for most cases, since it minimizes memory usage. The default value can be increased when clients send data over slow networks. See also -insert.maxQueueDuration (default 8)
|
||||
The maximum number of concurrent insert requests. Default value should work for most cases, since it minimizes the memory usage. The default value can be increased when clients send data over slow networks. See also -insert.maxQueueDuration (default 8)
|
||||
-maxInsertRequestSize size
|
||||
The maximum size in bytes of a single Prometheus remote_write API request
|
||||
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 33554432)
|
||||
|
|
@ -1025,15 +1025,15 @@ Below is the output for `/path/to/vmselect -help`:
|
|||
-cacheExpireDuration duration
|
||||
Items are removed from in-memory caches after they aren't accessed for this duration. Lower values may reduce memory usage at the cost of higher CPU usage. See also -prevCacheRemovalPercent (default 30m0s)
|
||||
-cluster.tls
|
||||
Whether to use TLS for connections to -storageNode. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Whether to use TLS for connections to -storageNode. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsCAFile string
|
||||
Path to TLS CA file to use for verifying certificates provided by -storageNode if -cluster.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to TLS CA file to use for verifying certificates provided by -storageNode if -cluster.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsCertFile string
|
||||
Path to client-side TLS certificate file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to client-side TLS certificate file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsInsecureSkipVerify
|
||||
Whether to skip verification of TLS certificates provided by -storageNode nodes if -cluster.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in enterprise version of VictoriaMetrics
|
||||
Whether to skip verification of TLS certificates provided by -storageNode nodes if -cluster.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsKeyFile string
|
||||
Path to client-side TLS key file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to client-side TLS key file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternative.disableCompression
|
||||
Whether to disable compression of the data sent to vmselect via -clusternativeListenAddr. This reduces CPU usage at the cost of higher network bandwidth usage
|
||||
-clusternative.maxConcurrentRequests int
|
||||
|
|
@ -1047,18 +1047,18 @@ Below is the output for `/path/to/vmselect -help`:
|
|||
-clusternative.maxTagValues int
|
||||
The maximum number of tag values returned per search at -clusternativeListenAddr (default 100000)
|
||||
-clusternative.tls
|
||||
Whether to use TLS when accepting connections at -clusternativeListenAddr. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
|
||||
Whether to use TLS when accepting connections at -clusternativeListenAddr. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternative.tlsCAFile string
|
||||
Path to TLS CA file to use for verifying certificates provided by vmselect, which connects at -clusternativeListenAddr if -clusternative.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
|
||||
Path to TLS CA file to use for verifying certificates provided by vmselect, which connects at -clusternativeListenAddr if -clusternative.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternative.tlsCertFile string
|
||||
Path to server-side TLS certificate file to use when accepting connections at -clusternativeListenAddr if -clusternative.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
|
||||
Path to server-side TLS certificate file to use when accepting connections at -clusternativeListenAddr if -clusternative.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternative.tlsCipherSuites array
|
||||
Optional list of TLS cipher suites used for connections at -clusternativeListenAddr if -clusternative.tls flag is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants
|
||||
Optional list of TLS cipher suites used for connections at -clusternativeListenAddr if -clusternative.tls flag is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-clusternative.tlsInsecureSkipVerify
|
||||
Whether to skip verification of TLS certificates provided by vmselect, which connects to -clusternativeListenAddr if -clusternative.tls flag is set. Note that disabled TLS certificate verification breaks security
|
||||
Whether to skip verification of TLS certificates provided by vmselect, which connects to -clusternativeListenAddr if -clusternative.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternative.tlsKeyFile string
|
||||
Path to server-side TLS key file to use when accepting connections at -clusternativeListenAddr if -clusternative.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
|
||||
Path to server-side TLS key file to use when accepting connections at -clusternativeListenAddr if -clusternative.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternativeListenAddr string
|
||||
TCP address to listen for requests from other vmselect nodes in multi-level cluster setup. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multi-level-cluster-setup . Usually :8401 should be set to match default vmstorage port for vmselect. Disabled work if empty
|
||||
-dedup.minScrapeInterval duration
|
||||
|
|
@ -1367,7 +1367,7 @@ Below is the output for `/path/to/vmstorage -help`:
|
|||
-loggerWarnsPerSecondLimit int
|
||||
Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
|
||||
-maxConcurrentInserts int
|
||||
The maximum number of concurrent insert requests. The default value should work for most cases, since it minimizes memory usage. The default value can be increased when clients send data over slow networks. See also -insert.maxQueueDuration (default 8)
|
||||
The maximum number of concurrent insert requests. Default value should work for most cases, since it minimizes the memory usage. The default value can be increased when clients send data over slow networks. See also -insert.maxQueueDuration (default 8)
|
||||
-memory.allowedBytes size
|
||||
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
|
||||
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
|
||||
|
|
|
|||
|
|
@ -113,6 +113,7 @@ Case studies:
|
|||
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
|
||||
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
|
||||
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
|
||||
* [Criteo](https://docs.victoriametrics.com/CaseStudies.html#criteo)
|
||||
* [Dig Security](https://docs.victoriametrics.com/CaseStudies.html#dig-security)
|
||||
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
|
||||
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
|
||||
|
|
@ -367,6 +368,8 @@ See the [example VMUI at VictoriaMetrics playground](https://play.victoriametric
|
|||
* queries with the biggest average execution duration;
|
||||
* queries that took the most summary time for execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/top_queries` HTTP endpoint.
|
||||
|
||||
## Active queries
|
||||
|
||||
[VMUI](#vmui) provides `active queries` tab, which shows currently execute queries.
|
||||
|
|
@ -376,6 +379,8 @@ It provides the following information per each query:
|
|||
- The duration of the query execution.
|
||||
- The client address, who initiated the query execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/active_queries` HTTP endpoint.
|
||||
|
||||
## Metrics explorer
|
||||
|
||||
[VMUI](#vmui) provides an ability to explore metrics exported by a particular `job` / `instance` in the following way:
|
||||
|
|
@ -407,14 +412,16 @@ matching the specified [series selector](https://prometheus.io/docs/prometheus/l
|
|||
|
||||
Cardinality explorer is built on top of [/api/v1/status/tsdb](#tsdb-stats).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## Cardinality explorer statistic inaccuracy
|
||||
|
||||
In [cluster version of VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html) each vmstorage tracks the stored time series individually.
|
||||
vmselect requests stats via [/api/v1/status/tsdb](#tsdb-stats) API from each vmstorage node and merges the results by summing per-series stats.
|
||||
This may lead to inflated values when samples for the same time series are spread across multiple vmstorage nodes
|
||||
due to [replication](#replication) or [rerouting](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html?highlight=re-routes#cluster-availability).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## How to apply new config to VictoriaMetrics
|
||||
|
||||
VictoriaMetrics is configured via command-line flags, so it must be restarted when new command-line flags should be applied:
|
||||
|
|
@ -619,6 +626,28 @@ Some plugins for Telegraf such as [fluentd](https://github.com/fangli/fluent-plu
|
|||
or [Juniper/jitmon](https://github.com/Juniper/jtimon) send `SHOW DATABASES` query to `/query` and expect a particular database name in the response.
|
||||
Comma-separated list of expected databases can be passed to VictoriaMetrics via `-influx.databaseNames` command-line flag.
|
||||
|
||||
### How to send data in InfluxDB v2 format
|
||||
|
||||
VictoriaMetrics exposes endpoint for InfluxDB v2 HTTP API at `/influx/api/v2/write` and `/api/v2/write`.
|
||||
|
||||
|
||||
In order to write data with InfluxDB line protocol to local VictoriaMetrics using `curl`:
|
||||
|
||||
<div class="with-copy" markdown="1">
|
||||
|
||||
```console
|
||||
curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/api/v2/write'
|
||||
```
|
||||
|
||||
</div>
|
||||
|
||||
The `/api/v1/export` endpoint should return the following response:
|
||||
|
||||
```json
|
||||
{"metric":{"__name__":"measurement_field1","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1695902762311]}
|
||||
{"metric":{"__name__":"measurement_field2","tag1":"value1","tag2":"value2"},"values":[1.23],"timestamps":[1695902762311]}
|
||||
```
|
||||
|
||||
## How to send data from Graphite-compatible agents such as [StatsD](https://github.com/etsy/statsd)
|
||||
|
||||
Enable Graphite receiver in VictoriaMetrics by setting `-graphiteListenAddr` command line flag. For instance,
|
||||
|
|
@ -833,7 +862,7 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
* `/api/v1/series/count` - returns the total number of time series in the database. Some notes:
|
||||
* the handler scans all the inverted index, so it can be slow if the database contains tens of millions of time series;
|
||||
* the handler may count [deleted time series](#how-to-delete-time-series) additionally to normal time series due to internal implementation restrictions;
|
||||
* `/api/v1/status/active_queries` - returns a list of currently running queries.
|
||||
* `/api/v1/status/active_queries` - returns the list of currently running queries. This list is also available at [`active queries` page at VMUI](#active-queries).
|
||||
* `/api/v1/status/top_queries` - returns the following query lists:
|
||||
* the most frequently executed queries - `topByCount`
|
||||
* queries with the biggest average execution duration - `topByAvgDuration`
|
||||
|
|
@ -843,6 +872,8 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
For example, request to `/api/v1/status/top_queries?topN=5&maxLifetime=30s` would return up to 5 queries per list, which were executed during the last 30 seconds.
|
||||
VictoriaMetrics tracks the last `-search.queryStats.lastQueriesCount` queries with durations at least `-search.queryStats.minQueryDuration`.
|
||||
|
||||
See also [`top queries` page at VMUI](#top-queries).
|
||||
|
||||
### Timestamp formats
|
||||
|
||||
VictoriaMetrics accepts the following formats for `time`, `start` and `end` query args
|
||||
|
|
@ -1657,9 +1688,10 @@ See [these docs](https://docs.victoriametrics.com/guides/guide-vmcluster-multipl
|
|||
which allow configuring multiple retentions for distinct sets of time series matching the configured [series filters](https://docs.victoriametrics.com/keyConcepts.html#filtering)
|
||||
via `-retentionFilter` command-line flag. This flag accepts `filter:duration` options, where `filter` must be
|
||||
a valid [series filter](https://docs.victoriametrics.com/keyConcepts.html#filtering), while the `duration`
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`. If series doesn't match
|
||||
any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention) command-line flag is applied to it.
|
||||
If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`.
|
||||
The `duration` of the `-retentionFilter` must be lower or equal to [-retentionPeriod](#retention) flag value.
|
||||
If series doesn't match any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention)
|
||||
command-line flag is applied to it. If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
|
||||
For example, the following config sets 3 days retention for time series with `team="juniors"` label,
|
||||
30 days retention for time series with `env="dev"` or `env="staging"` label and 1 year retention for the remaining time series:
|
||||
|
|
@ -1793,9 +1825,9 @@ Graphs on the dashboards contain useful hints - hover the `i` icon in the top le
|
|||
We recommend setting up [alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#alerts)
|
||||
via [vmalert](https://docs.victoriametrics.com/vmalert.html) or via Prometheus.
|
||||
|
||||
VictoriaMetrics exposes currently running queries and their execution times at `/api/v1/status/active_queries` page.
|
||||
VictoriaMetrics exposes currently running queries and their execution times at [`active queries` page](#active-queries).
|
||||
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at `/api/v1/status/top_queries` page.
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at [`top queries` page](#top-queries).
|
||||
|
||||
See also [VictoriaMetrics Monitoring](https://victoriametrics.com/blog/victoriametrics-monitoring/)
|
||||
and [troubleshooting docs](https://docs.victoriametrics.com/Troubleshooting.html).
|
||||
|
|
@ -1940,9 +1972,6 @@ and [cardinality explorer docs](#cardinality-explorer).
|
|||
has at least 20% of free space. The remaining amount of free space
|
||||
can be [monitored](#monitoring) via `vm_free_disk_space_bytes` metric. The total size of data
|
||||
stored on the disk can be monitored via sum of `vm_data_size_bytes` metrics.
|
||||
See also `vm_merge_need_free_disk_space` metrics, which are set to values higher than 0
|
||||
if background merge cannot be initiated due to free disk space shortage. The value shows the number of per-month partitions,
|
||||
which would start background merge if they had more free disk space.
|
||||
|
||||
* VictoriaMetrics buffers incoming data in memory for up to a few seconds before flushing it to persistent storage.
|
||||
This may lead to the following "issues":
|
||||
|
|
|
|||
|
|
@ -121,6 +121,7 @@ Case studies:
|
|||
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
|
||||
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
|
||||
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
|
||||
* [Criteo](https://docs.victoriametrics.com/CaseStudies.html#criteo)
|
||||
* [Dig Security](https://docs.victoriametrics.com/CaseStudies.html#dig-security)
|
||||
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
|
||||
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
|
||||
|
|
@ -375,6 +376,8 @@ See the [example VMUI at VictoriaMetrics playground](https://play.victoriametric
|
|||
* queries with the biggest average execution duration;
|
||||
* queries that took the most summary time for execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/top_queries` HTTP endpoint.
|
||||
|
||||
## Active queries
|
||||
|
||||
[VMUI](#vmui) provides `active queries` tab, which shows currently execute queries.
|
||||
|
|
@ -384,6 +387,8 @@ It provides the following information per each query:
|
|||
- The duration of the query execution.
|
||||
- The client address, who initiated the query execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/active_queries` HTTP endpoint.
|
||||
|
||||
## Metrics explorer
|
||||
|
||||
[VMUI](#vmui) provides an ability to explore metrics exported by a particular `job` / `instance` in the following way:
|
||||
|
|
@ -415,14 +420,16 @@ matching the specified [series selector](https://prometheus.io/docs/prometheus/l
|
|||
|
||||
Cardinality explorer is built on top of [/api/v1/status/tsdb](#tsdb-stats).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## Cardinality explorer statistic inaccuracy
|
||||
|
||||
In [cluster version of VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html) each vmstorage tracks the stored time series individually.
|
||||
vmselect requests stats via [/api/v1/status/tsdb](#tsdb-stats) API from each vmstorage node and merges the results by summing per-series stats.
|
||||
This may lead to inflated values when samples for the same time series are spread across multiple vmstorage nodes
|
||||
due to [replication](#replication) or [rerouting](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html?highlight=re-routes#cluster-availability).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## How to apply new config to VictoriaMetrics
|
||||
|
||||
VictoriaMetrics is configured via command-line flags, so it must be restarted when new command-line flags should be applied:
|
||||
|
|
@ -627,6 +634,28 @@ Some plugins for Telegraf such as [fluentd](https://github.com/fangli/fluent-plu
|
|||
or [Juniper/jitmon](https://github.com/Juniper/jtimon) send `SHOW DATABASES` query to `/query` and expect a particular database name in the response.
|
||||
Comma-separated list of expected databases can be passed to VictoriaMetrics via `-influx.databaseNames` command-line flag.
|
||||
|
||||
### How to send data in InfluxDB v2 format
|
||||
|
||||
VictoriaMetrics exposes endpoint for InfluxDB v2 HTTP API at `/influx/api/v2/write` and `/api/v2/write`.
|
||||
|
||||
|
||||
In order to write data with InfluxDB line protocol to local VictoriaMetrics using `curl`:
|
||||
|
||||
<div class="with-copy" markdown="1">
|
||||
|
||||
```console
|
||||
curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/api/v2/write'
|
||||
```
|
||||
|
||||
</div>
|
||||
|
||||
The `/api/v1/export` endpoint should return the following response:
|
||||
|
||||
```json
|
||||
{"metric":{"__name__":"measurement_field1","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1695902762311]}
|
||||
{"metric":{"__name__":"measurement_field2","tag1":"value1","tag2":"value2"},"values":[1.23],"timestamps":[1695902762311]}
|
||||
```
|
||||
|
||||
## How to send data from Graphite-compatible agents such as [StatsD](https://github.com/etsy/statsd)
|
||||
|
||||
Enable Graphite receiver in VictoriaMetrics by setting `-graphiteListenAddr` command line flag. For instance,
|
||||
|
|
@ -841,7 +870,7 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
* `/api/v1/series/count` - returns the total number of time series in the database. Some notes:
|
||||
* the handler scans all the inverted index, so it can be slow if the database contains tens of millions of time series;
|
||||
* the handler may count [deleted time series](#how-to-delete-time-series) additionally to normal time series due to internal implementation restrictions;
|
||||
* `/api/v1/status/active_queries` - returns a list of currently running queries.
|
||||
* `/api/v1/status/active_queries` - returns the list of currently running queries. This list is also available at [`active queries` page at VMUI](#active-queries).
|
||||
* `/api/v1/status/top_queries` - returns the following query lists:
|
||||
* the most frequently executed queries - `topByCount`
|
||||
* queries with the biggest average execution duration - `topByAvgDuration`
|
||||
|
|
@ -851,6 +880,8 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
For example, request to `/api/v1/status/top_queries?topN=5&maxLifetime=30s` would return up to 5 queries per list, which were executed during the last 30 seconds.
|
||||
VictoriaMetrics tracks the last `-search.queryStats.lastQueriesCount` queries with durations at least `-search.queryStats.minQueryDuration`.
|
||||
|
||||
See also [`top queries` page at VMUI](#top-queries).
|
||||
|
||||
### Timestamp formats
|
||||
|
||||
VictoriaMetrics accepts the following formats for `time`, `start` and `end` query args
|
||||
|
|
@ -1665,9 +1696,10 @@ See [these docs](https://docs.victoriametrics.com/guides/guide-vmcluster-multipl
|
|||
which allow configuring multiple retentions for distinct sets of time series matching the configured [series filters](https://docs.victoriametrics.com/keyConcepts.html#filtering)
|
||||
via `-retentionFilter` command-line flag. This flag accepts `filter:duration` options, where `filter` must be
|
||||
a valid [series filter](https://docs.victoriametrics.com/keyConcepts.html#filtering), while the `duration`
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`. If series doesn't match
|
||||
any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention) command-line flag is applied to it.
|
||||
If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`.
|
||||
The `duration` of the `-retentionFilter` must be lower or equal to [-retentionPeriod](#retention) flag value.
|
||||
If series doesn't match any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention)
|
||||
command-line flag is applied to it. If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
|
||||
For example, the following config sets 3 days retention for time series with `team="juniors"` label,
|
||||
30 days retention for time series with `env="dev"` or `env="staging"` label and 1 year retention for the remaining time series:
|
||||
|
|
@ -1801,9 +1833,9 @@ Graphs on the dashboards contain useful hints - hover the `i` icon in the top le
|
|||
We recommend setting up [alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#alerts)
|
||||
via [vmalert](https://docs.victoriametrics.com/vmalert.html) or via Prometheus.
|
||||
|
||||
VictoriaMetrics exposes currently running queries and their execution times at `/api/v1/status/active_queries` page.
|
||||
VictoriaMetrics exposes currently running queries and their execution times at [`active queries` page](#active-queries).
|
||||
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at `/api/v1/status/top_queries` page.
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at [`top queries` page](#top-queries).
|
||||
|
||||
See also [VictoriaMetrics Monitoring](https://victoriametrics.com/blog/victoriametrics-monitoring/)
|
||||
and [troubleshooting docs](https://docs.victoriametrics.com/Troubleshooting.html).
|
||||
|
|
@ -1948,9 +1980,6 @@ and [cardinality explorer docs](#cardinality-explorer).
|
|||
has at least 20% of free space. The remaining amount of free space
|
||||
can be [monitored](#monitoring) via `vm_free_disk_space_bytes` metric. The total size of data
|
||||
stored on the disk can be monitored via sum of `vm_data_size_bytes` metrics.
|
||||
See also `vm_merge_need_free_disk_space` metrics, which are set to values higher than 0
|
||||
if background merge cannot be initiated due to free disk space shortage. The value shows the number of per-month partitions,
|
||||
which would start background merge if they had more free disk space.
|
||||
|
||||
* VictoriaMetrics buffers incoming data in memory for up to a few seconds before flushing it to persistent storage.
|
||||
This may lead to the following "issues":
|
||||
|
|
|
|||
|
|
@ -296,29 +296,40 @@ There are the following most commons reasons for slow data ingestion in Victoria
|
|||
Some queries may take more time and resources (CPU, RAM, network bandwidth) than others.
|
||||
VictoriaMetrics logs slow queries if their execution time exceeds the duration passed
|
||||
to `-search.logSlowQueryDuration` command-line flag (5s by default).
|
||||
VictoriaMetrics also provides `/api/v1/status/top_queries` endpoint, which returns
|
||||
queries that took the most time to execute.
|
||||
See [these docs](https://docs.victoriametrics.com/#prometheus-querying-api-enhancements) for details.
|
||||
|
||||
There are the following solutions exist for slow queries:
|
||||
VictoriaMetrics provides [`top queries` page at VMUI](https://docs.victoriametrics.com/#top-queries), which shows
|
||||
queries that took the most time to execute.
|
||||
|
||||
There are the following solutions exist for improving performance of slow queries:
|
||||
|
||||
- Adding more CPU and memory to VictoriaMetrics, so it may perform the slow query faster.
|
||||
If you use cluster version of VictoriaMetrics, then migration of `vmselect` nodes to machines
|
||||
If you use cluster version of VictoriaMetrics, then migrating `vmselect` nodes to machines
|
||||
with more CPU and RAM should help improving speed for slow queries. Query performance
|
||||
is always limited by resources of one vmselect which processes the query. For example, if 2vCPU cores on `vmselect`
|
||||
is always limited by resources of one `vmselect` which processes the query. For example, if 2vCPU cores on `vmselect`
|
||||
isn't enough to process query fast enough, then migrating `vmselect` to a machine with 4vCPU cores should increase heavy query performance by up to 2x.
|
||||
If the line on `Concurrent select` graph form the [official Grafana dashboard for VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#monitoring)
|
||||
If the line on `concurrent select` graph form the [official Grafana dashboard for VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#monitoring)
|
||||
is close to the limit, then prefer adding more `vmselect` nodes to the cluster.
|
||||
Sometimes adding more `vmstorage` nodes also can help improving the speed for slow queries.
|
||||
|
||||
- Rewriting slow queries, so they become faster. Unfortunately it is hard determining
|
||||
whether the given query is slow by just looking at it.
|
||||
VictoriaMetrics provides [query tracing](https://docs.victoriametrics.com/#query-tracing) feature,
|
||||
which can help determine the source of slow query.
|
||||
See also [this article](https://valyala.medium.com/how-to-optimize-promql-and-metricsql-queries-85a1b75bf986),
|
||||
which explains how to determine and optimize slow queries.
|
||||
|
||||
In practice many slow queries are generated because of improper use of [subqueries](https://docs.victoriametrics.com/MetricsQL.html#subqueries).
|
||||
The main source of slow queries in practice is [alerting and recording rules](https://docs.victoriametrics.com/vmalert.html#rules)
|
||||
with long lookbehind windows in square brackets. These queries are frequently used in SLI/SLO calculations such as [Sloth](https://github.com/slok/sloth).
|
||||
|
||||
For example, `avg_over_time(up[30d]) > 0.99` needs to read and process
|
||||
all the [raw samples](https://docs.victoriametrics.com/keyConcepts.html#raw-samples)
|
||||
for `up` [time series](https://docs.victoriametrics.com/keyConcepts.html#time-series) over the last 30 days
|
||||
each time it executes. If this query is executed frequently, then it can take significant share of CPU, disk read IO, network bandwidth and RAM.
|
||||
Such queries can be optimized in the following ways:
|
||||
|
||||
- To reduce the lookbehind window in square brackets. For example, `avg_over_time(up[10d])` takes up to 3x less compute resources
|
||||
than `avg_over_time(up[30d])` at VictoriaMetrics.
|
||||
- To increase evaluation interval for alerting and recording rules, so they are executed less frequently.
|
||||
For example, increasing `-evaluationInterval` command-line flag value at [vmalert](https://docs.victoriametrics.com/vmalert.html)
|
||||
from `1m` to `2m` should reduce compute resource usage at VictoriaMetrics by 2x.
|
||||
|
||||
Another source of slow queries is improper use of [subqueries](https://docs.victoriametrics.com/MetricsQL.html#subqueries).
|
||||
It is recommended avoiding subqueries if you don't understand clearly how they work.
|
||||
It is easy to create a subquery without knowing about it.
|
||||
For example, `rate(sum(some_metric))` is implicitly transformed into the following subquery
|
||||
|
|
@ -335,6 +346,11 @@ There are the following solutions exist for slow queries:
|
|||
It is likely this query won't return the expected results. Instead, `sum(rate(some_metric))` must be used instead.
|
||||
See [this article](https://www.robustperception.io/rate-then-sum-never-sum-then-rate/) for more details.
|
||||
|
||||
VictoriaMetrics provides [query tracing](https://docs.victoriametrics.com/#query-tracing) feature,
|
||||
which can help determining the source of slow query.
|
||||
See also [this article](https://valyala.medium.com/how-to-optimize-promql-and-metricsql-queries-85a1b75bf986),
|
||||
which explains how to determine and optimize slow queries.
|
||||
|
||||
|
||||
## Out of memory errors
|
||||
|
||||
|
|
|
|||
|
|
@ -11,9 +11,11 @@ according to [these docs](https://docs.victoriametrics.com/VictoriaLogs/QuickSta
|
|||
* `vl_data_size_bytes{type="indexdb"}` - on-disk size for [log stream](https://docs.victoriametrics.com/VictoriaLogs/keyConcepts.html#stream-fields) indexes.
|
||||
* FEATURE: add `-insert.maxFieldsPerLine` command-line flag, which can be used for limiting the number of fields per line in logs sent to VictoriaLogs via ingestion protocols. This helps to avoid issues like [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762).
|
||||
* FEATURE: expose `vl_http_request_duration_seconds` histogram at the [/metrics](https://docs.victoriametrics.com/VictoriaLogs/#monitoring) page. Thanks to @crossoverJie for [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4934).
|
||||
* FEATURE: add support of `-storage.minFreeDiskSpaceBytes` command-line flag to allow switching to read-only mode when running out of disk space at `-storageDataPath`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4737).
|
||||
|
||||
* BUGFIX: fix possible panic when no data is written to VictoriaLogs for a long time. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4895). Thanks to @crossoverJie for filing and fixing the issue.
|
||||
* BUGFIX: add `/insert/loky/ready` endpoint, which is used by Promtail for healthchecks. This should remove `unsupported path requested: /insert/loki/ready` warning logs. See [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762#issuecomment-1690966722).
|
||||
* BUGFIX: prevent from panic during background merge when the number of columns in the resulting block exceeds the maximum allowed number of columns per block. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762).
|
||||
|
||||
## [v0.3.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.3.0-victorialogs)
|
||||
|
||||
|
|
|
|||
|
|
@ -239,6 +239,9 @@ Pass `-help` to VictoriaLogs in order to see the list of supported command-line
|
|||
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 1048576)
|
||||
-storageDataPath string
|
||||
Path to directory with the VictoriaLogs data; see https://docs.victoriametrics.com/VictoriaLogs/#storage (default "victoria-logs-data")
|
||||
-storage.minFreeDiskSpaceBytes size
|
||||
The minimum free disk space at -storageDataPath after which the storage stops accepting new data
|
||||
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 10000000)
|
||||
-tls
|
||||
Whether to enable TLS for incoming HTTP requests at -httpListenAddr (aka https). -tlsCertFile and -tlsKeyFile must be set if -tls is set
|
||||
-tlsCertFile string
|
||||
|
|
|
|||
|
|
@ -1,14 +1,22 @@
|
|||
---
|
||||
sort: 10
|
||||
weight: 10
|
||||
title: CHANGELOG
|
||||
---
|
||||
|
||||
# CHANGELOG
|
||||
|
||||
## Next release
|
||||
|
||||
### Features
|
||||
|
||||
- [vmoperator](https://docs.victoriametrics.com/operator/): upgrade vmagent/vmauth's default config-reloader image.
|
||||
- [vmoperator](./README.md): upgrade vmagent/vmauth's default config-reloader image.
|
||||
|
||||
### Fixes
|
||||
|
||||
- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): remove redundant annotation `operator.victoriametrics/last-applied-spec` from created workloads like vmstorage statefulset.
|
||||
- [vmcluster](./api.html#vmcluster): remove redundant annotation `operator.victoriametrics/last-applied-spec` from created workloads like vmstorage statefulset.
|
||||
- [vmoperator](./README.md): properly resize statefulset's multiple pvc when needed and allowable, before they could be updated with wrong size.
|
||||
- [vmoperator](./README.md): fix wrong api group of endpointsices, before vmagent won't able to access endpointsices resources with default rbac rule.
|
||||
|
||||
<a name="v0.38.0"></a>
|
||||
## [v0.38.0](https://github.com/VictoriaMetrics/operator/releases/tag/v0.38.0) - 11 Sep 2023
|
||||
|
|
@ -17,13 +25,13 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): [Enterprise] fixes ip_filters indent for url_prefix. Previously it wasn't possible to use ip_filters with multiple target refs
|
||||
- [vmoperator](https://docs.victoriametrics.com/operator/): turn off `EnableStrictSecurity` by default. Before, upgrade operator to v0.36.0+ could fail components with volume attached, see [this issue](https://github.com/VictoriaMetrics/operator/issues/749) for details.
|
||||
- [vmoperator](https://docs.victoriametrics.com/operator/): bump default version of VictoriaMetrics components to [1.93.4](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.4).
|
||||
- [vmuser](./api.md#vmuser): [Enterprise] fixes ip_filters indent for url_prefix. Previously it wasn't possible to use ip_filters with multiple target refs
|
||||
- [vmoperator](./README.md): turn off `EnableStrictSecurity` by default. Before, upgrade operator to v0.36.0+ could fail components with volume attached, see [this issue](https://github.com/VictoriaMetrics/operator/issues/749) for details.
|
||||
- [vmoperator](./README.md): bump default version of VictoriaMetrics components to [1.93.4](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.4).
|
||||
|
||||
### Features
|
||||
|
||||
- [vmoperator](https://docs.victoriametrics.com/operator/) add ability to print default values for all [operator variables](https://docs.victoriametrics.com/operator/vars.html). See [this issue](https://github.com/VictoriaMetrics/operator/issues/675) for details.
|
||||
- [vmoperator](./README.md) add ability to print default values for all [operator variables](./vars.md). See [this issue](https://github.com/VictoriaMetrics/operator/issues/675) for details.
|
||||
|
||||
<a name="v0.37.1"></a>
|
||||
## [v0.37.1](https://github.com/VictoriaMetrics/operator/releases/tag/v0.37.1) - 02 Sep 2023
|
||||
|
|
@ -39,18 +47,18 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fix unmarshalling for streaming aggregation `match` field.
|
||||
- [vmagent](./api.md#vmagent): fix unmarshalling for streaming aggregation `match` field.
|
||||
|
||||
### Features
|
||||
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): support [multiple if conditions](https://docs.victoriametrics.com/vmagent.html#relabeling:~:text=the%20if%20option%20may%20contain%20more%20than%20one%20filter) for relabeling. See [this issue](https://github.com/VictoriaMetrics/operator/issues/730) for details.
|
||||
- [vmagent](./api.md#vmagent): support [multiple if conditions](https://docs.victoriametrics.com/vmagent.html#relabeling:~:text=the%20if%20option%20may%20contain%20more%20than%20one%20filter) for relabeling. See [this issue](https://github.com/VictoriaMetrics/operator/issues/730) for details.
|
||||
|
||||
<a name="v0.36.1"></a>
|
||||
## [v0.36.1](https://github.com/VictoriaMetrics/operator/releases/tag/v0.36.0) - 25 Aug 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- [vmselect](https://docs.victoriametrics.com/operator/api.html#vmcluster): fix cache directory when `cacheDataPath` not specified, before it will use `/tmp` which is protect by default strict securityContext.
|
||||
- [vmselect](./api.md#vmcluster): fix cache directory when `cacheDataPath` not specified, before it will use `/tmp` which is protect by default strict securityContext.
|
||||
|
||||
### Features
|
||||
|
||||
|
|
@ -59,7 +67,7 @@
|
|||
|
||||
### Breaking changes
|
||||
|
||||
- **[vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): Field `OAuth2` was renamed to `oauth2` due to compatibility issue. If you defined `OAuth2` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `oauth2` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/689) for details.**
|
||||
- **[vmalert](./api.md#vmalert): Field `OAuth2` was renamed to `oauth2` due to compatibility issue. If you defined `OAuth2` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `oauth2` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/689) for details.**
|
||||
- **Affected fields:**
|
||||
- **`VMAlert.spec.datasource.OAuth2` -> `VMAlert.spec.datasource.oauth2`,**
|
||||
- **`VMAlert.spec.notifier.OAuth2` -> `VMAlert.spec.notifier.oauth2`,**
|
||||
|
|
@ -67,7 +75,7 @@
|
|||
- **`VMAlert.spec.remoteRead.OAuth2` -> `VMAlert.spec.remoteRead.oauth2`,**
|
||||
- **`VMAlert.spec.remoteWrite.OAuth2` -> `VMAlert.spec.remoteWrite.oauth2`,**
|
||||
|
||||
- **[vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): Field `bearerTokenFilePath` was renamed to `bearerTokenFile` due to compatibility issue. If you defined `bearerTokenFilePath` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `bearerTokenFile` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/688/) for details.**
|
||||
- **[vmalert](./api.md#vmalert): Field `bearerTokenFilePath` was renamed to `bearerTokenFile` due to compatibility issue. If you defined `bearerTokenFilePath` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `bearerTokenFile` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/688/) for details.**
|
||||
- **Affected fields:**
|
||||
- **`VMAlert.spec.datasource.bearerTokenFilePath` --> `VMAlert.spec.datasource.bearerTokenFile`,**
|
||||
- **`VMAlert.spec.notifier.bearerTokenFilePath` --> `VMAlert.spec.notifier.bearerTokenFile`,**
|
||||
|
|
@ -80,22 +88,22 @@
|
|||
- operator set resource requests for config-reloader container by default. See [this PR](https://github.com/VictoriaMetrics/operator/pull/695/) for details.
|
||||
- fix `attachMetadata` value miscovert for scrape objects. See [this issue](https://github.com/VictoriaMetrics/operator/issues/697) and [this PR](https://github.com/VictoriaMetrics/operator/pull/698) for details.
|
||||
- fix volumeClaimTemplates change check for objects that generate statefulset, like vmstorage, vmselect. Before, the statefulset won't be recreated if additional `claimTemplates` object changed. See [this issue](https://github.com/VictoriaMetrics/operator/issues/507) and [this PR](https://github.com/VictoriaMetrics/operator/pull/719) for details.
|
||||
- [vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): fix `tlsCAFile` argument value generation when using secret or configMap. See [this issue](https://github.com/VictoriaMetrics/operator/issues/699) and [this PR](https://github.com/VictoriaMetrics/operator/issues/699) for details.
|
||||
- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): fix default request memory and apply default resources if not set. See [this issue](https://github.com/VictoriaMetrics/operator/issues/706) and [this PR](https://github.com/VictoriaMetrics/operator/pull/710) for details.
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fix missing additional VolumeClaimTemplates when using `ClaimTemplates` under StatefulMode.
|
||||
- [vmalert](./api.md#vmalert): fix `tlsCAFile` argument value generation when using secret or configMap. See [this issue](https://github.com/VictoriaMetrics/operator/issues/699) and [this PR](https://github.com/VictoriaMetrics/operator/issues/699) for details.
|
||||
- [vmalertmanager](./api.md#vmalertmanager): fix default request memory and apply default resources if not set. See [this issue](https://github.com/VictoriaMetrics/operator/issues/706) and [this PR](https://github.com/VictoriaMetrics/operator/pull/710) for details.
|
||||
- [vmagent](./api.md#vmagent): fix missing additional VolumeClaimTemplates when using `ClaimTemplates` under StatefulMode.
|
||||
|
||||
### Features
|
||||
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmagent_stateful_with_sharding.yaml) for vmagent statefulmode.
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent)/[vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): adapt new features in streaming aggregation:
|
||||
- [vmagent](./api.md#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmagent_stateful_with_sharding.yaml) for vmagent statefulmode.
|
||||
- [vmagent](./api.md#vmagent)/[vmsingle](./api.md#vmsingle): adapt new features in streaming aggregation:
|
||||
- support `streamAggr.dropInput`, see [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4243) for details;
|
||||
- support list for `match` parameter, see [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635) for details;
|
||||
- support `staleness_interval`, see [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4667) for details.
|
||||
- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmcluster_with_additional_claim.yaml) for cluster with custom storage claims.
|
||||
- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): support `update_entries_limit` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/691) for details.
|
||||
- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): support `keep_firing_for` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/711) for details.
|
||||
- [vmoperator parameters](https://docs.victoriametrics.com/operator/vars.html): Add option `VM_ENABLESTRICTSECURITY` and enable strict security context by default. See [this issue](https://github.com/VictoriaMetrics/operator/issues/637), [this](https://github.com/VictoriaMetrics/operator/pull/692/) and [this](https://github.com/VictoriaMetrics/operator/pull/712) PR for details.
|
||||
- [vmoperator parameters](https://docs.victoriametrics.com/operator/vars.html): change option `VM_PSPAUTOCREATEENABLED` default value from `true` to `false` cause PodSecurityPolicy already got deprecated since [kubernetes v1.25](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#psp-v125). See [this pr](https://github.com/VictoriaMetrics/operator/pull/726) for details.
|
||||
- [vmcluster](./api.md#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmcluster_with_additional_claim.yaml) for cluster with custom storage claims.
|
||||
- [vmrule](./api.md#vmrule): support `update_entries_limit` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/691) for details.
|
||||
- [vmrule](./api.md#vmrule): support `keep_firing_for` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/711) for details.
|
||||
- [vmoperator parameters](./vars.md): Add option `VM_ENABLESTRICTSECURITY` and enable strict security context by default. See [this issue](https://github.com/VictoriaMetrics/operator/issues/637), [this](https://github.com/VictoriaMetrics/operator/pull/692/) and [this](https://github.com/VictoriaMetrics/operator/pull/712) PR for details.
|
||||
- [vmoperator parameters](./vars.md): change option `VM_PSPAUTOCREATEENABLED` default value from `true` to `false` cause PodSecurityPolicy already got deprecated since [kubernetes v1.25](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#psp-v125). See [this pr](https://github.com/VictoriaMetrics/operator/pull/726) for details.
|
||||
|
||||
[Changes][v0.36.0]
|
||||
|
||||
|
|
@ -104,7 +112,7 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fixes regression with remoteWrite authorization (basicAuth/token). When `UseCustomConfigReloader` option was set, operator incorrectly rendered mounts for `vmagent` container. https://github.com/VictoriaMetrics/operator/commit/f2b8cf701a33f91cef19848c857fd6efb7db59dd
|
||||
- [vmagent](./api.md#vmagent): fixes regression with remoteWrite authorization (basicAuth/token). When `UseCustomConfigReloader` option was set, operator incorrectly rendered mounts for `vmagent` container. https://github.com/VictoriaMetrics/operator/commit/f2b8cf701a33f91cef19848c857fd6efb7db59dd
|
||||
|
||||
[Changes][v0.35.1]
|
||||
|
||||
|
|
@ -114,19 +122,19 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
* [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): fix vmselect url_map in vmuser. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/655). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): correctly set default port for vmauth components discovery. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/658). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): remove rate limit on delete. In https://github.com/VictoriaMetrics/operator/pull/672. Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): fix spec change check. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmuser](./api.md#vmuser): fix vmselect url_map in vmuser. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/655). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmalert](./api.md#vmalert): correctly set default port for vmauth components discovery. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/658). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmuser](./api.md#vmuser): remove rate limit on delete. In https://github.com/VictoriaMetrics/operator/pull/672. Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmcluster](./api.md#vmcluster): fix spec change check. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* Correctly publish multi-arch release at https://github.com/VictoriaMetrics/operator/pull/681. Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
|
||||
### Features
|
||||
|
||||
* [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): add validation when generate static scrape config. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): add validation for slack receiver url. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/661). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth)/[vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): implement configuration initiation for custom config reloader. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/619). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmagent](./api.md#vmagent): add validation when generate static scrape config. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): add validation for slack receiver url. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/661). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmauth](./api.md#vmauth)/[vmagent](./api.md#vmagent): implement configuration initiation for custom config reloader. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/619). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* add more generators Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/668
|
||||
* [vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): add status field. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/670). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmsingle](./api.md#vmsingle): add status field. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/670). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
|
||||
[Changes][v0.35.0]
|
||||
|
||||
|
|
@ -136,9 +144,9 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): fail fast on misconfigured or missing kubernetes pods. It should prevent rare bug with cascade pod deletion. See this [issue](https://github.com/VictoriaMetrics/operator/issues/643) for details
|
||||
- [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth)/[vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): correctly renders initConfig image with global container registry domain. See this [issue](https://github.com/VictoriaMetrics/operator/issues/654) for details.
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): correctly set RBAC permissions for single namespace mode and custom config reloader image. See this [issue](https://github.com/VictoriaMetrics/operator/issues/653) for details.
|
||||
- [vmcluster](./api.md#vmcluster): fail fast on misconfigured or missing kubernetes pods. It should prevent rare bug with cascade pod deletion. See this [issue](https://github.com/VictoriaMetrics/operator/issues/643) for details
|
||||
- [vmauth](./api.md#vmauth)/[vmagent](./api.md#vmagent): correctly renders initConfig image with global container registry domain. See this [issue](https://github.com/VictoriaMetrics/operator/issues/654) for details.
|
||||
- [vmagent](./api.md#vmagent): correctly set RBAC permissions for single namespace mode and custom config reloader image. See this [issue](https://github.com/VictoriaMetrics/operator/issues/653) for details.
|
||||
|
||||
[Changes][v0.34.1]
|
||||
|
||||
|
|
@ -152,18 +160,18 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmnodescrape](https://docs.victoriametrics.com/operator/api.html#vmnodescrape): fixed selectors for Exists and NotExists operators with empty label Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/646
|
||||
- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): Add config for vmrule in validating webhook Thanks in https://github.com/VictoriaMetrics/operator/pull/650
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): skips misconfigured objects with missed secret references: https://github.com/VictoriaMetrics/operator/issues/648
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): correctly renders initContainer for configuration download: https://github.com/VictoriaMetrics/operator/issues/649
|
||||
- [vmnodescrape](./api.md#vmnodescrape): fixed selectors for Exists and NotExists operators with empty label Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/646
|
||||
- [vmrule](./api.md#vmrule): Add config for vmrule in validating webhook Thanks in https://github.com/VictoriaMetrics/operator/pull/650
|
||||
- [vmagent](./api.md#vmagent): skips misconfigured objects with missed secret references: https://github.com/VictoriaMetrics/operator/issues/648
|
||||
- [vmagent](./api.md#vmagent): correctly renders initContainer for configuration download: https://github.com/VictoriaMetrics/operator/issues/649
|
||||
|
||||
### Features
|
||||
|
||||
- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): Bump alertmanager to v0.25.0 Thanks [@tamcore](https://github.com/tamcore) in https://github.com/VictoriaMetrics/operator/pull/636
|
||||
- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): added `clusterNativePort` field to VMSelect/VMInsert for multi-level cluster setup ([#634](https://github.com/VictoriaMetrics/operator/issues/634)) Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/639
|
||||
- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): add notifierHeader field in vmrule spec Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/622
|
||||
- [vmpodscrape](https://docs.victoriametrics.com/operator/api.html#vmpodscrape): adds FilterRunning option as prometheus does in https://github.com/VictoriaMetrics/operator/pull/640
|
||||
- [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth): adds latest features in https://github.com/VictoriaMetrics/operator/pull/642
|
||||
- [vmalertmanager](./api.md#vmalertmanager): Bump alertmanager to v0.25.0 Thanks [@tamcore](https://github.com/tamcore) in https://github.com/VictoriaMetrics/operator/pull/636
|
||||
- [vmcluster](./api.md#vmcluster): added `clusterNativePort` field to VMSelect/VMInsert for multi-level cluster setup ([#634](https://github.com/VictoriaMetrics/operator/issues/634)) Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/639
|
||||
- [vmrule](./api.md#vmrule): add notifierHeader field in vmrule spec Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/622
|
||||
- [vmpodscrape](./api.md#vmpodscrape): adds FilterRunning option as prometheus does in https://github.com/VictoriaMetrics/operator/pull/640
|
||||
- [vmauth](./api.md#vmauth): adds latest features in https://github.com/VictoriaMetrics/operator/pull/642
|
||||
|
||||
[Changes][v0.34.0]
|
||||
|
||||
|
|
@ -173,22 +181,22 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): skip bad rules and improve logging for rules exceed max configmap size https://github.com/VictoriaMetrics/operator/commit/bb754d5c20bb371a197cd6ff5afac1ba86a4d92b
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): fixed error with headers in VMAlertmanagerConfig.Receivers.EmailConfigs.Headers unmarshalling. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/610
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fixed keepInput setting for streaming aggregation. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/618
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): fix webhook config maxAlerts not work. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/625
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): Remove single quotes from remote write headers. Thanks [@axelsccp](https://github.com/axelsccp) in https://github.com/VictoriaMetrics/operator/pull/613
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): fix parse route error and some comments. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/630
|
||||
- [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): properly removes finalizers for objects https://github.com/VictoriaMetrics/operator/commit/8f10113920a353f21fbcc8637076905f2e57bb34
|
||||
- [vmalert](./api.md#vmalert): skip bad rules and improve logging for rules exceed max configmap size https://github.com/VictoriaMetrics/operator/commit/bb754d5c20bb371a197cd6ff5afac1ba86a4d92b
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): fixed error with headers in VMAlertmanagerConfig.Receivers.EmailConfigs.Headers unmarshalling. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/610
|
||||
- [vmagent](./api.md#vmagent): fixed keepInput setting for streaming aggregation. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/618
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): fix webhook config maxAlerts not work. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/625
|
||||
- [vmagent](./api.md#vmagent): Remove single quotes from remote write headers. Thanks [@axelsccp](https://github.com/axelsccp) in https://github.com/VictoriaMetrics/operator/pull/613
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): fix parse route error and some comments. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/630
|
||||
- [vmuser](./api.md#vmuser): properly removes finalizers for objects https://github.com/VictoriaMetrics/operator/commit/8f10113920a353f21fbcc8637076905f2e57bb34
|
||||
|
||||
### Features
|
||||
|
||||
- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): add option to disable route continue enforce. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/621
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): support set require_tls to false. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/624
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): add sanity check. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/627
|
||||
- [vmalertmanager](./api.md#vmalertmanager): add option to disable route continue enforce. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/621
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): support set require_tls to false. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/624
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): add sanity check. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/627
|
||||
- Makefile: bump Alpine base image to latest v3.17.3. Thanks [@denisgolius](https://github.com/denisgolius) in https://github.com/VictoriaMetrics/operator/pull/628
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): support sound field in pushover config. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/631
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent)/[vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth): download initial config with initContainer https://github.com/VictoriaMetrics/operator/commit/612e7c8f40659731e7938ef9556eb088c67eb4b7
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): support sound field in pushover config. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/631
|
||||
- [vmagent](./api.md#vmagent)/[vmauth](./api.md#vmauth): download initial config with initContainer https://github.com/VictoriaMetrics/operator/commit/612e7c8f40659731e7938ef9556eb088c67eb4b7
|
||||
|
||||
[Changes][v0.33.0]
|
||||
|
||||
|
|
@ -199,7 +207,7 @@
|
|||
### Fixes
|
||||
|
||||
- config: fixes typo at default vm apps version https://github.com/VictoriaMetrics/operator/issues/608
|
||||
- [vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): conditionally adds stream aggregation config https://github.com/VictoriaMetrics/operator/commit/4a0ca54113afcde439ca4c77e22d3ef1c0d36241
|
||||
- [vmsingle](./api.md#vmsingle): conditionally adds stream aggregation config https://github.com/VictoriaMetrics/operator/commit/4a0ca54113afcde439ca4c77e22d3ef1c0d36241
|
||||
|
||||
[Changes][v0.32.1]
|
||||
|
||||
|
|
@ -213,10 +221,10 @@
|
|||
|
||||
### Features
|
||||
|
||||
- [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth): automatically configures `proxy-protocol` client and `reloadAuthKey` for `config-reloader` container. https://github.com/VictoriaMetrics/operator/commit/611819233bf595a4dbd04b07d7be24b7e994379c
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): adds `scrapeTimeout` global configuration for `VMAgent` https://github.com/VictoriaMetrics/operator/commit/d1d5024c6befa0961f8d56c82a0554935a4b1878
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) for `remoteWrite` targets https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
|
||||
- [vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) as global configuration for database https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
|
||||
- [vmauth](./api.md#vmauth): automatically configures `proxy-protocol` client and `reloadAuthKey` for `config-reloader` container. https://github.com/VictoriaMetrics/operator/commit/611819233bf595a4dbd04b07d7be24b7e994379c
|
||||
- [vmagent](./api.md#vmagent): adds `scrapeTimeout` global configuration for `VMAgent` https://github.com/VictoriaMetrics/operator/commit/d1d5024c6befa0961f8d56c82a0554935a4b1878
|
||||
- [vmagent](./api.md#vmagent): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) for `remoteWrite` targets https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
|
||||
- [vmsingle](./api.md#vmsingle): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) as global configuration for database https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
|
||||
|
||||
[Changes][v0.32.0]
|
||||
|
||||
|
|
@ -231,8 +239,8 @@
|
|||
|
||||
### Features
|
||||
|
||||
- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): Add support of vmalertmanager.spec.templates and autoreload dirs for templates and configmaps thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/590 https://github.com/VictoriaMetrics/operator/issues/592
|
||||
- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): Add support "%SHARD_NUM%" placeholder for vmagent sts/deployment Thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/508
|
||||
- [vmalertmanager](./api.md#vmalertmanager): Add support of vmalertmanager.spec.templates and autoreload dirs for templates and configmaps thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/590 https://github.com/VictoriaMetrics/operator/issues/592
|
||||
- [vmalertmanager](./api.md#vmalertmanager): Add support "%SHARD_NUM%" placeholder for vmagent sts/deployment Thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/508
|
||||
|
||||
[Changes][v0.31.0]
|
||||
|
||||
|
|
|
|||
|
|
@ -1,17 +1,14 @@
|
|||
---
|
||||
sort: 15
|
||||
weight: 15
|
||||
sort: 9
|
||||
weight: 9
|
||||
title: FAQ
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 15
|
||||
identifier: "faq-operator"
|
||||
aliases:
|
||||
- /operator/FAQ.html
|
||||
---
|
||||
|
||||
# FAQ
|
||||
# FAQ (Frequency Asked Questions)
|
||||
|
||||
## How do you monitor the operator itself?
|
||||
|
||||
You can read about vmoperator monitoring in [this document](./monitoring.md).
|
||||
|
||||
## How to change VMStorage PVC storage class
|
||||
|
||||
|
|
@ -29,3 +26,53 @@ With Operator deployment:
|
|||
1. Run `kubectl delete statefulset --cascade=orphan {vmstorage-sts}`
|
||||
1. Update VMCluster spec to use new storage class
|
||||
1. Apply cluster configuration
|
||||
|
||||
## How to override image registry
|
||||
|
||||
You can use `VM_CONTAINERREGISTRY` parameter for operator:
|
||||
|
||||
- See details about tuning [operator settings here](./setup.md#settings).
|
||||
- See [available operator settings](./vars.md) here.
|
||||
|
||||
## How to set up automatic backups?
|
||||
|
||||
You can read about backups:
|
||||
|
||||
- for `VMSingle`: [Backup automation](./resources/vmsingle.md#backup-automation)
|
||||
- for `VMCluster`: [Backup automation](./resources/vmcluster.md#backup-automation)
|
||||
|
||||
## How to migrate from Prometheus-operator to VictoriaMetrics operator?
|
||||
|
||||
You can read about migration from prometheus operator on [this page](./migration.md).
|
||||
|
||||
## How to turn off conversion for prometheus resources
|
||||
|
||||
You can read about it on [this page](./migration.md#objects-convesion).
|
||||
|
||||
## My VM objects are not deleted/changed when I delete/change Prometheus objects
|
||||
|
||||
You can read about it in following sections of "Migration from prometheus-operator" docs:
|
||||
|
||||
- [Deletion synchronization](./migration.md#deletion-synchronization)
|
||||
- [Update synchronization](./migration.md#update-synchronization)
|
||||
- [Labels synchronization](./migration.md#labels-synchronization)
|
||||
|
||||
## What permissions does an operator need to run in a cluster?
|
||||
|
||||
You can read about needed permissions for operator in [this document](./security.md#roles).
|
||||
|
||||
## How to know the version of VM components in the operator?
|
||||
|
||||
See [printDefaults mode](./configuration.md).
|
||||
|
||||
In addition, you can use [Release notes](https://github.com/VictoriaMetrics/operator/releases)
|
||||
or [CHANGELOG](https://github.com/VictoriaMetrics/operator/blob/master/docs/CHANGELOG.md).
|
||||
- that's where we describe default version of VictoriaMetrics components.
|
||||
|
||||
## How to run VictoriaMetrics operator with permissions for one namespace only?
|
||||
|
||||
See this document for details: [Configuration -> Namespaced mode](./configuration.md#namespaced-mode).
|
||||
|
||||
## What versions of Kubernetes is the operator compatible with?
|
||||
|
||||
Operator tested at kubernetes versions from 1.16 to 1.27.
|
||||
|
|
|
|||
|
|
@ -1,21 +1,91 @@
|
|||
---
|
||||
sort: 27
|
||||
sort: 0
|
||||
weight: 0
|
||||
title: VictoriaMetrics Operator
|
||||
disableToc: true
|
||||
---
|
||||
|
||||
# VictoriaMetrics Operator
|
||||
|
||||
1. [VictoriaMetrics Operator](VictoriaMetrics-Operator.html)
|
||||
1. [Additional Scrape Configuration](additional-scrape.html)
|
||||
1. [API Docs](api.html)
|
||||
1. [Authorization and exposing components](auth.html)
|
||||
1. [vmbackupmanager](backups.html)
|
||||
1. [Design](design.html)
|
||||
1. [High Availability](high-availability.html)
|
||||
1. [VMAlert, VMAgent, VMAlertmanager, VMSingle version](managing-versions.html)
|
||||
1. [Victoria Metrics Operator Quick Start](quick-start.html)
|
||||
1. [VMAgent relabel](relabeling.html)
|
||||
1. [CRD Validation](resources-validation.html)
|
||||
1. [Security](security.html)
|
||||
1. [Auto Generated vars for package config](vars.html)
|
||||
Operator serves to make running VictoriaMetrics applications on top of Kubernetes as easy as possible while preserving Kubernetes-native configuration options.
|
||||
|
||||
VictoriaMetrics Operator (`vmoperator`) is the classic kubernetes-operator for VictoriaMetrics with many [great features](#features).
|
||||
It allows you to manage Victoria Metrics components in Kubernetes or OpenShift clusters
|
||||
in a declarative style according to [GitOps](https://www.redhat.com/en/topics/devops/what-is-gitops)
|
||||
and [IaC](https://en.wikipedia.org/wiki/Infrastructure_as_code) concepts.
|
||||
|
||||
VictoriaMetrics also provides [helm charts](https://github.com/VictoriaMetrics/helm-charts) without operator.
|
||||
Operator makes the same, simplifies it and provides [advanced features](#features).
|
||||
|
||||
Learn more about [key concepts](#key-concepts) of `vmoperator` and follow the **[quick start guide](./quick-start.md)** for a better experience.
|
||||
|
||||
## Features of vmoperator
|
||||
|
||||
- Deployment and management in a kubernetes clusters of any number of VictoriaMetrics applications (like vmsingle/vmcluster instances and another components like vmauth, vmagent, vmalert, etc...)
|
||||
- Seamless [migration from prometheus-operator](./migration.md) with auto-conversion of prometheus [custom resources](#custom-resources)
|
||||
- Simple VictoriaMetrics cluster installation, configuring, upgrading and managing with [crd-objects](./resources/README.md).
|
||||
- Ability to delegate the configuration (parts of configuration) of applications monitoring to the end-users and managing access to different configurations or configuration sections.
|
||||
- Integration with VictoriaMetrics [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager.html) - advanced tools for making backups. Check [Backup automation for VMSingle](./resources/vmsingle.md#backup-automation) or [Backup automation for VMCluster](./resources/vmcluster.md#backup-automation).
|
||||
- Everything you need for monitoring out of the box in [k8s-stack helm chart](https://victoriametrics.github.io/helm-charts/charts/victoria-metrics-k8s-stack/) with ready-made usecases and solutions.
|
||||
- Ability to template your own deployment scenarios.
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### Kubernetes-operators
|
||||
|
||||
[Kubernetes-operators](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) are software extensions
|
||||
for Kubernetes that make use of [custom resources](#custom-resources) to manage applications and their components.
|
||||
Operators follow Kubernetes principles, notably the control loop.
|
||||
It can be said that operators are custom controllers for Kubernetes that allow you to create business logic for custom resources.
|
||||
|
||||
Design and implementation of `vmoperator` inspired by [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator).
|
||||
|
||||
Useful links:
|
||||
- [Custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
|
||||
- [Custom resource definitions](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)
|
||||
- [Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/)
|
||||
- [Operator best practices](https://sdk.operatorframework.io/docs/best-practices/)
|
||||
|
||||
### Custom resources
|
||||
|
||||
Kubernetes-Operators use [custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
|
||||
for interaction. Custom resources are a mechanism built into Kubernetes that allows you to create your own extensions for Kubernetes,
|
||||
working on the same principles as those built into Kubernetes APIs. Custom resources make Kubernetes so modular and extensible.
|
||||
|
||||
In addition, thanks to CRD ([Custom Resource Definitions](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)),
|
||||
the mechanism of custom resources allows you to declare an API in the format of the OpenAPI specification and verify that the resources correspond to this API.
|
||||
|
||||
### Reconciliation cycle
|
||||
|
||||
The main task of the operator is to bring the state of the cluster in line with what is declared by the user in the custom resources.
|
||||
This process of constant monitoring and adjustment is called the "Reconciliation cycle" - it is the operator's workflow.
|
||||
|
||||
The basic workflow of working with the operator can be simplified as the following diagram:
|
||||
|
||||
<img src="README_operator-workflow.png" width="1200">
|
||||
|
||||
- Operator declares and owns [resources of Victoria Metrics](./resources/README.md).
|
||||
- Kubernetes validates of the resource according to the specification from CRD (see more in [custom resources](#custom-resources)).
|
||||
- Operator subscribed to change events (`create`, `update`, `delete`) for related resources.
|
||||
- When an event occurs, the operator reacts and updates the state of the objects in the cluster.
|
||||
- For some objects in the cluster the reconciliation cycle is performed at a given interval, even without the occurrence of change events (see `VM_FORCERESYNCINTERVAL`).
|
||||
|
||||
### Next steps
|
||||
|
||||
- [Quick Start Guide](./quick-start.md)
|
||||
- [Setup](./setup.md)
|
||||
- [Security](./security.md)
|
||||
- [Configuration](./configuration.md)
|
||||
- [Migration from Prometheus](./migration.md)
|
||||
- [Monitoring](./monitoring.md)
|
||||
- [Authorization and exposing components](./auth.md)
|
||||
- [High Availability](./high-availability.md)
|
||||
- [Enterprise](./enterprise.md)
|
||||
- [Custom resources](./resources/README.md)
|
||||
|
||||
If you have any questions, check out our [FAQ](./FAQ.md)
|
||||
and feel free to can ask them:
|
||||
- [VictoriaMetrics Slack](https://victoriametrics.slack.com/)
|
||||
- [VictoriaMetrics Telegram](https://t.me/VictoriaMetrics_en)
|
||||
|
||||
If you have any suggestions or find a bug, please create an issue
|
||||
on [GitHub](https://github.com/VictoriaMetrics/operator/issues/new).
|
||||
|
|
|
|||
BIN
docs/operator/README_operator-workflow.png
Normal file
BIN
docs/operator/README_operator-workflow.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 805 KiB |
|
|
@ -1,15 +1,11 @@
|
|||
---
|
||||
sort: 16
|
||||
sort: 12
|
||||
weight: 12
|
||||
title: API Docs
|
||||
weight: 16
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 16
|
||||
aliases:
|
||||
- /operator/api.html
|
||||
---
|
||||
|
||||
<!-- this doc autogenerated - don't edit it manually -->
|
||||
|
||||
# API Docs
|
||||
|
||||
This Document documents the types introduced by the VictoriaMetrics to be consumed by users.
|
||||
|
|
|
|||
|
|
@ -1,27 +1,23 @@
|
|||
---
|
||||
sort: 4
|
||||
weight: 4
|
||||
sort: 7
|
||||
weight: 7
|
||||
title: Authorization and exposing components
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 4
|
||||
aliases:
|
||||
- /operator/auth.html
|
||||
---
|
||||
|
||||
# Authorization and exposing components
|
||||
|
||||
## Exposing components
|
||||
|
||||
CRD objects doesn't have `ingress` configuration.
|
||||
Instead, you can use [VMAuth](./resources/vmauth.md) as proxy between ingress-controller and VictoriaMetrics components.
|
||||
|
||||
CRD objects doesn't have `ingress` configuration. Instead, you can use `VMAuth` as proxy between ingress-controller and VM app components.
|
||||
It adds missing authorization and access control features and enforces it.
|
||||
It adds missing authorization and access control features and enforces it.
|
||||
|
||||
Access can be given with `VMUser` definition. It supports basic auth and bearer token authentication.
|
||||
Access can be given with [VMUser](./resources/vmuser.md) definition.
|
||||
|
||||
It supports basic auth and bearer token authentication:
|
||||
|
||||
```yaml
|
||||
cat << EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
|
|
@ -30,45 +26,43 @@ spec:
|
|||
userNamespaceSelector: {}
|
||||
userSelector: {}
|
||||
ingress: {}
|
||||
EOF
|
||||
unauthorizedAccessConfig: []
|
||||
```
|
||||
|
||||
Advanced configuration with cert-manager annotations:
|
||||
Advanced configuration with cert-manager annotations:
|
||||
|
||||
```yaml
|
||||
cat << EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
name: router-main
|
||||
name: router-main
|
||||
spec:
|
||||
podMetadata:
|
||||
labels:
|
||||
component: vmauth
|
||||
userSelector: {}
|
||||
userNamespaceSelector: {}
|
||||
replicaCount: 2
|
||||
resources:
|
||||
requests:
|
||||
cpu: "250m"
|
||||
memory: "350Mi"
|
||||
limits:
|
||||
cpu: "500m"
|
||||
memory: "850Mi"
|
||||
ingress:
|
||||
tlsSecretName: vmauth-tls
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: base
|
||||
class_name: nginx
|
||||
tlsHosts:
|
||||
- vm-access.example.com
|
||||
EOF
|
||||
podMetadata:
|
||||
labels:
|
||||
component: vmauth
|
||||
userSelector: {}
|
||||
userNamespaceSelector: {}
|
||||
replicaCount: 2
|
||||
resources:
|
||||
requests:
|
||||
cpu: "250m"
|
||||
memory: "350Mi"
|
||||
limits:
|
||||
cpu: "500m"
|
||||
memory: "850Mi"
|
||||
ingress:
|
||||
tlsSecretName: vmauth-tls
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: base
|
||||
class_name: nginx
|
||||
tlsHosts:
|
||||
- vm-access.example.com
|
||||
```
|
||||
|
||||
|
||||
simple static routing with read-only access to vmagent for username - `user-1` with password `Asafs124142`
|
||||
Simple static routing with read-only access to vmagent for username - `user-1` with password `Asafs124142`:
|
||||
|
||||
```yaml
|
||||
# curl vmauth:8427/metrics -u 'user-1:Asafs124142'
|
||||
cat << EOF | kubectl apply -f
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMUser
|
||||
metadata:
|
||||
|
|
@ -79,14 +73,12 @@ spec:
|
|||
- static:
|
||||
url: http://vmagent-base.default.svc:8429
|
||||
paths: ["/targets/api/v1","/targets","/metrics"]
|
||||
EOF
|
||||
```
|
||||
|
||||
With bearer token access:
|
||||
With bearer token access:
|
||||
|
||||
```yaml
|
||||
# curl vmauth:8427/metrics -H 'Authorization: Bearer Asafs124142'
|
||||
cat << EOF | kubectl apply -f
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMUser
|
||||
metadata:
|
||||
|
|
@ -97,13 +89,12 @@ spec:
|
|||
- static:
|
||||
url: http://vmagent-base.default.svc:8429
|
||||
paths: ["/targets/api/v1","/targets","/metrics"]
|
||||
EOF
|
||||
```
|
||||
|
||||
It's also possible to use service discovery for objects:
|
||||
It's also possible to use service discovery for objects:
|
||||
|
||||
```yaml
|
||||
# curl vmauth:8427/metrics -H 'Authorization: Bearer Asafs124142'
|
||||
cat << EOF | kubectl apply -f
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMUser
|
||||
metadata:
|
||||
|
|
@ -116,12 +107,11 @@ spec:
|
|||
name: base
|
||||
namespace: default
|
||||
paths: ["/targets/api/v1","/targets","/metrics"]
|
||||
EOF
|
||||
```
|
||||
|
||||
Cluster components supports auto path generation for single tenant view:
|
||||
Cluster components supports auto path generation for single tenant view:
|
||||
|
||||
```yaml
|
||||
cat << EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMUser
|
||||
metadata:
|
||||
|
|
@ -143,17 +133,15 @@ spec:
|
|||
url: http://vmselect-test-persistent.default.svc:8481/
|
||||
paths:
|
||||
- /internal/resetRollupResultCache
|
||||
EOF
|
||||
```
|
||||
|
||||
For each `VMUser` operator generates corresponding secret with username/password or bearer token at the same namespace as `VMUser`.
|
||||
For each `VMUser` operator generates corresponding secret with username/password or bearer token at the same namespace as `VMUser`.
|
||||
|
||||
## Basic auth for targets
|
||||
|
||||
To authenticate a `VMServiceScrape`s over a metrics endpoint use [`basicAuth`](https://docs.victoriametrics.com/operator/api.html#basicauth)
|
||||
To authenticate a `VMServiceScrape`s over a metrics endpoint use [`basicAuth`](./api.md#basicauth):
|
||||
|
||||
```yaml
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMServiceScrape
|
||||
metadata:
|
||||
|
|
@ -173,11 +161,9 @@ spec:
|
|||
selector:
|
||||
matchLabels:
|
||||
app: myapp
|
||||
EOF
|
||||
```
|
||||
|
||||
```yaml
|
||||
cat <<EOF | kubectl apply -f -
|
||||
---
|
||||
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
|
|
@ -186,5 +172,14 @@ data:
|
|||
password: dG9vcg== # toor
|
||||
user: YWRtaW4= # admin
|
||||
type: Opaque
|
||||
EOF
|
||||
```
|
||||
|
||||
## Unauthorized access
|
||||
|
||||
You can expose some routes without authorization with `unauthorizedAccessConfig`.
|
||||
|
||||
Check more details in [VMAuth docs -> Unauthorized access](./resources/vmauth.md#unauthorized-access).
|
||||
|
||||
More details about features of `VMAuth` and `VMUser` you can read in:
|
||||
- [VMAuth docs](./resources/vmauth.md),
|
||||
- [VMUser docs](./resources/vmuser.md).
|
||||
|
|
|
|||
260
docs/operator/configuration.md
Normal file
260
docs/operator/configuration.md
Normal file
|
|
@ -0,0 +1,260 @@
|
|||
---
|
||||
sort: 4
|
||||
weight: 4
|
||||
title: Configuration
|
||||
---
|
||||
|
||||
# Configuration
|
||||
|
||||
Operator configured by env variables, list of it can be found
|
||||
on [Variables](./vars.md) page.
|
||||
|
||||
It defines default configuration options, like images for components, timeouts, features.
|
||||
|
||||
In addition, the operator has a special startup mode for outputting all variables, their types and default values.
|
||||
For instance, with this mode you can know versions of VM components, which are used by default:
|
||||
|
||||
```console
|
||||
./operator --printDefaults
|
||||
|
||||
# This application is configured via the environment. The following environment variables can be used:
|
||||
#
|
||||
# KEY TYPE DEFAULT REQUIRED DESCRIPTION
|
||||
# VM_USECUSTOMCONFIGRELOADER True or False false
|
||||
# VM_CUSTOMCONFIGRELOADERIMAGE String victoriametrics/operator:config-reloader-v0.32.0
|
||||
# VM_VMALERTDEFAULT_IMAGE String victoriametrics/vmalert
|
||||
# VM_VMALERTDEFAULT_VERSION String v1.93.3
|
||||
# VM_VMALERTDEFAULT_USEDEFAULTRESOURCES True or False true
|
||||
# VM_VMALERTDEFAULT_RESOURCE_LIMIT_MEM String 500Mi
|
||||
# VM_VMALERTDEFAULT_RESOURCE_LIMIT_CPU String 200m
|
||||
# ...
|
||||
```
|
||||
|
||||
You can choose output format for variables with `--printFormat` flag, possible values: `json`, `yaml`, `list` and `table` (default):
|
||||
|
||||
```console
|
||||
.operator --printDefaults --printFormat=json
|
||||
|
||||
# {
|
||||
# 'VM_USECUSTOMCONFIGRELOADER': 'false',
|
||||
# 'VM_CUSTOMCONFIGRELOADERIMAGE': 'victoriametrics/operator:config-reloader-v0.32.0',
|
||||
# 'VM_VMALERTDEFAULT_IMAGE': 'victoriametrics/vmalert',
|
||||
# 'VM_VMALERTDEFAULT_VERSION': 'v1.93.3',
|
||||
# ...
|
||||
# 'VM_FORCERESYNCINTERVAL': '60s',
|
||||
# 'VM_ENABLESTRICTSECURITY': 'true'
|
||||
# }
|
||||
```
|
||||
|
||||
## Conversion of prometheus-operator objects
|
||||
|
||||
You can read detailed instructions about configuring prometheus-objects conversion in [this document](./migration.md).
|
||||
|
||||
## Helm-charts
|
||||
|
||||
In [helm-charts](https://github.com/VictoriaMetrics/helm-charts) some important configuration parameters are implemented as separate flags in `values.yaml`:
|
||||
|
||||
### victoria-metrics-k8s-stack
|
||||
|
||||
For possible values refer to [parameters](https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-k8s-stack#parameters).
|
||||
|
||||
Also, checkout [here possible ENV variables](./vars.md) to configure operator behaviour.
|
||||
ENV variables can be set in the `victoria-metrics-operator.env` section.
|
||||
|
||||
```yaml
|
||||
# values.yaml
|
||||
|
||||
victoria-metrics-operator:
|
||||
image:
|
||||
# -- Image repository
|
||||
repository: victoriametrics/operator
|
||||
# -- Image tag
|
||||
tag: v0.35.0
|
||||
# -- Image pull policy
|
||||
pullPolicy: IfNotPresent
|
||||
|
||||
# -- Tells helm to remove CRD after chart remove
|
||||
cleanupCRD: true
|
||||
cleanupImage:
|
||||
repository: gcr.io/google_containers/hyperkube
|
||||
tag: v1.18.0
|
||||
pullPolicy: IfNotPresent
|
||||
|
||||
operator:
|
||||
# -- By default, operator converts prometheus-operator objects.
|
||||
disable_prometheus_converter: false
|
||||
# -- Compare-options and sync-options for prometheus objects converted by operator for properly use with ArgoCD
|
||||
prometheus_converter_add_argocd_ignore_annotations: false
|
||||
# -- Enables ownership reference for converted prometheus-operator objects,
|
||||
# it will remove corresponding victoria-metrics objects in case of deletion prometheus one.
|
||||
enable_converter_ownership: false
|
||||
# -- By default, operator creates psp for its objects.
|
||||
psp_auto_creation_enabled: true
|
||||
# -- Enables custom config-reloader, bundled with operator.
|
||||
# It should reduce vmagent and vmauth config sync-time and make it predictable.
|
||||
useCustomConfigReloader: false
|
||||
|
||||
# -- extra settings for the operator deployment. full list Ref: [https://github.com/VictoriaMetrics/operator/blob/master/vars.md](https://github.com/VictoriaMetrics/operator/blob/master/vars.md)
|
||||
env:
|
||||
# -- default version for vmsingle
|
||||
- name: VM_VMSINGLEDEFAULT_VERSION
|
||||
value: v1.43.0
|
||||
# -- container registry name prefix, e.g. docker.io
|
||||
- name: VM_CONTAINERREGISTRY
|
||||
value: ""
|
||||
# -- image for custom reloader (see the useCustomConfigReloader parameter)
|
||||
- name: VM_CUSTOMCONFIGRELOADERIMAGE
|
||||
value: victoriametrics/operator:config-reloader-v0.32.0
|
||||
|
||||
# By default, the operator will watch all the namespaces
|
||||
# If you want to override this behavior, specify the namespace it needs to watch separated by a comma.
|
||||
# Ex: my_namespace1,my_namespace2
|
||||
watchNamespace: ""
|
||||
|
||||
# Count of operator instances (can be increased for HA mode)
|
||||
replicaCount: 1
|
||||
|
||||
# -- VM operator log level
|
||||
# -- possible values: info and error.
|
||||
logLevel: "info"
|
||||
|
||||
# -- Resource object
|
||||
resources:
|
||||
{}
|
||||
# limits:
|
||||
# cpu: 120m
|
||||
# memory: 320Mi
|
||||
# requests:
|
||||
# cpu: 80m
|
||||
# memory: 120Mi
|
||||
```
|
||||
|
||||
### victoria-metrics-operator
|
||||
|
||||
For possible values refer to [parameters](https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-operator#parameters).
|
||||
|
||||
Also, checkout [here possible ENV variables](./vars.md) to configure operator behaviour.
|
||||
ENV variables can be set in the `env` section.
|
||||
|
||||
```yaml
|
||||
# values.yaml
|
||||
|
||||
image:
|
||||
# -- Image repository
|
||||
repository: victoriametrics/operator
|
||||
# -- Image tag
|
||||
tag: v0.35.0
|
||||
# -- Image pull policy
|
||||
pullPolicy: IfNotPresent
|
||||
|
||||
operator:
|
||||
# -- By default, operator converts prometheus-operator objects.
|
||||
disable_prometheus_converter: false
|
||||
# -- Compare-options and sync-options for prometheus objects converted by operator for properly use with ArgoCD
|
||||
prometheus_converter_add_argocd_ignore_annotations: false
|
||||
# -- Enables ownership reference for converted prometheus-operator objects,
|
||||
# it will remove corresponding victoria-metrics objects in case of deletion prometheus one.
|
||||
enable_converter_ownership: false
|
||||
# -- By default, operator creates psp for its objects.
|
||||
psp_auto_creation_enabled: true
|
||||
# -- Enables custom config-reloader, bundled with operator.
|
||||
# It should reduce vmagent and vmauth config sync-time and make it predictable.
|
||||
useCustomConfigReloader: false
|
||||
|
||||
# -- extra settings for the operator deployment. full list Ref: [https://github.com/VictoriaMetrics/operator/blob/master/vars.md](https://github.com/VictoriaMetrics/operator/blob/master/vars.md)
|
||||
env:
|
||||
# -- default version for vmsingle
|
||||
- name: VM_VMSINGLEDEFAULT_VERSION
|
||||
value: v1.43.0
|
||||
# -- container registry name prefix, e.g. docker.io
|
||||
- name: VM_CONTAINERREGISTRY
|
||||
value: ""
|
||||
# -- image for custom reloader (see the useCustomConfigReloader parameter)
|
||||
- name: VM_CUSTOMCONFIGRELOADERIMAGE
|
||||
value: victoriametrics/operator:config-reloader-v0.32.0
|
||||
|
||||
# By default, the operator will watch all the namespaces
|
||||
# If you want to override this behavior, specify the namespace it needs to watch separated by a comma.
|
||||
# Ex: my_namespace1,my_namespace2
|
||||
watchNamespace: ""
|
||||
|
||||
# Count of operator instances (can be increased for HA mode)
|
||||
replicaCount: 1
|
||||
|
||||
# -- VM operator log level
|
||||
# -- possible values: info and error.
|
||||
logLevel: "info"
|
||||
|
||||
# -- Resource object
|
||||
resources:
|
||||
{}
|
||||
# limits:
|
||||
# cpu: 120m
|
||||
# memory: 320Mi
|
||||
# requests:
|
||||
# cpu: 80m
|
||||
# memory: 120Mi
|
||||
```
|
||||
|
||||
## Namespaced mode
|
||||
|
||||
By default, the operator will watch all namespaces, but it can be configured to watch only specific namespace.
|
||||
|
||||
If you want to override this behavior, specify the namespace:
|
||||
|
||||
- in the `WATCH_NAMESPACE` environment variable.
|
||||
- in the `watchNamespace` field in the `values.yaml` file of helm-charts.
|
||||
|
||||
The operator supports only single namespace for watching.
|
||||
|
||||
You can find example of RBAC manifests for single-namespace mode in
|
||||
[this file](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/operator_rbac_for_single_namespace.yaml).
|
||||
|
||||
## Monitoring of cluster components
|
||||
|
||||
By default, operator creates [VMServiceScrape](./resources/vmservicescrape.md)
|
||||
object for each component that it manages.
|
||||
|
||||
You can disable this behaviour with `VM_DASABLESELFSERVICASCRAPECREATION` environment variable:
|
||||
|
||||
```shell
|
||||
VM_DASABLESELFSERVICASCRAPECREATION=false
|
||||
```
|
||||
|
||||
Also, you can override default configuration for self-scraping with `ServiceScrapeSpec` field in each deployable resource
|
||||
(`vmcluster/select`, `vmcluster/insert`, `vmcluster/storage`, `vmagent`, `vmalert`, `vmalertmanager`, `vmauth`, `vmsingle`):
|
||||
|
||||
## CRD Validation
|
||||
|
||||
Operator supports validation admission webhook [docs](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/)
|
||||
|
||||
It checks resources configuration and returns errors to caller before resource will be created at kubernetes api.
|
||||
This should reduce errors and simplify debugging.
|
||||
|
||||
Validation hooks at operator side must be enabled with flags:
|
||||
|
||||
```console
|
||||
./operator
|
||||
--webhook.enable
|
||||
# optional configuration for certDir and tls names.
|
||||
--webhook.certDir=/tmp/k8s-webhook-server/serving-certs/
|
||||
--webhook.keyName=tls.key
|
||||
--webhook.certName=tls.crt
|
||||
```
|
||||
|
||||
You have to mount correct certificates at give directory.
|
||||
It can be simplified with cert-manager and kustomize command:
|
||||
|
||||
```console
|
||||
kustomize build config/deployments/webhook/
|
||||
```
|
||||
|
||||
### Requirements
|
||||
|
||||
- Valid certificate with key must be provided to operator
|
||||
- Valid CABundle must be added to the `ValidatingWebhookConfiguration`
|
||||
|
||||
### Useful links
|
||||
|
||||
- [k8s admission webhooks](https://banzaicloud.com/blog/k8s-admission-webhooks/)
|
||||
- [olm webhooks](https://docs.openshift.com/container-platform/4.5/operators/user/olm-webhooks.html)
|
||||
36
docs/operator/enterprise.md
Normal file
36
docs/operator/enterprise.md
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
sort: 13
|
||||
weight: 13
|
||||
title: Enterprise features
|
||||
---
|
||||
|
||||
# Using operator with enterprise features
|
||||
|
||||
Operator doesn't have enterprise version for itself, but it supports
|
||||
[enterprise features for VictoriaMetrics components](https://docs.victoriametrics.com/enterprise.html):
|
||||
|
||||
- [VMAgent Enterprise features](./resources/vmagent.md#enterprise-features):
|
||||
- [Reading metrics from kafka](./resources/vmagent.md#reading-metrics-from-kafka)
|
||||
- [Writing metrics to kafka](./resources/vmagent.md#writing-metrics-to-kafka)
|
||||
- [VMAlert Enterprise features](./resources/vmalert.md#enterprise-features):
|
||||
- [Reading rules from object storage](./resources/vmalert.md#reading-rules-from-object-storage)
|
||||
- [Multitenancy](./resources/vmalert.md#multitenancy)
|
||||
- [VMAuth Enterprise features](./resources/vmauth.md#enterprise-features)
|
||||
- [IP Filters](./resources/vmauth.md#ip-filters)
|
||||
- [VMCluster Enterprise features](./resources/vmcluster.md#enterprise-features)
|
||||
- [Downsampling](./resources/vmcluster.md#downsampling)
|
||||
- [Multiple retentions / Retention filters](./resources/vmcluster.md#retention-filters)
|
||||
- [Advanced per-tenant statistic](./resources/vmcluster.md#advanced-per-tenant-statistic)
|
||||
- [mTLS protection](./resources/vmcluster.md#mtls-protection)
|
||||
- [Backup atomation](./resources/vmcluster.md#backup-atomation)
|
||||
- [VMRule Enterprise features](./resources/vmrule.md#enterprise-features)
|
||||
- [Multitenancy](./resources/vmrule.md#multitenancy)
|
||||
- [VMSingle Enterprise features](./resources/vmsingle.md#enterprise-features)
|
||||
- [Downsampling](./resources/vmsingle.md#downsampling)
|
||||
- [Retention filters](./resources/vmsingle.md#retention-filters)
|
||||
- [Backup atomation](./resources/vmsingle.md#backup-atomation)
|
||||
- [VMUser Enterprise features](./resources/vmuser.md#enterprise-features)
|
||||
- [IP Filters](./resources/vmuser.md#ip-filters)
|
||||
|
||||
More information about enterprise features you can read
|
||||
on [VictoriaMetrics Enterprise page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
|
||||
|
|
@ -1,381 +1,43 @@
|
|||
---
|
||||
sort: 7
|
||||
weight: 7
|
||||
sort: 8
|
||||
weight: 8
|
||||
title: High Availability
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 7
|
||||
aliases:
|
||||
- /operator/high-availability.html
|
||||
---
|
||||
|
||||
# High Availability
|
||||
|
||||
High availability is not only important for customer-facing software but if the monitoring infrastructure is not highly available, then there is a risk that operations people are not notified of alerts. Therefore, high availability must be just as thought through for the monitoring stack, as for anything else.
|
||||
High availability is not only important for customer-facing software but if the monitoring infrastructure is not highly available, then there is a risk that operations people are not notified of alerts.
|
||||
Therefore, high availability must be just as thought through for the monitoring stack, as for anything else.
|
||||
|
||||
## VMAgent
|
||||
## Components
|
||||
|
||||
To run VMAgent in a highly available manner you have to configure deduplication at Victoria Metrics first [doc](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/Single-server-VictoriaMetrics.md#deduplication)
|
||||
VictoriaMetrics operator support high availability for each component of the monitoring stack:
|
||||
|
||||
Then increase replicas for VMAgent.
|
||||
- [VMAgent](./resources/vmagent.md#high-availability)
|
||||
- [VMAlert](./resources/vmalert.md#high-availability)
|
||||
- [VMAlertmanager](./resources/vmalertmanager.md#high-availability)
|
||||
- [VMAuth](./resources/vmauth.md#high-availability)
|
||||
- [VMCluster](./resources/vmcluster.md#high-availability)
|
||||
|
||||
create `VMSingle` with dedup flag:
|
||||
More details you can find in the section **[High Availability for resources](./resources/README.md#high-availability)**.
|
||||
|
||||
```yaml
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMSingle
|
||||
metadata:
|
||||
name: example-vmsingle-persisted
|
||||
spec:
|
||||
retentionPeriod: "1"
|
||||
extraArgs:
|
||||
dedup.minScrapeInterval: 60s
|
||||
EOF
|
||||
```
|
||||
## Operator
|
||||
|
||||
create `VMAgent` with 2 replicas:
|
||||
VictoriaMetrics operator can be safely scaled horizontally, but only one replica of the operator can
|
||||
process [the reconciliation](./README.md#reconciliation-cycle) at a time -
|
||||
it uses a leader election mechanism to ensure that only one replica is active at a time.
|
||||
|
||||
```yaml
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: example-vmagent
|
||||
spec:
|
||||
serviceScrapeNamespaceSelector: {}
|
||||
podScrapeNamespaceSelector: {}
|
||||
podScrapeSelector: {}
|
||||
serviceScrapeSelector: {}
|
||||
scrapeInterval: 60s
|
||||
vmAgentExternalLabelName: vmagent-ha
|
||||
replicaCount: 2
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
|
||||
EOF
|
||||
If one of replicas of the operator will be failed, then another replica will be elected as a leader and will continue to work -
|
||||
operator replication affects how quickly this happens.
|
||||
|
||||
```
|
||||
[CRD validation](./configuration.md#crd-validation) workload is fully
|
||||
distributed among the available operator replicas.
|
||||
|
||||
### Sharding
|
||||
In addition, you can safely use for operator such features
|
||||
as [assigning and distributing to nodes](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/)
|
||||
(like [node selector](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector),
|
||||
[affinity and anti-affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity),
|
||||
[topology spread constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#pod-topology-spread-constraints),
|
||||
[taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/), etc...)
|
||||
|
||||
Sharding for `VMAgent` distributes scraping between multiple deployments of `VMAgent`.
|
||||
more info https://victoriametrics.github.io/vmagent.html#scraping-big-number-of-targets
|
||||
|
||||
Example usage:
|
||||
|
||||
```yaml
|
||||
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: example-vmagent
|
||||
spec:
|
||||
serviceScrapeNamespaceSelector: {}
|
||||
podScrapeNamespaceSelector: {}
|
||||
podScrapeSelector: {}
|
||||
serviceScrapeSelector: {}
|
||||
scrapeInterval: 60s
|
||||
vmAgentExternalLabelName: vmagent-ha
|
||||
shardCount: 5
|
||||
replicaCount: 2
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
preferredDuringSchedulingIgnoredDuringExecution:
|
||||
- podAffinityTerm:
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
shard-num: '%SHARD_NUM%'
|
||||
topologyKey: kubernetes.io/hostname
|
||||
EOF
|
||||
```
|
||||
|
||||
This configuration produces 5 deployments with 2 replicas at each. Each deployment has its own shard num
|
||||
and scrapes only 1/5 of all targets.
|
||||
|
||||
You can use special placeholder `%SHARD_NUM%` in any field of `VMAgent` specification.
|
||||
and operator will replace it with current shard num of vmagent when creating deployment or statefullset for vmagent.
|
||||
|
||||
In the example above, the `%SHARD_NUM%` placeholder is used in the `podAntiAffinity` section,
|
||||
which recommend to scheduler that pods with the same shard num (label `shard-num` in the pod template)
|
||||
are not deployed on the same node. You can use another `topologyKey` for availability zone or region instead of nodes.
|
||||
|
||||
### StatefulMode
|
||||
|
||||
In `StatefulMode` `VMAgent` doesn't lose state of the PersistentQueue (file-based buffer size for unsent data) on pod restarts.
|
||||
|
||||
Operator creates `StatefulSet` and, with provided `PersistentVolumeClaimTemplate` at `StatefulStorage` configuration param, metrics queue is stored on disk.
|
||||
|
||||
Example of configuration:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: example-persistent
|
||||
spec:
|
||||
selectAllByDefault: true
|
||||
vmAgentExternalLabelName: vmagent-ha
|
||||
replicaCount: 2
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
|
||||
statefulMode: true
|
||||
statefulStorage:
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
resources:
|
||||
requests:
|
||||
storage: 20Gi
|
||||
```
|
||||
|
||||
## VMAlert
|
||||
|
||||
It can be launched with multiple replicas without an additional configuration, alertmanager is responsible for alert deduplication.
|
||||
Note, if you want to use `VMAlert` with high-available `VMAlertmanager`, which has more than 1 replica. You have to specify all pod fqdns
|
||||
at `VMAlert.spec.notifiers.[url]`. Or you can use service discovery for notifier, examples:
|
||||
|
||||
alertmanager:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: vmalertmanager-example-alertmanager
|
||||
labels:
|
||||
app: vm-operator
|
||||
type: Opaque
|
||||
stringData:
|
||||
alertmanager.yaml: |
|
||||
global:
|
||||
resolve_timeout: 5m
|
||||
route:
|
||||
group_by: ['job']
|
||||
group_wait: 30s
|
||||
group_interval: 5m
|
||||
repeat_interval: 12h
|
||||
receiver: 'webhook'
|
||||
receivers:
|
||||
- name: 'webhook'
|
||||
webhook_configs:
|
||||
- url: 'http://alertmanagerwh:30500/'
|
||||
|
||||
---
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanager
|
||||
metadata:
|
||||
name: example
|
||||
namespace: default
|
||||
labels:
|
||||
usage: dedicated
|
||||
spec:
|
||||
replicaCount: 2
|
||||
configSecret: vmalertmanager-example-alertmanager
|
||||
configSelector: {}
|
||||
configNamespaceSelector: {}
|
||||
```
|
||||
vmalert with fqdns:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: example-ha
|
||||
namespace: default
|
||||
spec:
|
||||
datasource:
|
||||
url: http://vmsingle-example.default.svc:8429
|
||||
notifiers:
|
||||
- url: http://vmalertmanager-example-0.vmalertmanager-example.default.svc:9093
|
||||
- url: http://vmalertmanager-example-1.vmalertmanager-example.default.svc:9093
|
||||
```
|
||||
|
||||
vmalert with service discovery:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: example-ha
|
||||
namespace: default
|
||||
spec:
|
||||
datasource:
|
||||
url: http://vmsingle-example.default.svc:8429
|
||||
notifiers:
|
||||
- selector:
|
||||
namespaceSelector:
|
||||
matchNames:
|
||||
- default
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
usage: dedicated
|
||||
```
|
||||
|
||||
|
||||
## VMSingle
|
||||
|
||||
It doesn't support high availability by default, for such purpose use VMCluster or duplicate the setup.
|
||||
|
||||
|
||||
## VMCluster
|
||||
|
||||
The cluster version provides a full set of high availability features - metrics replication, node failover, horizontal scaling.
|
||||
|
||||
For using the cluster version you have to create the corresponding CRD object:
|
||||
|
||||
```yaml
|
||||
cat << EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: example-vmcluster-persistent
|
||||
spec:
|
||||
retentionPeriod: "4"
|
||||
replicationFactor: 2
|
||||
vmstorage:
|
||||
replicaCount: 2
|
||||
storageDataPath: "/vm-data"
|
||||
podMetadata:
|
||||
labels:
|
||||
owner: infra
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchExpressions:
|
||||
- key: "app.kubernetes.io/name"
|
||||
operator: In
|
||||
values:
|
||||
- "vmstorage"
|
||||
topologyKey: "kubernetes.io/hostname"
|
||||
storage:
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
resources:
|
||||
limits:
|
||||
cpu: "2"
|
||||
memory: 2048Mi
|
||||
vmselect:
|
||||
replicaCount: 2
|
||||
cacheMountPath: "/select-cache"
|
||||
podMetadata:
|
||||
labels:
|
||||
owner: infra
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchExpressions:
|
||||
- key: "app.kubernetes.io/name"
|
||||
operator: In
|
||||
values:
|
||||
- "vmselect"
|
||||
topologyKey: "kubernetes.io/hostname"
|
||||
|
||||
storage:
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
resources:
|
||||
requests:
|
||||
storage: 2Gi
|
||||
resources:
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: "500Mi"
|
||||
vminsert:
|
||||
replicaCount: 2
|
||||
podMetadata:
|
||||
labels:
|
||||
owner: infra
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchExpressions:
|
||||
- key: "app.kubernetes.io/name"
|
||||
operator: In
|
||||
values:
|
||||
- "vminsert"
|
||||
topologyKey: "kubernetes.io/hostname"
|
||||
resources:
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: "500Mi"
|
||||
|
||||
EOF
|
||||
```
|
||||
|
||||
Then wait for the cluster becomes ready
|
||||
|
||||
```console
|
||||
kubectl get vmclusters -w
|
||||
NAME INSERT COUNT STORAGE COUNT SELECT COUNT AGE STATUS
|
||||
example-vmcluster-persistent 2 2 2 2s expanding
|
||||
example-vmcluster-persistent 2 2 2 30s operational
|
||||
```
|
||||
|
||||
Get links for connection by executing the command:
|
||||
|
||||
```console
|
||||
kubectl get svc -l app.kubernetes.io/instance=example-vmcluster-persistent
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
vminsert-example-vmcluster-persistent ClusterIP 10.96.34.94 <none> 8480/TCP 69s
|
||||
vmselect-example-vmcluster-persistent ClusterIP None <none> 8481/TCP 79s
|
||||
vmstorage-example-vmcluster-persistent ClusterIP None <none> 8482/TCP,8400/TCP,8401/TCP 85s
|
||||
```
|
||||
|
||||
Now you can connect vmagent to vminsert and vmalert to vmselect
|
||||
|
||||
>NOTE do not forget to create rbac for vmagent
|
||||
|
||||
```yaml
|
||||
cat << EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: example-vmagent
|
||||
spec:
|
||||
serviceScrapeNamespaceSelector: {}
|
||||
serviceScrapeSelector: {}
|
||||
podScrapeNamespaceSelector: {}
|
||||
podScrapeSelector: {}
|
||||
# Add fields here
|
||||
replicaCount: 1
|
||||
remoteWrite:
|
||||
- url: "http://vminsert-example-vmcluster-persistent.default.svc.cluster.local:8480/insert/0/prometheus/api/v1/write"
|
||||
EOF
|
||||
```
|
||||
|
||||
Config for vmalert
|
||||
|
||||
```yaml
|
||||
cat << EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: example-vmalert
|
||||
spec:
|
||||
# Add fields here
|
||||
replicas: 1
|
||||
datasource:
|
||||
url: "http://vmselect-example-vmcluster-persistent.default.svc.cluster.local:8481/select/0/prometheus"
|
||||
notifier:
|
||||
url: "http://alertmanager-operated.default.svc:9093"
|
||||
evaluationInterval: "10s"
|
||||
ruleSelector: {}
|
||||
EOF
|
||||
```
|
||||
|
||||
|
||||
## Alertmanager
|
||||
|
||||
The final step of the high availability scheme is Alertmanager, when an alert triggers, actually fire alerts against *all* instances of an Alertmanager cluster.
|
||||
|
||||
The Alertmanager, starting with the `v0.5.0` release, ships with a high availability mode. It implements a gossip protocol to synchronize instances of an Alertmanager cluster regarding notifications that have been sent out, to prevent duplicate notifications. It is an AP (available and partition tolerant) system. Being an AP system means that notifications are guaranteed to be sent at least once.
|
||||
|
||||
The Victoria Metrics Operator ensures that Alertmanager clusters are properly configured to run highly available on Kubernetes.
|
||||
In addition, don't forget about [monitoring for the operator](./monitoring.md).
|
||||
|
|
|
|||
203
docs/operator/migration.md
Normal file
203
docs/operator/migration.md
Normal file
|
|
@ -0,0 +1,203 @@
|
|||
---
|
||||
sort: 5
|
||||
weight: 5
|
||||
title: Migration from Prometheus
|
||||
---
|
||||
|
||||
# Migration from prometheus-operator
|
||||
|
||||
Design and implementation inspired by [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator).
|
||||
It's great a tool for managing monitoring configuration of your applications. VictoriaMetrics operator has api capability with it.
|
||||
|
||||
So you can use familiar CRD objects: `ServiceMonitor`, `PodMonitor`, `PrometheusRule`, `Probe` and `AlertmanagerConfig`.
|
||||
|
||||
Or you can use VictoriaMetrics CRDs:
|
||||
|
||||
- `VMServiceScrape` (instead of `ServiceMonitor`) - defines scraping metrics configuration from pods backed by services. [See details](./resources/vmservicescrape.md).
|
||||
- `VMPodScrape` (instead of `PodMonitor`) - defines scraping metrics configuration from pods. [See details](./resources/vmpodscrape.md).
|
||||
- `VMRule` (instead of `PrometheusRule`) - defines alerting or recording rules. [See details](./resources/vmrule.md).
|
||||
- `VMProbe` (instead of `Probe`) - defines a probing configuration for targets with blackbox exporter. [See details](./resources/vmprobe.md).
|
||||
- `VMAlertmanagerConfig` (instead of `AlertmanagerConfig`) - defines a configuration for AlertManager. [See details](./resources/vmalertmanagerconfig.md).
|
||||
|
||||
Note that Prometheus CRDs are not supplied with the VictoriaMetrics operator,
|
||||
so you need to [install them separately](https://github.com/prometheus-operator/prometheus-operator/releases).
|
||||
VictoriaMetrics operator supports conversion from Prometheus CRD of
|
||||
version `monitoring.coreos.com/v1` for kinds `ServiceMonitor`, `PodMonitor`, `PrometheusRule`, `Probe`
|
||||
and version `monitoring.coreos.com/v1alpha1` for kind `AlertmanagerConfig`.
|
||||
|
||||
The default behavior of the operator is as follows:
|
||||
|
||||
- It **converts** all existing Prometheus `ServiceMonitor`, `PodMonitor`, `PrometheusRule` and `Probe` objects into corresponding VictoriaMetrics Operator objects.
|
||||
- It **syncs** updates (including labels) from Prometheus `ServiceMonitor`, `PodMonitor`, `PrometheusRule` and `Probe` objects to corresponding VictoriaMetrics Operator objects.
|
||||
- It **DOES NOT delete** converted objects after original ones are deleted.
|
||||
|
||||
With this configuration removing prometheus-operator API objects wouldn't delete any converted objects. So you can safely migrate or run two operators at the same time.
|
||||
|
||||
You can change default behavior with operator configuration - [see details below](#objects-conversion).
|
||||
|
||||
## Objects conversion
|
||||
|
||||
By default, the vmoperator converts all existing [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator)
|
||||
API objects into corresponding VictoriaMetrics Operator objects ([see above](#migration-from-prometheus-operator)),
|
||||
i.e. creates resources of VictoriaMetrics similar to Prometheus resources in the same namespace.
|
||||
|
||||
You can control this behaviour by setting env variable for operator:
|
||||
|
||||
```console
|
||||
# disable convertion for each object
|
||||
VM_ENABLEDPROMETHEUSCONVERTER_PODMONITOR=false
|
||||
VM_ENABLEDPROMETHEUSCONVERTER_SERVICESCRAPE=false
|
||||
VM_ENABLEDPROMETHEUSCONVERTER_PROMETHEUSRULE=false
|
||||
VM_ENABLEDPROMETHEUSCONVERTER_PROBE=false
|
||||
```
|
||||
|
||||
For [victoria-metrics-operator helm-chart](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-operator/README.md) you can use following way:
|
||||
|
||||
```yaml
|
||||
# values.yaml
|
||||
|
||||
# ...
|
||||
operator:
|
||||
# -- By default, operator converts prometheus-operator objects.
|
||||
disable_prometheus_converter: true
|
||||
# ...
|
||||
```
|
||||
|
||||
Otherwise, VictoriaMetrics Operator would try to discover prometheus-operator API and convert it.
|
||||
|
||||
<img src="migration_prometheus-conversion.png" width="1200">
|
||||
|
||||
For more information about the operator's workflow, see [this doc](./README.md).
|
||||
|
||||
## Deletion synchronization
|
||||
|
||||
By default, the operator doesn't make converted objects disappear after original ones are deleted. To change this behaviour
|
||||
configure adding `OwnerReferences` to converted objects with following [operator parameter](./setup.md#settings):
|
||||
|
||||
```console
|
||||
VM_ENABLEDPROMETHEUSCONVERTEROWNERREFERENCES=true
|
||||
```
|
||||
|
||||
For [victoria-metrics-operator helm-chart](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-operator/README.md) you can use following way:
|
||||
|
||||
```yaml
|
||||
# values.yaml
|
||||
|
||||
# ...
|
||||
operator:
|
||||
# -- Enables ownership reference for converted prometheus-operator objects,
|
||||
# it will remove corresponding victoria-metrics objects in case of deletion prometheus one.
|
||||
enable_converter_ownership: true
|
||||
# ...
|
||||
```
|
||||
|
||||
Converted objects will be linked to the original ones and will be deleted by kubernetes after the original ones are deleted.
|
||||
|
||||
## Update synchronization
|
||||
|
||||
Conversion of api objects can be controlled by annotations, added to `VMObject`s.
|
||||
|
||||
Annotation `operator.victoriametrics.com/ignore-prometheus-updates` controls updates from Prometheus api objects.
|
||||
|
||||
By default, it set to `disabled`. You define it to `enabled` state and all updates from Prometheus api objects will be ignored.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMServiceScrape
|
||||
metadata:
|
||||
annotations:
|
||||
meta.helm.sh/release-name: prometheus
|
||||
operator.victoriametrics.com/ignore-prometheus-updates: enabled
|
||||
labels:
|
||||
release: prometheus
|
||||
name: prometheus-monitor
|
||||
spec:
|
||||
endpoints: []
|
||||
```
|
||||
|
||||
Annotation `operator.victoriametrics.com/ignore-prometheus-updates` can be set on one of the resources:
|
||||
|
||||
- [VMServiceScrape](./resources/vmservicescrape.md)
|
||||
- [VMPodScrape](./resources/vmpodscrape.md)
|
||||
- [VMRule](./resources/vmrule.md)
|
||||
- [VMProbe](./resources/vmprobe.md)
|
||||
- [VMAlertmanagerConfig](./resources/vmalertmanagerconfig.md)
|
||||
|
||||
And annotation doesn't make sense for [VMStaticScrape](./resources/vmstaticscrape.md)
|
||||
and [VMNodeScrape](./resources/vmnodescrape.md) because these objects are not created as a result of conversion.
|
||||
|
||||
## Labels and annotations synchronization
|
||||
|
||||
Conversion of api objects can be controlled by annotations, added to `VMObject`s.
|
||||
|
||||
Annotation `operator.victoriametrics.com/merge-meta-strategy` controls syncing of metadata labels and annotations
|
||||
between `VMObject`s and `Prometheus` api objects during updates to `Prometheus` objects.
|
||||
|
||||
By default, it has `prefer-prometheus`. And annotations and labels will be used from `Prometheus` objects, manually set values will be dropped.
|
||||
|
||||
You can set it to `prefer-victoriametrics`. In this case all labels and annotations applied to `Prometheus` object will be ignored and `VMObject` will use own values.
|
||||
|
||||
Two additional strategies annotations -`merge-victoriametrics-priority` and `merge-prometheus-priority` merges labelSets into one combined labelSet, with priority.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMServiceScrape
|
||||
metadata:
|
||||
annotations:
|
||||
meta.helm.sh/release-name: prometheus
|
||||
operator.victoriametrics.com/merge-meta-strategy: prefer-victoriametrics
|
||||
labels:
|
||||
release: prometheus
|
||||
name: prometheus-monitor
|
||||
spec:
|
||||
endpoints: []
|
||||
```
|
||||
|
||||
Annotation `operator.victoriametrics.com/merge-meta-strategy` can be set on one of the resources:
|
||||
|
||||
- [VMServiceScrape](./resources/vmservicescrape.md)
|
||||
- [VMPodScrape](./resources/vmpodscrape.md)
|
||||
- [VMRule](./resources/vmrule.md)
|
||||
- [VMProbe](./resources/vmprobe.md)
|
||||
- [VMAlertmanagerConfig](./resources/vmalertmanagerconfig.md)
|
||||
|
||||
And annotation doesn't make sense for [VMStaticScrape](./resources/vmstaticscrape.md)
|
||||
and [VMNodeScrape](./resources/vmnodescrape.md) because these objects are not created as a result of conversion.
|
||||
|
||||
You can filter labels for syncing
|
||||
with [operator parameter](./setup.md#settings) `VM_FILTERPROMETHEUSCONVERTERLABELPREFIXES`:
|
||||
|
||||
```console
|
||||
# it excludes all labels that start with "helm.sh" or "argoproj.io" from synchronization
|
||||
VM_FILTERPROMETHEUSCONVERTERLABELPREFIXES=helm.sh,argoproj.io
|
||||
```
|
||||
|
||||
In the same way, annotations with specified prefixes can be excluded from synchronization
|
||||
with [operator parameter](./setup.md#settings) `VM_FILTERPROMETHEUSCONVERTERANNOTATIONPREFIXES`:
|
||||
|
||||
```console
|
||||
# it excludes all annotations that start with "helm.sh" or "argoproj.io" from synchronization
|
||||
VM_FILTERPROMETHEUSCONVERTERANNOTATIONPREFIXES=helm.sh,argoproj.io
|
||||
```
|
||||
|
||||
## Using converter with ArgoCD
|
||||
|
||||
If you use ArgoCD, you can allow ignoring objects at ArgoCD converted from Prometheus CRD
|
||||
with [operator parameter](./setup.md#settings) `VM_PROMETHEUSCONVERTERADDARGOCDIGNOREANNOTATIONS`.
|
||||
|
||||
It helps to properly use converter with ArgoCD and should help prevent out-of-sync issues with argo-cd based deployments:
|
||||
|
||||
```console
|
||||
# adds compare-options and sync-options for prometheus objects converted by operator
|
||||
VM_PROMETHEUSCONVERTERADDARGOCDIGNOREANNOTATIONS=true
|
||||
```
|
||||
|
||||
## Data migration
|
||||
|
||||
You can use [vmctl](https://docs.victoriametrics.com/vmctl.html) for migrating your data from Prometheus to VictoriaMetrics.
|
||||
|
||||
See [this doc](https://docs.victoriametrics.com/vmctl.html#migrating-data-from-prometheus) for more details.
|
||||
BIN
docs/operator/migration_prometheus-conversion.png
Normal file
BIN
docs/operator/migration_prometheus-conversion.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 636 KiB |
72
docs/operator/monitoring.md
Normal file
72
docs/operator/monitoring.md
Normal file
|
|
@ -0,0 +1,72 @@
|
|||
---
|
||||
sort: 6
|
||||
weight: 6
|
||||
title: Monitoring
|
||||
---
|
||||
|
||||
# Monitoring of VictoriaMetrics Operator
|
||||
|
||||
VictoriaMetrics operator exports internal metrics in Prometheus exposition format at `/metrics` page.
|
||||
|
||||
These metrics can be scraped via [vmagent](./resources/vmagent.md) or Prometheus.
|
||||
|
||||
## Dashboard
|
||||
|
||||
Official Grafana dashboard available for [vmoperator](https://grafana.com/grafana/dashboards/17869-victoriametrics-operator/).
|
||||
|
||||
<img src="monitoring_operator-dashboard.jpeg" width=1200>
|
||||
|
||||
Graphs on the dashboards contain useful hints - hover the `i` icon in the top left corner of each graph to read it.
|
||||
|
||||
<!-- TODO: alerts for operator -->
|
||||
|
||||
## Configuration
|
||||
|
||||
### Helm-chart victoria-metrics-k8s-stack
|
||||
|
||||
In [victoria-metrics-k8s-stack](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-k8s-stack/README.md) helm-chart operator self-scrapes metrics by default.
|
||||
|
||||
This helm-chart also includes [official grafana dashboard for operator](#dashboard).
|
||||
|
||||
### Helm-chart victoria-metrics-operator
|
||||
|
||||
With [victoria-metrics-operator](https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-operator/README.md) you can use following parameter in `values.yaml`:
|
||||
|
||||
```yaml
|
||||
# values.yaml
|
||||
#...
|
||||
# -- configures monitoring with serviceScrape. VMServiceScrape must be pre-installed
|
||||
serviceMonitor:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
This parameter makes helm-chart to create a scrape-object for installed operator instance.
|
||||
|
||||
You will also need to deploy a (vmsingle)[./resources/vmsingle.md] where the metrics will be collected.
|
||||
|
||||
### Pure operator installation
|
||||
|
||||
With pure operator installation you can use config with separate vmsingle and scrape object for operator like that:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMServiceScrape
|
||||
metadata:
|
||||
name: vmoperator
|
||||
namespace: monitoring
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/instance: vm-operator
|
||||
app.kubernetes.io/name: victoria-metrics-operator
|
||||
endpoints:
|
||||
- port: http
|
||||
namespaceSelector:
|
||||
matchNames:
|
||||
- monitoring
|
||||
```
|
||||
|
||||
See more info about object [VMServiceScrape](./resources/vmservicescrape.md).
|
||||
|
||||
You will also need a [vmsingle](https://docs.victoriametrics.com/vmoperatos/resources/vmsingle.html) where the metrics will be collected.
|
||||
|
||||
File diff suppressed because it is too large
Load diff
BIN
docs/operator/quick-start_alert-1.png
Normal file
BIN
docs/operator/quick-start_alert-1.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 312 KiB |
BIN
docs/operator/quick-start_alert-2.png
Normal file
BIN
docs/operator/quick-start_alert-2.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 378 KiB |
BIN
docs/operator/quick-start_cluster-scheme.png
Normal file
BIN
docs/operator/quick-start_cluster-scheme.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 405 KiB |
BIN
docs/operator/quick-start_select-1.png
Normal file
BIN
docs/operator/quick-start_select-1.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 371 KiB |
BIN
docs/operator/quick-start_select-2.png
Normal file
BIN
docs/operator/quick-start_select-2.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 604 KiB |
BIN
docs/operator/quick-start_values.png
Normal file
BIN
docs/operator/quick-start_values.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 944 KiB |
220
docs/operator/resources/README.md
Normal file
220
docs/operator/resources/README.md
Normal file
|
|
@ -0,0 +1,220 @@
|
|||
---
|
||||
sort: 14
|
||||
weight: 14
|
||||
title: Custom resources
|
||||
---
|
||||
|
||||
# Custom resource kinds
|
||||
|
||||
This documentation section describes the design and interaction between the custom resource definitions (CRD) that the Victoria
|
||||
Metrics Operator introduces.
|
||||
|
||||
[Operator](../README.md) introduces the following custom resources:
|
||||
|
||||
- [VMAgent](./vmagent.md)
|
||||
- [VMAlert](./vmalert.md)
|
||||
- [VMAlertManager](./vmalertmanager.md)
|
||||
- [VMAlertManagerConfig](./vmalertmanagerconfig.md)
|
||||
- [VMAuth](./vmauth.md)
|
||||
- [VMCluster](./vmcluster.md)
|
||||
- [VMNodeScrape](./vmnodescrape.md)
|
||||
- [VMPodScrape](./vmpodscrape.md)
|
||||
- [VMProbe](./vmprobe.md)
|
||||
- [VMRule](./vmrule.md)
|
||||
- [VMServiceScrape](./vmservicescrape.md)
|
||||
- [VMStaticScrape](./vmstaticscrape.md)
|
||||
- [VMSingle](./vmsingle.md)
|
||||
- [VMUser](./vmuser.md)
|
||||
|
||||
Here is the scheme of relations between the custom resources:
|
||||
|
||||
<img src="README_cr-relations.png" width="1200">
|
||||
|
||||
## Specification
|
||||
|
||||
You can find the specification for the custom resources on **[API Docs](../api.md)**.
|
||||
|
||||
### Extra arguments
|
||||
|
||||
If you can't find necessary field in the specification of custom resource,
|
||||
you can use `extraArgs` field for passing additional arguments to the application.
|
||||
|
||||
Field `extraArgs` is supported for the following custom resources:
|
||||
|
||||
- [VMAgent spec](../api.md#vmagentspec)
|
||||
- [VMAlert spec](../api.md#vmalertspec)
|
||||
- [VMAlertManager spec](../api.md#vmalertmanagerspec)
|
||||
- [VMAuth spec](../api.md#vmauthspec)
|
||||
- [VMCluster/vmselect spec](../api.md#vmselect)
|
||||
- [VMCluster/vminsert spec](../api.md#vminsert)
|
||||
- [VMCluster/vmstorage spec](../api.md#vmstorage)
|
||||
- [VMSingle spec](../api.md#vmsinglespec)
|
||||
|
||||
Supported flags for each application can be found the in the corresponding documentation:
|
||||
|
||||
- [VMAgent flags](https://docs.victoriametrics.com/vmagent.html#advanced-usage)
|
||||
- [VMAlert](https://docs.victoriametrics.com/vmalert.html#configuration)
|
||||
- [VMAuth](https://docs.victoriametrics.com/vmauth.html#advanced-usage)
|
||||
- [VMCluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#list-of-command-line-flags)
|
||||
- [VMSingle](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#list-of-command-line-flags)
|
||||
|
||||
Usage example:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMSingle
|
||||
metadata:
|
||||
name: vmsingle-example-exrtaargs
|
||||
spec:
|
||||
retentionPeriod: "1"
|
||||
extraArgs:
|
||||
dedup.minScrapeInterval: 60s
|
||||
# ...
|
||||
```
|
||||
|
||||
### Extra environment variables
|
||||
|
||||
Flag can be replaced with environment variable, it's useful for retrieving value from secret.
|
||||
You can use `extraEnvs` field for passing additional arguments to the application.
|
||||
|
||||
Usage example:
|
||||
|
||||
```yaml
|
||||
kind: VMSingle
|
||||
metadata:
|
||||
name: vmsingle-example--exrtaenvs
|
||||
spec:
|
||||
retentionPeriod: "1"
|
||||
extraEnvs:
|
||||
- name: DEDUP_MINSCRAPEINTERVAL
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: vm-secret
|
||||
key: dedup
|
||||
```
|
||||
|
||||
This feature really useful for using with
|
||||
[`-envflag.enable` command-line argument](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#environment-variables).
|
||||
|
||||
## Examples
|
||||
|
||||
Page for every custom resource contains examples section:
|
||||
|
||||
- [VMAgent examples](./vmagent.md#examples)
|
||||
- [VMAlert examples](./vmalert.md#examples)
|
||||
- [VMAlertmanager examples](./vmalertmanager.md#examples)
|
||||
- [VMAlertmanagerConfig examples](./vmalertmanagerconfig.md#examples)
|
||||
- [VMAuth examples](./vmauth.md#examples)
|
||||
- [VMCluster examples](./vmcluster.md#examples)
|
||||
- [VMNodeScrape examples](./vmnodescrape.md#examples)
|
||||
- [VMPodScrape examples](./vmpodscrape.md#examples)
|
||||
- [VMProbe examples](./vmprobe.md#examples)
|
||||
- [VMRule examples](./vmrule.md#examples)
|
||||
- [VMServiceScrape examples](./vmservicescrape.md#examples)
|
||||
- [VMStaticScrape examples](./vmstaticscrape.md#examples)
|
||||
- [VMSingle examples](./vmsingle.md#examples)
|
||||
- [VMUser examples](./vmuser.md#examples)
|
||||
|
||||
In addition, you can find examples of the custom resources for VIctoriMetrics operator in
|
||||
the **[examples directory](https://github.com/VictoriaMetrics/operator/tree/master/config/examples) of operator repository**.
|
||||
|
||||
## Managing versions of VM
|
||||
|
||||
Every custom resource with deployable application has a fields for specifying version (docker image) of component:
|
||||
|
||||
- [Managing versions for VMAgent](./vmagent.md#version-management)
|
||||
- [Managing versions for VMAlert](./vmalert.md#version-management)
|
||||
- [Managing versions for VMAlertmanager](./vmalertmanager.md#version-management)
|
||||
- [Managing versions for VMAuth](./vmauth.md#version-management)
|
||||
- [Managing versions for VMCluster](./vmcluster.md#version-management)
|
||||
- [Managing versions for VMSingle](./vmsingle.md#version-management)
|
||||
|
||||
## High availability
|
||||
|
||||
VictoriaMetrics operator support high availability for each component of the monitoring stack:
|
||||
|
||||
- [VMAgent](./vmagent.md#high-availability)
|
||||
- [VMAlert](./vmalert.md#high-availability)
|
||||
- [VMAlertmanager](./vmalertmanager.md#high-availability)
|
||||
- [VMAuth](./vmauth.md#high-availability)
|
||||
- [VMCluster](./vmcluster.md#high-availability)
|
||||
|
||||
In addition, these CRD support common features, that can be used to increase high availability - resources above have the following fields:
|
||||
|
||||
- `affinity` - to schedule pods on different nodes ([affinity and anti-affinity in kubernetes docs](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)),
|
||||
- `tolerations` - to schedule pods on nodes with taints ([taints and tolerations in kubernetes docs](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/)),
|
||||
- `nodeSelector` - to schedule pods on nodes with specific labels ([node selector in kubernetes docs](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector)),
|
||||
- `topologySpreadConstraints` - to schedule pods on different nodes in the same topology ([topology spread constraints in kubernetes docs](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#pod-topology-spread-constraints)).
|
||||
|
||||
See details about these fields in the [Specification](#specification).
|
||||
|
||||
## Enterprise features
|
||||
|
||||
Operator supports following [Enterprise features for VictoriaMetrics components](https://docs.victoriametrics.com/enterprise.html):
|
||||
|
||||
- [VMAgent Enterprise features](./vmagent.md#enterprise-features):
|
||||
- [Reading metrics from kafka](./vmagent.md#reading-metrics-from-kafka)
|
||||
- [Writing metrics to kafka](./vmagent.md#writing-metrics-to-kafka)
|
||||
- [VMAlert Enterprise features](./vmalert.md#enterprise-features):
|
||||
- [Reading rules from object storage](./vmalert.md#reading-rules-from-object-storage)
|
||||
- [Multitenancy](./vmalert.md#multitenancy)
|
||||
- [VMAuth Enterprise features](./vmauth.md#enterprise-features)
|
||||
- [IP Filters](./vmauth.md#ip-filters)
|
||||
- [VMCluster Enterprise features](./vmcluster.md#enterprise-features)
|
||||
- [Downsampling](./vmcluster.md#downsampling)
|
||||
- [Multiple retentions / Retention filters](./vmcluster.md#retention-filters)
|
||||
- [Advanced per-tenant statistic](./vmcluster.md#advanced-per-tenant-statistic)
|
||||
- [mTLS protection](./vmcluster.md#mtls-protection)
|
||||
- [Backup atomation](./vmcluster.md#backup-atomation)
|
||||
- [VMRule Enterprise features](./vmrule.md#enterprise-features)
|
||||
- [Multitenancy](./vmrule.md#multitenancy)
|
||||
- [VMSingle Enterprise features](./vmsingle.md#enterprise-features)
|
||||
- [Downsampling](./vmsingle.md#downsampling)
|
||||
- [Retention filters](./vmsingle.md#retention-filters)
|
||||
- [Backup atomation](./vmsingle.md#backup-atomation)
|
||||
- [VMUser Enterprise features](./vmuser.md#enterprise-features)
|
||||
- [IP Filters](./vmuser.md#ip-filters)
|
||||
|
||||
More information about enterprise features you can read
|
||||
on [VictoriaMetrics Enterprise page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
|
||||
|
||||
## Configuration synchronization
|
||||
|
||||
### Basic concepts
|
||||
|
||||
VictoriaMetrics applications, like many other applications with configuration file deployed at Kubernetes, uses `ConfigMaps` and `Secrets` for configuration files.
|
||||
Usually, it's out of application scope to watch for configuration on-disk changes.
|
||||
Applications reload their configuration by a signal from a user or some other tool, that knows how to watch for updates.
|
||||
At Kubernetes, the most popular design for this case is a sidecar container, that watches for configuration file changes and sends an HTTP request to the application.
|
||||
|
||||
`Configmap` or `Secret` that mounted at `Pod` holds a copy of its content.
|
||||
Kubernetes component `kubelet` is responsible for content synchronization between an object at Kubernetes API and a file served on disk.
|
||||
It's not efficient to sync its content immediately, and `kubelet` eventually synchronizes it. There is a configuration option, that controls this period.
|
||||
|
||||
That's why, applications managed by operator don't receive changes immediately. It usually takes 1-2 min, before content will be updated.
|
||||
|
||||
It may trigger errors when an application was deleted, but [`VMAgent`](./README.mdvmagent) still tries to scrape it.
|
||||
|
||||
### Possible mitigations
|
||||
|
||||
The naive solution for this case decrease the synchronization period. But it configures globally and may be hard for operator users.
|
||||
|
||||
That's why operator uses a few hacks.
|
||||
|
||||
For `ConfigMap` updates, operator changes annotation with a time of `Configmap` content update. It triggers `ConfigMap`'s content synchronization by kubelet immediately.
|
||||
It's the case for `VMAlert`, it uses `ConfigMap` as a configuration source.
|
||||
|
||||
For `Secret` it doesn't work. And operator offers its implementation for side-car container. It can be configured with env variable for operator:
|
||||
|
||||
```
|
||||
- name: VM_USECUSTOMCONFIGRELOADER
|
||||
value: "true"
|
||||
```
|
||||
|
||||
If it's defined, operator uses own [config-reloader](https://github.com/VictoriaMetrics/operator/tree/master/internal/config-reloader)
|
||||
instead of [prometheus-config-reload](https://github.com/prometheus-operator/prometheus-operator/tree/main/cmd/prometheus-config-reloader).
|
||||
|
||||
It watches corresponding `Secret` for changes with Kubernetes API watch call and writes content into emptyDir.
|
||||
This emptyDir shared with the application.
|
||||
In case of content changes, `config-reloader` sends HTTP requests to the application.
|
||||
It greatly reduces the time for configuration synchronization.
|
||||
BIN
docs/operator/resources/README_cr-relations.png
Normal file
BIN
docs/operator/resources/README_cr-relations.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.3 MiB |
720
docs/operator/resources/vmagent.md
Normal file
720
docs/operator/resources/vmagent.md
Normal file
|
|
@ -0,0 +1,720 @@
|
|||
# VMAgent
|
||||
|
||||
`VMAgent` represents agent, which helps you collect metrics from various sources and stores them in VictoriaMetrics.
|
||||
The `VMAgent` CRD declaratively defines a desired [VMAgent](https://docs.victoriametrics.com/vmagent)
|
||||
setup to run in a Kubernetes cluster.
|
||||
|
||||
It requires access to Kubernetes API and you can create RBAC for it first, it can be found
|
||||
at [`examples/vmagent_rbac.yaml`](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmagent_rbac.yaml)
|
||||
Or you can use default rbac account, that will be created for `VMAgent` by operator automatically.
|
||||
|
||||
For each `VMAgent` resource Operator deploys a properly configured `Deployment` in the same namespace.
|
||||
The VMAgent `Pod`s are configured to mount a `Secret` prefixed with `<VMAgent-name>` containing the configuration
|
||||
for VMAgent.
|
||||
|
||||
For each `VMAgent` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
|
||||
name `<VMAgent-name>`.
|
||||
|
||||
The CRD specifies which `VMServiceScrape` should be covered by the deployed VMAgent instances based on label selection.
|
||||
The Operator then generates a configuration based on the included `VMServiceScrape`s and updates the `Secret` which
|
||||
contains the configuration. It continuously does so for all changes that are made to the `VMServiceScrape`s or the
|
||||
`VMAgent` resource itself.
|
||||
|
||||
If no selection of `VMServiceScrape`s is provided - Operator leaves management of the `Secret` to the user,
|
||||
so user can set custom configuration while still benefiting from the Operator's capabilities of managing VMAgent setups.
|
||||
|
||||
## Specification
|
||||
|
||||
You can see the full actual specification of the `VMAgent` resource in the **[API docs -> VMAgent](../api.md#vmagent)**.
|
||||
|
||||
If you can't find necessary field in the specification of the custom resource,
|
||||
see [Extra arguments section](./README.md#extra-arguments).
|
||||
|
||||
Also, you can check out the [examples](#examples) section.
|
||||
|
||||
## Scraping
|
||||
|
||||
`VMAgent` supports scraping targets with:
|
||||
|
||||
- [VMServiceScrape](./vmservicescrape.md),
|
||||
- [VMPodScrape](./vmpodscrape.md),
|
||||
- [VMNodeScrape](./vmnodescrape.md),
|
||||
- [VMStaticScrape](./vmstaticscrape.md),
|
||||
- [VMProbe](./vmprobe.md).
|
||||
|
||||
These objects tell VMAgent from which targets and how to collect metrics and
|
||||
generate part of [VMAgent](./vmagent.md) scrape configuration.
|
||||
|
||||
For filtering scrape objects `VMAgent` uses selectors.
|
||||
Selectors are defined with suffixes - `NamespaceSelector` and `Selector` for each type of scrape objects in spec of `VMAgent`:
|
||||
|
||||
- `serviceScrapeNamespaceSelector` and `serviceScrapeSelector` for selecting [VMServiceScrape](./vmservicescrape.md) objects,
|
||||
- `podScrapeNamespaceSelector` and `podScrapeSelector` for selecting [VMPodScrape](./vmpodscrape.md) objects,
|
||||
- `probeNamespaceSelector` and `probeSelector` for selecting [VMProbe](./vmprobe.md) objects,
|
||||
- `staticScrapeNamespaceSelector` and `staticScrapeSelector` for selecting [VMStaticScrape](./vmstaticscrape.md) objects,
|
||||
- `nodeScrapeNamespaceSelector` and `nodeScrapeSelector` for selecting [VMNodeScrape](./vmnodescrape.md) objects.
|
||||
|
||||
It allows configuring objects access control across namespaces and different environments.
|
||||
Specification of selectors you can see in [this doc](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#labelselector-v1-meta).
|
||||
|
||||
In addition to the above selectors, the filtering of objects in a cluster is affected by the field `selectAllByDefault` of `VMAgent` spec and environment variable `WATCH_NAMESPACE` for operator.
|
||||
|
||||
Following rules are applied:
|
||||
|
||||
- If `*NamespaceSelector` and `*Selector` both undefined, then by default select nothing. With option set - `spec.selectAllByDefault: true`, select all objects of given type.
|
||||
- If `*NamespaceSelector` defined, `*Selector` undefined, then all objects are matching at namespaces for given `*NamespaceSelector`.
|
||||
- If `*NamespaceSelector` undefined, `*Selector` defined, then all objects at `VMAgent`'s namespaces are matching for given `*Selector`.
|
||||
- If `*NamespaceSelector` and `*Selector` both defined, then only objects at namespaces matched `*NamespaceSelector` for given `*Selector` are matching.
|
||||
|
||||
Here's a more visual and more detailed view:
|
||||
|
||||
| `*NamespaceSelector` | `*Selector` | `selectAllByDefault` | `WATCH_NAMESPACE` | Selected objects |
|
||||
|----------------------|-------------|----------------------|-------------------|-------------------------------------------------------------------------------------------------------|
|
||||
| undefined | undefined | false | undefined | nothing |
|
||||
| undefined | undefined | **true** | undefined | all objects of given type (`*`) in the cluster |
|
||||
| **defined** | undefined | any | undefined | all objects of given type (`*`) at namespaces for given `*NamespaceSelector` |
|
||||
| undefined | **defined** | any | undefined | all objects of given type (`*`) only at `VMAgent`'s namespace are matching for given `Selector |
|
||||
| **defined** | **defined** | any | undefined | all objects of given type (`*`) only at namespaces matched `*NamespaceSelector` for given `*Selector` |
|
||||
| any | undefined | any | **defined** | all objects of given type (`*`) only at `VMAgent`'s namespace |
|
||||
| any | **defined** | any | **defined** | all objects of given type (`*`) only at `VMAgent`'s namespace for given `*Selector` |
|
||||
|
||||
More details about `WATCH_NAMESPACE` variable you can read in [this doc](../configuration.md#namespaced-mode).
|
||||
|
||||
Here are some examples of `VMAgent` configuration with selectors:
|
||||
|
||||
```yaml
|
||||
# select all scrape objects in the cluster
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-select-all
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
|
||||
---
|
||||
|
||||
# select all scrape objects in specific namespace (my-namespace)
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-select-ns
|
||||
spec:
|
||||
# ...
|
||||
serviceScrapeNamespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: my-namespace
|
||||
podScrapeNamespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: my-namespace
|
||||
nodeScrapeNamespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: my-namespace
|
||||
staticScrapeNamespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: my-namespace
|
||||
probeNamespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: my-namespace
|
||||
```
|
||||
|
||||
## High availability
|
||||
|
||||
<!-- TODO: health checks -->
|
||||
|
||||
### Replication and deduplication
|
||||
|
||||
To run VMAgent in a highly available manner at first you have to configure deduplication in Victoria Metrics
|
||||
according [this doc for VMSingle](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#deduplication)
|
||||
or [this doc for VMCluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#deduplication).
|
||||
|
||||
You can do it with `extraArgs` on [`VMSingle`](./vmsingle.md):
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMSingle
|
||||
metadata:
|
||||
name: vmsingle-example
|
||||
spec:
|
||||
# ...
|
||||
extraArgs:
|
||||
dedup.minScrapeInterval: 30s
|
||||
# ...
|
||||
```
|
||||
|
||||
For [`VMCluster`](./vmcluster.md) you can do it with `vmstorage.extraArgs` and `vmselect.extraArgs`:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: vmcluster-example
|
||||
spec:
|
||||
# ...
|
||||
vmselect:
|
||||
extraArgs:
|
||||
dedup.minScrapeInterval: 30s
|
||||
# ...
|
||||
vmstorage:
|
||||
extraArgs:
|
||||
dedup.minScrapeInterval: 30s
|
||||
# ...
|
||||
```
|
||||
|
||||
Deduplication is automatically enabled with `replicationFactor > 1` on `VMCLuster`.
|
||||
|
||||
After enabling deduplication you can increase replicas for VMAgent.
|
||||
|
||||
For instance, let's create `VMAgent` with 2 replicas:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-ha-example
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
vmAgentExternalLabelName: vmagent_ha
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example.default.svc:8429/api/v1/write"
|
||||
# Replication:
|
||||
scrapeInterval: 30s
|
||||
replicaCount: 2
|
||||
# ...
|
||||
```
|
||||
|
||||
Now, even if something happens to one of the vmagent, you'll still have the data.
|
||||
|
||||
### StatefulMode
|
||||
|
||||
VMAgent supports [persistent buffering](https://docs.victoriametrics.com/vmagent.html#replication-and-high-availability)
|
||||
for sending data to remote storage. By default, operator set `-remoteWrite.tmpDataPath` for `VMAgent` to `/tmp` (that use k8s ephemeral storage)
|
||||
and `VMAgent` loses state of the PersistentQueue on pod restarts.
|
||||
|
||||
In `StatefulMode` `VMAgent` doesn't lose state of the PersistentQueue (file-based buffer size for unsent data) on pod restarts.
|
||||
Operator creates `StatefulSet` and, with provided `PersistentVolumeClaimTemplate` at `StatefulStorage` configuration param, metrics queue is stored on disk.
|
||||
|
||||
Example of configuration for `StatefulMode`:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-ha-example
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
vmAgentExternalLabelName: vmagent_ha
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example.default.svc:8429/api/v1/write"
|
||||
# Replication:
|
||||
scrapeInterval: 30s
|
||||
replicaCount: 2
|
||||
# StatefulMode:
|
||||
statefulMode: true
|
||||
statefulStorage:
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
resources:
|
||||
requests:
|
||||
storage: 20Gi
|
||||
# ...
|
||||
```
|
||||
|
||||
### Sharding
|
||||
|
||||
Operator supports sharding with [cluster mode of vmagent](https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets)
|
||||
for **scraping big number of targets**.
|
||||
|
||||
Sharding for `VMAgent` distributes scraping between multiple deployments of `VMAgent`.
|
||||
|
||||
Example usage (it is a complete example of `VMAgent` with high availability features):
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-ha-example
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
vmAgentExternalLabelName: vmagent_ha
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example.default.svc:8429/api/v1/write"
|
||||
# Replication:
|
||||
scrapeInterval: 30s
|
||||
replicaCount: 2
|
||||
# StatefulMode:
|
||||
statefulMode: true
|
||||
statefulStorage:
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
resources:
|
||||
requests:
|
||||
storage: 20Gi
|
||||
# Sharding
|
||||
shardCount: 5
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
preferredDuringSchedulingIgnoredDuringExecution:
|
||||
- podAffinityTerm:
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
shard-num: '%SHARD_NUM%'
|
||||
topologyKey: kubernetes.io/hostname
|
||||
# ...
|
||||
```
|
||||
|
||||
This configuration produces `5` deployments with `2` replicas at each.
|
||||
Each deployment has its own shard num and scrapes only `1/5` of all targets.
|
||||
|
||||
Also, you can use special placeholder `%SHARD_NUM%` in fields of `VMAgent` specification
|
||||
and operator will replace it with current shard num of vmagent when creating deployment or statefullset for vmagent.
|
||||
|
||||
In the example above, the `%SHARD_NUM%` placeholder is used in the `podAntiAffinity` section,
|
||||
which recommend to scheduler that pods with the same shard num (label `shard-num` in the pod template)
|
||||
are not deployed on the same node. You can use another `topologyKey` for availability zone or region instead of nodes.
|
||||
|
||||
**Note** that at the moment operator doesn't use `-promscrape.cluster.replicationFactor` parameter of `VMAgent` and
|
||||
creates `replicaCount` of replicas for each shard (which leads greater resource consumption).
|
||||
This will be fixed in the future, more details can be seen in [this issue](https://github.com/VictoriaMetrics/operator/issues/604).
|
||||
|
||||
Also see [this example](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmagent_stateful_with_sharding.yaml).
|
||||
|
||||
## Additional scrape configuration
|
||||
|
||||
AdditionalScrapeConfigs is an additional way to add scrape targets in `VMAgent` CRD.
|
||||
|
||||
There are two options for adding targets into `VMAgent`:
|
||||
|
||||
- [inline configuration into CRD](#inline-additional-scrape-configuration-in-vmagent-crd),
|
||||
- [defining it as a Kubernetes Secret](#define-additional-scrape-configuration-as-a-kubernetes-secret).
|
||||
|
||||
No validation happens during the creation of configuration. However, you must validate job specs, and it must follow job spec configuration.
|
||||
Please check [scrape_configs documentation](https://docs.victoriametrics.com/sd_configs.html#scrape_configs) as references.
|
||||
|
||||
### Inline Additional Scrape Configuration in VMAgent CRD
|
||||
|
||||
You need to add scrape configuration directly to the `vmagent spec.inlineScrapeConfig`. It is raw text in YAML format.
|
||||
See example below
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-example
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
inlineScrapeConfig: |
|
||||
- job_name: "prometheus"
|
||||
static_configs:
|
||||
- targets: ["localhost:9090"]
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example.default.svc:8429/api/v1/write"
|
||||
# ...
|
||||
```
|
||||
|
||||
**Note**: Do not use passwords and tokens with inlineScrapeConfig use Secret instead.
|
||||
|
||||
## Define Additional Scrape Configuration as a Kubernetes Secret
|
||||
|
||||
You need to define Kubernetes Secret with a key.
|
||||
|
||||
The key is `prometheus-additional.yaml` in the example below:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: additional-scrape-configs
|
||||
stringData:
|
||||
prometheus-additional.yaml: |
|
||||
- job_name: "prometheus"
|
||||
static_configs:
|
||||
- targets: ["localhost:9090"]
|
||||
```
|
||||
|
||||
After that, you need to specify the secret's name and key in VMAgent CRD in `additionalScrapeConfigs` section:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-example
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
additionalScrapeConfigs:
|
||||
name: additional-scrape-configs
|
||||
key: prometheus-additional.yaml
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example.default.svc:8429/api/v1/write"
|
||||
# ...
|
||||
```
|
||||
|
||||
**Note**: You can specify only one Secret in the VMAgent CRD configuration so use it for all additional scrape configurations.
|
||||
|
||||
## Relabeling
|
||||
|
||||
`VMAgent` supports global relabeling for all metrics and per remoteWrite target relabel config.
|
||||
|
||||
Note in some cases, you don't need relabeling, `key=value` label pairs can be added to the all scrapped metrics with `spec.externalLabels` for `VMAgent`:
|
||||
|
||||
```yaml
|
||||
# simple label add config
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-example
|
||||
spec:
|
||||
externalLabels:
|
||||
clusterid: some_cluster
|
||||
```
|
||||
|
||||
`VMAgent` CR supports relabeling with [custom configMap](#relabeling-config-in-configmap)
|
||||
or [inline defined at CRD](#inline-relabeling-config).
|
||||
|
||||
### Relabeling config in Configmap
|
||||
|
||||
Quick tour how to create `ConfigMap` with relabeling configuration:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: vmagent-relabel
|
||||
data:
|
||||
global-relabel.yaml: |
|
||||
- target_label: bar
|
||||
- source_labels: [aa]
|
||||
separator: "foobar"
|
||||
regex: "foo.+bar"
|
||||
target_label: aaa
|
||||
replacement: "xxx"
|
||||
- action: keep
|
||||
source_labels: [aaa]
|
||||
- action: drop
|
||||
source_labels: [aaa]
|
||||
target-1-relabel.yaml: |
|
||||
- action: keep_if_equal
|
||||
source_labels: [foo, bar]
|
||||
- action: drop_if_equal
|
||||
source_labels: [foo, bar]
|
||||
```
|
||||
|
||||
Second, add `relabelConfig` to `VMagent` spec for global relabeling with name of `Configmap` - `vmagent-relabel` and key `global-relabel.yaml`.
|
||||
|
||||
For relabeling per remoteWrite target, add `urlRelabelConfig` name of `Configmap` - `vmagent-relabel`
|
||||
and key `target-1-relabel.yaml` to one of remoteWrite target for relabeling only for those target:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-example
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
relabelConfig:
|
||||
name: "vmagent-relabel"
|
||||
key: "global-relabel.yaml"
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
|
||||
- url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
|
||||
urlRelabelConfig:
|
||||
name: "vmagent-relabel"
|
||||
key: "target-1-relabel.yaml"
|
||||
```
|
||||
|
||||
### Inline relabeling config
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-example
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
inlineRelabelConfig:
|
||||
- target_label: bar
|
||||
- source_labels: [aa]
|
||||
separator: "foobar"
|
||||
regex: "foo.+bar"
|
||||
target_label: aaa
|
||||
replacement: "xxx"
|
||||
- action: keep
|
||||
source_labels: [aaa]
|
||||
- action: drop
|
||||
source_labels: [aaa]
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
|
||||
- url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
|
||||
inlineUrlRelabelConfig:
|
||||
- action: keep_if_equal
|
||||
source_labels: [foo, bar]
|
||||
- action: drop_if_equal
|
||||
source_labels: [foo, bar]
|
||||
```
|
||||
|
||||
### Combined example
|
||||
|
||||
It's also possible to use both features in combination.
|
||||
|
||||
First will be added relabeling configs from `inlineRelabelConfig`, then `relabelConfig` from configmap.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: vmagent-relabel
|
||||
data:
|
||||
global-relabel.yaml: |
|
||||
- target_label: bar
|
||||
- source_labels: [aa]
|
||||
separator: "foobar"
|
||||
regex: "foo.+bar"
|
||||
target_label: aaa
|
||||
replacement: "xxx"
|
||||
- action: keep
|
||||
source_labels: [aaa]
|
||||
- action: drop
|
||||
source_labels: [aaa]
|
||||
target-1-relabel.yaml: |
|
||||
- action: keep_if_equal
|
||||
source_labels: [foo, bar]
|
||||
- action: drop_if_equal
|
||||
source_labels: [foo, bar]
|
||||
```
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: example-vmagent
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
inlineRelabelConfig:
|
||||
- target_label: bar1
|
||||
- source_labels: [aa]
|
||||
relabelConfig:
|
||||
name: "vmagent-relabel"
|
||||
key: "global-relabel.yaml"
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
|
||||
- url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
|
||||
urlRelabelConfig:
|
||||
name: "vmagent-relabel"
|
||||
key: "target-1-relabel.yaml"
|
||||
inlineUrlRelabelConfig:
|
||||
- action: keep_if_equal
|
||||
source_labels: [foo1, bar2]
|
||||
```
|
||||
|
||||
Resulted configmap, mounted to `VMAgent` pod:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
data:
|
||||
global_relabeling.yaml: |
|
||||
- target_label: bar1
|
||||
- source_labels:
|
||||
- aa
|
||||
- target_label: bar
|
||||
- source_labels: [aa]
|
||||
separator: "foobar"
|
||||
regex: "foo.+bar"
|
||||
target_label: aaa
|
||||
replacement: "xxx"
|
||||
- action: keep
|
||||
source_labels: [aaa]
|
||||
- action: drop
|
||||
source_labels: [aaa]
|
||||
url_rebaling-1.yaml: |
|
||||
- source_labels:
|
||||
- foo1
|
||||
- bar2
|
||||
action: keep_if_equal
|
||||
- action: keep_if_equal
|
||||
source_labels: [foo, bar]
|
||||
- action: drop_if_equal
|
||||
source_labels: [foo, bar]
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
finalizers:
|
||||
- apps.victoriametrics.com/finalizer
|
||||
labels:
|
||||
app.kubernetes.io/component: monitoring
|
||||
app.kubernetes.io/instance: example-vmagent
|
||||
app.kubernetes.io/name: vmagent
|
||||
managed-by: vm-operator
|
||||
name: relabelings-assets-vmagent-example-vmagent
|
||||
namespace: default
|
||||
ownerReferences:
|
||||
- apiVersion: operator.victoriametrics.com/v1beta1
|
||||
blockOwnerDeletion: true
|
||||
controller: true
|
||||
kind: VMAgent
|
||||
name: example-vmagent
|
||||
uid: 7e9fb838-65da-4443-a43b-c00cd6c4db5b
|
||||
```
|
||||
|
||||
### Additional information
|
||||
|
||||
`VMAgent` also has some extra options for relabeling actions, you can check it [docs](https://docs.victoriametrics.com/vmagent#relabeling).
|
||||
|
||||
## Version management
|
||||
|
||||
To set `VMAgent` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: example-vmagent
|
||||
spec:
|
||||
image:
|
||||
repository: victoriametrics/victoria-metrics
|
||||
tag: v1.93.4
|
||||
pullPolicy: Always
|
||||
# ...
|
||||
```
|
||||
|
||||
Also, you can specify `imagePullSecrets` if you are pulling images from private repo:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: example-vmagent
|
||||
spec:
|
||||
image:
|
||||
repository: victoriametrics/victoria-metrics
|
||||
tag: v1.93.4
|
||||
pullPolicy: Always
|
||||
imagePullSecrets:
|
||||
- name: my-repo-secret
|
||||
# ...
|
||||
```
|
||||
|
||||
## Enterprise features
|
||||
|
||||
VMAgent supports feature [Kafka integration](https://docs.victoriametrics.com/vmagent.html#kafka-integration)
|
||||
from [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
|
||||
|
||||
For using Enterprise version of [vmagent](https://docs.victoriametrics.com/vmagent.html)
|
||||
you need to change version of `vmagent` to version with `-enterprise` suffix using [Version management](#version-management).
|
||||
|
||||
All the enterprise apps require `-eula` command-line flag to be passed to them.
|
||||
This flag acknowledges that your usage fits one of the cases listed on [this page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
|
||||
So you can use [extraArgs](./README.md#extra-arguments) for passing this flag to `VMAgent`:
|
||||
|
||||
After that you can pass [Kafka integration](https://docs.victoriametrics.com/vmagent.html#kafka-integration)
|
||||
flags to `VMAgent` with [extraArgs](./README.md#extra-arguments).
|
||||
|
||||
### Reading metrics from Kafka
|
||||
|
||||
Here are complete example for [Reading metrics from Kafka](https://docs.victoriametrics.com/vmagent.html#reading-metrics-from-kafka):
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-ent-example
|
||||
spec:
|
||||
# enabling enterprise features
|
||||
image:
|
||||
# enterprise version of vmagent
|
||||
tag: v1.93.5-enterprise
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmagent enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# using enterprise features: reading metrics from kafka
|
||||
# more details about kafka integration you can read on https://docs.victoriametrics.com/vmagent.html#kafka-integration
|
||||
# more details about these and other flags you can read on https://docs.victoriametrics.com/vmagent.html#command-line-flags-for-kafka-consumer
|
||||
kafka.consumer.topic.brokers: localhost:9092
|
||||
kafka.consumer.topic.format: influx
|
||||
kafka.consumer.topic: metrics-by-telegraf
|
||||
kafka.consumer.topic.groupID: some-id
|
||||
|
||||
# ...other fields...
|
||||
```
|
||||
|
||||
### Writing metrics to Kafka
|
||||
|
||||
Here are complete example for [Writing metrics to Kafka](https://docs.victoriametrics.com/vmagent.html#writing-metrics-to-kafka):
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-ent-example
|
||||
spec:
|
||||
# enabling enterprise features
|
||||
image:
|
||||
# enterprise version of vmagent
|
||||
tag: v1.93.5-enterprise
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmagent enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# using enterprise features: writing metrics to Kafka
|
||||
# more details about kafka integration you can read on https://docs.victoriametrics.com/vmagent.html#kafka-integration
|
||||
remoteWrite:
|
||||
# sasl with username and password
|
||||
- url: kafka://broker-1:9092/?topic=prom-rw-1&security.protocol=SASL_SSL&sasl.mechanisms=PLAIN
|
||||
# it requires to create kubernetes secret `kafka-basic-auth` with keys `username` and `password` in the same namespace
|
||||
basicAuth:
|
||||
username:
|
||||
name: kafka-basic-auth
|
||||
key: username
|
||||
password:
|
||||
name: kafka-basic-auth
|
||||
key: password
|
||||
# sasl with username and password from secret and tls
|
||||
- url: kafka://localhost:9092/?topic=prom-rw-2&security.protocol=SSL
|
||||
# it requires to create kubernetes secret `kafka-tls` with keys `ca.pem`, `cert.pem` and `key.pem` in the same namespace
|
||||
tlsConfig:
|
||||
ca:
|
||||
secret:
|
||||
name: kafka-tls
|
||||
key: ca.pem
|
||||
cert:
|
||||
secret:
|
||||
name: kafka-tls
|
||||
key: cert.pem
|
||||
keySecret:
|
||||
name: kafka-tls
|
||||
key: key.pem
|
||||
|
||||
# ...other fields...
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
```yaml
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: vmagent-example
|
||||
spec:
|
||||
selectAllByDefault: true
|
||||
replicaCount: 1
|
||||
scrapeInterval: 30s
|
||||
scrapeTimeout: 10s
|
||||
vmAgentExternalLabelName: example
|
||||
externalLabels:
|
||||
cluster: my-cluster
|
||||
remoteWrite:
|
||||
- url: "http://vmsingle-example.default.svc:8428/api/v1/write"
|
||||
inlineRelabelConfig:
|
||||
- action: labeldrop
|
||||
regex: "temp.*"
|
||||
```
|
||||
362
docs/operator/resources/vmalert.md
Normal file
362
docs/operator/resources/vmalert.md
Normal file
|
|
@ -0,0 +1,362 @@
|
|||
# VMAlert
|
||||
|
||||
`VMAlert` - executes a list of given [alerting](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/)
|
||||
or [recording](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) rules against configured address.
|
||||
|
||||
The `VMAlert` CRD declaratively defines a desired [VMAlert](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmalert)
|
||||
setup to run in a Kubernetes cluster.
|
||||
|
||||
It has few required config options - `datasource` and `notifier` are required, for other config parameters
|
||||
check [doc](../api.md#vmalert).
|
||||
|
||||
For each `VMAlert` resource, the Operator deploys a properly configured `Deployment` in the same namespace.
|
||||
The VMAlert `Pod`s are configured to mount a list of `Configmaps` prefixed with `<VMAlert-name>-number` containing
|
||||
the configuration for alerting rules.
|
||||
|
||||
For each `VMAlert` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
|
||||
name `<VMAlert-name>`.
|
||||
|
||||
## Specification
|
||||
|
||||
You can see the full actual specification of the `VMAlert` resource in the **[API docs -> VMAlert](../api.md#vmalert)**.
|
||||
|
||||
If you can't find necessary field in the specification of the custom resource,
|
||||
see [Extra arguments section](./README.md#extra-arguments).
|
||||
|
||||
Also, you can check out the [examples](#examples) section.
|
||||
|
||||
## Rules
|
||||
|
||||
The CRD specifies which `VMRule`s should be covered by the deployed `VMAlert` instances based on label selection.
|
||||
The Operator then generates a configuration based on the included `VMRule`s and updates the `Configmaps` containing
|
||||
the configuration. It continuously does so for all changes that are made to `VMRule`s or to the `VMAlert` resource itself.
|
||||
|
||||
Alerting rules are filtered by selectors `ruleNamespaceSelector` and `ruleSelector` in `VMAlert` CRD definition.
|
||||
For selecting rules from all namespaces you must specify it to empty value:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
ruleNamespaceSelector: {}
|
||||
```
|
||||
|
||||
[VMRUle](./vmrule.md) objects generate part of `VMAlert` configuration.
|
||||
|
||||
For filtering rules `VMAlert` uses selectors `ruleNamespaceSelector` and `ruleSelector`.
|
||||
It allows configuring rules access control across namespaces and different environments.
|
||||
Specification of selectors you can see in [this doc](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#labelselector-v1-meta).
|
||||
|
||||
In addition to the above selectors, the filtering of objects in a cluster is affected by the field `selectAllByDefault` of `VMAlert` spec and environment variable `WATCH_NAMESPACE` for operator.
|
||||
|
||||
Following rules are applied:
|
||||
|
||||
- If `ruleNamespaceSelector` and `ruleSelector` both undefined, then by default select nothing. With option set - `spec.selectAllByDefault: true`, select all vmrules.
|
||||
- If `ruleNamespaceSelector` defined, `ruleSelector` undefined, then all vmrules are matching at namespaces for given `ruleNamespaceSelector`.
|
||||
- If `ruleNamespaceSelector` undefined, `ruleSelector` defined, then all vmrules at `VMAgent`'s namespaces are matching for given `ruleSelector`.
|
||||
- If `ruleNamespaceSelector` and `ruleSelector` both defined, then only vmrules at namespaces matched `ruleNamespaceSelector` for given `ruleSelector` are matching.
|
||||
|
||||
Here's a more visual and more detailed view:
|
||||
|
||||
| `ruleNamespaceSelector` | `ruleSelector` | `selectAllByDefault` | `WATCH_NAMESPACE` | Selected rules |
|
||||
|-------------------------|----------------|----------------------|-------------------|------------------------------------------------------------------------------------------------------|
|
||||
| undefined | undefined | false | undefined | nothing |
|
||||
| undefined | undefined | **true** | undefined | all vmrules in the cluster |
|
||||
| **defined** | undefined | any | undefined | all vmrules are matching at namespaces for given `ruleNamespaceSelector` |
|
||||
| undefined | **defined** | any | undefined | all vmrules only at `VMAlert`'s namespace are matching for given `ruleSelector` |
|
||||
| **defined** | **defined** | any | undefined | all vmrules only at namespaces matched `ruleNamespaceSelector` for given `ruleSelector` are matching |
|
||||
| any | undefined | any | **defined** | all vmrules only at `VMAlert`'s namespace |
|
||||
| any | **defined** | any | **defined** | all vmrules only at `VMAlert`'s namespace for given `ruleSelector` are matching |
|
||||
|
||||
More details about `WATCH_NAMESPACE` variable you can read in [this doc](../configuration.md#namespaced-mode).
|
||||
|
||||
Here are some examples of `VMAlert` configuration with selectors:
|
||||
|
||||
```yaml
|
||||
# select all rule objects in the cluster
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: vmalert-select-all
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
|
||||
---
|
||||
|
||||
# select all rule objects in specific namespace (my-namespace)
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: vmalert-select-ns
|
||||
spec:
|
||||
# ...
|
||||
ruleNamespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: my-namespace
|
||||
```
|
||||
|
||||
## High availability
|
||||
|
||||
`VMAlert` can be launched with multiple replicas without an additional configuration as far [alertmanager](./vmalertmanager.md) is responsible for alert deduplication.
|
||||
|
||||
Note, if you want to use `VMAlert` with high-available [`VMAlertmanager`](./vmalertmanager.md), which has more than 1 replica.
|
||||
You have to specify all pod fqdns at `VMAlert.spec.notifiers.[url]`. Or you can use service discovery for notifier, examples:
|
||||
|
||||
- alertmanager:
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: vmalertmanager-example-alertmanager
|
||||
labels:
|
||||
app: vm-operator
|
||||
type: Opaque
|
||||
stringData:
|
||||
alertmanager.yaml: |
|
||||
global:
|
||||
resolve_timeout: 5m
|
||||
route:
|
||||
group_by: ['job']
|
||||
group_wait: 30s
|
||||
group_interval: 5m
|
||||
repeat_interval: 12h
|
||||
receiver: 'webhook'
|
||||
receivers:
|
||||
- name: 'webhook'
|
||||
webhook_configs:
|
||||
- url: 'http://alertmanagerwh:30500/'
|
||||
# ...
|
||||
|
||||
---
|
||||
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanager
|
||||
metadata:
|
||||
name: example
|
||||
namespace: default
|
||||
labels:
|
||||
usage: dedicated
|
||||
spec:
|
||||
replicaCount: 2
|
||||
configSecret: vmalertmanager-example-alertmanager
|
||||
configSelector: {}
|
||||
configNamespaceSelector: {}
|
||||
# ...
|
||||
```
|
||||
- vmalert with fqdns:
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: example-ha
|
||||
namespace: default
|
||||
spec:
|
||||
replicaCount: 2
|
||||
datasource:
|
||||
url: http://vmsingle-example.default.svc:8429
|
||||
notifiers:
|
||||
- url: http://vmalertmanager-example-0.vmalertmanager-example.default.svc:9093
|
||||
- url: http://vmalertmanager-example-1.vmalertmanager-example.default.svc:9093
|
||||
evaluationInterval: "10s"
|
||||
ruleSelector: {}
|
||||
# ...
|
||||
```
|
||||
- vmalert with service discovery:
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: example-ha
|
||||
namespace: default
|
||||
spec:
|
||||
replicaCount: 2
|
||||
datasource:
|
||||
url: http://vmsingle-example.default.svc:8429
|
||||
notifiers:
|
||||
- selector:
|
||||
namespaceSelector:
|
||||
matchNames:
|
||||
- default
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
usage: dedicated
|
||||
evaluationInterval: "10s"
|
||||
ruleSelector: {}
|
||||
# ...
|
||||
```
|
||||
|
||||
In addition, you need to specify `remoteWrite` and `remoteRead` urls for restoring alert states after restarts:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: example-ha
|
||||
namespace: default
|
||||
spec:
|
||||
replicaCount: 2
|
||||
evaluationInterval: "10s"
|
||||
selectAllByDefault: true
|
||||
datasource:
|
||||
url: http://vmselect-demo.vm.svc:8481/select/0/prometheus
|
||||
notifiers:
|
||||
- url: http://vmalertmanager-example-0.vmalertmanager-example.default.svc:9093
|
||||
- url: http://vmalertmanager-example-1.vmalertmanager-example.default.svc:9093
|
||||
remoteWrite:
|
||||
url: http://vminsert-demo.vm.svc:8480/insert/0/prometheus
|
||||
remoteRead:
|
||||
url: http://vmselect-demo.vm.svc:8481/select/0/prometheus
|
||||
```
|
||||
|
||||
More details about `remoteWrite` and `remoteRead` you can read in [vmalert docs](https://docs.victoriametrics.com/vmalert.html#alerts-state-on-restarts).
|
||||
|
||||
## Version management
|
||||
|
||||
To set `VMAlert` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: example-vmalert
|
||||
spec:
|
||||
image:
|
||||
repository: victoriametrics/victoria-metrics
|
||||
tag: v1.93.4
|
||||
pullPolicy: Always
|
||||
# ...
|
||||
```
|
||||
|
||||
Also, you can specify `imagePullSecrets` if you are pulling images from private repo:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: example-vmalert
|
||||
spec:
|
||||
image:
|
||||
repository: victoriametrics/victoria-metrics
|
||||
tag: v1.93.4
|
||||
pullPolicy: Always
|
||||
imagePullSecrets:
|
||||
- name: my-repo-secret
|
||||
# ...
|
||||
```
|
||||
|
||||
## Enterprise features
|
||||
|
||||
VMAlert supports features [Reading rules from object storage](https://docs.victoriametrics.com/vmalert.html#reading-rules-from-object-storage)
|
||||
and [Multitenancy](https://docs.victoriametrics.com/vmalert.html#multitenancy)
|
||||
from [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
|
||||
|
||||
For using Enterprise version of [vmalert](https://docs.victoriametrics.com/vmalert.html)
|
||||
you need to change version of `VMAlert` to version with `-enterprise` suffix using [Version management](#version-management).
|
||||
|
||||
All the enterprise apps require `-eula` command-line flag to be passed to them.
|
||||
This flag acknowledges that your usage fits one of the cases listed on [this page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
|
||||
So you can use [extraArgs](./README.md#extra-arguments) for passing this flag to `VMAlert`:
|
||||
|
||||
### Reading rules from object storage
|
||||
|
||||
After that you can pass `-rule` command-line argument with `s3://` or `gs://`
|
||||
to `VMAlert` with [extraArgs](./README.md#extra-arguments).
|
||||
|
||||
More details about reading rules from object storage you can read in [vmalert docs](https://docs.victoriametrics.com/vmalert.html#reading-rules-from-object-storage).
|
||||
|
||||
Here are complete example for [Reading rules from object storage](https://docs.victoriametrics.com/vmalert.html#reading-rules-from-object-storage):
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: vmalert-ent-example
|
||||
spec:
|
||||
# enabling enterprise features
|
||||
image:
|
||||
# enterprise version of vmalert
|
||||
tag: v1.93.5-enterprise
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmalert enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# using enterprise features: Reading rules from object storage
|
||||
# more details about reading rules from object storage you can read on https://docs.victoriametrics.com/vmalert.html#reading-rules-from-object-storage
|
||||
rule: s3://bucket/dir/alert.rules
|
||||
|
||||
# ...other fields...
|
||||
```
|
||||
|
||||
### Multitenancy
|
||||
|
||||
After enabling enterprise version you can use [Multitenancy](https://docs.victoriametrics.com/vmalert.html#multitenancy)
|
||||
feature in `VMAlert`.
|
||||
|
||||
For that you need to set `clusterMode` commad-line flag
|
||||
with [extraArgs](./README.md#extra-arguments)
|
||||
and specify `tenant` field for groups
|
||||
in [VMRule](./vmrule.md#enterprise-features):
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: vmalert-ent-example
|
||||
spec:
|
||||
# enabling enterprise features
|
||||
image:
|
||||
# enterprise version of vmalert
|
||||
tag: v1.93.5-enterprise
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmalert enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# using enterprise features: Multitenancy
|
||||
# more details about multitenancy you can read on https://docs.victoriametrics.com/vmalert.html#multitenancy
|
||||
clusterMode: true
|
||||
|
||||
# ...other fields...
|
||||
|
||||
---
|
||||
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMRule
|
||||
metadata:
|
||||
name: vmrule-ent-example
|
||||
spec:
|
||||
groups:
|
||||
- name: vmalert-1
|
||||
rules:
|
||||
# using enterprise features: Multitenancy
|
||||
# more details about multitenancy you can read on https://docs.victoriametrics.com/vmalert.html#multitenancy
|
||||
- tenant: 1
|
||||
alert: vmalert config reload error
|
||||
expr: delta(vmalert_config_last_reload_errors_total[5m]) > 0
|
||||
for: 10s
|
||||
labels:
|
||||
severity: major
|
||||
job: "{{ $labels.job }}"
|
||||
annotations:
|
||||
value: "{{ $value }}"
|
||||
description: 'error reloading vmalert config, reload count for 5 min {{ $value }}'
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlert
|
||||
metadata:
|
||||
name: example-vmalert
|
||||
spec:
|
||||
replicaCount: 1
|
||||
datasource:
|
||||
url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429"
|
||||
notifier:
|
||||
url: "http://vmalertmanager-example-alertmanager.default.svc:9093"
|
||||
evaluationInterval: "30s"
|
||||
selectAllByDefault: true
|
||||
```
|
||||
270
docs/operator/resources/vmalertmanager.md
Normal file
270
docs/operator/resources/vmalertmanager.md
Normal file
|
|
@ -0,0 +1,270 @@
|
|||
# VMAlertmanager
|
||||
|
||||
`VMAlertmanager` - represents [alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/) configuration.
|
||||
|
||||
The `VMAlertmanager` CRD declaratively defines a desired Alertmanager setup to run in a Kubernetes cluster.
|
||||
It provides options to configure replication and persistent storage.
|
||||
|
||||
For each `Alertmanager` resource, the Operator deploys a properly configured `StatefulSet` in the same namespace.
|
||||
The Alertmanager pods are configured to include a `Secret` called `<alertmanager-name>` which holds the used
|
||||
configuration file in the key `alertmanager.yaml`.
|
||||
|
||||
When there are two or more configured replicas the Operator runs the Alertmanager instances in high availability mode.
|
||||
|
||||
## Specification
|
||||
|
||||
You can see the full actual specification of the `VMAlertmanager` resource in the **[API docs -> VMAlert](../api.md#vmalertmanager)**.
|
||||
|
||||
If you can't find necessary field in the specification of the custom resource,
|
||||
see [Extra arguments section](./README.md#extra-arguments).
|
||||
|
||||
Also, you can check out the [examples](#examples) section.
|
||||
|
||||
## Configuration
|
||||
|
||||
The operator generates a configuration file for `VMAlertmanager` based on user input at the definition of `CRD`.
|
||||
|
||||
Generated config stored at `Secret` created by the operator, it has the following name template `vmalertmanager-CRD_NAME-config`.
|
||||
|
||||
This configuration file is mounted at `VMAlertmanager` `Pod`. A special side-car container tracks its changes and sends config-reload signals to `alertmanager` container.
|
||||
|
||||
### Using secret
|
||||
|
||||
Basically, you can use the global configuration defined at manually created `Secret`. This `Secret` must be created before `VMAlertmanager`.
|
||||
|
||||
Name of the `Secret` must be defined at `VMAlertmanager` `spec.configSecret` option:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: vmalertmanager-example-alertmanager
|
||||
labels:
|
||||
app: vm-operator
|
||||
type: Opaque
|
||||
stringData:
|
||||
alertmanager.yaml: |
|
||||
global:
|
||||
resolve_timeout: 5m
|
||||
route:
|
||||
receiver: 'webhook'
|
||||
receivers:
|
||||
- name: 'webhook'
|
||||
webhook_configs:
|
||||
- url: 'http://alertmanagerwh:30500/'
|
||||
|
||||
---
|
||||
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanager
|
||||
metadata:
|
||||
name: example-alertmanager
|
||||
spec:
|
||||
replicaCount: 2
|
||||
configSecret: vmalertmanager-example-alertmanager
|
||||
```
|
||||
|
||||
### Using inline raw config
|
||||
|
||||
Also, if there is no secret data at configuration, or you just want to redefine some global variables for `alertmanager`.
|
||||
You can define configuration at `spec.configRawYaml` section of `VMAlertmanager` configuration:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanager
|
||||
metadata:
|
||||
name: example-alertmanager
|
||||
spec:
|
||||
replicaCount: 2
|
||||
configRawYaml: |
|
||||
global:
|
||||
resolve_timeout: 5m
|
||||
route:
|
||||
receiver: 'default'
|
||||
group_interval: 5m
|
||||
repeat_interval: 12h
|
||||
receivers:
|
||||
- name: 'default'
|
||||
```
|
||||
|
||||
If both `configSecret` and `configRawYaml` are defined, only configuration from `configRawYaml` will be used. Values from `configRawYaml` will be ignored.
|
||||
|
||||
### Using VMAlertmanagerConfig
|
||||
|
||||
See details at [VMAlertmanagerConfig](./vmalertmanagerconfig.md).
|
||||
|
||||
The CRD specifies which `VMAlertmanagerConfig`s should be covered by the deployed `VMAlertmanager` instances based on label selection.
|
||||
The Operator then generates a configuration based on the included `VMAlertmanagerConfig`s and updates the `Configmaps` containing
|
||||
the configuration. It continuously does so for all changes that are made to `VMAlertmanagerConfig`s or to the `VMAlertmanager` resource itself.
|
||||
|
||||
Configs are filtered by selectors `configNamespaceSelector` and `configSelector` in `VMAlertmanager` CRD definition.
|
||||
For selecting rules from all namespaces you must specify it to empty value:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
configNamespaceSelector: {}
|
||||
```
|
||||
|
||||
[VMAlertmanagerConfig](./vmalertmanagerconfig.md) objects are
|
||||
generates part of [VMAlertmanager](./vmalertmanager.md) configuration.
|
||||
|
||||
For filtering rules `VMAlertmanager` uses selectors `configNamespaceSelector` and `configSelector`.
|
||||
It allows configuring rules access control across namespaces and different environments.
|
||||
Specification of selectors you can see in [this doc](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#labelselector-v1-meta).
|
||||
|
||||
In addition to the above selectors, the filtering of objects in a cluster is affected by the field `selectAllByDefault`
|
||||
of `VMAlertmanager` spec and environment variable `WATCH_NAMESPACE` for operator.
|
||||
|
||||
Following rules are applied:
|
||||
|
||||
- If `configNamespaceSelector` and `configSelector` both undefined, then by default select nothing. With option set - `spec.selectAllByDefault: true`, select all vmalertmanagerconfigs.
|
||||
- If `configNamespaceSelector` defined, `configSelector` undefined, then all vmalertmaangerconfigs are matching at namespaces for given `configNamespaceSelector`.
|
||||
- If `configNamespaceSelector` undefined, `configSelector` defined, then all vmalertmaangerconfigs at `VMAgent`'s namespaces are matching for given `configSelector`.
|
||||
- If `configNamespaceSelector` and `configSelector` both defined, then only vmalertmaangerconfigs at namespaces matched `configNamespaceSelector` for given `configSelector` are matching.
|
||||
|
||||
Here's a more visual and more detailed view:
|
||||
|
||||
| `configNamespaceSelector` | `configSelector` | `selectAllByDefault` | `WATCH_NAMESPACE` | Selected rules |
|
||||
|---------------------------|------------------|----------------------|-------------------|------------------------------------------------------------------------------------------------------------------------|
|
||||
| undefined | undefined | false | undefined | nothing |
|
||||
| undefined | undefined | **true** | undefined | all vmalertmaangerconfigs in the cluster |
|
||||
| **defined** | undefined | any | undefined | all vmalertmaangerconfigs are matching at namespaces for given `configNamespaceSelector` |
|
||||
| undefined | **defined** | any | undefined | all vmalertmaangerconfigs only at `VMAlertmanager`'s namespace are matching for given `ruleSelector` |
|
||||
| **defined** | **defined** | any | undefined | all vmalertmaangerconfigs only at namespaces matched `configNamespaceSelector` for given `configSelector` are matching |
|
||||
| any | undefined | any | **defined** | all vmalertmaangerconfigs only at `VMAlertmanager`'s namespace |
|
||||
| any | **defined** | any | **defined** | all vmalertmaangerconfigs only at `VMAlertmanager`'s namespace for given `configSelector` are matching |
|
||||
|
||||
More details about `WATCH_NAMESPACE` variable you can read in [this doc](../configuration.md#namespaced-mode).
|
||||
|
||||
Here are some examples of `VMAlertmanager` configuration with selectors:
|
||||
|
||||
```yaml
|
||||
# select all config objects in the cluster
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanager
|
||||
metadata:
|
||||
name: vmalertmanager-select-all
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
|
||||
---
|
||||
|
||||
# select all config objects in specific namespace (my-namespace)
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanager
|
||||
metadata:
|
||||
name: vmalertmanager-select-ns
|
||||
spec:
|
||||
# ...
|
||||
configNamespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: my-namespace
|
||||
```
|
||||
|
||||
### Extra configuration files
|
||||
|
||||
`VMAlertmanager` specification has the following fields, that can be used to configure without editing raw configuration file:
|
||||
|
||||
- `spec.templates` - list of keys in `ConfigMaps`, that contains template files for `alertmanager`, e.g.:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanager
|
||||
metadata:
|
||||
name: example-alertmanager
|
||||
spec:
|
||||
replicaCount: 2
|
||||
templates:
|
||||
- Name: alertmanager-templates
|
||||
Key: my-template-1.tmpl
|
||||
- Name: alertmanager-templates
|
||||
Key: my-template-2.tmpl
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: alertmanager-templates
|
||||
data:
|
||||
my-template-1.tmpl: |
|
||||
{{ define "hello" -}}
|
||||
hello, Victoria!
|
||||
{{- end }}
|
||||
my-template-2.tmpl: """
|
||||
```
|
||||
|
||||
These templates will be automatically added to `VMAlertmanager` configuration and will be automatically reloaded on changes in source `ConfigMap`.
|
||||
- `spec.configMaps` - list of `ConfigMap` names (in the same namespace) that will be mounted at `VMAlertmanager`
|
||||
workload and will be automatically reloaded on changes in source `ConfigMap`. Mount path is `/etc/vm/configs/<configmap-name>`.
|
||||
|
||||
### Behavior without provided config
|
||||
|
||||
If no configuration is provided, operator configures stub configuration with blackhole route.
|
||||
|
||||
## High Availability
|
||||
|
||||
The final step of the high availability scheme is Alertmanager, when an alert triggers, actually fire alerts against *all* instances of an Alertmanager cluster.
|
||||
|
||||
The Alertmanager, starting with the `v0.5.0` release, ships with a high availability mode.
|
||||
It implements a gossip protocol to synchronize instances of an Alertmanager cluster
|
||||
regarding notifications that have been sent out, to prevent duplicate notifications.
|
||||
It is an AP (available and partition tolerant) system. Being an AP system means that notifications are guaranteed to be sent at least once.
|
||||
|
||||
The Victoria Metrics Operator ensures that Alertmanager clusters are properly configured to run highly available on Kubernetes.
|
||||
|
||||
## Version management
|
||||
|
||||
To set `VMAlertmanager` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanager
|
||||
metadata:
|
||||
name: example-vmalertmanager
|
||||
spec:
|
||||
image:
|
||||
repository: prom/alertmanager
|
||||
tag: v0.25.0
|
||||
pullPolicy: Always
|
||||
# ...
|
||||
```
|
||||
|
||||
Also, you can specify `imagePullSecrets` if you are pulling images from private repo:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanager
|
||||
metadata:
|
||||
name: example-vmalertmanager
|
||||
spec:
|
||||
image:
|
||||
repository: prom/alertmanager
|
||||
tag: v0.25.0
|
||||
pullPolicy: Always
|
||||
imagePullSecrets:
|
||||
- name: my-repo-secret
|
||||
# ...
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanager
|
||||
metadata:
|
||||
name: vmalertmanager-example
|
||||
spec:
|
||||
replicaCount: 1
|
||||
configRawYaml: |
|
||||
global:
|
||||
resolve_timeout: 5m
|
||||
route:
|
||||
group_wait: 30s
|
||||
group_interval: 5m
|
||||
repeat_interval: 12h
|
||||
receiver: 'webhook'
|
||||
receivers:
|
||||
- name: 'webhook'
|
||||
webhook_configs:
|
||||
- url: 'http://localhost:30502/'
|
||||
```
|
||||
101
docs/operator/resources/vmalertmanagerconfig.md
Normal file
101
docs/operator/resources/vmalertmanagerconfig.md
Normal file
|
|
@ -0,0 +1,101 @@
|
|||
# VMAlertmanagerConfig
|
||||
|
||||
The `VMAlertmanagerConfig` provides way to configure [VMAlertmanager](./vmalertmanager.md)
|
||||
configuration with CRD. It allows to define different configuration parts, which will be merged by operator into config.
|
||||
|
||||
It behaves like other config parts - `VMServiceScrape` and etc.
|
||||
|
||||
Read [Usage](#usage) and [Special case](#special-case) before using.
|
||||
|
||||
## Specification
|
||||
|
||||
You can see the full actual specification of the `VMAlertmanagerConfig` resource in
|
||||
the **[API docs -> VMAlertmanagerConfig](../api.md#vmalertmanagerconfig)**.
|
||||
|
||||
Also, you can check out the [examples](#examples) section.
|
||||
|
||||
## Usage
|
||||
|
||||
`VMAlertmanagerConfig` allows delegating notification configuration to the kubernetes cluster users.
|
||||
The application owner may configure notifications by defining it at `VMAlertmanagerConfig`.
|
||||
|
||||
With the combination of `VMRule` and `VMServiceScrape` it allows delegating configuration observability to application owners, and uses popular `GitOps` practice.
|
||||
|
||||
Operator combines `VMAlertmanagerConfig`s into a single configuration file for `VMAlertmanager`.
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanagerConfig
|
||||
metadata:
|
||||
name: example-email-web
|
||||
namespace: production
|
||||
spec:
|
||||
route:
|
||||
receiver: email
|
||||
group_interval: 1m
|
||||
routes:
|
||||
- receiver: email
|
||||
matchers:
|
||||
- {severity =~ "warning|critical", app_name = "blog"}
|
||||
receivers:
|
||||
- name: email
|
||||
email_configs:
|
||||
- to: some-email@example.com
|
||||
from: alerting@example.com
|
||||
smarthost: example.com:25
|
||||
text: ALARM
|
||||
```
|
||||
|
||||
#### Special Case
|
||||
|
||||
VMAlertmanagerConfig has enforced namespace matcher.
|
||||
Alerts must have a proper namespace label, with the same value as name of namespace for VMAlertmanagerConfig.
|
||||
|
||||
It can be disabled, by setting the following value to the VMAlertmanager: `spec.disableNamespaceMatcher: true`.
|
||||
|
||||
## Examples
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAlertmanagerConfig
|
||||
metadata:
|
||||
name: example
|
||||
namespace: default
|
||||
spec:
|
||||
inhibit_rules:
|
||||
- equals: []
|
||||
target_matchers: []
|
||||
source_matchers: []
|
||||
route:
|
||||
routes:
|
||||
- receiver: webhook
|
||||
continue: true
|
||||
receiver: email
|
||||
group_by: []
|
||||
continue: false
|
||||
matchers:
|
||||
- job = "alertmanager"
|
||||
group_wait: 30s
|
||||
group_interval: 45s
|
||||
repeat_interval: 1h
|
||||
mute_time_intervals:
|
||||
- name: base
|
||||
time_intervals:
|
||||
- times:
|
||||
- start_time: ""
|
||||
end_time: ""
|
||||
weekdays: []
|
||||
days_of_month: []
|
||||
months: []
|
||||
years: []
|
||||
receivers:
|
||||
email_configs: []
|
||||
webhook_configs:
|
||||
- url: http://some-other-wh
|
||||
pagerduty_configs: []
|
||||
pushover_configs: []
|
||||
slack_configs: []
|
||||
opsgenie_configs: []
|
||||
victorops_configs: []
|
||||
wechat_configs: []
|
||||
```
|
||||
237
docs/operator/resources/vmauth.md
Normal file
237
docs/operator/resources/vmauth.md
Normal file
|
|
@ -0,0 +1,237 @@
|
|||
# VMAuth
|
||||
|
||||
The `VMAuth` CRD provides mechanism for exposing application with authorization to outside world or to other applications inside kubernetes cluster.
|
||||
|
||||
For first case, user can configure `ingress` setting at `VMAuth` CRD. For second one, operator will create secret with `username` and `password` at `VMUser` CRD name.
|
||||
So it will be possible to access these credentials from any application by targeting corresponding kubernetes secret.
|
||||
|
||||
## Specification
|
||||
|
||||
You can see the full actual specification of the `VMAuth` resource in
|
||||
the **[API docs -> VMAuth](../api.md#vmauth)**.
|
||||
|
||||
If you can't find necessary field in the specification of the custom resource,
|
||||
see [Extra arguments section](./README.md#extra-arguments).
|
||||
|
||||
Also, you can check out the [examples](#examples) section.
|
||||
|
||||
## Users
|
||||
|
||||
The CRD specifies which `VMUser`s should be covered by the deployed `VMAuth` instances based on label selection.
|
||||
The Operator then generates a configuration based on the included `VMUser`s and updates the `Configmaps` containing
|
||||
the configuration. It continuously does so for all changes that are made to `VMUser`s or to the `VMAuth` resource itself.
|
||||
|
||||
[VMUser](./vmrule.md) objects generate part of `VMAuth` configuration.
|
||||
|
||||
For filtering users `VMAuth` uses selectors `userNamespaceSelector` and `userSelector`.
|
||||
It allows configuring rules access control across namespaces and different environments.
|
||||
Specification of selectors you can see in [this doc](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#labelselector-v1-meta).
|
||||
|
||||
In addition to the above selectors, the filtering of objects in a cluster is affected by the field `selectAllByDefault` of `VMAuth` spec and environment variable `WATCH_NAMESPACE` for operator.
|
||||
|
||||
Following rules are applied:
|
||||
|
||||
- If `userNamespaceSelector` and `userSelector` both undefined, then by default select nothing. With option set - `spec.selectAllByDefault: true`, select all vmusers.
|
||||
- If `userNamespaceSelector` defined, `userSelector` undefined, then all vmusers are matching at namespaces for given `userNamespaceSelector`.
|
||||
- If `userNamespaceSelector` undefined, `userSelector` defined, then all vmusers at `VMAgent`'s namespaces are matching for given `userSelector`.
|
||||
- If `userNamespaceSelector` and `userSelector` both defined, then only vmusers at namespaces matched `userNamespaceSelector` for given `userSelector` are matching.
|
||||
|
||||
Here's a more visual and more detailed view:
|
||||
|
||||
| `userNamespaceSelector` | `userSelector` | `selectAllByDefault` | `WATCH_NAMESPACE` | Selected rules |
|
||||
|-------------------------|----------------|----------------------|-------------------|------------------------------------------------------------------------------------------------------|
|
||||
| undefined | undefined | false | undefined | nothing |
|
||||
| undefined | undefined | **true** | undefined | all vmusers in the cluster |
|
||||
| **defined** | undefined | any | undefined | all vmusers are matching at namespaces for given `userNamespaceSelector` |
|
||||
| undefined | **defined** | any | undefined | all vmusers only at `VMAuth`'s namespace are matching for given `userSelector` |
|
||||
| **defined** | **defined** | any | undefined | all vmusers only at namespaces matched `userNamespaceSelector` for given `userSelector` are matching |
|
||||
| any | undefined | any | **defined** | all vmusers only at `VMAuth`'s namespace |
|
||||
| any | **defined** | any | **defined** | all vmusers only at `VMAuth`'s namespace for given `userSelector` are matching |
|
||||
|
||||
More details about `WATCH_NAMESPACE` variable you can read in [this doc](../configuration.md#namespaced-mode).
|
||||
|
||||
Here are some examples of `VMAuth` configuration with selectors:
|
||||
|
||||
```yaml
|
||||
# select all user objects in the cluster
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
name: vmauth-select-all
|
||||
spec:
|
||||
# ...
|
||||
selectAllByDefault: true
|
||||
|
||||
---
|
||||
|
||||
# select all user objects in specific namespace (my-namespace)
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
name: vmauth-select-ns
|
||||
spec:
|
||||
# ...
|
||||
userNamespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: my-namespace
|
||||
```
|
||||
|
||||
## Unauthorized access
|
||||
|
||||
You can configure `VMAuth` to allow unauthorized access for specified routes with `unauthorizedAccessConfig` field.
|
||||
|
||||
For instance:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
name: vmauth-unauthorized-example
|
||||
spec:
|
||||
unauthorizedAccessConfig:
|
||||
- paths: ["/metrics"]
|
||||
urls:
|
||||
- http://vmsingle-example.default.svc:8428
|
||||
```
|
||||
|
||||
In this example every user can access `/metrics` route and get vmsingle metrics without authorization.
|
||||
|
||||
In addition, `unauthorizedAccessConfig` in [Enterprise version](#enterprise-features) supports [IP Filters](#ip-filters)
|
||||
with `ip_filters` field.
|
||||
|
||||
## High availability
|
||||
|
||||
The `VMAuth` resource is stateless, so it can be scaled horizontally by increasing the number of replicas:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
name: vmauth-example
|
||||
spec:
|
||||
replicas: 3
|
||||
# ...
|
||||
```
|
||||
|
||||
## Version management
|
||||
|
||||
To set `VMAuth` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
name: example-vmauth
|
||||
spec:
|
||||
image:
|
||||
repository: victoriametrics/victoria-metrics
|
||||
tag: v1.93.4
|
||||
pullPolicy: Always
|
||||
# ...
|
||||
```
|
||||
|
||||
Also, you can specify `imagePullSecrets` if you are pulling images from private repo:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
name: example-vmauth
|
||||
spec:
|
||||
image:
|
||||
repository: victoriametrics/victoria-metrics
|
||||
tag: v1.93.4
|
||||
pullPolicy: Always
|
||||
imagePullSecrets:
|
||||
- name: my-repo-secret
|
||||
# ...
|
||||
```
|
||||
|
||||
## Enterprise features
|
||||
|
||||
Custom resource `VMAuth` supports feature [IP filters](https://docs.victoriametrics.com/vmauth.html#ip-filters)
|
||||
from [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
|
||||
|
||||
For using Enterprise version of [vmauth](https://docs.victoriametrics.com/vmauth.html)
|
||||
you need to change version of `vmauth` to version with `-enterprise` suffix using [Version management](#version-management).
|
||||
|
||||
All the enterprise apps require `-eula` command-line flag to be passed to them.
|
||||
This flag acknowledges that your usage fits one of the cases listed on [this page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
|
||||
So you can use [extraArgs](./README.md#extra-arguments) for passing this flag to `VMAuth`:
|
||||
|
||||
### IP Filters
|
||||
|
||||
After that you can use [IP filters for `VMUser`](./vmuser.md#enterprise-features)
|
||||
and field `ip_filters` for `VMAuth`.
|
||||
|
||||
Here are complete example with described above:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
name: vmauth-ent-example
|
||||
spec:
|
||||
# enabling enterprise features
|
||||
image:
|
||||
# enterprise version of vmauth
|
||||
tag: v1.93.5-enterprise
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmauth enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# using enterprise features: ip filters for vmauth
|
||||
# more details about ip filters you can read in https://docs.victoriametrics.com/vmauth.html#ip-filters
|
||||
ip_filters:
|
||||
allow_list:
|
||||
- 10.0.0.0/24
|
||||
- 1.2.3.4
|
||||
deny_list:
|
||||
- 5.6.7.8
|
||||
# allow read vmsingle metrics without authorization for users from internal network
|
||||
unauthorizedAccessConfig:
|
||||
- paths: ["/metrics"]
|
||||
urls: ["http://vmsingle-example.default.svc:8428"]
|
||||
ip_filters:
|
||||
allow_list:
|
||||
- 192.168.0.0/16
|
||||
- 10.0.0.0/8
|
||||
|
||||
# ...other fields...
|
||||
|
||||
---
|
||||
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMUser
|
||||
metadata:
|
||||
name: vmuser-ent-example
|
||||
spec:
|
||||
username: simple-user
|
||||
password: simple-password
|
||||
|
||||
# using enterprise features: ip filters for vmuser
|
||||
# more details about ip filters you can read in https://docs.victoriametrics.com/vmuser.html#enterprise-features
|
||||
ip_filters:
|
||||
allow_list:
|
||||
- 10.0.0.0/24
|
||||
- 1.2.3.4
|
||||
deny_list:
|
||||
- 5.6.7.8
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
name: example
|
||||
namespace: default
|
||||
spec:
|
||||
selectAllByDefault: true
|
||||
ingress:
|
||||
class_name: nginx # <-- change this to your ingress-controller
|
||||
host: vm-demo.k8s.orb.local # <-- change this to your domain
|
||||
```
|
||||
612
docs/operator/resources/vmcluster.md
Normal file
612
docs/operator/resources/vmcluster.md
Normal file
|
|
@ -0,0 +1,612 @@
|
|||
# VMCluster
|
||||
|
||||
`VMCluster` represents a high-available and fault-tolerant version of VictoriaMetrics database.
|
||||
The `VMCluster` CRD defines a [cluster version VM](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html).
|
||||
|
||||
For each `VMCluster` resource, the Operator creates:
|
||||
|
||||
- `VMStorage` as `StatefulSet`,
|
||||
- `VMSelect` as `StatefulSet`
|
||||
- and `VMInsert` as deployment.
|
||||
|
||||
For `VMStorage` and `VMSelect` headless services are created. `VMInsert` is created as service with clusterIP.
|
||||
|
||||
There is a strict order for these objects creation and reconciliation:
|
||||
|
||||
1. `VMStorage` is synced - the Operator waits until all its pods are ready;
|
||||
1. Then it syncs `VMSelect` with the same manner;
|
||||
1. `VMInsert` is the last object to sync.
|
||||
|
||||
All [statefulsets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) are created
|
||||
with [OnDelete](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#on-delete) update type.
|
||||
It allows to manually manage the rolling update process for Operator by deleting pods one by one and waiting for the ready status.
|
||||
|
||||
Rolling update process may be configured by the operator env variables.
|
||||
The most important is `VM_PODWAITREADYTIMEOUT=80s` - it controls how long to wait for pod's ready status.
|
||||
|
||||
## Specification
|
||||
|
||||
You can see the full actual specification of the `VMCluster` resource in the **[API docs -> VMCluster](../api.md#vmcluster)**.
|
||||
|
||||
If you can't find necessary field in the specification of the custom resource,
|
||||
see [Extra arguments section](./README.md#extra-arguments).
|
||||
|
||||
Also, you can check out the [examples](#examples) section.
|
||||
|
||||
## High availability
|
||||
|
||||
The cluster version provides a full set of high availability features - metrics replication, node failover, horizontal scaling.
|
||||
|
||||
First, we recommend familiarizing yourself with the high availability tools provided by "VictoriaMetrics Cluster" itself:
|
||||
|
||||
- [High availability](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#high-availability),
|
||||
- [Cluster availability](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#cluster-availability),
|
||||
- [Replication and data safety](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#replication-and-data-safety).
|
||||
|
||||
`VMCluster` supports all listed in the above-mentioned articles parameters and features:
|
||||
|
||||
- `replicationFactor` - the number of replicas for each metric.
|
||||
- for every component of cluster (`vmstorage` / `vmselect` / `vminsert`):
|
||||
- `replicaCount` - the number of replicas for components of cluster.
|
||||
- `affinity` - the affinity (the pod's scheduling constraints) for components pods. See more details in [kubernetes docs](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity).
|
||||
- `topologySpreadConstraints` - controls how pods are spread across your cluster among failure-domains such as regions, zones, nodes, and other user-defined topology domains. See more details in [kubernetes docs](https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/).
|
||||
|
||||
In addition, operator:
|
||||
|
||||
- uses k8s services or vmauth for load balancing between `vminsert` and `vmselect` components,
|
||||
- uses health checks for to determine the readiness of components for work after restart,
|
||||
- allows to horizontally scale all cluster components just by changing `replicaCount` field.
|
||||
|
||||
Here is an example of a `VMCluster` resource with HA features:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: example-vmcluster-persistent
|
||||
spec:
|
||||
replicationFactor: 2
|
||||
vmstorage:
|
||||
replicaCount: 10
|
||||
storageDataPath: "/vm-data"
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchExpressions:
|
||||
- key: "app.kubernetes.io/name"
|
||||
operator: In
|
||||
values:
|
||||
- "vmstorage"
|
||||
topologyKey: "kubernetes.io/hostname"
|
||||
storage:
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
resources:
|
||||
limits:
|
||||
cpu: "2"
|
||||
memory: 2048Mi
|
||||
vmselect:
|
||||
replicaCount: 3
|
||||
cacheMountPath: "/select-cache"
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchExpressions:
|
||||
- key: "app.kubernetes.io/name"
|
||||
operator: In
|
||||
values:
|
||||
- "vmselect"
|
||||
topologyKey: "kubernetes.io/hostname"
|
||||
storage:
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
resources:
|
||||
requests:
|
||||
storage: 2Gi
|
||||
resources:
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: "500Mi"
|
||||
vminsert:
|
||||
replicaCount: 4
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchExpressions:
|
||||
- key: "app.kubernetes.io/name"
|
||||
operator: In
|
||||
values:
|
||||
- "vminsert"
|
||||
topologyKey: "kubernetes.io/hostname"
|
||||
resources:
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: "500Mi"
|
||||
```
|
||||
|
||||
## Version management
|
||||
|
||||
For `VMCluster` you can specify tag name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) and repository setting per cluster object:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: example-vmcluster
|
||||
spec:
|
||||
vmstorage:
|
||||
replicaCount: 2
|
||||
image:
|
||||
repository: victoriametrics/vmstorage
|
||||
tag: v1.93.4-cluster
|
||||
pullPolicy: Always
|
||||
vmselect:
|
||||
replicaCount: 2
|
||||
image:
|
||||
repository: victoriametrics/vmselect
|
||||
tag: v1.93.4-cluster
|
||||
pullPolicy: Always
|
||||
vminsert:
|
||||
replicaCount: 2
|
||||
image:
|
||||
repository: victoriametrics/vminsert
|
||||
tag: v1.93.4-cluster
|
||||
pullPolicy: Always
|
||||
```
|
||||
|
||||
Also, you can specify `imagePullSecrets` if you are pulling images from private repo,
|
||||
but `imagePullSecrets` is global setting for all `VMCluster` specification:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: example-vmcluster
|
||||
spec:
|
||||
vmstorage:
|
||||
replicaCount: 2
|
||||
image:
|
||||
repository: victoriametrics/vmstorage
|
||||
tag: v1.93.4-cluster
|
||||
pullPolicy: Always
|
||||
vmselect:
|
||||
replicaCount: 2
|
||||
image:
|
||||
repository: victoriametrics/vmselect
|
||||
tag: v1.93.4-cluster
|
||||
pullPolicy: Always
|
||||
vminsert:
|
||||
replicaCount: 2
|
||||
image:
|
||||
repository: victoriametrics/vminsert
|
||||
tag: v1.93.4-cluster
|
||||
pullPolicy: Always
|
||||
imagePullSecrets:
|
||||
- name: my-repo-secret
|
||||
# ...
|
||||
```
|
||||
|
||||
## Enterprise features
|
||||
|
||||
VMCluster supports following features
|
||||
from [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise):
|
||||
|
||||
- [Downsampling](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#downsampling)
|
||||
- [Multiple retentions / Retention filters](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#retention-filters)
|
||||
- [Advanced per-tenant statistic](https://docs.victoriametrics.com/PerTenantStatistic.html)
|
||||
- [mTLS for cluster components](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection)
|
||||
- [Backup automation](https://docs.victoriametrics.com/vmbackupmanager.html)
|
||||
|
||||
VMCluster doesn't support yet feature
|
||||
[Automatic discovery for vmstorage nodes](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#automatic-vmstorage-discovery).
|
||||
|
||||
For using Enterprise version of [vmcluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html)
|
||||
you need to change version of `VMCluster` to version with `-enterprise` suffix using [Version management](#version-management).
|
||||
|
||||
All the enterprise apps require `-eula` command-line flag to be passed to them.
|
||||
This flag acknowledges that your usage fits one of the cases listed on [this page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
|
||||
So you can use [extraArgs](./README.md#extra-arguments) for passing this flag to `VMCluster`.
|
||||
|
||||
### Downsampling
|
||||
|
||||
After that you can pass [Downsampling](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#downsampling)
|
||||
flag to `VMCluster/vmselect` and `VMCluster/vmstorage` with [extraArgs](./README.md#extra-arguments) too.
|
||||
|
||||
Here are complete example for [Downsampling](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#downsampling):
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: vmcluster-ent-example
|
||||
spec:
|
||||
|
||||
vmselect:
|
||||
# enabling enterprise features for vmselect
|
||||
image:
|
||||
# enterprise version of vmselect
|
||||
tag: v1.93.5-enterprise-cluster
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmselect enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# using enterprise features: Downsampling
|
||||
# more details about downsampling you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#downsampling
|
||||
downsampling.period: 30d:5m,180d:1h,1y:6h,2y:1d
|
||||
|
||||
vmstorage:
|
||||
# enabling enterprise features for vmstorage
|
||||
image:
|
||||
# enterprise version of vmstorage
|
||||
tag: v1.93.5-enterprise-cluster
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmstorage enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# using enterprise features: Downsampling
|
||||
# more details about downsampling you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#downsampling
|
||||
downsampling.period: 30d:5m,180d:1h,1y:6h,2y:1d
|
||||
|
||||
# ...other fields...
|
||||
```
|
||||
|
||||
### Retention filters
|
||||
|
||||
You can pass [Retention filters](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#retention-filters)
|
||||
flag to `VMCluster/vmstorage` with [extraArgs](./README.md#extra-arguments).
|
||||
|
||||
Here are complete example for [Retention filters](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#retention-filters):
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: vmcluster-ent-example
|
||||
spec:
|
||||
|
||||
vmstorage:
|
||||
# enabling enterprise features for vmstorage
|
||||
image:
|
||||
# enterprise version of vmstorage
|
||||
tag: v1.93.5-enterprise-cluster
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmstorage enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# using enterprise features: Retention filters
|
||||
# more details about retention filters you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#retention-filters
|
||||
retentionFilter: '{vm_account_id="5",env="dev"}:5d,{vm_account_id="5",env="prod"}:5y'
|
||||
|
||||
# ...other fields...
|
||||
```
|
||||
|
||||
### Advanced per-tenant statistic
|
||||
|
||||
For using [Advanced per-tenant statistic](https://docs.victoriametrics.com/PerTenantStatistic.html)
|
||||
you only need to [enable Enterprise version of vmcluster components](#enterprise-features)
|
||||
and operator will automatically create
|
||||
[Scrape objects](./vmagent.md#scraping) for cluster components.
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: vmcluster-ent-example
|
||||
spec:
|
||||
|
||||
vmselect:
|
||||
# enabling enterprise features for vmselect
|
||||
image:
|
||||
# enterprise version of vmselect
|
||||
tag: v1.93.5-enterprise-cluster
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmselect enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
vminsert:
|
||||
# enabling enterprise features for vminsert
|
||||
image:
|
||||
# enterprise version of vminsert
|
||||
tag: v1.93.5-enterprise-cluster
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vminsert enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
vmstorage:
|
||||
# enabling enterprise features for vmstorage
|
||||
image:
|
||||
# enterprise version of vmstorage
|
||||
tag: v1.93.5-enterprise-cluster
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmstorage enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# ...other fields...
|
||||
```
|
||||
|
||||
After that [VMAgent](./vmagent.md) will automatically
|
||||
scrape [Advanced per-tenant statistic](https://docs.victoriametrics.com/PerTenantStatistic.html) for cluster components.
|
||||
|
||||
### mTLS protection
|
||||
|
||||
You can pass [mTLS protection](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection)
|
||||
flags to `VMCluster/vmstorage`, `VMCluster/vmselect` and `VMCluster/vminsert` with [extraArgs](./README.md#extra-arguments) and mount secret files
|
||||
with `extraVolumes` and `extraVolumeMounts` fields.
|
||||
|
||||
Here are complete example for [mTLS protection](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection)
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: vmcluster-ent-example
|
||||
spec:
|
||||
|
||||
vmselect:
|
||||
# enabling enterprise features for vmselect
|
||||
image:
|
||||
# enterprise version of vmselect
|
||||
tag: v1.93.5-enterprise-cluster
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmselect enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# using enterprise features: mTLS protection
|
||||
# more details about mTLS protection you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
|
||||
cluster.tls: true
|
||||
cluster.tlsCAFile: /etc/mtls/ca.crt
|
||||
cluster.tlsCertFile: /etc/mtls/vmselect.crt
|
||||
cluster.tlsKeyFile: /etc/mtls/vmselect.key
|
||||
extraVolumes:
|
||||
- name: mtls
|
||||
secret:
|
||||
secretName: mtls
|
||||
extraVolumeMounts:
|
||||
- name: mtls
|
||||
mountPath: /etc/mtls
|
||||
|
||||
vminsert:
|
||||
# enabling enterprise features for vminsert
|
||||
image:
|
||||
# enterprise version of vminsert
|
||||
tag: v1.93.5-enterprise-cluster
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vminsert enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# using enterprise features: mTLS protection
|
||||
# more details about mTLS protection you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
|
||||
cluster.tls: true
|
||||
cluster.tlsCAFile: /etc/mtls/ca.crt
|
||||
cluster.tlsCertFile: /etc/mtls/vminsert.crt
|
||||
cluster.tlsKeyFile: /etc/mtls/vminsert.key
|
||||
extraVolumes:
|
||||
- name: mtls
|
||||
secret:
|
||||
secretName: mtls
|
||||
extraVolumeMounts:
|
||||
- name: mtls
|
||||
mountPath: /etc/mtls
|
||||
|
||||
vmstorage:
|
||||
# enabling enterprise features for vmstorage
|
||||
image:
|
||||
# enterprise version of vmstorage
|
||||
tag: v1.93.5-enterprise-cluster
|
||||
env:
|
||||
- name: POD
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
extraArgs:
|
||||
# should be true and means that you have the legal right to run a vmstorage enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
eula: true
|
||||
|
||||
# using enterprise features: mTLS protection
|
||||
# more details about mTLS protection you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
|
||||
cluster.tls: true
|
||||
cluster.tlsCAFile: /etc/mtls/ca.crt
|
||||
cluster.tlsCertFile: /etc/mtls/$(POD).crt
|
||||
cluster.tlsKeyFile: /etc/mtls/$(POD).key
|
||||
extraVolumes:
|
||||
- name: mtls
|
||||
secret:
|
||||
secretName: mtls
|
||||
extraVolumeMounts:
|
||||
- name: mtls
|
||||
mountPath: /etc/mtls
|
||||
|
||||
# ...other fields...
|
||||
|
||||
---
|
||||
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: mtls
|
||||
namespace: default
|
||||
stringData:
|
||||
ca.crt: |
|
||||
-----BEGIN CERTIFICATE-----
|
||||
...
|
||||
-----END CERTIFICATE-----
|
||||
mtls-vmstorage-0.crt: |
|
||||
-----BEGIN CERTIFICATE-----
|
||||
...
|
||||
-----END CERTIFICATE-----
|
||||
mtls-vmstorage-0.key: |
|
||||
-----BEGIN PRIVATE KEY-----
|
||||
...
|
||||
-----END PRIVATE KEY-----
|
||||
mtls-vmstorage-1.crt: |
|
||||
-----BEGIN CERTIFICATE-----
|
||||
...
|
||||
-----END CERTIFICATE-----
|
||||
mtls-vmstorage-1.key: |
|
||||
-----BEGIN PRIVATE KEY-----
|
||||
...
|
||||
-----END PRIVATE KEY-----
|
||||
vminsert.crt: |
|
||||
-----BEGIN CERTIFICATE-----
|
||||
...
|
||||
-----END CERTIFICATE-----
|
||||
vminsert.key: |
|
||||
-----BEGIN PRIVATE KEY-----
|
||||
...
|
||||
-----END PRIVATE KEY-----
|
||||
vmselect.crt: |
|
||||
-----BEGIN CERTIFICATE-----
|
||||
...
|
||||
-----END CERTIFICATE-----
|
||||
vmselect.key: |
|
||||
-----BEGIN PRIVATE KEY-----
|
||||
...
|
||||
-----END PRIVATE KEY-----
|
||||
|
||||
```
|
||||
|
||||
Example commands for generating certificates you can read
|
||||
on [this page](https://gist.github.com/f41gh7/76ed8e5fb1ebb9737fe746bae9175ee6#generate-self-signed-ca-with-key).
|
||||
|
||||
### Backup automation
|
||||
|
||||
You can check [vmbackupmanager documentation](https://docs.victoriametrics.com/vmbackupmanager.html) for backup automation.
|
||||
It contains a description of the service and its features. This section covers vmbackumanager integration in vmoperator.
|
||||
|
||||
`VMCluster` has built-in backup configuration, it uses `vmbackupmanager` - proprietary tool for backups.
|
||||
It supports incremental backups (hourly, daily, weekly, monthly) with popular object storages (aws s3, google cloud storage).
|
||||
|
||||
Here is a complete example for backup configuration:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: vmcluster-ent-example
|
||||
spec:
|
||||
|
||||
vmstorage:
|
||||
vmBackup:
|
||||
# should be true and means that you have the legal right to run a vmstorage enterprise
|
||||
# that can either be a signed contract or an email with confirmation to run the service in a trial period
|
||||
# https://victoriametrics.com/legal/esa/
|
||||
acceptEULA: true
|
||||
|
||||
# using enterprise features: Backup automation
|
||||
# more details about backup automation you can read on https://docs.victoriametrics.com/vmbackupmanager.html
|
||||
destination: "s3://your_bucket/folder"
|
||||
credentialsSecret:
|
||||
name: remote-storage-keys
|
||||
key: credentials
|
||||
|
||||
# ...other fields...
|
||||
|
||||
---
|
||||
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: remote-storage-keys
|
||||
type: Opaque
|
||||
stringData:
|
||||
credentials: |-
|
||||
[default]
|
||||
aws_access_key_id = your_access_key_id
|
||||
aws_secret_access_key = your_secret_access_key
|
||||
```
|
||||
|
||||
**NOTE**: for cluster version operator adds suffix for destination: `"s3://your_bucket/folder"`, it becomes `"s3://your_bucket/folder/$(POD_NAME)"`.
|
||||
It's needed to make consistent backups for each storage node.
|
||||
|
||||
You can read more about backup configuration options and mechanics [here](https://docs.victoriametrics.com/vmbackupmanager.html)
|
||||
|
||||
Possible configuration options for backup crd can be found at [link](../api.md#vmbackup)
|
||||
|
||||
**Using VMBackupmanager for restoring backups** in Kubernetes environment is described [here](https://docs.victoriametrics.com/vmbackupmanager.html#how-to-restore-in-kubernetes).
|
||||
|
||||
Also see VMCLuster example spec [here](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmcluster_with_backuper.yaml).
|
||||
|
||||
## Examples
|
||||
|
||||
### Minimal example without persistence
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: vmcluster-example-minimal
|
||||
spec:
|
||||
# ...
|
||||
retentionPeriod: "1"
|
||||
vmstorage:
|
||||
replicaCount: 2
|
||||
vmselect:
|
||||
replicaCount: 2
|
||||
vminsert:
|
||||
replicaCount: 2
|
||||
```
|
||||
|
||||
### With persistence
|
||||
|
||||
```yaml
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: vmcluster-example-persistent
|
||||
spec:
|
||||
# ...
|
||||
retentionPeriod: "4"
|
||||
replicationFactor: 2
|
||||
vmstorage:
|
||||
replicaCount: 2
|
||||
storageDataPath: "/vm-data"
|
||||
storage:
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
storageClassName: standard
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
resources:
|
||||
limits:
|
||||
cpu: "0.5"
|
||||
memory: 500Mi
|
||||
vmselect:
|
||||
replicaCount: 2
|
||||
cacheMountPath: "/select-cache"
|
||||
storage:
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
resources:
|
||||
requests:
|
||||
storage: 2Gi
|
||||
resources:
|
||||
limits:
|
||||
cpu: "0.3"
|
||||
memory: "300Mi"
|
||||
vminsert:
|
||||
replicaCount: 2
|
||||
```
|
||||
46
docs/operator/resources/vmnodescrape.md
Normal file
46
docs/operator/resources/vmnodescrape.md
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
# VMNodeScrape
|
||||
|
||||
The `VMNodeScrape` CRD provides discovery mechanism for scraping metrics kubernetes nodes,
|
||||
it is useful for node exporters monitoring.
|
||||
|
||||
`VMNodeScrape` object generates part of [VMAgent](./vmagent.md) configuration.
|
||||
It has various options for scraping configuration of target (with basic auth,tls access, by specific port name etc.).
|
||||
|
||||
By specifying configuration at CRD, operator generates config
|
||||
for [VMAgent](./vmagent.md) and syncs it. It's useful for cadvisor scraping,
|
||||
node-exporter or other node-based exporters. `VMAgent` `nodeScrapeSelector` must match `VMNodeScrape` labels.
|
||||
|
||||
More information about selectors you can find in [this doc](./vmagent.md#scraping).
|
||||
|
||||
## Specification
|
||||
|
||||
You can see the full actual specification of the `VMNodeScrape` resource in
|
||||
the **[API docs -> VMNodeScrape](../api.md#vmnodescrape)**.
|
||||
|
||||
Also, you can check out the [examples](#examples) section.
|
||||
|
||||
## Examples
|
||||
|
||||
### Cadvisor scraping
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMNodeScrape
|
||||
metadata:
|
||||
name: cadvisor-metrics
|
||||
spec:
|
||||
scheme: "https"
|
||||
tlsConfig:
|
||||
insecureSkipVerify: true
|
||||
caFile: "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
|
||||
bearerTokenFile: "/var/run/secrets/kubernetes.io/serviceaccount/token"
|
||||
relabelConfigs:
|
||||
- action: labelmap
|
||||
regex: __meta_kubernetes_node_label_(.+)
|
||||
- targetLabel: __address__
|
||||
replacement: kubernetes.default.svc:443
|
||||
- sourceLabels: [__meta_kubernetes_node_name]
|
||||
regex: (.+)
|
||||
targetLabel: __metrics_path__
|
||||
replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
|
||||
```
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue