mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-11-21 14:44:00 +00:00
Merge branch 'public-single-node' into issue-4972-add-missing-tsids
This commit is contained in:
Aliaksandr Valialkin
commit
a3b1f28b9a
315 changed files with 13331 additions and 9790 deletions
104
README.md
104
README.md
|
|
@ -110,6 +110,7 @@ Case studies:
|
|||
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
|
||||
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
|
||||
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
|
||||
* [Criteo](https://docs.victoriametrics.com/CaseStudies.html#criteo)
|
||||
* [Dig Security](https://docs.victoriametrics.com/CaseStudies.html#dig-security)
|
||||
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
|
||||
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
|
||||
|
|
@ -364,6 +365,8 @@ See the [example VMUI at VictoriaMetrics playground](https://play.victoriametric
|
|||
* queries with the biggest average execution duration;
|
||||
* queries that took the most summary time for execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/top_queries` HTTP endpoint.
|
||||
|
||||
## Active queries
|
||||
|
||||
[VMUI](#vmui) provides `active queries` tab, which shows currently execute queries.
|
||||
|
|
@ -373,6 +376,8 @@ It provides the following information per each query:
|
|||
- The duration of the query execution.
|
||||
- The client address, who initiated the query execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/active_queries` HTTP endpoint.
|
||||
|
||||
## Metrics explorer
|
||||
|
||||
[VMUI](#vmui) provides an ability to explore metrics exported by a particular `job` / `instance` in the following way:
|
||||
|
|
@ -404,14 +409,16 @@ matching the specified [series selector](https://prometheus.io/docs/prometheus/l
|
|||
|
||||
Cardinality explorer is built on top of [/api/v1/status/tsdb](#tsdb-stats).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## Cardinality explorer statistic inaccuracy
|
||||
|
||||
In [cluster version of VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html) each vmstorage tracks the stored time series individually.
|
||||
vmselect requests stats via [/api/v1/status/tsdb](#tsdb-stats) API from each vmstorage node and merges the results by summing per-series stats.
|
||||
This may lead to inflated values when samples for the same time series are spread across multiple vmstorage nodes
|
||||
due to [replication](#replication) or [rerouting](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html?highlight=re-routes#cluster-availability).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## How to apply new config to VictoriaMetrics
|
||||
|
||||
VictoriaMetrics is configured via command-line flags, so it must be restarted when new command-line flags should be applied:
|
||||
|
|
@ -616,6 +623,28 @@ Some plugins for Telegraf such as [fluentd](https://github.com/fangli/fluent-plu
|
|||
or [Juniper/jitmon](https://github.com/Juniper/jtimon) send `SHOW DATABASES` query to `/query` and expect a particular database name in the response.
|
||||
Comma-separated list of expected databases can be passed to VictoriaMetrics via `-influx.databaseNames` command-line flag.
|
||||
|
||||
### How to send data in InfluxDB v2 format
|
||||
|
||||
VictoriaMetrics exposes endpoint for InfluxDB v2 HTTP API at `/influx/api/v2/write` and `/api/v2/write`.
|
||||
|
||||
|
||||
In order to write data with InfluxDB line protocol to local VictoriaMetrics using `curl`:
|
||||
|
||||
<div class="with-copy" markdown="1">
|
||||
|
||||
```console
|
||||
curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/api/v2/write'
|
||||
```
|
||||
|
||||
</div>
|
||||
|
||||
The `/api/v1/export` endpoint should return the following response:
|
||||
|
||||
```json
|
||||
{"metric":{"__name__":"measurement_field1","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1695902762311]}
|
||||
{"metric":{"__name__":"measurement_field2","tag1":"value1","tag2":"value2"},"values":[1.23],"timestamps":[1695902762311]}
|
||||
```
|
||||
|
||||
## How to send data from Graphite-compatible agents such as [StatsD](https://github.com/etsy/statsd)
|
||||
|
||||
Enable Graphite receiver in VictoriaMetrics by setting `-graphiteListenAddr` command line flag. For instance,
|
||||
|
|
@ -830,7 +859,7 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
* `/api/v1/series/count` - returns the total number of time series in the database. Some notes:
|
||||
* the handler scans all the inverted index, so it can be slow if the database contains tens of millions of time series;
|
||||
* the handler may count [deleted time series](#how-to-delete-time-series) additionally to normal time series due to internal implementation restrictions;
|
||||
* `/api/v1/status/active_queries` - returns a list of currently running queries.
|
||||
* `/api/v1/status/active_queries` - returns the list of currently running queries. This list is also available at [`active queries` page at VMUI](#active-queries).
|
||||
* `/api/v1/status/top_queries` - returns the following query lists:
|
||||
* the most frequently executed queries - `topByCount`
|
||||
* queries with the biggest average execution duration - `topByAvgDuration`
|
||||
|
|
@ -840,6 +869,8 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
For example, request to `/api/v1/status/top_queries?topN=5&maxLifetime=30s` would return up to 5 queries per list, which were executed during the last 30 seconds.
|
||||
VictoriaMetrics tracks the last `-search.queryStats.lastQueriesCount` queries with durations at least `-search.queryStats.minQueryDuration`.
|
||||
|
||||
See also [`top queries` page at VMUI](#top-queries).
|
||||
|
||||
### Timestamp formats
|
||||
|
||||
VictoriaMetrics accepts the following formats for `time`, `start` and `end` query args
|
||||
|
|
@ -1066,7 +1097,9 @@ VictoriaMetrics provides the following handlers for exporting data:
|
|||
Send a request to `http://<victoriametrics-addr>:8428/api/v1/export?match[]=<timeseries_selector_for_export>`,
|
||||
where `<timeseries_selector_for_export>` may contain any [time series selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors)
|
||||
for metrics to export. Use `{__name__!=""}` selector for fetching all the time series.
|
||||
The response would contain all the data for the selected time series in [JSON streaming format](http://ndjson.org/).
|
||||
|
||||
The response would contain all the data for the selected time series in JSON line format - see [these docs](#json-line-format) for details on this format.
|
||||
|
||||
Each JSON line contains samples for a single time series. An example output:
|
||||
|
||||
```json
|
||||
|
|
@ -1196,6 +1229,8 @@ check for changes in `vm_rows_invalid_total` (exported by server side) metric.
|
|||
|
||||
### How to import data in JSON line format
|
||||
|
||||
VictoriaMetrics accepts metrics data in JSON line format at `/api/v1/import` endpoint. See [these docs](#json-line-format) for details on this format.
|
||||
|
||||
Example for importing data obtained via [/api/v1/export](#how-to-export-data-in-json-line-format):
|
||||
|
||||
```console
|
||||
|
|
@ -1360,13 +1395,54 @@ Note that it could be required to flush response cache after importing historica
|
|||
|
||||
VictoriaMetrics also may scrape Prometheus targets - see [these docs](#how-to-scrape-prometheus-exporters-such-as-node-exporter).
|
||||
|
||||
## Sending data via OpenTelemetry
|
||||
### Sending data via OpenTelemetry
|
||||
|
||||
VictoriaMetrics supports data ingestion via [OpenTelemetry protocol for metrics](https://github.com/open-telemetry/opentelemetry-specification/blob/ffddc289462dfe0c2041e3ca42a7b1df805706de/specification/metrics/data-model.md) at `/opentelemetry/api/v1/push` path.
|
||||
|
||||
VictoriaMetrics expects `protobuf`-encoded requests at `/opentelemetry/api/v1/push`.
|
||||
Set HTTP request header `Content-Encoding: gzip` when sending gzip-compressed data to `/opentelemetry/api/v1/push`.
|
||||
|
||||
## JSON line format
|
||||
|
||||
VictoriaMetrics accepts data in JSON line format at [/api/v1/import](#how-to-import-data-in-json-line-format)
|
||||
and exports data in this format at [/api/v1/export](#how-to-export-data-in-json-line-format).
|
||||
|
||||
The format follows [JSON streaming concept](http://ndjson.org/), e.g. each line contains JSON object with metrics data in the following format:
|
||||
|
||||
```
|
||||
{
|
||||
// metric contans metric name plus labels for a particular time series
|
||||
"metric":{
|
||||
"__name__": "metric_name", // <- this is metric name
|
||||
|
||||
// Other labels for the time series
|
||||
|
||||
"label1": "value1",
|
||||
"label2": "value2",
|
||||
...
|
||||
"labelN": "valueN"
|
||||
},
|
||||
|
||||
// values contains raw sample values for the given time series
|
||||
"values": [1, 2.345, -678],
|
||||
|
||||
// timestamps contains raw sample UNIX timestamps in milliseconds for the given time series
|
||||
// every timestamp is associated with the value at the corresponding position
|
||||
"timestamps": [1549891472010,1549891487724,1549891503438]
|
||||
}
|
||||
```
|
||||
|
||||
Note that every JSON object must be written in a single line, e.g. all the newline chars must be removed from it.
|
||||
Every line length is limited by the value passed to `-import.maxLineLen` command-line flag (by default this is 100MB).
|
||||
|
||||
It is recommended passing 1K-10K samples per line for achieving the maximum data ingestion performance at [/api/v1/import](#how-to-import-data-in-json-line-format).
|
||||
Too long JSON lines may increase RAM usage at VictoriaMetrics side.
|
||||
|
||||
It is OK to split [raw samples](https://docs.victoriametrics.com/keyConcepts.html#raw-samples)
|
||||
for the same [time series](https://docs.victoriametrics.com/keyConcepts.html#time-series) across multiple lines.
|
||||
|
||||
The number of lines in JSON line document can be arbitrary.
|
||||
|
||||
## Relabeling
|
||||
|
||||
VictoriaMetrics supports Prometheus-compatible relabeling for all the ingested metrics if `-relabelConfig` command-line flag points
|
||||
|
|
@ -1654,9 +1730,10 @@ See [these docs](https://docs.victoriametrics.com/guides/guide-vmcluster-multipl
|
|||
which allow configuring multiple retentions for distinct sets of time series matching the configured [series filters](https://docs.victoriametrics.com/keyConcepts.html#filtering)
|
||||
via `-retentionFilter` command-line flag. This flag accepts `filter:duration` options, where `filter` must be
|
||||
a valid [series filter](https://docs.victoriametrics.com/keyConcepts.html#filtering), while the `duration`
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`. If series doesn't match
|
||||
any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention) command-line flag is applied to it.
|
||||
If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`.
|
||||
The `duration` of the `-retentionFilter` must be lower or equal to [-retentionPeriod](#retention) flag value.
|
||||
If series doesn't match any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention)
|
||||
command-line flag is applied to it. If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
|
||||
For example, the following config sets 3 days retention for time series with `team="juniors"` label,
|
||||
30 days retention for time series with `env="dev"` or `env="staging"` label and 1 year retention for the remaining time series:
|
||||
|
|
@ -1790,9 +1867,9 @@ Graphs on the dashboards contain useful hints - hover the `i` icon in the top le
|
|||
We recommend setting up [alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#alerts)
|
||||
via [vmalert](https://docs.victoriametrics.com/vmalert.html) or via Prometheus.
|
||||
|
||||
VictoriaMetrics exposes currently running queries and their execution times at `/api/v1/status/active_queries` page.
|
||||
VictoriaMetrics exposes currently running queries and their execution times at [`active queries` page](#active-queries).
|
||||
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at `/api/v1/status/top_queries` page.
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at [`top queries` page](#top-queries).
|
||||
|
||||
See also [VictoriaMetrics Monitoring](https://victoriametrics.com/blog/victoriametrics-monitoring/)
|
||||
and [troubleshooting docs](https://docs.victoriametrics.com/Troubleshooting.html).
|
||||
|
|
@ -1937,9 +2014,6 @@ and [cardinality explorer docs](#cardinality-explorer).
|
|||
has at least 20% of free space. The remaining amount of free space
|
||||
can be [monitored](#monitoring) via `vm_free_disk_space_bytes` metric. The total size of data
|
||||
stored on the disk can be monitored via sum of `vm_data_size_bytes` metrics.
|
||||
See also `vm_merge_need_free_disk_space` metrics, which are set to values higher than 0
|
||||
if background merge cannot be initiated due to free disk space shortage. The value shows the number of per-month partitions,
|
||||
which would start background merge if they had more free disk space.
|
||||
|
||||
* VictoriaMetrics buffers incoming data in memory for up to a few seconds before flushing it to persistent storage.
|
||||
This may lead to the following "issues":
|
||||
|
|
@ -2299,6 +2373,8 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
|
|||
Prefix for environment variables if -envflag.enable is set
|
||||
-eula
|
||||
Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-filestream.disableFadvise
|
||||
Whether to disable fadvise() syscall when reading large data files. The fadvise() syscall prevents from eviction of recently accessed data from OS page cache during background merges and backups. In some rare cases it is better to disable the syscall if it uses too much CPU
|
||||
-finalMergeDelay duration
|
||||
The delay before starting final merge for per-month partition after no new data is ingested into it. Final merge may require additional disk IO and CPU resources. Final merge may increase query speed and reduce disk space usage in some cases. Zero value disables final merge
|
||||
-flagsAuthKey string
|
||||
|
|
|
|||
|
|
@ -12,6 +12,8 @@ import (
|
|||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vlinsert/insertutils"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vlstorage"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/bufferedwriter"
|
||||
|
|
@ -22,7 +24,6 @@ import (
|
|||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logstorage"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/common"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/writeconcurrencylimiter"
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
)
|
||||
|
||||
var (
|
||||
|
|
@ -93,16 +94,20 @@ func RequestHandler(path string, w http.ResponseWriter, r *http.Request) bool {
|
|||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
if err := vlstorage.CanWriteData(); err != nil {
|
||||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
lr := logstorage.GetLogRows(cp.StreamFields, cp.IgnoreFields)
|
||||
processLogMessage := cp.GetProcessLogMessageFunc(lr)
|
||||
isGzip := r.Header.Get("Content-Encoding") == "gzip"
|
||||
n, err := readBulkRequest(r.Body, isGzip, cp.TimeField, cp.MsgField, processLogMessage)
|
||||
vlstorage.MustAddRows(lr)
|
||||
logstorage.PutLogRows(lr)
|
||||
if err != nil {
|
||||
logger.Warnf("cannot decode log message #%d in /_bulk request: %s", n, err)
|
||||
return true
|
||||
}
|
||||
vlstorage.MustAddRows(lr)
|
||||
logstorage.PutLogRows(lr)
|
||||
|
||||
tookMs := time.Since(startTime).Milliseconds()
|
||||
bw := bufferedwriter.Get(w)
|
||||
|
|
@ -220,6 +225,7 @@ func readBulkLine(sc *bufio.Scanner, timeField, msgField string,
|
|||
p.RenameField(msgField, "_msg")
|
||||
processLogMessage(ts, p.Fields)
|
||||
logjson.PutParser(p)
|
||||
|
||||
return true, nil
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -36,6 +36,10 @@ func RequestHandler(w http.ResponseWriter, r *http.Request) bool {
|
|||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
if err := vlstorage.CanWriteData(); err != nil {
|
||||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
lr := logstorage.GetLogRows(cp.StreamFields, cp.IgnoreFields)
|
||||
processLogMessage := cp.GetProcessLogMessageFunc(lr)
|
||||
|
||||
|
|
@ -115,6 +119,7 @@ func readLine(sc *bufio.Scanner, timeField, msgField string, processLogMessage f
|
|||
p.RenameField(msgField, "_msg")
|
||||
processLogMessage(ts, p.Fields)
|
||||
logjson.PutParser(p)
|
||||
|
||||
return true, nil
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -47,15 +47,20 @@ func handleJSON(r *http.Request, w http.ResponseWriter) bool {
|
|||
httpserver.Errorf(w, r, "cannot parse common params from request: %s", err)
|
||||
return true
|
||||
}
|
||||
if err := vlstorage.CanWriteData(); err != nil {
|
||||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
lr := logstorage.GetLogRows(cp.StreamFields, cp.IgnoreFields)
|
||||
processLogMessage := cp.GetProcessLogMessageFunc(lr)
|
||||
n, err := parseJSONRequest(data, processLogMessage)
|
||||
vlstorage.MustAddRows(lr)
|
||||
logstorage.PutLogRows(lr)
|
||||
if err != nil {
|
||||
httpserver.Errorf(w, r, "cannot parse Loki request: %s", err)
|
||||
httpserver.Errorf(w, r, "cannot parse Loki json request: %s", err)
|
||||
return true
|
||||
}
|
||||
|
||||
rowsIngestedJSONTotal.Add(n)
|
||||
|
||||
// update lokiRequestJSONDuration only for successfully parsed requests
|
||||
|
|
@ -166,7 +171,6 @@ func parseJSONRequest(data []byte, processLogMessage func(timestamp int64, field
|
|||
Value: bytesutil.ToUnsafeString(msg),
|
||||
})
|
||||
processLogMessage(ts, fields)
|
||||
|
||||
}
|
||||
rowsIngested += len(lines)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -39,13 +39,17 @@ func handleProtobuf(r *http.Request, w http.ResponseWriter) bool {
|
|||
httpserver.Errorf(w, r, "cannot parse common params from request: %s", err)
|
||||
return true
|
||||
}
|
||||
if err := vlstorage.CanWriteData(); err != nil {
|
||||
httpserver.Errorf(w, r, "%s", err)
|
||||
return true
|
||||
}
|
||||
lr := logstorage.GetLogRows(cp.StreamFields, cp.IgnoreFields)
|
||||
processLogMessage := cp.GetProcessLogMessageFunc(lr)
|
||||
n, err := parseProtobufRequest(data, processLogMessage)
|
||||
vlstorage.MustAddRows(lr)
|
||||
logstorage.PutLogRows(lr)
|
||||
if err != nil {
|
||||
httpserver.Errorf(w, r, "cannot parse loki request: %s", err)
|
||||
httpserver.Errorf(w, r, "cannot parse Loki protobuf request: %s", err)
|
||||
return true
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -6,8 +6,9 @@ import (
|
|||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logstorage"
|
||||
"github.com/golang/snappy"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logstorage"
|
||||
)
|
||||
|
||||
func BenchmarkParseProtobufRequest(b *testing.B) {
|
||||
|
|
|
|||
|
|
@ -1,14 +1,14 @@
|
|||
{
|
||||
"files": {
|
||||
"main.css": "./static/css/main.17914339.css",
|
||||
"main.js": "./static/js/main.b6509627.js",
|
||||
"main.css": "./static/css/main.3258b6c4.css",
|
||||
"main.js": "./static/js/main.41b816cc.js",
|
||||
"static/js/522.b5ae4365.chunk.js": "./static/js/522.b5ae4365.chunk.js",
|
||||
"static/media/Lato-Regular.ttf": "./static/media/Lato-Regular.d714fec1633b69a9c2e9.ttf",
|
||||
"static/media/Lato-Bold.ttf": "./static/media/Lato-Bold.32360ba4b57802daa4d6.ttf",
|
||||
"index.html": "./index.html"
|
||||
},
|
||||
"entrypoints": [
|
||||
"static/css/main.17914339.css",
|
||||
"static/js/main.b6509627.js"
|
||||
"static/css/main.3258b6c4.css",
|
||||
"static/js/main.41b816cc.js"
|
||||
]
|
||||
}
|
||||
|
|
@ -1 +1 @@
|
|||
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.b6509627.js"></script><link href="./static/css/main.17914339.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
|
||||
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.41b816cc.js"></script><link href="./static/css/main.3258b6c4.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
|
||||
File diff suppressed because one or more lines are too long
1
app/vlselect/vmui/static/css/main.3258b6c4.css
Normal file
1
app/vlselect/vmui/static/css/main.3258b6c4.css
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -3,14 +3,17 @@ package vlstorage
|
|||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/flagutil"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logstorage"
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
)
|
||||
|
||||
var (
|
||||
|
|
@ -29,6 +32,8 @@ var (
|
|||
"see https://docs.victoriametrics.com/VictoriaLogs/keyConcepts.html#stream-fields ; see also -logIngestedRows")
|
||||
logIngestedRows = flag.Bool("logIngestedRows", false, "Whether to log all the ingested log entries; this can be useful for debugging of data ingestion; "+
|
||||
"see https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/ ; see also -logNewStreams")
|
||||
minFreeDiskSpaceBytes = flagutil.NewBytes("storage.minFreeDiskSpaceBytes", 10e6, "The minimum free disk space at -storageDataPath after which "+
|
||||
"the storage stops accepting new data")
|
||||
)
|
||||
|
||||
// Init initializes vlstorage.
|
||||
|
|
@ -43,11 +48,12 @@ func Init() {
|
|||
logger.Fatalf("-retentionPeriod cannot be smaller than a day; got %s", retentionPeriod)
|
||||
}
|
||||
cfg := &logstorage.StorageConfig{
|
||||
Retention: retentionPeriod.Duration(),
|
||||
FlushInterval: *inmemoryDataFlushInterval,
|
||||
FutureRetention: futureRetention.Duration(),
|
||||
LogNewStreams: *logNewStreams,
|
||||
LogIngestedRows: *logIngestedRows,
|
||||
Retention: retentionPeriod.Duration(),
|
||||
FlushInterval: *inmemoryDataFlushInterval,
|
||||
FutureRetention: futureRetention.Duration(),
|
||||
LogNewStreams: *logNewStreams,
|
||||
LogIngestedRows: *logIngestedRows,
|
||||
MinFreeDiskSpaceBytes: minFreeDiskSpaceBytes.N,
|
||||
}
|
||||
logger.Infof("opening storage at -storageDataPath=%s", *storageDataPath)
|
||||
startTime := time.Now()
|
||||
|
|
@ -74,7 +80,21 @@ func Stop() {
|
|||
var strg *logstorage.Storage
|
||||
var storageMetrics *metrics.Set
|
||||
|
||||
// CanWriteData returns non-nil error if it cannot write data to vlstorage.
|
||||
func CanWriteData() error {
|
||||
if strg.IsReadOnly() {
|
||||
return &httpserver.ErrorWithStatusCode{
|
||||
Err: fmt.Errorf("cannot add rows into storage in read-only mode; the storage can be in read-only mode "+
|
||||
"because of lack of free disk space at -storageDataPath=%s", *storageDataPath),
|
||||
StatusCode: http.StatusTooManyRequests,
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// MustAddRows adds lr to vlstorage
|
||||
//
|
||||
// It is advised to call CanWriteData() before calling MustAddRows()
|
||||
func MustAddRows(lr *logstorage.LogRows) {
|
||||
strg.MustAddRows(lr)
|
||||
}
|
||||
|
|
@ -107,6 +127,12 @@ func initStorageMetrics(strg *logstorage.Storage) *metrics.Set {
|
|||
ms.NewGauge(fmt.Sprintf(`vl_free_disk_space_bytes{path=%q}`, *storageDataPath), func() float64 {
|
||||
return float64(fs.MustGetFreeSpace(*storageDataPath))
|
||||
})
|
||||
ms.NewGauge(fmt.Sprintf(`vl_storage_is_read_only{path=%q}`, *storageDataPath), func() float64 {
|
||||
if m().IsReadOnly {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
})
|
||||
|
||||
ms.NewGauge(`vl_active_merges{type="inmemory"}`, func() float64 {
|
||||
return float64(m().InmemoryActiveMerges)
|
||||
|
|
|
|||
|
|
@ -3,7 +3,8 @@
|
|||
`vmagent` is a tiny agent which helps you collect metrics from various sources,
|
||||
[relabel and filter the collected metrics](#relabeling)
|
||||
and store them in [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics)
|
||||
or any other storage systems via Prometheus `remote_write` protocol.
|
||||
or any other storage systems via Prometheus `remote_write` protocol
|
||||
or via [VictoriaMetrics `remote_write` protocol](#victoriametrics-remote-write-protocol).
|
||||
|
||||
See [Quick Start](#quick-start) for details.
|
||||
|
||||
|
|
|
|||
|
|
@ -11,6 +11,8 @@ import (
|
|||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmagent/csvimport"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmagent/datadog"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmagent/graphite"
|
||||
|
|
@ -39,7 +41,6 @@ import (
|
|||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/promscrape"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/common"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/pushmetrics"
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
)
|
||||
|
||||
var (
|
||||
|
|
@ -208,7 +209,7 @@ func getAuthTokenFromPath(path string) (*auth.Token, error) {
|
|||
if p.Suffix != "opentsdb/api/put" {
|
||||
return nil, fmt.Errorf("unsupported path requested: %q; expecting 'opentsdb/api/put'", p.Suffix)
|
||||
}
|
||||
return auth.NewToken(p.AuthToken)
|
||||
return auth.NewTokenPossibleMultitenant(p.AuthToken)
|
||||
}
|
||||
|
||||
func requestHandler(w http.ResponseWriter, r *http.Request) bool {
|
||||
|
|
@ -251,7 +252,7 @@ func requestHandler(w http.ResponseWriter, r *http.Request) bool {
|
|||
w.WriteHeader(statusCode)
|
||||
return true
|
||||
}
|
||||
if strings.HasPrefix(path, "datadog/") {
|
||||
if strings.HasPrefix(path, "/datadog/") {
|
||||
// Trim suffix from paths starting from /datadog/ in order to support legacy DataDog agent.
|
||||
// See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2670
|
||||
path = strings.TrimSuffix(path, "/")
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ func InsertHandler(at *auth.Token, req *http.Request) error {
|
|||
return err
|
||||
}
|
||||
isGzipped := req.Header.Get("Content-Encoding") == "gzip"
|
||||
return stream.Parse(req.Body, defaultTimestamp, isGzipped, func(rows []parser.Row) error {
|
||||
return stream.Parse(req.Body, defaultTimestamp, isGzipped, true, func(rows []parser.Row) error {
|
||||
return insertRows(at, rows, extraLabels)
|
||||
}, func(s string) {
|
||||
httpserver.LogError(req, s)
|
||||
|
|
|
|||
|
|
@ -526,7 +526,7 @@ Alertmanagers.
|
|||
|
||||
To avoid recording rules results and alerts state duplication in VictoriaMetrics server
|
||||
don't forget to configure [deduplication](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#deduplication).
|
||||
The recommended value for `-dedup.minScrapeInterval` must be multiple of vmalert's `evaluation_interval`.
|
||||
The recommended value for `-dedup.minScrapeInterval` must be multiple of vmalert's `-evaluationInterval`.
|
||||
If you observe inconsistent or "jumping" values in series produced by vmalert, try disabling `-datasource.queryTimeAlignment`
|
||||
command line flag. Because of alignment, two or more vmalert HA pairs will produce results with the same timestamps.
|
||||
But due of backfilling (data delivered to the datasource with some delay) values of such results may differ,
|
||||
|
|
@ -778,7 +778,7 @@ may get empty response from the datasource and produce empty recording rules or
|
|||
|
||||
Try the following recommendations to reduce the chance of hitting the data delay issue:
|
||||
|
||||
* Always configure group's `evaluationInterval` to be bigger or at least equal to
|
||||
* Always configure group's `-evaluationInterval` to be bigger or at least equal to
|
||||
[time series resolution](https://docs.victoriametrics.com/keyConcepts.html#time-series-resolution);
|
||||
* Ensure that `[duration]` value is at least twice bigger than
|
||||
[time series resolution](https://docs.victoriametrics.com/keyConcepts.html#time-series-resolution). For example,
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ The auth config can be reloaded via the following ways:
|
|||
and apply new changes every 5 seconds.
|
||||
|
||||
Docker images for `vmauth` are available [here](https://hub.docker.com/r/victoriametrics/vmauth/tags).
|
||||
See how `vmauth` used in [docker-compose env](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/README.md#victoriametrics-cluster).
|
||||
|
||||
Pass `-help` to `vmauth` in order to see all the supported command-line flags with their descriptions.
|
||||
|
||||
|
|
|
|||
|
|
@ -136,7 +136,8 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-
|
|||
## Troubleshooting
|
||||
|
||||
* If the backup is slow, then try setting higher value for `-concurrency` flag. This will increase the number of concurrent workers that upload data to backup storage.
|
||||
* If `vmbackup` eats all the network bandwidth, then set `-maxBytesPerSecond` to the desired value.
|
||||
* If `vmbackup` eats all the network bandwidth or CPU, then either decrease the `-concurrency` command-line flag value or set `-maxBytesPerSecond` command-line flag value to lower value.
|
||||
* If `vmbackup` consumes all the CPU on systems with big number of CPU cores, then try running it with `-filestream.disableFadvise` command-line flag.
|
||||
* If `vmbackup` has been interrupted due to temporary error, then just restart it with the same args. It will resume the backup process.
|
||||
* Backups created from [single-node VictoriaMetrics](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html) cannot be restored
|
||||
at [cluster VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html) and vice versa.
|
||||
|
|
@ -292,6 +293,8 @@ Run `vmbackup -help` in order to see all the available options:
|
|||
Prefix for environment variables if -envflag.enable is set
|
||||
-eula
|
||||
Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-filestream.disableFadvise
|
||||
Whether to disable fadvise() syscall when reading large data files. The fadvise() syscall prevents from eviction of recently accessed data from OS page cache during background merges and backups. In some rare cases it is better to disable the syscall if it uses too much CPU
|
||||
-flagsAuthKey string
|
||||
Auth key for /flags endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-fs.disableMmap
|
||||
|
|
|
|||
|
|
@ -373,6 +373,7 @@ Clusters here are referred to as `source` and `destination`.
|
|||
```console
|
||||
$ /vmbackupmanager-prod restore create s3://source_cluster/vmstorage-source-0/daily/2023-04-07
|
||||
```
|
||||
1. Restart `vmstorage` pods of *destination* cluster. On pod start `vmbackupmanager` will restore data from the specified backup.
|
||||
|
||||
## Monitoring
|
||||
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ func InsertHandler(req *http.Request) error {
|
|||
return err
|
||||
}
|
||||
isGzipped := req.Header.Get("Content-Encoding") == "gzip"
|
||||
return stream.Parse(req.Body, defaultTimestamp, isGzipped, func(rows []parser.Row) error {
|
||||
return stream.Parse(req.Body, defaultTimestamp, isGzipped, true, func(rows []parser.Row) error {
|
||||
return insertRows(rows, extraLabels)
|
||||
}, func(s string) {
|
||||
httpserver.LogError(req, s)
|
||||
|
|
|
|||
|
|
@ -111,6 +111,12 @@ func maySortResults(e metricsql.Expr) bool {
|
|||
"bottomk_max", "bottomk_min", "bottomk_avg", "bottomk_median", "bottomk_last":
|
||||
return false
|
||||
}
|
||||
case *metricsql.BinaryOpExpr:
|
||||
if strings.ToLower(v.Op) == "or" {
|
||||
// Do not sort results for `a or b` in the same way as Prometheus does.
|
||||
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
|
|
|||
|
|
@ -5,10 +5,11 @@ import (
|
|||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/netstorage"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/searchutils"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
)
|
||||
|
||||
func TestEscapeDots(t *testing.T) {
|
||||
|
|
@ -245,6 +246,27 @@ func TestExecSuccess(t *testing.T) {
|
|||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("bitmap_and(NaN, 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_and(NaN, 1)`
|
||||
f(q, nil)
|
||||
})
|
||||
t.Run("bitmap_and(1, NaN)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_and(1, NaN)`
|
||||
f(q, nil)
|
||||
})
|
||||
t.Run("bitmap_and(round(rand(1) > 0.5, 1), 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_and(round(rand(1) > 0.5, 1), 1)`
|
||||
r := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{1, 1, 1, nan, nan, 1},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("bitmap_or(0xA2, 0x11)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_or(0xA2, 0x11)`
|
||||
|
|
@ -267,6 +289,22 @@ func TestExecSuccess(t *testing.T) {
|
|||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("bitmap_or(NaN, 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_or(NaN, 1)`
|
||||
f(q, nil)
|
||||
})
|
||||
t.Run("bitmap_or(round(rand(1) > 0.5, 1), 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_or(round(rand(1) > 0.5, 1), 1)`
|
||||
r := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{1, 1, 1, nan, nan, 1},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("bitmap_xor(0xB3, 0x11)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_xor(0xB3, 0x11)`
|
||||
|
|
@ -289,6 +327,22 @@ func TestExecSuccess(t *testing.T) {
|
|||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("bitmap_xor(NaN, 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_xor(NaN, 1)`
|
||||
f(q, nil)
|
||||
})
|
||||
t.Run("bitmap_xor(round(rand(1) > 0.5, 1), 1)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `bitmap_xor(round(rand(1) > 0.5, 1), 1)`
|
||||
r := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{0, 0, 0, nan, nan, 0},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
resultExpected := []netstorage.Result{r}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run("timezone_offset(UTC)", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `timezone_offset("UTC")`
|
||||
|
|
@ -7669,7 +7723,7 @@ func TestExecSuccess(t *testing.T) {
|
|||
})
|
||||
t.Run(`aggr_over_time(multi-func)`, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `sort(aggr_over_time(("min_over_time", "count_over_time", "max_over_time"), round(rand(0),0.1)[:10s]))`
|
||||
q := `sort(aggr_over_time(("min_over_time", "median_over_time", "max_over_time"), round(rand(0),0.1)[:10s]))`
|
||||
r1 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{0, 0, 0, 0, 0, 0},
|
||||
|
|
@ -7681,21 +7735,21 @@ func TestExecSuccess(t *testing.T) {
|
|||
}}
|
||||
r2 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{0.8, 0.9, 1, 0.9, 1, 0.9},
|
||||
Values: []float64{0.4, 0.5, 0.5, 0.75, 0.6, 0.45},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
r2.MetricName.Tags = []storage.Tag{{
|
||||
Key: []byte("rollup"),
|
||||
Value: []byte("max_over_time"),
|
||||
Value: []byte("median_over_time"),
|
||||
}}
|
||||
r3 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{20, 20, 20, 20, 20, 20},
|
||||
Values: []float64{0.8, 0.9, 1, 0.9, 1, 0.9},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
r3.MetricName.Tags = []storage.Tag{{
|
||||
Key: []byte("rollup"),
|
||||
Value: []byte("count_over_time"),
|
||||
Value: []byte("max_over_time"),
|
||||
}}
|
||||
resultExpected := []netstorage.Result{r1, r2, r3}
|
||||
f(q, resultExpected)
|
||||
|
|
@ -8479,11 +8533,11 @@ func TestExecSuccess(t *testing.T) {
|
|||
})
|
||||
t.Run(`result sorting`, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `label_set(1, "instance", "localhost:1001", "type", "free")
|
||||
or label_set(1, "instance", "localhost:1001", "type", "buffers")
|
||||
or label_set(1, "instance", "localhost:1000", "type", "buffers")
|
||||
or label_set(1, "instance", "localhost:1000", "type", "free")
|
||||
`
|
||||
q := `(label_set(1, "instance", "localhost:1001", "type", "free"),
|
||||
label_set(1, "instance", "localhost:1001", "type", "buffers"),
|
||||
label_set(1, "instance", "localhost:1000", "type", "buffers"),
|
||||
label_set(1, "instance", "localhost:1000", "type", "free"),
|
||||
)`
|
||||
r1 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{1, 1, 1, 1, 1, 1},
|
||||
|
|
@ -8515,6 +8569,34 @@ func TestExecSuccess(t *testing.T) {
|
|||
resultExpected := []netstorage.Result{r1, r2, r3, r4}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run(`no_sorting_for_or`, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `label_set(2, "foo", "bar") or label_set(1, "foo", "baz")`
|
||||
r1 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{2, 2, 2, 2, 2, 2},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
r1.MetricName.Tags = []storage.Tag{
|
||||
{
|
||||
Key: []byte("foo"),
|
||||
Value: []byte("bar"),
|
||||
},
|
||||
}
|
||||
r2 := netstorage.Result{
|
||||
MetricName: metricNameExpected,
|
||||
Values: []float64{1, 1, 1, 1, 1, 1},
|
||||
Timestamps: timestampsExpected,
|
||||
}
|
||||
r2.MetricName.Tags = []storage.Tag{
|
||||
{
|
||||
Key: []byte("foo"),
|
||||
Value: []byte("baz"),
|
||||
},
|
||||
}
|
||||
resultExpected := []netstorage.Result{r1, r2}
|
||||
f(q, resultExpected)
|
||||
})
|
||||
t.Run(`sort_by_label_numeric(multiple_labels_only_string)`, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
q := `sort_by_label_numeric((
|
||||
|
|
|
|||
|
|
@ -7,11 +7,12 @@ import (
|
|||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
)
|
||||
|
||||
var minStalenessInterval = flag.Duration("search.minStalenessInterval", 0, "The minimum interval for staleness calculations. "+
|
||||
|
|
@ -58,6 +59,7 @@ var rollupFuncs = map[string]newRollupFunc{
|
|||
"lifetime": newRollupFuncOneArg(rollupLifetime),
|
||||
"mad_over_time": newRollupFuncOneArg(rollupMAD),
|
||||
"max_over_time": newRollupFuncOneArg(rollupMax),
|
||||
"median_over_time": newRollupFuncOneArg(rollupMedian),
|
||||
"min_over_time": newRollupFuncOneArg(rollupMin),
|
||||
"mode_over_time": newRollupFuncOneArg(rollupModeOverTime),
|
||||
"predict_linear": newRollupPredictLinear,
|
||||
|
|
@ -125,6 +127,7 @@ var rollupAggrFuncs = map[string]rollupFunc{
|
|||
"lifetime": rollupLifetime,
|
||||
"mad_over_time": rollupMAD,
|
||||
"max_over_time": rollupMax,
|
||||
"median_over_time": rollupMedian,
|
||||
"min_over_time": rollupMin,
|
||||
"mode_over_time": rollupModeOverTime,
|
||||
"present_over_time": rollupPresent,
|
||||
|
|
@ -224,6 +227,7 @@ var rollupFuncsKeepMetricName = map[string]bool{
|
|||
"holt_winters": true,
|
||||
"last_over_time": true,
|
||||
"max_over_time": true,
|
||||
"median_over_time": true,
|
||||
"min_over_time": true,
|
||||
"mode_over_time": true,
|
||||
"predict_linear": true,
|
||||
|
|
@ -1396,6 +1400,10 @@ func rollupMax(rfa *rollupFuncArg) float64 {
|
|||
return maxValue
|
||||
}
|
||||
|
||||
func rollupMedian(rfa *rollupFuncArg) float64 {
|
||||
return quantile(0.5, rfa.values)
|
||||
}
|
||||
|
||||
func rollupTmin(rfa *rollupFuncArg) float64 {
|
||||
// There is no need in handling NaNs here, since they must be cleaned up
|
||||
// before calling rollup funcs.
|
||||
|
|
|
|||
|
|
@ -11,12 +11,13 @@ import (
|
|||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/app/vmselect/searchutils"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/storage"
|
||||
"github.com/VictoriaMetrics/metricsql"
|
||||
)
|
||||
|
||||
var transformFuncs = map[string]transformFunc{
|
||||
|
|
@ -2589,7 +2590,12 @@ func newTransformBitmap(bitmapFunc func(a, b uint64) uint64) func(tfa *transform
|
|||
}
|
||||
tf := func(values []float64) {
|
||||
for i, v := range values {
|
||||
values[i] = float64(bitmapFunc(uint64(v), uint64(ns[i])))
|
||||
w := ns[i]
|
||||
result := nan
|
||||
if !math.IsNaN(v) && !math.IsNaN(w) {
|
||||
result = float64(bitmapFunc(uint64(v), uint64(w)))
|
||||
}
|
||||
values[i] = result
|
||||
}
|
||||
}
|
||||
return doTransformValues(args[0], tf, tfa.fe)
|
||||
|
|
|
|||
|
|
@ -1,14 +1,14 @@
|
|||
{
|
||||
"files": {
|
||||
"main.css": "./static/css/main.e95426eb.css",
|
||||
"main.js": "./static/js/main.8d3e794d.js",
|
||||
"main.css": "./static/css/main.4ab6595e.css",
|
||||
"main.js": "./static/js/main.dbf8fb4f.js",
|
||||
"static/js/522.b5ae4365.chunk.js": "./static/js/522.b5ae4365.chunk.js",
|
||||
"static/media/Lato-Regular.ttf": "./static/media/Lato-Regular.d714fec1633b69a9c2e9.ttf",
|
||||
"static/media/Lato-Bold.ttf": "./static/media/Lato-Bold.32360ba4b57802daa4d6.ttf",
|
||||
"index.html": "./index.html"
|
||||
},
|
||||
"entrypoints": [
|
||||
"static/css/main.e95426eb.css",
|
||||
"static/js/main.8d3e794d.js"
|
||||
"static/css/main.4ab6595e.css",
|
||||
"static/js/main.dbf8fb4f.js"
|
||||
]
|
||||
}
|
||||
|
|
@ -1 +1 @@
|
|||
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.8d3e794d.js"></script><link href="./static/css/main.e95426eb.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
|
||||
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=5"/><meta name="theme-color" content="#000000"/><meta name="description" content="UI for VictoriaMetrics"/><link rel="apple-touch-icon" href="./apple-touch-icon.png"/><link rel="icon" type="image/png" sizes="32x32" href="./favicon-32x32.png"><link rel="manifest" href="./manifest.json"/><title>VM UI</title><script src="./dashboards/index.js" type="module"></script><meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" content="./preview.jpg"><meta name="twitter:title" content="UI for VictoriaMetrics"><meta name="twitter:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta name="twitter:site" content="@VictoriaMetrics"><meta property="og:title" content="Metric explorer for VictoriaMetrics"><meta property="og:description" content="Explore and troubleshoot your VictoriaMetrics data"><meta property="og:image" content="./preview.jpg"><meta property="og:type" content="website"><script defer="defer" src="./static/js/main.dbf8fb4f.js"></script><link href="./static/css/main.4ab6595e.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
|
||||
1
app/vmselect/vmui/static/css/main.4ab6595e.css
Normal file
1
app/vmselect/vmui/static/css/main.4ab6595e.css
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
2
app/vmselect/vmui/static/js/main.dbf8fb4f.js
Normal file
2
app/vmselect/vmui/static/js/main.dbf8fb4f.js
Normal file
File diff suppressed because one or more lines are too long
|
|
@ -585,11 +585,6 @@ func registerStorageMetrics(strg *storage.Storage) {
|
|||
return float64(idbm().ItemsAddedSizeBytes)
|
||||
})
|
||||
|
||||
// See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686
|
||||
metrics.NewGauge(`vm_merge_need_free_disk_space`, func() float64 {
|
||||
return float64(tm().MergeNeedFreeDiskSpace)
|
||||
})
|
||||
|
||||
metrics.NewGauge(`vm_pending_rows{type="storage"}`, func() float64 {
|
||||
return float64(tm().PendingRows)
|
||||
})
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ interface Hyperlink {
|
|||
children?: ReactNode;
|
||||
colored?: boolean;
|
||||
underlined?: boolean;
|
||||
withIcon?: boolean;
|
||||
}
|
||||
|
||||
const Hyperlink: FC<Hyperlink> = ({
|
||||
|
|
@ -15,14 +16,16 @@ const Hyperlink: FC<Hyperlink> = ({
|
|||
href,
|
||||
children,
|
||||
colored = true,
|
||||
underlined = false
|
||||
underlined = false,
|
||||
withIcon = false,
|
||||
}) => (
|
||||
<a
|
||||
href={href}
|
||||
className={classNames({
|
||||
"vm-link": true,
|
||||
"vm-link_colored": colored,
|
||||
"vm-link_underlined": underlined
|
||||
"vm-link_underlined": underlined,
|
||||
"vm-link_with-icon": withIcon,
|
||||
})}
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
|
|
|
|||
|
|
@ -430,3 +430,23 @@ export const ListIcon = () => (
|
|||
<path d="M3 14h4v-4H3v4zm0 5h4v-4H3v4zM3 9h4V5H3v4zm5 5h13v-4H8v4zm0 5h13v-4H8v4zM8 5v4h13V5H8z"></path>
|
||||
</svg>
|
||||
);
|
||||
|
||||
export const StarBorderIcon = () => (
|
||||
<svg
|
||||
viewBox="0 0 24 24"
|
||||
fill="currentColor"
|
||||
>
|
||||
<path
|
||||
d="m22 9.24-7.19-.62L12 2 9.19 8.63 2 9.24l5.46 4.73L5.82 21 12 17.27 18.18 21l-1.63-7.03L22 9.24zM12 15.4l-3.76 2.27 1-4.28-3.32-2.88 4.38-.38L12 6.1l1.71 4.04 4.38.38-3.32 2.88 1 4.28L12 15.4z"
|
||||
></path>
|
||||
</svg>
|
||||
);
|
||||
|
||||
export const StarIcon = () => (
|
||||
<svg
|
||||
viewBox="0 0 24 24"
|
||||
fill="currentColor"
|
||||
>
|
||||
<path d="M12 17.27 18.18 21l-1.64-7.03L22 9.24l-7.19-.61L12 2 9.19 8.63 2 9.24l5.46 4.73L5.82 21z"></path>
|
||||
</svg>
|
||||
);
|
||||
|
|
|
|||
|
|
@ -32,6 +32,10 @@ $padding-modal: $padding-medium;
|
|||
&-header {
|
||||
padding: $padding-small $padding-small $padding-small $padding-global;
|
||||
margin-bottom: $padding-global;
|
||||
|
||||
&__title {
|
||||
max-width: 80vw;
|
||||
}
|
||||
}
|
||||
|
||||
&-body {
|
||||
|
|
@ -69,6 +73,10 @@ $padding-modal: $padding-medium;
|
|||
&__title {
|
||||
font-weight: bold;
|
||||
user-select: none;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
max-width: 50vw;
|
||||
}
|
||||
|
||||
&__close {
|
||||
|
|
|
|||
|
|
@ -8,8 +8,10 @@ import classNames from "classnames";
|
|||
import { useAppState } from "../../../state/common/StateContext";
|
||||
import useDeviceDetect from "../../../hooks/useDeviceDetect";
|
||||
import Button from "../../Main/Button/Button";
|
||||
import { humanizeSeconds } from "../../../utils/time";
|
||||
|
||||
interface RecursiveProps {
|
||||
isRoot?: boolean;
|
||||
trace: Trace;
|
||||
totalMsec: number;
|
||||
}
|
||||
|
|
@ -18,7 +20,7 @@ interface OpenLevels {
|
|||
[x: number]: boolean
|
||||
}
|
||||
|
||||
const NestedNav: FC<RecursiveProps> = ({ trace, totalMsec }) => {
|
||||
const NestedNav: FC<RecursiveProps> = ({ isRoot, trace, totalMsec }) => {
|
||||
const { isDarkTheme } = useAppState();
|
||||
const { isMobile } = useDeviceDetect();
|
||||
const [openLevels, setOpenLevels] = useState({} as OpenLevels);
|
||||
|
|
@ -27,6 +29,8 @@ const NestedNav: FC<RecursiveProps> = ({ trace, totalMsec }) => {
|
|||
const [isExpanded, setIsExpanded] = useState(false);
|
||||
const [showFullMessage, setShowFullMessage] = useState(false);
|
||||
|
||||
const duration = humanizeSeconds(trace.duration / 1000) || `${trace.duration}ms`;
|
||||
|
||||
useEffect(() => {
|
||||
if (!messageRef.current) return;
|
||||
const contentElement = messageRef.current;
|
||||
|
|
@ -40,24 +44,29 @@ const NestedNav: FC<RecursiveProps> = ({ trace, totalMsec }) => {
|
|||
setShowFullMessage(prev => !prev);
|
||||
};
|
||||
|
||||
const hasChildren = trace.children && !!trace.children.length;
|
||||
const progress = trace.duration / totalMsec * 100;
|
||||
const handleListClick = (level: number) => () => {
|
||||
if (!hasChildren) return;
|
||||
setOpenLevels((prevState:OpenLevels) => {
|
||||
return { ...prevState, [level]: !prevState[level] };
|
||||
});
|
||||
};
|
||||
const hasChildren = trace.children && !!trace.children.length;
|
||||
const progress = trace.duration / totalMsec * 100;
|
||||
|
||||
return (
|
||||
<div
|
||||
className={classNames({
|
||||
"vm-nested-nav": true,
|
||||
"vm-nested-nav_root": isRoot,
|
||||
"vm-nested-nav_dark": isDarkTheme,
|
||||
"vm-nested-nav_mobile": isMobile,
|
||||
})}
|
||||
>
|
||||
<div
|
||||
className="vm-nested-nav-header"
|
||||
className={classNames({
|
||||
"vm-nested-nav-header": true,
|
||||
"vm-nested-nav-header_open": openLevels[trace.idValue],
|
||||
})}
|
||||
onClick={handleListClick(trace.idValue)}
|
||||
>
|
||||
{hasChildren && (
|
||||
|
|
@ -84,7 +93,7 @@ const NestedNav: FC<RecursiveProps> = ({ trace, totalMsec }) => {
|
|||
</div>
|
||||
<div className="vm-nested-nav-header-bottom">
|
||||
<div className="vm-nested-nav-header-bottom__duration">
|
||||
{`duration: ${trace.duration} ms`}
|
||||
{`duration: ${duration}`}
|
||||
</div>
|
||||
{(isExpanded || showFullMessage) && (
|
||||
<Button
|
||||
|
|
@ -97,15 +106,17 @@ const NestedNav: FC<RecursiveProps> = ({ trace, totalMsec }) => {
|
|||
)}
|
||||
</div>
|
||||
</div>
|
||||
{openLevels[trace.idValue] && <div>
|
||||
{hasChildren && trace.children.map((trace) => (
|
||||
<NestedNav
|
||||
key={trace.duration}
|
||||
trace={trace}
|
||||
totalMsec={totalMsec}
|
||||
/>
|
||||
))}
|
||||
</div>}
|
||||
{openLevels[trace.idValue] && (
|
||||
<div className="vm-nested-nav__childrens">
|
||||
{hasChildren && trace.children.map((trace) => (
|
||||
<NestedNav
|
||||
key={trace.duration}
|
||||
trace={trace}
|
||||
totalMsec={totalMsec}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,29 +1,71 @@
|
|||
@use "src/styles/variables" as *;
|
||||
|
||||
$color-base-nested-nav: $color-tropical-blue;
|
||||
$color-base-nested-nav-dark: $color-background-body;
|
||||
|
||||
.vm-nested-nav {
|
||||
margin-left: $padding-medium;
|
||||
position: relative;
|
||||
margin-left: $padding-global;
|
||||
border-radius: $border-radius-small;
|
||||
background-color: rgba($color-tropical-blue, 0.4);
|
||||
|
||||
&_dark &-header {
|
||||
background-color: $color-base-nested-nav-dark;
|
||||
|
||||
&:after, &:before {
|
||||
background-color: $color-base-nested-nav-dark;
|
||||
}
|
||||
|
||||
&:hover {
|
||||
box-shadow: rgba($color-white, 0.08) 0 0 0 1px;
|
||||
}
|
||||
}
|
||||
|
||||
&_mobile {
|
||||
margin-left: $padding-small;
|
||||
}
|
||||
|
||||
&_dark {
|
||||
background-color: rgba($color-black, 0.1);
|
||||
&_root > &-header {
|
||||
&:before,
|
||||
&:after {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
|
||||
&-header {
|
||||
position: relative;
|
||||
display: grid;
|
||||
grid-template-columns: auto 1fr;
|
||||
gap: $padding-small;
|
||||
padding: $padding-small;
|
||||
border-radius: $border-radius-small;
|
||||
transition: background-color 200ms ease-in-out;
|
||||
transition: box-shadow 200ms ease-in-out;
|
||||
cursor: pointer;
|
||||
background-color: rgba($color-base-nested-nav, 0.4);
|
||||
margin-bottom: $padding-small;
|
||||
z-index: 2;
|
||||
|
||||
&:after {
|
||||
content: "";
|
||||
position: absolute;
|
||||
top: calc(50% - 1px);
|
||||
height: 2px;
|
||||
width: $padding-small;
|
||||
background-color: $color-base-nested-nav;
|
||||
left: calc(-1 * $padding-small);
|
||||
}
|
||||
|
||||
&:before {
|
||||
content: "";
|
||||
position: absolute;
|
||||
bottom: 50%;
|
||||
left: calc($padding-global / -2);
|
||||
height: calc(50% + $padding-small);
|
||||
width: 2px;
|
||||
background-color: $color-base-nested-nav;
|
||||
}
|
||||
|
||||
&:hover {
|
||||
background-color: $color-hover-black;
|
||||
box-shadow: rgba($color-black, 0.08) 0 0 0 1px;
|
||||
}
|
||||
|
||||
&__icon {
|
||||
|
|
@ -32,9 +74,11 @@
|
|||
justify-content: center;
|
||||
width: 20px;
|
||||
transition: transform 200ms ease-in-out;
|
||||
transform: rotate(-90deg);
|
||||
color: $color-text-secondary;
|
||||
|
||||
&_open {
|
||||
transform: rotate(180deg);
|
||||
transform: rotate(0);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -72,4 +116,22 @@
|
|||
}
|
||||
}
|
||||
}
|
||||
|
||||
&__childrens > .vm-nested-nav:not(:last-child) {
|
||||
&:before {
|
||||
content: "";
|
||||
position: absolute;
|
||||
top: 0;
|
||||
left: calc($padding-global / -2);
|
||||
height: 100%;
|
||||
width: 2px;
|
||||
background-color: $color-base-nested-nav;
|
||||
}
|
||||
}
|
||||
|
||||
&__childrens > .vm-nested-nav_dark:not(:last-child) {
|
||||
&:before {
|
||||
background-color: $color-base-nested-nav-dark;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -89,6 +89,7 @@ const TracingsView: FC<TraceViewProps> = ({ traces, jsonEditor = false, onDelete
|
|||
})}
|
||||
>
|
||||
<NestedNav
|
||||
isRoot
|
||||
trace={trace}
|
||||
totalMsec={trace.duration}
|
||||
/>
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import { GraphSize, SeriesItemStats } from "../types";
|
||||
|
||||
export const MAX_QUERY_FIELDS = 4;
|
||||
export const MAX_QUERIES_HISTORY = 25;
|
||||
export const DEFAULT_MAX_SERIES = {
|
||||
table: 100,
|
||||
chart: 20,
|
||||
|
|
|
|||
|
|
@ -125,7 +125,7 @@ export const useFetchQuery = ({
|
|||
}
|
||||
|
||||
isHistogramResult = isDisplayChart && isHistogramData(resp.data.result);
|
||||
seriesLimit = isHistogramResult ? Infinity : Math.max(totalLength, defaultLimit);
|
||||
seriesLimit = isHistogramResult ? Infinity : defaultLimit;
|
||||
const freeTempSize = seriesLimit - tempData.length;
|
||||
resp.data.result.slice(0, freeTempSize).forEach((d: MetricBase) => {
|
||||
d.group = counter;
|
||||
|
|
@ -140,7 +140,7 @@ export const useFetchQuery = ({
|
|||
counter++;
|
||||
}
|
||||
|
||||
const limitText = `Showing ${seriesLimit} series out of ${totalLength} series due to performance reasons. Please narrow down the query, so it returns less series`;
|
||||
const limitText = `Showing ${tempData.length} series out of ${totalLength} series due to performance reasons. Please narrow down the query, so it returns less series`;
|
||||
setWarning(totalLength > seriesLimit ? limitText : "");
|
||||
isDisplayChart ? setGraphData(tempData as MetricResult[]) : setLiveData(tempData as InstantMetricResult[]);
|
||||
setTraces(tempTraces);
|
||||
|
|
|
|||
|
|
@ -11,8 +11,9 @@ import { useSearchParams } from "react-router-dom";
|
|||
import CardinalityTotals, { CardinalityTotalsProps } from "../CardinalityTotals/CardinalityTotals";
|
||||
import useSearchParamsFromObject from "../../../hooks/useSearchParamsFromObject";
|
||||
import useStateSearchParams from "../../../hooks/useStateSearchParams";
|
||||
import Hyperlink from "../../../components/Main/Hyperlink/Hyperlink";
|
||||
|
||||
const CardinalityConfigurator: FC<CardinalityTotalsProps> = ({ isPrometheus, ...props }) => {
|
||||
const CardinalityConfigurator: FC<CardinalityTotalsProps> = ({ isPrometheus, isCluster, ...props }) => {
|
||||
const { isMobile } = useDeviceDetect();
|
||||
const [searchParams] = useSearchParams();
|
||||
const { setSearchParamsFromKeys } = useSearchParamsFromObject();
|
||||
|
|
@ -105,19 +106,29 @@ const CardinalityConfigurator: FC<CardinalityTotalsProps> = ({ isPrometheus, ...
|
|||
<div className="vm-cardinality-configurator-bottom">
|
||||
<CardinalityTotals
|
||||
isPrometheus={isPrometheus}
|
||||
isCluster={isCluster}
|
||||
{...props}
|
||||
/>
|
||||
{isCluster &&
|
||||
<div className="vm-cardinality-configurator-bottom-helpful">
|
||||
<Hyperlink
|
||||
href="https://docs.victoriametrics.com/#cardinality-explorer-statistic-inaccurancy"
|
||||
withIcon={true}
|
||||
>
|
||||
<WikiIcon/>
|
||||
Statistic inaccuracy explanation
|
||||
</Hyperlink>
|
||||
</div>
|
||||
}
|
||||
|
||||
<div className="vm-cardinality-configurator-bottom-helpful">
|
||||
<a
|
||||
className="vm-link vm-link_with-icon"
|
||||
target="_blank"
|
||||
<Hyperlink
|
||||
href="https://docs.victoriametrics.com/#cardinality-explorer"
|
||||
rel="help noreferrer"
|
||||
withIcon={true}
|
||||
>
|
||||
<WikiIcon/>
|
||||
Documentation
|
||||
</a>
|
||||
</Hyperlink>
|
||||
</div>
|
||||
|
||||
<div className="vm-cardinality-configurator-bottom__execute">
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ export interface CardinalityTotalsProps {
|
|||
totalLabelValuePairs: number;
|
||||
seriesCountByMetricName: TopHeapEntry[];
|
||||
isPrometheus?: boolean;
|
||||
isCluster: boolean;
|
||||
}
|
||||
|
||||
const CardinalityTotals: FC<CardinalityTotalsProps> = ({
|
||||
|
|
@ -21,7 +22,7 @@ const CardinalityTotals: FC<CardinalityTotalsProps> = ({
|
|||
totalSeriesPrev = 0,
|
||||
totalSeriesAll = 0,
|
||||
seriesCountByMetricName = [],
|
||||
isPrometheus
|
||||
isPrometheus,
|
||||
}) => {
|
||||
const { isMobile } = useDeviceDetect();
|
||||
|
||||
|
|
@ -50,7 +51,7 @@ const CardinalityTotals: FC<CardinalityTotalsProps> = ({
|
|||
value: isNaN(progress) ? "-" : `${progress.toFixed(2)}%`,
|
||||
display: isMetric,
|
||||
info: "The share of these series in the total number of time series."
|
||||
}
|
||||
},
|
||||
].filter(t => t.display);
|
||||
|
||||
if (!totals.length) {
|
||||
|
|
|
|||
|
|
@ -7,12 +7,14 @@ import AppConfigurator from "../appConfigurator";
|
|||
import { useSearchParams } from "react-router-dom";
|
||||
import dayjs from "dayjs";
|
||||
import { DATE_FORMAT } from "../../../constants/date";
|
||||
import { getTenantIdFromUrl } from "../../../utils/tenants";
|
||||
|
||||
export const useFetchQuery = (): {
|
||||
fetchUrl?: string[],
|
||||
isLoading: boolean,
|
||||
error?: ErrorTypes | string
|
||||
appConfigurator: AppConfigurator,
|
||||
isCluster: boolean,
|
||||
} => {
|
||||
const appConfigurator = new AppConfigurator();
|
||||
|
||||
|
|
@ -26,6 +28,7 @@ export const useFetchQuery = (): {
|
|||
const [isLoading, setIsLoading] = useState(false);
|
||||
const [error, setError] = useState<ErrorTypes | string>();
|
||||
const [tsdbStatus, setTSDBStatus] = useState<TSDBStatus>(appConfigurator.defaultTSDBStatus);
|
||||
const [isCluster, setIsCluster] = useState<boolean>(false);
|
||||
|
||||
const getResponseJson = async (url: string) => {
|
||||
const response = await fetch(url);
|
||||
|
|
@ -115,6 +118,12 @@ export const useFetchQuery = (): {
|
|||
}
|
||||
}, [error]);
|
||||
|
||||
useEffect(() => {
|
||||
const id = getTenantIdFromUrl(serverUrl);
|
||||
setIsCluster(!!id);
|
||||
}, [serverUrl]);
|
||||
|
||||
|
||||
appConfigurator.tsdbStatusData = tsdbStatus;
|
||||
return { isLoading, appConfigurator: appConfigurator, error };
|
||||
return { isLoading, appConfigurator: appConfigurator, error, isCluster };
|
||||
};
|
||||
|
|
|
|||
|
|
@ -31,7 +31,7 @@ const CardinalityPanel: FC = () => {
|
|||
const match = searchParams.get("match") || "";
|
||||
const focusLabel = searchParams.get("focusLabel") || "";
|
||||
|
||||
const { isLoading, appConfigurator, error } = useFetchQuery();
|
||||
const { isLoading, appConfigurator, error, isCluster } = useFetchQuery();
|
||||
const { tsdbStatusData, getDefaultState, tablesHeaders, sectionsTips } = appConfigurator;
|
||||
const defaultState = getDefaultState(match, focusLabel);
|
||||
|
||||
|
|
@ -62,6 +62,7 @@ const CardinalityPanel: FC = () => {
|
|||
totalSeriesAll={tsdbStatusData.totalSeriesByAll}
|
||||
totalLabelValuePairs={tsdbStatusData.totalLabelValuePairs}
|
||||
seriesCountByMetricName={tsdbStatusData.seriesCountByMetricName}
|
||||
isCluster={isCluster}
|
||||
/>
|
||||
|
||||
{showTips && (
|
||||
|
|
@ -69,7 +70,7 @@ const CardinalityPanel: FC = () => {
|
|||
{!match && !focusLabel && <TipHighNumberOfSeries/>}
|
||||
{match && !focusLabel && <TipCardinalityOfSingle/>}
|
||||
{!match && !focusLabel && <TipHighNumberOfValues/>}
|
||||
{focusLabel && <TipCardinalityOfLabel/>}
|
||||
{focusLabel && <TipCardinalityOfLabel />}
|
||||
</div>
|
||||
)}
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import React, { FC, StateUpdater, useEffect, useState } from "preact/compat";
|
|||
import QueryEditor from "../../../components/Configurators/QueryEditor/QueryEditor";
|
||||
import AdditionalSettings from "../../../components/Configurators/AdditionalSettings/AdditionalSettings";
|
||||
import usePrevious from "../../../hooks/usePrevious";
|
||||
import { MAX_QUERY_FIELDS } from "../../../constants/graph";
|
||||
import { MAX_QUERIES_HISTORY, MAX_QUERY_FIELDS } from "../../../constants/graph";
|
||||
import { useQueryDispatch, useQueryState } from "../../../state/query/QueryStateContext";
|
||||
import { useTimeDispatch } from "../../../state/time/TimeStateContext";
|
||||
import {
|
||||
|
|
@ -22,7 +22,7 @@ import { arrayEquals } from "../../../utils/array";
|
|||
import useDeviceDetect from "../../../hooks/useDeviceDetect";
|
||||
import { QueryStats } from "../../../api/types";
|
||||
import { usePrettifyQuery } from "./hooks/usePrettifyQuery";
|
||||
import QueryHistoryList from "../QueryHistory/QueryHistoryList";
|
||||
import QueryHistory from "../QueryHistory/QueryHistory";
|
||||
|
||||
export interface QueryConfiguratorProps {
|
||||
queryErrors: string[];
|
||||
|
|
@ -66,7 +66,7 @@ const QueryConfigurator: FC<QueryConfiguratorProps> = ({
|
|||
const newValues = !queryEqual && q ? [...h.values, q] : h.values;
|
||||
|
||||
// limit the history
|
||||
if (newValues.length > 25) newValues.shift();
|
||||
if (newValues.length > MAX_QUERIES_HISTORY) newValues.shift();
|
||||
|
||||
return {
|
||||
index: h.values.length - Number(queryEqual),
|
||||
|
|
@ -243,10 +243,7 @@ const QueryConfigurator: FC<QueryConfiguratorProps> = ({
|
|||
<div className="vm-query-configurator-settings">
|
||||
<AdditionalSettings/>
|
||||
<div className="vm-query-configurator-settings__buttons">
|
||||
<QueryHistoryList
|
||||
history={queryHistory}
|
||||
handleSelectQuery={handleSelectHistory}
|
||||
/>
|
||||
<QueryHistory handleSelectQuery={handleSelectHistory}/>
|
||||
{stateQuery.length < MAX_QUERY_FIELDS && (
|
||||
<Button
|
||||
variant="outlined"
|
||||
|
|
|
|||
|
|
@ -0,0 +1,188 @@
|
|||
import React, { FC, useEffect, useMemo, useState } from "preact/compat";
|
||||
import Button from "../../../components/Main/Button/Button";
|
||||
import { ClockIcon, DeleteIcon } from "../../../components/Main/Icons";
|
||||
import Tooltip from "../../../components/Main/Tooltip/Tooltip";
|
||||
import useBoolean from "../../../hooks/useBoolean";
|
||||
import Modal from "../../../components/Main/Modal/Modal";
|
||||
import Tabs from "../../../components/Main/Tabs/Tabs";
|
||||
import useDeviceDetect from "../../../hooks/useDeviceDetect";
|
||||
import useEventListener from "../../../hooks/useEventListener";
|
||||
import { useQueryState } from "../../../state/query/QueryStateContext";
|
||||
import { getQueriesFromStorage } from "./utils";
|
||||
import QueryHistoryItem from "./QueryHistoryItem";
|
||||
import classNames from "classnames";
|
||||
import "./style.scss";
|
||||
import { saveToStorage } from "../../../utils/storage";
|
||||
import { arrayEquals } from "../../../utils/array";
|
||||
|
||||
interface Props {
|
||||
handleSelectQuery: (query: string, index: number) => void
|
||||
}
|
||||
|
||||
export const HistoryTabTypes = {
|
||||
session: "session",
|
||||
storage: "saved",
|
||||
favorite: "favorite",
|
||||
};
|
||||
|
||||
export const historyTabs = [
|
||||
{ label: "Session history", value: HistoryTabTypes.session },
|
||||
{ label: "Saved history", value: HistoryTabTypes.storage },
|
||||
{ label: "Favorite queries", value: HistoryTabTypes.favorite },
|
||||
];
|
||||
|
||||
const QueryHistory: FC<Props> = ({ handleSelectQuery }) => {
|
||||
const { queryHistory: historyState } = useQueryState();
|
||||
const { isMobile } = useDeviceDetect();
|
||||
|
||||
const {
|
||||
value: openModal,
|
||||
setTrue: handleOpenModal,
|
||||
setFalse: handleCloseModal,
|
||||
} = useBoolean(false);
|
||||
|
||||
const [activeTab, setActiveTab] = useState(historyTabs[0].value);
|
||||
const [historyStorage, setHistoryStorage] = useState(getQueriesFromStorage("QUERY_HISTORY"));
|
||||
const [historyFavorites, setHistoryFavorites] = useState(getQueriesFromStorage("QUERY_FAVORITES"));
|
||||
|
||||
const historySession = useMemo(() => {
|
||||
return historyState.map((h) => h.values.filter(q => q).reverse());
|
||||
}, [historyState]);
|
||||
|
||||
const list = useMemo(() => {
|
||||
switch (activeTab) {
|
||||
case HistoryTabTypes.favorite:
|
||||
return historyFavorites;
|
||||
case HistoryTabTypes.storage:
|
||||
return historyStorage;
|
||||
default:
|
||||
return historySession;
|
||||
}
|
||||
}, [activeTab, historyFavorites, historyStorage, historySession]);
|
||||
|
||||
const isNoData = list?.every(s => !s.length);
|
||||
|
||||
const noDataText = useMemo(() => {
|
||||
switch (activeTab) {
|
||||
case HistoryTabTypes.favorite:
|
||||
return "Favorites queries are empty.\nTo see your favorites, mark a query as a favorite.";
|
||||
default:
|
||||
return "Query history is empty.\nTo see the history, please make a query.";
|
||||
}
|
||||
}, [activeTab]);
|
||||
|
||||
const handleRunQuery = (group: number) => (value: string) => {
|
||||
handleSelectQuery(value, group);
|
||||
handleCloseModal();
|
||||
};
|
||||
|
||||
const handleToggleFavorite = (value: string, isFavorite: boolean) => {
|
||||
setHistoryFavorites((prev) => {
|
||||
const values = prev[0] || [];
|
||||
if (isFavorite) return [values.filter(v => v !== value)];
|
||||
if (!isFavorite && !values.includes(value)) return [[...values, value]];
|
||||
return prev;
|
||||
});
|
||||
};
|
||||
|
||||
const updateStageHistory = () => {
|
||||
setHistoryStorage(getQueriesFromStorage("QUERY_HISTORY"));
|
||||
setHistoryFavorites(getQueriesFromStorage("QUERY_FAVORITES"));
|
||||
};
|
||||
|
||||
const handleClearStorage = () => {
|
||||
saveToStorage("QUERY_HISTORY", "");
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
const nextValue = historyFavorites[0] || [];
|
||||
const prevValue = getQueriesFromStorage("QUERY_FAVORITES")[0] || [];
|
||||
const isEqual = arrayEquals(nextValue, prevValue);
|
||||
if (isEqual) return;
|
||||
saveToStorage("QUERY_FAVORITES", JSON.stringify(historyFavorites));
|
||||
}, [historyFavorites]);
|
||||
|
||||
useEventListener("storage", updateStageHistory);
|
||||
|
||||
return (
|
||||
<>
|
||||
<Tooltip title={"Show history"}>
|
||||
<Button
|
||||
color="primary"
|
||||
variant="text"
|
||||
onClick={handleOpenModal}
|
||||
startIcon={<ClockIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
|
||||
{openModal && (
|
||||
<Modal
|
||||
title={"Query history"}
|
||||
onClose={handleCloseModal}
|
||||
>
|
||||
<div
|
||||
className={classNames({
|
||||
"vm-query-history": true,
|
||||
"vm-query-history_mobile": isMobile,
|
||||
})}
|
||||
>
|
||||
<div
|
||||
className={classNames({
|
||||
"vm-query-history__tabs": true,
|
||||
"vm-section-header__tabs": true,
|
||||
"vm-query-history__tabs_mobile": isMobile,
|
||||
})}
|
||||
>
|
||||
<Tabs
|
||||
activeItem={activeTab}
|
||||
items={historyTabs}
|
||||
onChange={setActiveTab}
|
||||
/>
|
||||
</div>
|
||||
<div className="vm-query-history-list">
|
||||
{isNoData && <div className="vm-query-history-list__no-data">{noDataText}</div>}
|
||||
{list.map((queries, group) => (
|
||||
<div key={group}>
|
||||
{list.length > 1 && (
|
||||
<div
|
||||
className={classNames({
|
||||
"vm-query-history-list__group-title": true,
|
||||
"vm-query-history-list__group-title_first": group === 0,
|
||||
})}
|
||||
>
|
||||
Query {group + 1}
|
||||
</div>
|
||||
)}
|
||||
{queries.map((query, index) => (
|
||||
<QueryHistoryItem
|
||||
key={index}
|
||||
query={query}
|
||||
favorites={historyFavorites.flat()}
|
||||
onRun={handleRunQuery(group)}
|
||||
onToggleFavorite={handleToggleFavorite}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
))}
|
||||
{(activeTab === HistoryTabTypes.storage) && !isNoData && (
|
||||
<div className="vm-query-history-footer">
|
||||
<Button
|
||||
color="error"
|
||||
variant="outlined"
|
||||
size="small"
|
||||
startIcon={<DeleteIcon/>}
|
||||
onClick={handleClearStorage}
|
||||
>
|
||||
clear history
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</Modal>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
export default QueryHistory;
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
import React, { FC, useMemo } from "preact/compat";
|
||||
import Button from "../../../components/Main/Button/Button";
|
||||
import { CopyIcon, PlayCircleOutlineIcon, StarBorderIcon, StarIcon } from "../../../components/Main/Icons";
|
||||
import Tooltip from "../../../components/Main/Tooltip/Tooltip";
|
||||
import useCopyToClipboard from "../../../hooks/useCopyToClipboard";
|
||||
import "./style.scss";
|
||||
|
||||
interface Props {
|
||||
query: string;
|
||||
favorites: string[];
|
||||
onRun: (query: string) => void;
|
||||
onToggleFavorite: (query: string, isFavorite: boolean) => void;
|
||||
}
|
||||
|
||||
const QueryHistoryItem: FC<Props> = ({ query, favorites, onRun, onToggleFavorite }) => {
|
||||
const copyToClipboard = useCopyToClipboard();
|
||||
const isFavorite = useMemo(() => favorites.includes(query), [query, favorites]);
|
||||
|
||||
const handleCopyQuery = async () => {
|
||||
await copyToClipboard(query, "Query has been copied");
|
||||
};
|
||||
|
||||
const handleRunQuery = () => {
|
||||
onRun(query);
|
||||
};
|
||||
|
||||
const handleToggleFavorite = () => {
|
||||
onToggleFavorite(query, isFavorite);
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="vm-query-history-item">
|
||||
<span className="vm-query-history-item__value">{query}</span>
|
||||
<div className="vm-query-history-item__buttons">
|
||||
<Tooltip title={"Execute query"}>
|
||||
<Button
|
||||
size="small"
|
||||
variant="text"
|
||||
onClick={handleRunQuery}
|
||||
startIcon={<PlayCircleOutlineIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip title={"Copy query"}>
|
||||
<Button
|
||||
size="small"
|
||||
variant="text"
|
||||
onClick={handleCopyQuery}
|
||||
startIcon={<CopyIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip title={isFavorite ? "Remove Favorite" : "Add to Favorites"}>
|
||||
<Button
|
||||
size="small"
|
||||
variant="text"
|
||||
color={isFavorite ? "warning" : "primary"}
|
||||
onClick={handleToggleFavorite}
|
||||
startIcon={isFavorite ? <StarIcon/> : <StarBorderIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
export default QueryHistoryItem;
|
||||
|
|
@ -1,114 +0,0 @@
|
|||
import React, { FC, useMemo } from "preact/compat";
|
||||
import Button from "../../../components/Main/Button/Button";
|
||||
import { ClockIcon, CopyIcon, PlayCircleOutlineIcon } from "../../../components/Main/Icons";
|
||||
import Tooltip from "../../../components/Main/Tooltip/Tooltip";
|
||||
import { QueryHistory } from "../../../state/query/reducer";
|
||||
import useBoolean from "../../../hooks/useBoolean";
|
||||
import Modal from "../../../components/Main/Modal/Modal";
|
||||
import "./style.scss";
|
||||
import Tabs from "../../../components/Main/Tabs/Tabs";
|
||||
import { useState } from "react";
|
||||
import useCopyToClipboard from "../../../hooks/useCopyToClipboard";
|
||||
import useDeviceDetect from "../../../hooks/useDeviceDetect";
|
||||
import classNames from "classnames";
|
||||
|
||||
interface QueryHistoryProps {
|
||||
history: QueryHistory[];
|
||||
handleSelectQuery: (query: string, index: number) => void
|
||||
}
|
||||
|
||||
const QueryHistoryList: FC<QueryHistoryProps> = ({ history, handleSelectQuery }) => {
|
||||
const { isMobile } = useDeviceDetect();
|
||||
const copyToClipboard = useCopyToClipboard();
|
||||
const {
|
||||
value: openModal,
|
||||
setTrue: handleOpenModal,
|
||||
setFalse: handleCloseModal,
|
||||
} = useBoolean(false);
|
||||
|
||||
const [activeTab, setActiveTab] = useState("0");
|
||||
const tabs = useMemo(() => history.map((item, i) => ({
|
||||
value: `${i}`,
|
||||
label: `Query ${i+1}`,
|
||||
})), [history]);
|
||||
|
||||
const queries = useMemo(() => {
|
||||
const historyItem = history[+activeTab];
|
||||
return historyItem ? historyItem.values.filter(q => q).reverse() : [];
|
||||
}, [activeTab, history]);
|
||||
|
||||
const handleCopyQuery = (value: string) => async () => {
|
||||
await copyToClipboard(value, "Query has been copied");
|
||||
};
|
||||
|
||||
const handleRunQuery = (value: string, index: number) => () => {
|
||||
handleSelectQuery(value, index);
|
||||
handleCloseModal();
|
||||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<Tooltip title={"Show history"}>
|
||||
<Button
|
||||
color="primary"
|
||||
variant="text"
|
||||
onClick={handleOpenModal}
|
||||
startIcon={<ClockIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
|
||||
{openModal && (
|
||||
<Modal
|
||||
title={"Query history"}
|
||||
onClose={handleCloseModal}
|
||||
>
|
||||
<div className="vm-query-history">
|
||||
<div
|
||||
className={classNames({
|
||||
"vm-query-history__tabs": true,
|
||||
"vm-section-header__tabs": true,
|
||||
"vm-query-history__tabs_mobile": isMobile,
|
||||
})}
|
||||
>
|
||||
<Tabs
|
||||
activeItem={activeTab}
|
||||
items={tabs}
|
||||
onChange={setActiveTab}
|
||||
/>
|
||||
</div>
|
||||
<div className="vm-query-history-list">
|
||||
{queries.map((query, index) => (
|
||||
<div
|
||||
className="vm-query-history-list-item"
|
||||
key={index}
|
||||
>
|
||||
<span className="vm-query-history-list-item__value">{query}</span>
|
||||
<div className="vm-query-history-list-item__buttons">
|
||||
<Tooltip title={"Execute query"}>
|
||||
<Button
|
||||
size="small"
|
||||
variant="text"
|
||||
onClick={handleRunQuery(query, +activeTab)}
|
||||
startIcon={<PlayCircleOutlineIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
<Tooltip title={"Copy query"}>
|
||||
<Button
|
||||
size="small"
|
||||
variant="text"
|
||||
onClick={handleCopyQuery(query)}
|
||||
startIcon={<CopyIcon/>}
|
||||
/>
|
||||
</Tooltip>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</Modal>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
export default QueryHistoryList;
|
||||
|
|
@ -2,11 +2,17 @@
|
|||
|
||||
.vm-query-history {
|
||||
max-width: 80vw;
|
||||
min-width: 40vw;
|
||||
min-width: 500px;
|
||||
|
||||
&_mobile {
|
||||
max-width: 100vw;
|
||||
min-width: 100vw;
|
||||
}
|
||||
|
||||
&__tabs {
|
||||
margin: (-$padding-medium) (-$padding-medium) 0;
|
||||
padding: 0 $padding-medium;
|
||||
padding: 0 $padding-small;
|
||||
border-bottom: $border-divider;
|
||||
|
||||
&_mobile {
|
||||
margin: (-$padding-global) (-$padding-medium) 0;
|
||||
|
|
@ -17,29 +23,51 @@
|
|||
display: grid;
|
||||
align-items: flex-start;
|
||||
|
||||
&-item {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr auto;
|
||||
gap: $padding-small;
|
||||
align-items: center;
|
||||
&__group-title {
|
||||
font-weight: bold;
|
||||
margin: 0 (-$padding-medium) 0;
|
||||
padding: $padding-small calc($padding-medium + $padding-small);
|
||||
border-bottom: $border-divider;
|
||||
padding: $padding-medium $padding-global $padding-small;
|
||||
|
||||
&:first-child {
|
||||
border-top: $border-divider;
|
||||
}
|
||||
|
||||
&__value {
|
||||
white-space: pre-wrap;
|
||||
overflow-wrap: anywhere;
|
||||
font-family: $font-family-monospace;
|
||||
}
|
||||
|
||||
&__buttons {
|
||||
display: flex;
|
||||
gap: $padding-small;
|
||||
&_first {
|
||||
padding-top: $padding-global;
|
||||
}
|
||||
}
|
||||
|
||||
&__no-data {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: $padding-large $padding-global;
|
||||
color: $color-text-secondary;
|
||||
text-align: center;
|
||||
line-height: $font-size-large;
|
||||
white-space: pre-line;
|
||||
}
|
||||
}
|
||||
|
||||
&-item {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr auto;
|
||||
gap: $padding-small;
|
||||
align-items: center;
|
||||
margin: 0 (-$padding-medium) 0;
|
||||
padding: $padding-small $padding-medium;
|
||||
border-bottom: $border-divider;
|
||||
|
||||
&__value {
|
||||
white-space: pre-wrap;
|
||||
overflow-wrap: anywhere;
|
||||
font-family: $font-family-monospace;
|
||||
}
|
||||
|
||||
&__buttons {
|
||||
display: flex;
|
||||
}
|
||||
}
|
||||
|
||||
&-footer {
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
padding-top: $padding-medium;
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,27 @@
|
|||
import { getFromStorage, saveToStorage, StorageKeys } from "../../../utils/storage";
|
||||
import { QueryHistoryType } from "../../../state/query/reducer";
|
||||
import { MAX_QUERIES_HISTORY, MAX_QUERY_FIELDS } from "../../../constants/graph";
|
||||
|
||||
export const getQueriesFromStorage = (key: StorageKeys) => {
|
||||
const list = getFromStorage(key) as string;
|
||||
return list ? JSON.parse(list) as string[][] : [];
|
||||
};
|
||||
|
||||
export const setQueriesToStorage = (history: QueryHistoryType[]) => {
|
||||
// For localStorage, avoid splitting into query fields because when working from multiple tabs can cause confusion.
|
||||
// For convenience, we maintain the original structure of `string[][]`
|
||||
const lastValues = history.map(h => h.values[h.index]);
|
||||
const storageValues = getQueriesFromStorage("QUERY_HISTORY");
|
||||
if (!storageValues[0]) storageValues[0] = [];
|
||||
|
||||
const values = storageValues[0];
|
||||
const TOTAL_LIMIT = MAX_QUERIES_HISTORY * MAX_QUERY_FIELDS;
|
||||
|
||||
lastValues.forEach((v) => {
|
||||
const already = values.includes(v);
|
||||
if (!already && v) values.unshift(v);
|
||||
if (values.length > TOTAL_LIMIT) values.shift();
|
||||
});
|
||||
|
||||
saveToStorage("QUERY_HISTORY", JSON.stringify(storageValues));
|
||||
};
|
||||
|
|
@ -27,7 +27,7 @@ export const initialCustomPanelState: CustomPanelState = {
|
|||
displayType: (displayType?.value || "chart") as DisplayType,
|
||||
nocache: false,
|
||||
isTracingEnabled: false,
|
||||
seriesLimits: limitsStorage ? JSON.parse(getFromStorage("SERIES_LIMITS") as string) : DEFAULT_MAX_SERIES,
|
||||
seriesLimits: limitsStorage ? JSON.parse(limitsStorage) : DEFAULT_MAX_SERIES,
|
||||
tableCompact: getFromStorage("TABLE_COMPACT") as boolean || false,
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -1,22 +1,23 @@
|
|||
import { getFromStorage, saveToStorage } from "../../utils/storage";
|
||||
import { getQueryArray } from "../../utils/query-string";
|
||||
import { setQueriesToStorage } from "../../pages/CustomPanel/QueryHistory/utils";
|
||||
|
||||
export interface QueryHistory {
|
||||
export interface QueryHistoryType {
|
||||
index: number;
|
||||
values: string[];
|
||||
}
|
||||
|
||||
export interface QueryState {
|
||||
query: string[];
|
||||
queryHistory: QueryHistory[];
|
||||
queryHistory: QueryHistoryType[];
|
||||
autocomplete: boolean;
|
||||
|
||||
}
|
||||
|
||||
export type QueryAction =
|
||||
| { type: "SET_QUERY", payload: string[] }
|
||||
| { type: "SET_QUERY_HISTORY_BY_INDEX", payload: {value: QueryHistory, queryNumber: number} }
|
||||
| { type: "SET_QUERY_HISTORY", payload: QueryHistory[] }
|
||||
| { type: "SET_QUERY_HISTORY_BY_INDEX", payload: {value: QueryHistoryType, queryNumber: number} }
|
||||
| { type: "SET_QUERY_HISTORY", payload: QueryHistoryType[] }
|
||||
| { type: "TOGGLE_AUTOCOMPLETE"}
|
||||
|
||||
const query = getQueryArray();
|
||||
|
|
@ -34,6 +35,7 @@ export function reducer(state: QueryState, action: QueryAction): QueryState {
|
|||
query: action.payload.map(q => q)
|
||||
};
|
||||
case "SET_QUERY_HISTORY":
|
||||
setQueriesToStorage(action.payload);
|
||||
return {
|
||||
...state,
|
||||
queryHistory: action.payload
|
||||
|
|
|
|||
|
|
@ -1,7 +1,4 @@
|
|||
export type StorageKeys = "BASIC_AUTH_DATA"
|
||||
| "BEARER_AUTH_DATA"
|
||||
| "AUTH_TYPE"
|
||||
| "AUTOCOMPLETE"
|
||||
export type StorageKeys = "AUTOCOMPLETE"
|
||||
| "NO_CACHE"
|
||||
| "QUERY_TRACING"
|
||||
| "SERIES_LIMITS"
|
||||
|
|
@ -10,6 +7,8 @@ export type StorageKeys = "BASIC_AUTH_DATA"
|
|||
| "THEME"
|
||||
| "LOGS_LIMIT"
|
||||
| "EXPLORE_METRICS_TIPS"
|
||||
| "QUERY_HISTORY"
|
||||
| "QUERY_FAVORITES"
|
||||
|
||||
export const saveToStorage = (key: StorageKeys, value: string | boolean | Record<string, unknown>): void => {
|
||||
if (value) {
|
||||
|
|
@ -22,7 +21,7 @@ export const saveToStorage = (key: StorageKeys, value: string | boolean | Record
|
|||
};
|
||||
|
||||
// TODO: make this aware of data type that is stored
|
||||
export const getFromStorage = (key: StorageKeys): undefined | boolean | string | Record<string, unknown> => {
|
||||
export const getFromStorage = (key: StorageKeys): undefined | boolean | string | Record<string, unknown> => {
|
||||
const valueObj = window.localStorage.getItem(key);
|
||||
if (valueObj === null) {
|
||||
return undefined;
|
||||
|
|
@ -36,6 +35,3 @@ export const getFromStorage = (key: StorageKeys): undefined | boolean | string |
|
|||
};
|
||||
|
||||
export const removeFromStorage = (keys: StorageKeys[]): void => keys.forEach(k => window.localStorage.removeItem(k));
|
||||
|
||||
export const authKeys: StorageKeys[] = ["BASIC_AUTH_DATA", "BEARER_AUTH_DATA"];
|
||||
|
||||
|
|
|
|||
|
|
@ -30,6 +30,10 @@ const shortDurations = supportedDurations.map(d => d.short);
|
|||
|
||||
export const roundToMilliseconds = (num: number): number => Math.round(num*1000)/1000;
|
||||
|
||||
export const humanizeSeconds = (num: number): string => {
|
||||
return getDurationFromMilliseconds(dayjs.duration(num, "seconds").asMilliseconds());
|
||||
};
|
||||
|
||||
export const roundStep = (step: number) => {
|
||||
let result = roundToMilliseconds(step);
|
||||
const integerStep = Math.round(step);
|
||||
|
|
@ -46,8 +50,7 @@ export const roundStep = (step: number) => {
|
|||
if (step < 1 && step > 0.01) {
|
||||
result = Math.round(step * 40) / 40; // float to thousandths multiple of 5
|
||||
}
|
||||
|
||||
const humanize = getDurationFromMilliseconds(dayjs.duration(result || 0.001, "seconds").asMilliseconds());
|
||||
const humanize = humanizeSeconds(result || 0.001);
|
||||
return humanize.replace(/\s/g, "");
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -2,8 +2,8 @@
|
|||
|
||||
DOCKER_NAMESPACE ?= victoriametrics
|
||||
|
||||
ROOT_IMAGE ?= alpine:3.18.3
|
||||
CERTS_IMAGE := alpine:3.18.3
|
||||
ROOT_IMAGE ?= alpine:3.18.4
|
||||
CERTS_IMAGE := alpine:3.18.4
|
||||
|
||||
GO_BUILDER_IMAGE := golang:1.21.1-alpine
|
||||
BUILDER_IMAGE := local/builder:2.0.0-$(shell echo $(GO_BUILDER_IMAGE) | tr :/ __)-1
|
||||
|
|
|
|||
|
|
@ -42,30 +42,36 @@ The communication scheme between components is the following:
|
|||
and recording rules back to it;
|
||||
* [alertmanager](#alertmanager) is configured to receive notifications from `vmalert`.
|
||||
|
||||
To access `vmalert` use link [http://localhost:8428/vmalert](http://localhost:8428/vmalert/).
|
||||
To access Grafana use link [http://localhost:3000](http://localhost:3000).
|
||||
|
||||
To access [vmui](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui)
|
||||
use link [http://localhost:8428/vmui](http://localhost:8428/vmui).
|
||||
|
||||
To access `vmalert` use link [http://localhost:8428/vmalert](http://localhost:8428/vmalert/).
|
||||
|
||||
|
||||
## VictoriaMetrics cluster
|
||||
|
||||
VictoriaMetrics cluster environment consists of `vminsert`, `vmstorage` and `vmselect` components.
|
||||
`vmselect` has exposed port `:8481`, `vminsert` has exposed port `:8480` and the rest of components
|
||||
are available only inside the environment.
|
||||
`vminsert` has exposed port `:8480`, access to `vmselect` components goes through `vmauth` on port `:8427`,
|
||||
and the rest of components are available only inside the environment.
|
||||
|
||||
The communication scheme between components is the following:
|
||||
* [vmagent](#vmagent) sends scraped metrics to `vminsert`;
|
||||
* `vminsert` forwards data to `vmstorage`;
|
||||
* `vmselect` is connected to `vmstorage` for querying data;
|
||||
* [grafana](#grafana) is configured with datasource pointing to `vmselect`;
|
||||
* [vmalert](#vmalert) is configured to query `vmselect` and send alerts state
|
||||
* `vmselect`s are connected to `vmstorage` for querying data;
|
||||
* [vmauth](#vmauth) balances incoming read requests among `vmselect`s;
|
||||
* [grafana](#grafana) is configured with datasource pointing to `vmauth`;
|
||||
* [vmalert](#vmalert) is configured to query `vmselect`s via `vmauth` and send alerts state
|
||||
and recording rules to `vminsert`;
|
||||
* [alertmanager](#alertmanager) is configured to receive notifications from `vmalert`.
|
||||
|
||||
To access `vmalert` use link [http://localhost:8481/select/0/prometheus/vmalert](http://localhost:8481/select/0/prometheus/vmalert/).
|
||||
To access Grafana use link [http://localhost:3000](http://localhost:3000).
|
||||
|
||||
To access [vmui](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui)
|
||||
use link [http://localhost:8481/select/0/prometheus/vmui](http://localhost:8481/select/0/prometheus/vmui).
|
||||
To access [vmui](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui)
|
||||
use link [http://localhost:8427/select/0/prometheus/vmui/](http://localhost:8427/select/0/prometheus/vmui/).
|
||||
|
||||
To access `vmalert` use link [http://localhost:8427/select/0/prometheus/vmalert/](http://localhost:8427/select/0/prometheus/vmalert/).
|
||||
|
||||
## vmagent
|
||||
|
||||
|
|
@ -75,6 +81,13 @@ with listed targets for scraping.
|
|||
|
||||
[Web interface link](http://localhost:8429/).
|
||||
|
||||
## vmauth
|
||||
|
||||
[vmauth](https://docs.victoriametrics.com/vmauth.html) acts as a [balancer](https://docs.victoriametrics.com/vmauth.html#load-balancing)
|
||||
to spread the load across `vmselect`'s. [Grafana](#grafana) and [vmalert](#vmalert) use vmauth for read queries.
|
||||
vmauth config is available [here](ttps://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/auth-cluster.yml)
|
||||
|
||||
|
||||
## vmalert
|
||||
|
||||
vmalert evaluates alerting rules [alerts.yml](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/alerts.yml)
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ groups:
|
|||
# note the `job` filter and update accordingly to your setup
|
||||
rules:
|
||||
- alert: TooManyRestarts
|
||||
expr: changes(process_start_time_seconds{job=~"victoriametrics.*|vmselect.*|vminsert.*|vmstorage.*|vmagent.*|vmalert.*|vmsingle.*|vmalertmanager.*"}[15m]) > 2
|
||||
expr: changes(process_start_time_seconds{job=~".*(victoriametrics|vmselect|vminsert|vmstorage|vmagent|vmalert|vmsingle|vmalertmanager|vmauth).*"}[15m]) > 2
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
|
|
@ -16,7 +16,7 @@ groups:
|
|||
It might be crashlooping."
|
||||
|
||||
- alert: ServiceDown
|
||||
expr: up{job=~"victoriametrics.*|vmselect.*|vminsert.*|vmstorage.*|vmagent.*|vmalert.*|vmsingle.*|vmalertmanager.*"} == 0
|
||||
expr: up{job=~".*(victoriametrics|vmselect|vminsert|vmstorage|vmagent|vmalert|vmsingle|vmalertmanager|vmauth).*"} == 0
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
|
|
|
|||
6
deployment/docker/auth-cluster.yml
Normal file
6
deployment/docker/auth-cluster.yml
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
# balance load among vmselects
|
||||
# see https://docs.victoriametrics.com/vmauth.html#load-balancing
|
||||
unauthorized_user:
|
||||
url_prefix:
|
||||
- http://vmselect-1:8481
|
||||
- http://vmselect-2:8481
|
||||
|
|
@ -2,7 +2,7 @@ version: '3.5'
|
|||
services:
|
||||
vmagent:
|
||||
container_name: vmagent
|
||||
image: victoriametrics/vmagent:v1.93.4
|
||||
image: victoriametrics/vmagent:v1.94.0
|
||||
depends_on:
|
||||
- "vminsert"
|
||||
ports:
|
||||
|
|
@ -19,7 +19,7 @@ services:
|
|||
container_name: grafana
|
||||
image: grafana/grafana:9.2.7
|
||||
depends_on:
|
||||
- "vmselect"
|
||||
- "vmauth"
|
||||
ports:
|
||||
- 3000:3000
|
||||
restart: always
|
||||
|
|
@ -32,7 +32,7 @@ services:
|
|||
|
||||
vmstorage-1:
|
||||
container_name: vmstorage-1
|
||||
image: victoriametrics/vmstorage:v1.93.4-cluster
|
||||
image: victoriametrics/vmstorage:v1.94.0-cluster
|
||||
ports:
|
||||
- 8482
|
||||
- 8400
|
||||
|
|
@ -44,7 +44,7 @@ services:
|
|||
restart: always
|
||||
vmstorage-2:
|
||||
container_name: vmstorage-2
|
||||
image: victoriametrics/vmstorage:v1.93.4-cluster
|
||||
image: victoriametrics/vmstorage:v1.94.0-cluster
|
||||
ports:
|
||||
- 8482
|
||||
- 8400
|
||||
|
|
@ -54,9 +54,10 @@ services:
|
|||
command:
|
||||
- '--storageDataPath=/storage'
|
||||
restart: always
|
||||
|
||||
vminsert:
|
||||
container_name: vminsert
|
||||
image: victoriametrics/vminsert:v1.93.4-cluster
|
||||
image: victoriametrics/vminsert:v1.94.0-cluster
|
||||
depends_on:
|
||||
- "vmstorage-1"
|
||||
- "vmstorage-2"
|
||||
|
|
@ -66,9 +67,10 @@ services:
|
|||
ports:
|
||||
- 8480:8480
|
||||
restart: always
|
||||
vmselect:
|
||||
container_name: vmselect
|
||||
image: victoriametrics/vmselect:v1.93.4-cluster
|
||||
|
||||
vmselect-1:
|
||||
container_name: vmselect-1
|
||||
image: victoriametrics/vmselect:v1.94.0-cluster
|
||||
depends_on:
|
||||
- "vmstorage-1"
|
||||
- "vmstorage-2"
|
||||
|
|
@ -77,14 +79,43 @@ services:
|
|||
- '--storageNode=vmstorage-2:8401'
|
||||
- '--vmalert.proxyURL=http://vmalert:8880'
|
||||
ports:
|
||||
- 8481:8481
|
||||
- 8481
|
||||
restart: always
|
||||
|
||||
vmselect-2:
|
||||
container_name: vmselect-2
|
||||
image: victoriametrics/vmselect:v1.94.0-cluster
|
||||
depends_on:
|
||||
- "vmstorage-1"
|
||||
- "vmstorage-2"
|
||||
command:
|
||||
- '--storageNode=vmstorage-1:8401'
|
||||
- '--storageNode=vmstorage-2:8401'
|
||||
- '--vmalert.proxyURL=http://vmalert:8880'
|
||||
ports:
|
||||
- 8481
|
||||
restart: always
|
||||
|
||||
vmauth:
|
||||
container_name: vmauth
|
||||
image: victoriametrics/vmauth:v1.94.0
|
||||
depends_on:
|
||||
- "vmselect-1"
|
||||
- "vmselect-2"
|
||||
volumes:
|
||||
- ./auth-cluster.yml:/etc/auth.yml
|
||||
# - /var/run/docker.sock:/var/run/docker.sock
|
||||
command:
|
||||
- '--auth.config=/etc/auth.yml'
|
||||
ports:
|
||||
- 8427:8427
|
||||
restart: always
|
||||
|
||||
vmalert:
|
||||
container_name: vmalert
|
||||
image: victoriametrics/vmalert:v1.93.4
|
||||
image: victoriametrics/vmalert:v1.94.0
|
||||
depends_on:
|
||||
- "vmselect"
|
||||
- "vmauth"
|
||||
ports:
|
||||
- 8880:8880
|
||||
volumes:
|
||||
|
|
@ -93,8 +124,8 @@ services:
|
|||
- ./alerts-vmagent.yml:/etc/alerts/alerts-vmagent.yml
|
||||
- ./alerts-vmalert.yml:/etc/alerts/alerts-vmalert.yml
|
||||
command:
|
||||
- '--datasource.url=http://vmselect:8481/select/0/prometheus'
|
||||
- '--remoteRead.url=http://vmselect:8481/select/0/prometheus'
|
||||
- '--datasource.url=http://vmauth:8427/select/0/prometheus'
|
||||
- '--remoteRead.url=http://vmauth:8427/select/0/prometheus'
|
||||
- '--remoteWrite.url=http://vminsert:8480/insert/0/prometheus'
|
||||
- '--notifier.url=http://alertmanager:9093/'
|
||||
- '--rule=/etc/alerts/*.yml'
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ version: "3.5"
|
|||
services:
|
||||
vmagent:
|
||||
container_name: vmagent
|
||||
image: victoriametrics/vmagent:v1.93.4
|
||||
image: victoriametrics/vmagent:v1.94.0
|
||||
depends_on:
|
||||
- "victoriametrics"
|
||||
ports:
|
||||
|
|
@ -18,7 +18,7 @@ services:
|
|||
restart: always
|
||||
victoriametrics:
|
||||
container_name: victoriametrics
|
||||
image: victoriametrics/victoria-metrics:v1.93.4
|
||||
image: victoriametrics/victoria-metrics:v1.94.0
|
||||
ports:
|
||||
- 8428:8428
|
||||
- 8089:8089
|
||||
|
|
@ -57,7 +57,7 @@ services:
|
|||
restart: always
|
||||
vmalert:
|
||||
container_name: vmalert
|
||||
image: victoriametrics/vmalert:v1.93.4
|
||||
image: victoriametrics/vmalert:v1.94.0
|
||||
depends_on:
|
||||
- "victoriametrics"
|
||||
- "alertmanager"
|
||||
|
|
@ -106,7 +106,7 @@ services:
|
|||
- vm_net
|
||||
victorialogs:
|
||||
container_name: victorialogs
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.3.0-victorialogs
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.4.1-victorialogs
|
||||
command:
|
||||
- "--storageDataPath=/vlogs"
|
||||
- "--httpListenAddr=:9428"
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ scrape_configs:
|
|||
- targets: ['vminsert:8480']
|
||||
- job_name: 'vmselect'
|
||||
static_configs:
|
||||
- targets: ['vmselect:8481']
|
||||
- targets: ['vmselect-1:8481', 'vmselect-2:8481']
|
||||
- job_name: 'vmstorage'
|
||||
static_configs:
|
||||
- targets: ['vmstorage-1:8482', 'vmstorage-2:8482']
|
||||
|
|
@ -10,5 +10,5 @@ datasources:
|
|||
- name: VictoriaMetrics - cluster
|
||||
type: prometheus
|
||||
access: proxy
|
||||
url: http://vmselect:8481/select/0/prometheus
|
||||
url: http://vmauth:8427/select/0/prometheus
|
||||
isDefault: false
|
||||
|
|
@ -22,7 +22,7 @@ services:
|
|||
- -beat.uri=http://filebeat-victorialogs:5066
|
||||
|
||||
victorialogs:
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.3.0-victorialogs
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.4.1-victorialogs
|
||||
volumes:
|
||||
- victorialogs-filebeat-docker-vl:/vlogs
|
||||
ports:
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ services:
|
|||
- "5140:5140"
|
||||
|
||||
victorialogs:
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.3.0-victorialogs
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.4.1-victorialogs
|
||||
volumes:
|
||||
- victorialogs-filebeat-syslog-vl:/vlogs
|
||||
ports:
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ services:
|
|||
- "5140:5140"
|
||||
|
||||
victorialogs:
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.3.0-victorialogs
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.4.1-victorialogs
|
||||
volumes:
|
||||
- victorialogs-fluentbit-vl:/vlogs
|
||||
ports:
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ services:
|
|||
- "5140:5140"
|
||||
|
||||
victorialogs:
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.3.0-victorialogs
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.4.1-victorialogs
|
||||
volumes:
|
||||
- victorialogs-logstash-vl:/vlogs
|
||||
ports:
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ services:
|
|||
- "5140:5140"
|
||||
|
||||
vlogs:
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.3.0-victorialogs
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.4.1-victorialogs
|
||||
volumes:
|
||||
- victorialogs-promtail-docker:/vlogs
|
||||
ports:
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ services:
|
|||
condition: service_healthy
|
||||
|
||||
victorialogs:
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.3.0-victorialogs
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.4.1-victorialogs
|
||||
volumes:
|
||||
- victorialogs-vector-docker-vl:/vlogs
|
||||
ports:
|
||||
|
|
|
|||
|
|
@ -44,7 +44,7 @@ services:
|
|||
|
||||
# Run `make package-victoria-logs` to build victoria-logs image
|
||||
vlogs:
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.3.0-victorialogs
|
||||
image: docker.io/victoriametrics/victoria-logs:v0.4.1-victorialogs
|
||||
volumes:
|
||||
- vlogs:/vlogs
|
||||
ports:
|
||||
|
|
@ -105,7 +105,7 @@ services:
|
|||
- '--config=/config.yml'
|
||||
|
||||
vmsingle:
|
||||
image: victoriametrics/victoria-metrics:v1.93.4
|
||||
image: victoriametrics/victoria-metrics:v1.94.0
|
||||
ports:
|
||||
- '8428:8428'
|
||||
command:
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
4. Set variables `DIGITALOCEAN_API_TOKEN` with `VM_VERSION` for `packer` environment and run make from example below:
|
||||
|
||||
```console
|
||||
make release-victoria-metrics-digitalocean-oneclick-droplet DIGITALOCEAN_API_TOKEN="dop_v23_2e46f4759ceeeba0d0248" VM_VERSION="1.93.4"
|
||||
make release-victoria-metrics-digitalocean-oneclick-droplet DIGITALOCEAN_API_TOKEN="dop_v23_2e46f4759ceeeba0d0248" VM_VERSION="1.94.0"
|
||||
```
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -19,8 +19,8 @@ On the server:
|
|||
* VictoriaMetrics is running on ports: 8428, 8089, 4242, 2003 and they are bound to the local interface.
|
||||
|
||||
********************************************************************************
|
||||
# This image includes 1.93.4 version of VictoriaMetrics.
|
||||
# See Release notes https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.4
|
||||
# This image includes 1.94.0 version of VictoriaMetrics.
|
||||
# See Release notes https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.94.0
|
||||
|
||||
# Welcome to VictoriaMetrics droplet!
|
||||
|
||||
|
|
|
|||
|
|
@ -137,3 +137,5 @@ See also [case studies](https://docs.victoriametrics.com/CaseStudies.html).
|
|||
* [VictoriaMetrics Meetup December 2022](https://www.youtube.com/watch?v=Mesc6JBFNhQ). See also [slides for "VictoriaMetrics 2022: new features" talk](https://docs.google.com/presentation/d/1jI7XZoodmuzLymdu4MToG9onAKQjzCNwMO2NDupyUkQ/edit?usp=sharing).
|
||||
* [Comparing Thanos to VictoriaMetrics cluster](https://faun.pub/comparing-thanos-to-victoriametrics-cluster-b193bea1683)
|
||||
* [Evaluation performance and correctness: VictoriaMetrics response](https://valyala.medium.com/evaluating-performance-and-correctness-victoriametrics-response-e27315627e87)
|
||||
* [How to reduce expenses on monitoring slides](https://www.slideshare.net/RomanKhavronenko/how-to-reduce-expenses-on-monitoringpdf)
|
||||
* [Writing a TSDB from Scratch: Performance Optimization](https://www.youtube.com/watch?v=NdjuW98ep_w)
|
||||
|
|
|
|||
|
|
@ -21,9 +21,20 @@ The following `tip` changes can be tested by building VictoriaMetrics components
|
|||
* [How to build vmauth](https://docs.victoriametrics.com/vmauth.html#how-to-build-from-sources)
|
||||
* [How to build vmctl](https://docs.victoriametrics.com/vmctl.html#how-to-build)
|
||||
|
||||
Metrics of the latest version of VictoriaMetrics cluster are available for viewing at our
|
||||
[sandbox](https://play-grafana.victoriametrics.com/d/oS7Bi_0Wz_vm/victoriametrics-cluster-vm).
|
||||
The sandbox cluster installation is running under the constant load generated by
|
||||
[prometheus-benchmark](https://github.com/VictoriaMetrics/prometheus-benchmark) and used for testing latest releases.
|
||||
|
||||
## tip
|
||||
|
||||
* FEATURE: [vmbackup](https://docs.victoriametrics.com/vmbackup.html): add `-filestream.disableFadvise` command-line flag, which can be used for disabling `fadvise` syscall during backup upload to the remote storage. By default `vmbackup` uses `fadvise` syscall in order to prevent from eviction of recently accessed data from the [OS page cache](https://en.wikipedia.org/wiki/Page_cache) when backing up large files. Sometimes the `fadvise` syscall may take significant amounts of CPU when the backup is performed with large value of `-concurrency` command-line flag on systems with big number of CPU cores. In this case it is better to manually disable `fadvise` syscall by passing `-filestream.disableFadvise` command-line flag to `vmbackup`. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5120) for details.
|
||||
* FEATURE: [Alerting rules for VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#alerts): account for `vmauth` component for alerts `ServiceDown` and `TooManyRestarts`.
|
||||
|
||||
## [v1.94.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.94.0)
|
||||
|
||||
Released at 2023-10-02
|
||||
|
||||
* FEATURE: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): add support for numbers with underscore delimiters such as `1_234_567_890` and `1.234_567_890`. These numbers are easier to read than `1234567890` and `1.234567890`.
|
||||
* FEATURE: [vmbackup](https://docs.victoriametrics.com/vmbackup.html): add support for server-side copy of existing backups. See [these docs](https://docs.victoriametrics.com/vmbackup.html#server-side-copy-of-the-existing-backup) for details.
|
||||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add the option to see the latest 25 queries. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4718).
|
||||
|
|
@ -37,6 +48,8 @@ The following `tip` changes can be tested by building VictoriaMetrics components
|
|||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add button for auto-formatting PromQL/MetricsQL queries. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4681). Thanks to @aramattamara for the [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4694).
|
||||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): improve accessibility score to 100 according to [Google's Lighthouse](https://developer.chrome.com/docs/lighthouse/accessibility/) tests.
|
||||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): organize `min`, `max`, `median` values on the chart legend and tooltips for better visibility.
|
||||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add explanation about [cardinality explorer](https://docs.victoriametrics.com/#cardinality-explorer) statistic inaccuracy in VictoriaMetrics cluster. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3070).
|
||||
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add storage of query history in `localStorage`. See [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5022).
|
||||
* FEATURE: dashboards: provide copies of Grafana dashboards alternated with VictoriaMetrics datasource at [dashboards/vm](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/dashboards/vm).
|
||||
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth.html): added ability to set, override and clear request and response headers on a per-user and per-path basis. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4825) and [these docs](https://docs.victoriametrics.com/vmauth.html#auth-config) for details.
|
||||
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth.html): add ability to retry requests to the [remaining backends](https://docs.victoriametrics.com/vmauth.html#load-balancing) if they return response status codes specified in the `retry_status_codes` list. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893).
|
||||
|
|
@ -45,12 +58,20 @@ The following `tip` changes can be tested by building VictoriaMetrics components
|
|||
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): add `eval_offset` attribute for [Groups](https://docs.victoriametrics.com/vmalert.html#groups). If specified, Group will be evaluated at the exact time offset on the range of [0...evaluationInterval]. The setting might be useful for cron-like rules which must be evaluated at specific moments of time. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3409) for details.
|
||||
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): validate [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html) function names in alerting and recording rules when `vmalert` runs with `-dryRun` command-line flag. Previously it was allowed to use unknown (aka invalid) MetricsQL function names there. For example, `foo()` was counted as a valid query. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4933).
|
||||
* FEATURE: limit the length of string params in log messages to 500 chars. Longer string params are replaced with the `first_250_chars..last_250_chars`. This prevents from too long log lines, which can be emitted by VictoriaMetrics components.
|
||||
* FEATURE: [docker compose environment](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker): add `vmauth` component to cluster's docker-compose example for balancing load among multiple `vmselect` components.
|
||||
* FEATURE: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): make sure that `q2` series are returned after `q1` series in the results of `q1 or q2` query, in the same way as Prometheus does. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763).
|
||||
* FEATURE: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): return empty result from [`bitmap_and(a, b)`](https://docs.victoriametrics.com/MetricsQL.html#bitmap_and), [`bitmap_or(a, b)`](https://docs.victoriametrics.com/MetricsQL.html#bitmap_or) and [`bitmap_xor(a, b)`](https://docs.victoriametrics.com/MetricsQL.html#bitmap_xor) if `a` or `b` have no value at the particular timestamp. Previously `0` was returned in this case. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4996).
|
||||
* FEATURE: stop exposing `vm_merge_need_free_disk_space` metric, since it has been appeared that it confuses users while doesn't bring any useful information. See [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686#issuecomment-1733844128).
|
||||
|
||||
* BUGFIX: [Official Grafana dashboards for VictoriaMetrics](https://grafana.com/orgs/victoriametrics): fix display of ingested rows rate for `Samples ingested/s` and `Samples rate` panels for vmagent's dasbhoard. Previously, not all ingested protocols were accounted in these panels. An extra panel `Rows rate` was added to `Ingestion` section to display the split for rows ingested rate by protocol.
|
||||
* BUGFIX: [vmui](https://docs.victoriametrics.com/#vmui): fix the bug causing render looping when switching to heatmap.
|
||||
* BUGFIX: [VictoriaMetrics enterprise](https://docs.victoriametrics.com/enterprise.html) validate `-dedup.minScrapeInterval` value and `-downsampling.period` intervals are multiples of each other. See [these docs](https://docs.victoriametrics.com/#downsampling).
|
||||
* BUGFIX: [vmbackup](https://docs.victoriametrics.com/vmbackup.html): properly copy `appliedRetention.txt` files inside `<-storageDataPath>/{data}` folders during [incremental backups](https://docs.victoriametrics.com/vmbackup.html#incremental-backups). Previously the new `appliedRetention.txt` could be skipped during incremental backups, which could lead to increased load on storage after restoring from backup. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005).
|
||||
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent.html): suppress `context canceled` error messages in logs when `vmagent` is reloading service discovery config. This error could appear starting from [v1.93.5](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.5). See [this PR](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5048).
|
||||
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent.html): remove concurrency limit during parsing of scraped metrics, which was mistakenly applied to it. With this change cmd-line flag `-maxConcurrentInserts` won't have effect on scraping anymore.
|
||||
* BUGFIX: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): allow passing [median_over_time](https://docs.victoriametrics.com/MetricsQL.html#median_over_time) to [aggr_over_time](https://docs.victoriametrics.com/MetricsQL.html#aggr_over_time). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5034).
|
||||
* BUGFIX: [vminsert](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html): fix ingestion via [multitenant url](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multitenancy-via-labels) for opentsdbhttp. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5061). The bug has been introduced in [v1.93.2](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.2).
|
||||
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent.html): fix support of legacy DataDog agent, which adds trailing slashes to urls. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5078). Thanks to @maxb for spotting the issue.
|
||||
|
||||
## [v1.93.5](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.5)
|
||||
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ where you can chat with VictoriaMetrics users to get additional references, revi
|
|||
- [Brandwatch](#brandwatch)
|
||||
- [CERN](#cern)
|
||||
- [COLOPL](#colopl)
|
||||
- [Criteo](#criteo)
|
||||
- [Dig Security](#dig-security)
|
||||
- [Fly.io](#flyio)
|
||||
- [German Research Center for Artificial Intelligence](#german-research-center-for-artificial-intelligence)
|
||||
|
|
@ -242,6 +243,13 @@ after evaulating the following remote storage solutions for Prometheus:
|
|||
See [slides](https://speakerdeck.com/inletorder/monitoring-platform-with-victoria-metrics) and [video](https://www.youtube.com/watch?v=hUpHIluxw80)
|
||||
from `Large-scale, super-load system monitoring platform built with VictoriaMetrics` talk at [Prometheus Meetup Tokyo #3](https://prometheus.connpass.com/event/157721/).
|
||||
|
||||
## Criteo
|
||||
|
||||
[Criteo](https://www.criteo.com/) is a global technology company that helps marketers and media owners reach their goals through the world’s leading Commerce Media Platform.
|
||||
|
||||
See [this blog post](https://medium.com/criteo-engineering/victoriametrics-a-prometheus-remote-storage-solution-57081a3d8e61) on how Criteo started using VictoriaMetrics
|
||||
and why they prefer VictoriaMetrics over competing solutions.
|
||||
|
||||
## Dig Security
|
||||
|
||||
[Dig Security](https://www.dig.security) is a cloud data security startup with 50+ employees that provides real-time visibility, control, and protection of data assets.
|
||||
|
|
|
|||
|
|
@ -140,7 +140,7 @@ vmstorage-prod
|
|||
|
||||
### Development Builds
|
||||
|
||||
1. [Install go](https://golang.org/doc/install). The minimum supported version is Go 1.18.
|
||||
1. [Install go](https://golang.org/doc/install). The minimum supported version is Go 1.20.
|
||||
1. Run `make` from [the repository root](https://github.com/VictoriaMetrics/VictoriaMetrics). It should build `vmstorage`, `vmselect`
|
||||
and `vminsert` binaries and put them into the `bin` folder.
|
||||
|
||||
|
|
@ -821,15 +821,15 @@ Below is the output for `/path/to/vminsert -help`:
|
|||
-cacheExpireDuration duration
|
||||
Items are removed from in-memory caches after they aren't accessed for this duration. Lower values may reduce memory usage at the cost of higher CPU usage. See also -prevCacheRemovalPercent (default 30m0s)
|
||||
-cluster.tls
|
||||
Whether to use TLS for connections to -storageNode. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Whether to use TLS for connections to -storageNode. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsCAFile string
|
||||
Path to TLS CA file to use for verifying certificates provided by -storageNode if -cluster.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to TLS CA file to use for verifying certificates provided by -storageNode if -cluster.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsCertFile string
|
||||
Path to client-side TLS certificate file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to client-side TLS certificate file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsInsecureSkipVerify
|
||||
Whether to skip verification of TLS certificates provided by -storageNode nodes if -cluster.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in enterprise version of VictoriaMetrics
|
||||
Whether to skip verification of TLS certificates provided by -storageNode nodes if -cluster.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsKeyFile string
|
||||
Path to client-side TLS key file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to client-side TLS key file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternativeListenAddr string
|
||||
TCP address to listen for data from other vminsert nodes in multi-level cluster setup. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multi-level-cluster-setup . Usually :8400 should be set to match default vmstorage port for vminsert. Disabled work if empty
|
||||
-csvTrimTimestamp duration
|
||||
|
|
@ -937,7 +937,7 @@ Below is the output for `/path/to/vminsert -help`:
|
|||
-loggerWarnsPerSecondLimit int
|
||||
Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
|
||||
-maxConcurrentInserts int
|
||||
The maximum number of concurrent insert requests. The default value should work for most cases, since it minimizes memory usage. The default value can be increased when clients send data over slow networks. See also -insert.maxQueueDuration (default 8)
|
||||
The maximum number of concurrent insert requests. Default value should work for most cases, since it minimizes the memory usage. The default value can be increased when clients send data over slow networks. See also -insert.maxQueueDuration (default 8)
|
||||
-maxInsertRequestSize size
|
||||
The maximum size in bytes of a single Prometheus remote_write API request
|
||||
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 33554432)
|
||||
|
|
@ -1025,15 +1025,15 @@ Below is the output for `/path/to/vmselect -help`:
|
|||
-cacheExpireDuration duration
|
||||
Items are removed from in-memory caches after they aren't accessed for this duration. Lower values may reduce memory usage at the cost of higher CPU usage. See also -prevCacheRemovalPercent (default 30m0s)
|
||||
-cluster.tls
|
||||
Whether to use TLS for connections to -storageNode. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Whether to use TLS for connections to -storageNode. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsCAFile string
|
||||
Path to TLS CA file to use for verifying certificates provided by -storageNode if -cluster.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to TLS CA file to use for verifying certificates provided by -storageNode if -cluster.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsCertFile string
|
||||
Path to client-side TLS certificate file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to client-side TLS certificate file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsInsecureSkipVerify
|
||||
Whether to skip verification of TLS certificates provided by -storageNode nodes if -cluster.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in enterprise version of VictoriaMetrics
|
||||
Whether to skip verification of TLS certificates provided by -storageNode nodes if -cluster.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-cluster.tlsKeyFile string
|
||||
Path to client-side TLS key file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in enterprise version of VictoriaMetrics
|
||||
Path to client-side TLS key file to use when connecting to -storageNode if -cluster.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternative.disableCompression
|
||||
Whether to disable compression of the data sent to vmselect via -clusternativeListenAddr. This reduces CPU usage at the cost of higher network bandwidth usage
|
||||
-clusternative.maxConcurrentRequests int
|
||||
|
|
@ -1047,18 +1047,18 @@ Below is the output for `/path/to/vmselect -help`:
|
|||
-clusternative.maxTagValues int
|
||||
The maximum number of tag values returned per search at -clusternativeListenAddr (default 100000)
|
||||
-clusternative.tls
|
||||
Whether to use TLS when accepting connections at -clusternativeListenAddr. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
|
||||
Whether to use TLS when accepting connections at -clusternativeListenAddr. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternative.tlsCAFile string
|
||||
Path to TLS CA file to use for verifying certificates provided by vmselect, which connects at -clusternativeListenAddr if -clusternative.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
|
||||
Path to TLS CA file to use for verifying certificates provided by vmselect, which connects at -clusternativeListenAddr if -clusternative.tls flag is set. By default system CA is used. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternative.tlsCertFile string
|
||||
Path to server-side TLS certificate file to use when accepting connections at -clusternativeListenAddr if -clusternative.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
|
||||
Path to server-side TLS certificate file to use when accepting connections at -clusternativeListenAddr if -clusternative.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternative.tlsCipherSuites array
|
||||
Optional list of TLS cipher suites used for connections at -clusternativeListenAddr if -clusternative.tls flag is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants
|
||||
Optional list of TLS cipher suites used for connections at -clusternativeListenAddr if -clusternative.tls flag is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-clusternative.tlsInsecureSkipVerify
|
||||
Whether to skip verification of TLS certificates provided by vmselect, which connects to -clusternativeListenAddr if -clusternative.tls flag is set. Note that disabled TLS certificate verification breaks security
|
||||
Whether to skip verification of TLS certificates provided by vmselect, which connects to -clusternativeListenAddr if -clusternative.tls flag is set. Note that disabled TLS certificate verification breaks security. This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternative.tlsKeyFile string
|
||||
Path to server-side TLS key file to use when accepting connections at -clusternativeListenAddr if -clusternative.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
|
||||
Path to server-side TLS key file to use when accepting connections at -clusternativeListenAddr if -clusternative.tls flag is set. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-clusternativeListenAddr string
|
||||
TCP address to listen for requests from other vmselect nodes in multi-level cluster setup. See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multi-level-cluster-setup . Usually :8401 should be set to match default vmstorage port for vmselect. Disabled work if empty
|
||||
-dedup.minScrapeInterval duration
|
||||
|
|
@ -1304,6 +1304,8 @@ Below is the output for `/path/to/vmstorage -help`:
|
|||
Prefix for environment variables if -envflag.enable is set
|
||||
-eula
|
||||
Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-filestream.disableFadvise
|
||||
Whether to disable fadvise() syscall when reading large data files. The fadvise() syscall prevents from eviction of recently accessed data from OS page cache during background merges and backups. In some rare cases it is better to disable the syscall if it uses too much CPU
|
||||
-finalMergeDelay duration
|
||||
The delay before starting final merge for per-month partition after no new data is ingested into it. Final merge may require additional disk IO and CPU resources. Final merge may increase query speed and reduce disk space usage in some cases. Zero value disables final merge
|
||||
-flagsAuthKey string
|
||||
|
|
@ -1367,7 +1369,7 @@ Below is the output for `/path/to/vmstorage -help`:
|
|||
-loggerWarnsPerSecondLimit int
|
||||
Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
|
||||
-maxConcurrentInserts int
|
||||
The maximum number of concurrent insert requests. The default value should work for most cases, since it minimizes memory usage. The default value can be increased when clients send data over slow networks. See also -insert.maxQueueDuration (default 8)
|
||||
The maximum number of concurrent insert requests. Default value should work for most cases, since it minimizes the memory usage. The default value can be increased when clients send data over slow networks. See also -insert.maxQueueDuration (default 8)
|
||||
-memory.allowedBytes size
|
||||
Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
|
||||
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
sort: 100
|
||||
sort: -1 # hide page from menu
|
||||
weight: 100
|
||||
---
|
||||
|
||||
|
|
|
|||
104
docs/README.md
104
docs/README.md
|
|
@ -113,6 +113,7 @@ Case studies:
|
|||
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
|
||||
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
|
||||
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
|
||||
* [Criteo](https://docs.victoriametrics.com/CaseStudies.html#criteo)
|
||||
* [Dig Security](https://docs.victoriametrics.com/CaseStudies.html#dig-security)
|
||||
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
|
||||
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
|
||||
|
|
@ -367,6 +368,8 @@ See the [example VMUI at VictoriaMetrics playground](https://play.victoriametric
|
|||
* queries with the biggest average execution duration;
|
||||
* queries that took the most summary time for execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/top_queries` HTTP endpoint.
|
||||
|
||||
## Active queries
|
||||
|
||||
[VMUI](#vmui) provides `active queries` tab, which shows currently execute queries.
|
||||
|
|
@ -376,6 +379,8 @@ It provides the following information per each query:
|
|||
- The duration of the query execution.
|
||||
- The client address, who initiated the query execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/active_queries` HTTP endpoint.
|
||||
|
||||
## Metrics explorer
|
||||
|
||||
[VMUI](#vmui) provides an ability to explore metrics exported by a particular `job` / `instance` in the following way:
|
||||
|
|
@ -407,14 +412,16 @@ matching the specified [series selector](https://prometheus.io/docs/prometheus/l
|
|||
|
||||
Cardinality explorer is built on top of [/api/v1/status/tsdb](#tsdb-stats).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## Cardinality explorer statistic inaccuracy
|
||||
|
||||
In [cluster version of VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html) each vmstorage tracks the stored time series individually.
|
||||
vmselect requests stats via [/api/v1/status/tsdb](#tsdb-stats) API from each vmstorage node and merges the results by summing per-series stats.
|
||||
This may lead to inflated values when samples for the same time series are spread across multiple vmstorage nodes
|
||||
due to [replication](#replication) or [rerouting](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html?highlight=re-routes#cluster-availability).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## How to apply new config to VictoriaMetrics
|
||||
|
||||
VictoriaMetrics is configured via command-line flags, so it must be restarted when new command-line flags should be applied:
|
||||
|
|
@ -619,6 +626,28 @@ Some plugins for Telegraf such as [fluentd](https://github.com/fangli/fluent-plu
|
|||
or [Juniper/jitmon](https://github.com/Juniper/jtimon) send `SHOW DATABASES` query to `/query` and expect a particular database name in the response.
|
||||
Comma-separated list of expected databases can be passed to VictoriaMetrics via `-influx.databaseNames` command-line flag.
|
||||
|
||||
### How to send data in InfluxDB v2 format
|
||||
|
||||
VictoriaMetrics exposes endpoint for InfluxDB v2 HTTP API at `/influx/api/v2/write` and `/api/v2/write`.
|
||||
|
||||
|
||||
In order to write data with InfluxDB line protocol to local VictoriaMetrics using `curl`:
|
||||
|
||||
<div class="with-copy" markdown="1">
|
||||
|
||||
```console
|
||||
curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/api/v2/write'
|
||||
```
|
||||
|
||||
</div>
|
||||
|
||||
The `/api/v1/export` endpoint should return the following response:
|
||||
|
||||
```json
|
||||
{"metric":{"__name__":"measurement_field1","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1695902762311]}
|
||||
{"metric":{"__name__":"measurement_field2","tag1":"value1","tag2":"value2"},"values":[1.23],"timestamps":[1695902762311]}
|
||||
```
|
||||
|
||||
## How to send data from Graphite-compatible agents such as [StatsD](https://github.com/etsy/statsd)
|
||||
|
||||
Enable Graphite receiver in VictoriaMetrics by setting `-graphiteListenAddr` command line flag. For instance,
|
||||
|
|
@ -833,7 +862,7 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
* `/api/v1/series/count` - returns the total number of time series in the database. Some notes:
|
||||
* the handler scans all the inverted index, so it can be slow if the database contains tens of millions of time series;
|
||||
* the handler may count [deleted time series](#how-to-delete-time-series) additionally to normal time series due to internal implementation restrictions;
|
||||
* `/api/v1/status/active_queries` - returns a list of currently running queries.
|
||||
* `/api/v1/status/active_queries` - returns the list of currently running queries. This list is also available at [`active queries` page at VMUI](#active-queries).
|
||||
* `/api/v1/status/top_queries` - returns the following query lists:
|
||||
* the most frequently executed queries - `topByCount`
|
||||
* queries with the biggest average execution duration - `topByAvgDuration`
|
||||
|
|
@ -843,6 +872,8 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
For example, request to `/api/v1/status/top_queries?topN=5&maxLifetime=30s` would return up to 5 queries per list, which were executed during the last 30 seconds.
|
||||
VictoriaMetrics tracks the last `-search.queryStats.lastQueriesCount` queries with durations at least `-search.queryStats.minQueryDuration`.
|
||||
|
||||
See also [`top queries` page at VMUI](#top-queries).
|
||||
|
||||
### Timestamp formats
|
||||
|
||||
VictoriaMetrics accepts the following formats for `time`, `start` and `end` query args
|
||||
|
|
@ -1069,7 +1100,9 @@ VictoriaMetrics provides the following handlers for exporting data:
|
|||
Send a request to `http://<victoriametrics-addr>:8428/api/v1/export?match[]=<timeseries_selector_for_export>`,
|
||||
where `<timeseries_selector_for_export>` may contain any [time series selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors)
|
||||
for metrics to export. Use `{__name__!=""}` selector for fetching all the time series.
|
||||
The response would contain all the data for the selected time series in [JSON streaming format](http://ndjson.org/).
|
||||
|
||||
The response would contain all the data for the selected time series in JSON line format - see [these docs](#json-line-format) for details on this format.
|
||||
|
||||
Each JSON line contains samples for a single time series. An example output:
|
||||
|
||||
```json
|
||||
|
|
@ -1199,6 +1232,8 @@ check for changes in `vm_rows_invalid_total` (exported by server side) metric.
|
|||
|
||||
### How to import data in JSON line format
|
||||
|
||||
VictoriaMetrics accepts metrics data in JSON line format at `/api/v1/import` endpoint. See [these docs](#json-line-format) for details on this format.
|
||||
|
||||
Example for importing data obtained via [/api/v1/export](#how-to-export-data-in-json-line-format):
|
||||
|
||||
```console
|
||||
|
|
@ -1363,13 +1398,54 @@ Note that it could be required to flush response cache after importing historica
|
|||
|
||||
VictoriaMetrics also may scrape Prometheus targets - see [these docs](#how-to-scrape-prometheus-exporters-such-as-node-exporter).
|
||||
|
||||
## Sending data via OpenTelemetry
|
||||
### Sending data via OpenTelemetry
|
||||
|
||||
VictoriaMetrics supports data ingestion via [OpenTelemetry protocol for metrics](https://github.com/open-telemetry/opentelemetry-specification/blob/ffddc289462dfe0c2041e3ca42a7b1df805706de/specification/metrics/data-model.md) at `/opentelemetry/api/v1/push` path.
|
||||
|
||||
VictoriaMetrics expects `protobuf`-encoded requests at `/opentelemetry/api/v1/push`.
|
||||
Set HTTP request header `Content-Encoding: gzip` when sending gzip-compressed data to `/opentelemetry/api/v1/push`.
|
||||
|
||||
## JSON line format
|
||||
|
||||
VictoriaMetrics accepts data in JSON line format at [/api/v1/import](#how-to-import-data-in-json-line-format)
|
||||
and exports data in this format at [/api/v1/export](#how-to-export-data-in-json-line-format).
|
||||
|
||||
The format follows [JSON streaming concept](http://ndjson.org/), e.g. each line contains JSON object with metrics data in the following format:
|
||||
|
||||
```
|
||||
{
|
||||
// metric contans metric name plus labels for a particular time series
|
||||
"metric":{
|
||||
"__name__": "metric_name", // <- this is metric name
|
||||
|
||||
// Other labels for the time series
|
||||
|
||||
"label1": "value1",
|
||||
"label2": "value2",
|
||||
...
|
||||
"labelN": "valueN"
|
||||
},
|
||||
|
||||
// values contains raw sample values for the given time series
|
||||
"values": [1, 2.345, -678],
|
||||
|
||||
// timestamps contains raw sample UNIX timestamps in milliseconds for the given time series
|
||||
// every timestamp is associated with the value at the corresponding position
|
||||
"timestamps": [1549891472010,1549891487724,1549891503438]
|
||||
}
|
||||
```
|
||||
|
||||
Note that every JSON object must be written in a single line, e.g. all the newline chars must be removed from it.
|
||||
Every line length is limited by the value passed to `-import.maxLineLen` command-line flag (by default this is 100MB).
|
||||
|
||||
It is recommended passing 1K-10K samples per line for achieving the maximum data ingestion performance at [/api/v1/import](#how-to-import-data-in-json-line-format).
|
||||
Too long JSON lines may increase RAM usage at VictoriaMetrics side.
|
||||
|
||||
It is OK to split [raw samples](https://docs.victoriametrics.com/keyConcepts.html#raw-samples)
|
||||
for the same [time series](https://docs.victoriametrics.com/keyConcepts.html#time-series) across multiple lines.
|
||||
|
||||
The number of lines in JSON line document can be arbitrary.
|
||||
|
||||
## Relabeling
|
||||
|
||||
VictoriaMetrics supports Prometheus-compatible relabeling for all the ingested metrics if `-relabelConfig` command-line flag points
|
||||
|
|
@ -1657,9 +1733,10 @@ See [these docs](https://docs.victoriametrics.com/guides/guide-vmcluster-multipl
|
|||
which allow configuring multiple retentions for distinct sets of time series matching the configured [series filters](https://docs.victoriametrics.com/keyConcepts.html#filtering)
|
||||
via `-retentionFilter` command-line flag. This flag accepts `filter:duration` options, where `filter` must be
|
||||
a valid [series filter](https://docs.victoriametrics.com/keyConcepts.html#filtering), while the `duration`
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`. If series doesn't match
|
||||
any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention) command-line flag is applied to it.
|
||||
If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`.
|
||||
The `duration` of the `-retentionFilter` must be lower or equal to [-retentionPeriod](#retention) flag value.
|
||||
If series doesn't match any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention)
|
||||
command-line flag is applied to it. If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
|
||||
For example, the following config sets 3 days retention for time series with `team="juniors"` label,
|
||||
30 days retention for time series with `env="dev"` or `env="staging"` label and 1 year retention for the remaining time series:
|
||||
|
|
@ -1793,9 +1870,9 @@ Graphs on the dashboards contain useful hints - hover the `i` icon in the top le
|
|||
We recommend setting up [alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#alerts)
|
||||
via [vmalert](https://docs.victoriametrics.com/vmalert.html) or via Prometheus.
|
||||
|
||||
VictoriaMetrics exposes currently running queries and their execution times at `/api/v1/status/active_queries` page.
|
||||
VictoriaMetrics exposes currently running queries and their execution times at [`active queries` page](#active-queries).
|
||||
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at `/api/v1/status/top_queries` page.
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at [`top queries` page](#top-queries).
|
||||
|
||||
See also [VictoriaMetrics Monitoring](https://victoriametrics.com/blog/victoriametrics-monitoring/)
|
||||
and [troubleshooting docs](https://docs.victoriametrics.com/Troubleshooting.html).
|
||||
|
|
@ -1940,9 +2017,6 @@ and [cardinality explorer docs](#cardinality-explorer).
|
|||
has at least 20% of free space. The remaining amount of free space
|
||||
can be [monitored](#monitoring) via `vm_free_disk_space_bytes` metric. The total size of data
|
||||
stored on the disk can be monitored via sum of `vm_data_size_bytes` metrics.
|
||||
See also `vm_merge_need_free_disk_space` metrics, which are set to values higher than 0
|
||||
if background merge cannot be initiated due to free disk space shortage. The value shows the number of per-month partitions,
|
||||
which would start background merge if they had more free disk space.
|
||||
|
||||
* VictoriaMetrics buffers incoming data in memory for up to a few seconds before flushing it to persistent storage.
|
||||
This may lead to the following "issues":
|
||||
|
|
@ -2302,6 +2376,8 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
|
|||
Prefix for environment variables if -envflag.enable is set
|
||||
-eula
|
||||
Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-filestream.disableFadvise
|
||||
Whether to disable fadvise() syscall when reading large data files. The fadvise() syscall prevents from eviction of recently accessed data from OS page cache during background merges and backups. In some rare cases it is better to disable the syscall if it uses too much CPU
|
||||
-finalMergeDelay duration
|
||||
The delay before starting final merge for per-month partition after no new data is ingested into it. Final merge may require additional disk IO and CPU resources. Final merge may increase query speed and reduce disk space usage in some cases. Zero value disables final merge
|
||||
-flagsAuthKey string
|
||||
|
|
|
|||
|
|
@ -121,6 +121,7 @@ Case studies:
|
|||
* [Brandwatch](https://docs.victoriametrics.com/CaseStudies.html#brandwatch)
|
||||
* [CERN](https://docs.victoriametrics.com/CaseStudies.html#cern)
|
||||
* [COLOPL](https://docs.victoriametrics.com/CaseStudies.html#colopl)
|
||||
* [Criteo](https://docs.victoriametrics.com/CaseStudies.html#criteo)
|
||||
* [Dig Security](https://docs.victoriametrics.com/CaseStudies.html#dig-security)
|
||||
* [Fly.io](https://docs.victoriametrics.com/CaseStudies.html#flyio)
|
||||
* [German Research Center for Artificial Intelligence](https://docs.victoriametrics.com/CaseStudies.html#german-research-center-for-artificial-intelligence)
|
||||
|
|
@ -375,6 +376,8 @@ See the [example VMUI at VictoriaMetrics playground](https://play.victoriametric
|
|||
* queries with the biggest average execution duration;
|
||||
* queries that took the most summary time for execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/top_queries` HTTP endpoint.
|
||||
|
||||
## Active queries
|
||||
|
||||
[VMUI](#vmui) provides `active queries` tab, which shows currently execute queries.
|
||||
|
|
@ -384,6 +387,8 @@ It provides the following information per each query:
|
|||
- The duration of the query execution.
|
||||
- The client address, who initiated the query execution.
|
||||
|
||||
This information is obtained from the `/api/v1/status/active_queries` HTTP endpoint.
|
||||
|
||||
## Metrics explorer
|
||||
|
||||
[VMUI](#vmui) provides an ability to explore metrics exported by a particular `job` / `instance` in the following way:
|
||||
|
|
@ -415,14 +420,16 @@ matching the specified [series selector](https://prometheus.io/docs/prometheus/l
|
|||
|
||||
Cardinality explorer is built on top of [/api/v1/status/tsdb](#tsdb-stats).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## Cardinality explorer statistic inaccuracy
|
||||
|
||||
In [cluster version of VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html) each vmstorage tracks the stored time series individually.
|
||||
vmselect requests stats via [/api/v1/status/tsdb](#tsdb-stats) API from each vmstorage node and merges the results by summing per-series stats.
|
||||
This may lead to inflated values when samples for the same time series are spread across multiple vmstorage nodes
|
||||
due to [replication](#replication) or [rerouting](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html?highlight=re-routes#cluster-availability).
|
||||
|
||||
See [cardinality explorer playground](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality).
|
||||
See the example of using the cardinality explorer [here](https://victoriametrics.com/blog/cardinality-explorer/).
|
||||
|
||||
## How to apply new config to VictoriaMetrics
|
||||
|
||||
VictoriaMetrics is configured via command-line flags, so it must be restarted when new command-line flags should be applied:
|
||||
|
|
@ -627,6 +634,28 @@ Some plugins for Telegraf such as [fluentd](https://github.com/fangli/fluent-plu
|
|||
or [Juniper/jitmon](https://github.com/Juniper/jtimon) send `SHOW DATABASES` query to `/query` and expect a particular database name in the response.
|
||||
Comma-separated list of expected databases can be passed to VictoriaMetrics via `-influx.databaseNames` command-line flag.
|
||||
|
||||
### How to send data in InfluxDB v2 format
|
||||
|
||||
VictoriaMetrics exposes endpoint for InfluxDB v2 HTTP API at `/influx/api/v2/write` and `/api/v2/write`.
|
||||
|
||||
|
||||
In order to write data with InfluxDB line protocol to local VictoriaMetrics using `curl`:
|
||||
|
||||
<div class="with-copy" markdown="1">
|
||||
|
||||
```console
|
||||
curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/api/v2/write'
|
||||
```
|
||||
|
||||
</div>
|
||||
|
||||
The `/api/v1/export` endpoint should return the following response:
|
||||
|
||||
```json
|
||||
{"metric":{"__name__":"measurement_field1","tag1":"value1","tag2":"value2"},"values":[123],"timestamps":[1695902762311]}
|
||||
{"metric":{"__name__":"measurement_field2","tag1":"value1","tag2":"value2"},"values":[1.23],"timestamps":[1695902762311]}
|
||||
```
|
||||
|
||||
## How to send data from Graphite-compatible agents such as [StatsD](https://github.com/etsy/statsd)
|
||||
|
||||
Enable Graphite receiver in VictoriaMetrics by setting `-graphiteListenAddr` command line flag. For instance,
|
||||
|
|
@ -841,7 +870,7 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
* `/api/v1/series/count` - returns the total number of time series in the database. Some notes:
|
||||
* the handler scans all the inverted index, so it can be slow if the database contains tens of millions of time series;
|
||||
* the handler may count [deleted time series](#how-to-delete-time-series) additionally to normal time series due to internal implementation restrictions;
|
||||
* `/api/v1/status/active_queries` - returns a list of currently running queries.
|
||||
* `/api/v1/status/active_queries` - returns the list of currently running queries. This list is also available at [`active queries` page at VMUI](#active-queries).
|
||||
* `/api/v1/status/top_queries` - returns the following query lists:
|
||||
* the most frequently executed queries - `topByCount`
|
||||
* queries with the biggest average execution duration - `topByAvgDuration`
|
||||
|
|
@ -851,6 +880,8 @@ Additionally, VictoriaMetrics provides the following handlers:
|
|||
For example, request to `/api/v1/status/top_queries?topN=5&maxLifetime=30s` would return up to 5 queries per list, which were executed during the last 30 seconds.
|
||||
VictoriaMetrics tracks the last `-search.queryStats.lastQueriesCount` queries with durations at least `-search.queryStats.minQueryDuration`.
|
||||
|
||||
See also [`top queries` page at VMUI](#top-queries).
|
||||
|
||||
### Timestamp formats
|
||||
|
||||
VictoriaMetrics accepts the following formats for `time`, `start` and `end` query args
|
||||
|
|
@ -1077,7 +1108,9 @@ VictoriaMetrics provides the following handlers for exporting data:
|
|||
Send a request to `http://<victoriametrics-addr>:8428/api/v1/export?match[]=<timeseries_selector_for_export>`,
|
||||
where `<timeseries_selector_for_export>` may contain any [time series selector](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors)
|
||||
for metrics to export. Use `{__name__!=""}` selector for fetching all the time series.
|
||||
The response would contain all the data for the selected time series in [JSON streaming format](http://ndjson.org/).
|
||||
|
||||
The response would contain all the data for the selected time series in JSON line format - see [these docs](#json-line-format) for details on this format.
|
||||
|
||||
Each JSON line contains samples for a single time series. An example output:
|
||||
|
||||
```json
|
||||
|
|
@ -1207,6 +1240,8 @@ check for changes in `vm_rows_invalid_total` (exported by server side) metric.
|
|||
|
||||
### How to import data in JSON line format
|
||||
|
||||
VictoriaMetrics accepts metrics data in JSON line format at `/api/v1/import` endpoint. See [these docs](#json-line-format) for details on this format.
|
||||
|
||||
Example for importing data obtained via [/api/v1/export](#how-to-export-data-in-json-line-format):
|
||||
|
||||
```console
|
||||
|
|
@ -1371,13 +1406,54 @@ Note that it could be required to flush response cache after importing historica
|
|||
|
||||
VictoriaMetrics also may scrape Prometheus targets - see [these docs](#how-to-scrape-prometheus-exporters-such-as-node-exporter).
|
||||
|
||||
## Sending data via OpenTelemetry
|
||||
### Sending data via OpenTelemetry
|
||||
|
||||
VictoriaMetrics supports data ingestion via [OpenTelemetry protocol for metrics](https://github.com/open-telemetry/opentelemetry-specification/blob/ffddc289462dfe0c2041e3ca42a7b1df805706de/specification/metrics/data-model.md) at `/opentelemetry/api/v1/push` path.
|
||||
|
||||
VictoriaMetrics expects `protobuf`-encoded requests at `/opentelemetry/api/v1/push`.
|
||||
Set HTTP request header `Content-Encoding: gzip` when sending gzip-compressed data to `/opentelemetry/api/v1/push`.
|
||||
|
||||
## JSON line format
|
||||
|
||||
VictoriaMetrics accepts data in JSON line format at [/api/v1/import](#how-to-import-data-in-json-line-format)
|
||||
and exports data in this format at [/api/v1/export](#how-to-export-data-in-json-line-format).
|
||||
|
||||
The format follows [JSON streaming concept](http://ndjson.org/), e.g. each line contains JSON object with metrics data in the following format:
|
||||
|
||||
```
|
||||
{
|
||||
// metric contans metric name plus labels for a particular time series
|
||||
"metric":{
|
||||
"__name__": "metric_name", // <- this is metric name
|
||||
|
||||
// Other labels for the time series
|
||||
|
||||
"label1": "value1",
|
||||
"label2": "value2",
|
||||
...
|
||||
"labelN": "valueN"
|
||||
},
|
||||
|
||||
// values contains raw sample values for the given time series
|
||||
"values": [1, 2.345, -678],
|
||||
|
||||
// timestamps contains raw sample UNIX timestamps in milliseconds for the given time series
|
||||
// every timestamp is associated with the value at the corresponding position
|
||||
"timestamps": [1549891472010,1549891487724,1549891503438]
|
||||
}
|
||||
```
|
||||
|
||||
Note that every JSON object must be written in a single line, e.g. all the newline chars must be removed from it.
|
||||
Every line length is limited by the value passed to `-import.maxLineLen` command-line flag (by default this is 100MB).
|
||||
|
||||
It is recommended passing 1K-10K samples per line for achieving the maximum data ingestion performance at [/api/v1/import](#how-to-import-data-in-json-line-format).
|
||||
Too long JSON lines may increase RAM usage at VictoriaMetrics side.
|
||||
|
||||
It is OK to split [raw samples](https://docs.victoriametrics.com/keyConcepts.html#raw-samples)
|
||||
for the same [time series](https://docs.victoriametrics.com/keyConcepts.html#time-series) across multiple lines.
|
||||
|
||||
The number of lines in JSON line document can be arbitrary.
|
||||
|
||||
## Relabeling
|
||||
|
||||
VictoriaMetrics supports Prometheus-compatible relabeling for all the ingested metrics if `-relabelConfig` command-line flag points
|
||||
|
|
@ -1665,9 +1741,10 @@ See [these docs](https://docs.victoriametrics.com/guides/guide-vmcluster-multipl
|
|||
which allow configuring multiple retentions for distinct sets of time series matching the configured [series filters](https://docs.victoriametrics.com/keyConcepts.html#filtering)
|
||||
via `-retentionFilter` command-line flag. This flag accepts `filter:duration` options, where `filter` must be
|
||||
a valid [series filter](https://docs.victoriametrics.com/keyConcepts.html#filtering), while the `duration`
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`. If series doesn't match
|
||||
any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention) command-line flag is applied to it.
|
||||
If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
must contain valid [retention](#retention) for time series matching the given `filter`.
|
||||
The `duration` of the `-retentionFilter` must be lower or equal to [-retentionPeriod](#retention) flag value.
|
||||
If series doesn't match any configured `-retentionFilter`, then the retention configured via [-retentionPeriod](#retention)
|
||||
command-line flag is applied to it. If series matches multiple configured retention filters, then the smallest retention is applied.
|
||||
|
||||
For example, the following config sets 3 days retention for time series with `team="juniors"` label,
|
||||
30 days retention for time series with `env="dev"` or `env="staging"` label and 1 year retention for the remaining time series:
|
||||
|
|
@ -1801,9 +1878,9 @@ Graphs on the dashboards contain useful hints - hover the `i` icon in the top le
|
|||
We recommend setting up [alerts](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker#alerts)
|
||||
via [vmalert](https://docs.victoriametrics.com/vmalert.html) or via Prometheus.
|
||||
|
||||
VictoriaMetrics exposes currently running queries and their execution times at `/api/v1/status/active_queries` page.
|
||||
VictoriaMetrics exposes currently running queries and their execution times at [`active queries` page](#active-queries).
|
||||
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at `/api/v1/status/top_queries` page.
|
||||
VictoriaMetrics exposes queries, which take the most time to execute, at [`top queries` page](#top-queries).
|
||||
|
||||
See also [VictoriaMetrics Monitoring](https://victoriametrics.com/blog/victoriametrics-monitoring/)
|
||||
and [troubleshooting docs](https://docs.victoriametrics.com/Troubleshooting.html).
|
||||
|
|
@ -1948,9 +2025,6 @@ and [cardinality explorer docs](#cardinality-explorer).
|
|||
has at least 20% of free space. The remaining amount of free space
|
||||
can be [monitored](#monitoring) via `vm_free_disk_space_bytes` metric. The total size of data
|
||||
stored on the disk can be monitored via sum of `vm_data_size_bytes` metrics.
|
||||
See also `vm_merge_need_free_disk_space` metrics, which are set to values higher than 0
|
||||
if background merge cannot be initiated due to free disk space shortage. The value shows the number of per-month partitions,
|
||||
which would start background merge if they had more free disk space.
|
||||
|
||||
* VictoriaMetrics buffers incoming data in memory for up to a few seconds before flushing it to persistent storage.
|
||||
This may lead to the following "issues":
|
||||
|
|
@ -2310,6 +2384,8 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
|
|||
Prefix for environment variables if -envflag.enable is set
|
||||
-eula
|
||||
Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in VictoriaMetrics enterprise. See https://docs.victoriametrics.com/enterprise.html
|
||||
-filestream.disableFadvise
|
||||
Whether to disable fadvise() syscall when reading large data files. The fadvise() syscall prevents from eviction of recently accessed data from OS page cache during background merges and backups. In some rare cases it is better to disable the syscall if it uses too much CPU
|
||||
-finalMergeDelay duration
|
||||
The delay before starting final merge for per-month partition after no new data is ingested into it. Final merge may require additional disk IO and CPU resources. Final merge may increase query speed and reduce disk space usage in some cases. Zero value disables final merge
|
||||
-flagsAuthKey string
|
||||
|
|
|
|||
|
|
@ -296,29 +296,40 @@ There are the following most commons reasons for slow data ingestion in Victoria
|
|||
Some queries may take more time and resources (CPU, RAM, network bandwidth) than others.
|
||||
VictoriaMetrics logs slow queries if their execution time exceeds the duration passed
|
||||
to `-search.logSlowQueryDuration` command-line flag (5s by default).
|
||||
VictoriaMetrics also provides `/api/v1/status/top_queries` endpoint, which returns
|
||||
queries that took the most time to execute.
|
||||
See [these docs](https://docs.victoriametrics.com/#prometheus-querying-api-enhancements) for details.
|
||||
|
||||
There are the following solutions exist for slow queries:
|
||||
VictoriaMetrics provides [`top queries` page at VMUI](https://docs.victoriametrics.com/#top-queries), which shows
|
||||
queries that took the most time to execute.
|
||||
|
||||
There are the following solutions exist for improving performance of slow queries:
|
||||
|
||||
- Adding more CPU and memory to VictoriaMetrics, so it may perform the slow query faster.
|
||||
If you use cluster version of VictoriaMetrics, then migration of `vmselect` nodes to machines
|
||||
If you use cluster version of VictoriaMetrics, then migrating `vmselect` nodes to machines
|
||||
with more CPU and RAM should help improving speed for slow queries. Query performance
|
||||
is always limited by resources of one vmselect which processes the query. For example, if 2vCPU cores on `vmselect`
|
||||
is always limited by resources of one `vmselect` which processes the query. For example, if 2vCPU cores on `vmselect`
|
||||
isn't enough to process query fast enough, then migrating `vmselect` to a machine with 4vCPU cores should increase heavy query performance by up to 2x.
|
||||
If the line on `Concurrent select` graph form the [official Grafana dashboard for VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#monitoring)
|
||||
If the line on `concurrent select` graph form the [official Grafana dashboard for VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#monitoring)
|
||||
is close to the limit, then prefer adding more `vmselect` nodes to the cluster.
|
||||
Sometimes adding more `vmstorage` nodes also can help improving the speed for slow queries.
|
||||
|
||||
- Rewriting slow queries, so they become faster. Unfortunately it is hard determining
|
||||
whether the given query is slow by just looking at it.
|
||||
VictoriaMetrics provides [query tracing](https://docs.victoriametrics.com/#query-tracing) feature,
|
||||
which can help determine the source of slow query.
|
||||
See also [this article](https://valyala.medium.com/how-to-optimize-promql-and-metricsql-queries-85a1b75bf986),
|
||||
which explains how to determine and optimize slow queries.
|
||||
|
||||
In practice many slow queries are generated because of improper use of [subqueries](https://docs.victoriametrics.com/MetricsQL.html#subqueries).
|
||||
The main source of slow queries in practice is [alerting and recording rules](https://docs.victoriametrics.com/vmalert.html#rules)
|
||||
with long lookbehind windows in square brackets. These queries are frequently used in SLI/SLO calculations such as [Sloth](https://github.com/slok/sloth).
|
||||
|
||||
For example, `avg_over_time(up[30d]) > 0.99` needs to read and process
|
||||
all the [raw samples](https://docs.victoriametrics.com/keyConcepts.html#raw-samples)
|
||||
for `up` [time series](https://docs.victoriametrics.com/keyConcepts.html#time-series) over the last 30 days
|
||||
each time it executes. If this query is executed frequently, then it can take significant share of CPU, disk read IO, network bandwidth and RAM.
|
||||
Such queries can be optimized in the following ways:
|
||||
|
||||
- To reduce the lookbehind window in square brackets. For example, `avg_over_time(up[10d])` takes up to 3x less compute resources
|
||||
than `avg_over_time(up[30d])` at VictoriaMetrics.
|
||||
- To increase evaluation interval for alerting and recording rules, so they are executed less frequently.
|
||||
For example, increasing `-evaluationInterval` command-line flag value at [vmalert](https://docs.victoriametrics.com/vmalert.html)
|
||||
from `1m` to `2m` should reduce compute resource usage at VictoriaMetrics by 2x.
|
||||
|
||||
Another source of slow queries is improper use of [subqueries](https://docs.victoriametrics.com/MetricsQL.html#subqueries).
|
||||
It is recommended avoiding subqueries if you don't understand clearly how they work.
|
||||
It is easy to create a subquery without knowing about it.
|
||||
For example, `rate(sum(some_metric))` is implicitly transformed into the following subquery
|
||||
|
|
@ -335,6 +346,11 @@ There are the following solutions exist for slow queries:
|
|||
It is likely this query won't return the expected results. Instead, `sum(rate(some_metric))` must be used instead.
|
||||
See [this article](https://www.robustperception.io/rate-then-sum-never-sum-then-rate/) for more details.
|
||||
|
||||
VictoriaMetrics provides [query tracing](https://docs.victoriametrics.com/#query-tracing) feature,
|
||||
which can help determining the source of slow query.
|
||||
See also [this article](https://valyala.medium.com/how-to-optimize-promql-and-metricsql-queries-85a1b75bf986),
|
||||
which explains how to determine and optimize slow queries.
|
||||
|
||||
|
||||
## Out of memory errors
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,16 @@
|
|||
---
|
||||
sort: 6
|
||||
weight: 6
|
||||
title: VictoriaLogs changelog
|
||||
menu:
|
||||
docs:
|
||||
identifier: "victorialogs-changelog"
|
||||
parent: "victorialogs"
|
||||
weight: 6
|
||||
aliases:
|
||||
- /VictoriaLogs/CHANGELOG.html
|
||||
---
|
||||
|
||||
# VictoriaLogs changelog
|
||||
|
||||
The following `tip` changes can be tested by building VictoriaLogs from the latest commit of [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/) repository
|
||||
|
|
@ -5,15 +18,25 @@ according to [these docs](https://docs.victoriametrics.com/VictoriaLogs/QuickSta
|
|||
|
||||
## tip
|
||||
|
||||
## [v0.4.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.4.1-victorialogs)
|
||||
|
||||
* BUGFIX: fix the free space verification process in VictoriaLogs that was erroneously shifting to read-only mode, despite there being sufficient free space available. See [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5112) issue.
|
||||
|
||||
## [v0.4.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.4.0-victorialogs)
|
||||
|
||||
Released at 2023-10-03
|
||||
|
||||
* FEATURE: add `-elasticsearch.version` command-line flag, which can be used for specifying Elasticsearch version returned by VictoriaLogs to Filebeat at [elasticsearch bulk API](https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/#elasticsearch-bulk-api). This helps resolving [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4777).
|
||||
* FEATURE: expose the following metrics at [/metrics](monitoring) page:
|
||||
* `vl_data_size_bytes{type="storage"}` - on-disk size for data excluding [log stream](https://docs.victoriametrics.com/VictoriaLogs/keyConcepts.html#stream-fields) indexes.
|
||||
* `vl_data_size_bytes{type="indexdb"}` - on-disk size for [log stream](https://docs.victoriametrics.com/VictoriaLogs/keyConcepts.html#stream-fields) indexes.
|
||||
* FEATURE: add `-insert.maxFieldsPerLine` command-line flag, which can be used for limiting the number of fields per line in logs sent to VictoriaLogs via ingestion protocols. This helps to avoid issues like [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762).
|
||||
* FEATURE: expose `vl_http_request_duration_seconds` histogram at the [/metrics](https://docs.victoriametrics.com/VictoriaLogs/#monitoring) page. Thanks to @crossoverJie for [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4934).
|
||||
* FEATURE: add support of `-storage.minFreeDiskSpaceBytes` command-line flag to allow switching to read-only mode when running out of disk space at `-storageDataPath`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4737).
|
||||
|
||||
* BUGFIX: fix possible panic when no data is written to VictoriaLogs for a long time. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4895). Thanks to @crossoverJie for filing and fixing the issue.
|
||||
* BUGFIX: add `/insert/loky/ready` endpoint, which is used by Promtail for healthchecks. This should remove `unsupported path requested: /insert/loki/ready` warning logs. See [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762#issuecomment-1690966722).
|
||||
* BUGFIX: prevent from panic during background merge when the number of columns in the resulting block exceeds the maximum allowed number of columns per block. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762).
|
||||
|
||||
## [v0.3.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.3.0-victorialogs)
|
||||
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ weight: 1
|
|||
title: VictoriaLogs FAQ
|
||||
menu:
|
||||
docs:
|
||||
identifier: "victorialogs-faq"
|
||||
parent: "victorialogs"
|
||||
weight: 1
|
||||
aliases:
|
||||
|
|
|
|||
|
|
@ -33,8 +33,8 @@ Just download archive for the needed Operating system and architecture, unpack i
|
|||
For example, the following commands download VictoriaLogs archive for Linux/amd64, unpack and run it:
|
||||
|
||||
```bash
|
||||
curl -L -O https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v0.3.0-victorialogs/victoria-logs-linux-amd64-v0.3.0-victorialogs.tar.gz
|
||||
tar xzf victoria-logs-linux-amd64-v0.3.0-victorialogs.tar.gz
|
||||
curl -L -O https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v0.4.1-victorialogs/victoria-logs-linux-amd64-v0.4.1-victorialogs.tar.gz
|
||||
tar xzf victoria-logs-linux-amd64-v0.4.1-victorialogs.tar.gz
|
||||
./victoria-logs-prod
|
||||
```
|
||||
|
||||
|
|
@ -58,7 +58,7 @@ Here is the command to run VictoriaLogs in a Docker container:
|
|||
|
||||
```bash
|
||||
docker run --rm -it -p 9428:9428 -v ./victoria-logs-data:/victoria-logs-data \
|
||||
docker.io/victoriametrics/victoria-logs:v0.3.0-victorialogs
|
||||
docker.io/victoriametrics/victoria-logs:v0.4.1-victorialogs
|
||||
```
|
||||
|
||||
See also:
|
||||
|
|
|
|||
|
|
@ -239,6 +239,9 @@ Pass `-help` to VictoriaLogs in order to see the list of supported command-line
|
|||
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 1048576)
|
||||
-storageDataPath string
|
||||
Path to directory with the VictoriaLogs data; see https://docs.victoriametrics.com/VictoriaLogs/#storage (default "victoria-logs-data")
|
||||
-storage.minFreeDiskSpaceBytes size
|
||||
The minimum free disk space at -storageDataPath after which the storage stops accepting new data
|
||||
Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 10000000)
|
||||
-tls
|
||||
Whether to enable TLS for incoming HTTP requests at -httpListenAddr (aka https). -tlsCertFile and -tlsKeyFile must be set if -tls is set
|
||||
-tlsCertFile string
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
sort: 6
|
||||
sort: 7
|
||||
title: Data ingestion
|
||||
weight: 6
|
||||
weight: 7
|
||||
menu:
|
||||
docs:
|
||||
identifier: victorialogs-data-ingestion
|
||||
parent: "victorialogs"
|
||||
weight: 6
|
||||
weight: 7
|
||||
---
|
||||
|
||||
# Data ingestion
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
sort: 7
|
||||
sort: 8
|
||||
title: Querying
|
||||
weight: 7
|
||||
weight: 8
|
||||
menu:
|
||||
docs:
|
||||
identifier: victorialogs-querying
|
||||
parent: "victorialogs"
|
||||
weight: 7
|
||||
weight: 8
|
||||
---
|
||||
|
||||
# Querying
|
||||
|
|
|
|||
|
|
@ -37,6 +37,11 @@
|
|||
color: #b35cec !important;
|
||||
}
|
||||
|
||||
// hidden in menu pages
|
||||
[data-sort="-1"] {
|
||||
display: none;
|
||||
}
|
||||
|
||||
// Content
|
||||
.markdown-body h1,
|
||||
.markdown-body h2,
|
||||
|
|
|
|||
|
|
@ -1,14 +1,27 @@
|
|||
---
|
||||
sort: 10
|
||||
weight: 10
|
||||
title: CHANGELOG
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 10
|
||||
identifier: "operator-changelog"
|
||||
---
|
||||
|
||||
# CHANGELOG
|
||||
|
||||
## Next release
|
||||
|
||||
### Features
|
||||
|
||||
- [vmoperator](https://docs.victoriametrics.com/operator/): upgrade vmagent/vmauth's default config-reloader image.
|
||||
- [vmoperator](./README.md): upgrade vmagent/vmauth's default config-reloader image.
|
||||
|
||||
### Fixes
|
||||
|
||||
- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): remove redundant annotation `operator.victoriametrics/last-applied-spec` from created workloads like vmstorage statefulset.
|
||||
- [vmcluster](./api.html#vmcluster): remove redundant annotation `operator.victoriametrics/last-applied-spec` from created workloads like vmstorage statefulset.
|
||||
- [vmoperator](./README.md): properly resize statefulset's multiple pvc when needed and allowable, before they could be updated with wrong size.
|
||||
- [vmoperator](./README.md): fix wrong api group of endpointsices, before vmagent won't able to access endpointsices resources with default rbac rule.
|
||||
|
||||
<a name="v0.38.0"></a>
|
||||
## [v0.38.0](https://github.com/VictoriaMetrics/operator/releases/tag/v0.38.0) - 11 Sep 2023
|
||||
|
|
@ -17,13 +30,13 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): [Enterprise] fixes ip_filters indent for url_prefix. Previously it wasn't possible to use ip_filters with multiple target refs
|
||||
- [vmoperator](https://docs.victoriametrics.com/operator/): turn off `EnableStrictSecurity` by default. Before, upgrade operator to v0.36.0+ could fail components with volume attached, see [this issue](https://github.com/VictoriaMetrics/operator/issues/749) for details.
|
||||
- [vmoperator](https://docs.victoriametrics.com/operator/): bump default version of VictoriaMetrics components to [1.93.4](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.4).
|
||||
- [vmuser](./api.md#vmuser): [Enterprise] fixes ip_filters indent for url_prefix. Previously it wasn't possible to use ip_filters with multiple target refs
|
||||
- [vmoperator](./README.md): turn off `EnableStrictSecurity` by default. Before, upgrade operator to v0.36.0+ could fail components with volume attached, see [this issue](https://github.com/VictoriaMetrics/operator/issues/749) for details.
|
||||
- [vmoperator](./README.md): bump default version of VictoriaMetrics components to [1.93.4](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.4).
|
||||
|
||||
### Features
|
||||
|
||||
- [vmoperator](https://docs.victoriametrics.com/operator/) add ability to print default values for all [operator variables](https://docs.victoriametrics.com/operator/vars.html). See [this issue](https://github.com/VictoriaMetrics/operator/issues/675) for details.
|
||||
- [vmoperator](./README.md) add ability to print default values for all [operator variables](./vars.md). See [this issue](https://github.com/VictoriaMetrics/operator/issues/675) for details.
|
||||
|
||||
<a name="v0.37.1"></a>
|
||||
## [v0.37.1](https://github.com/VictoriaMetrics/operator/releases/tag/v0.37.1) - 02 Sep 2023
|
||||
|
|
@ -39,18 +52,18 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fix unmarshalling for streaming aggregation `match` field.
|
||||
- [vmagent](./api.md#vmagent): fix unmarshalling for streaming aggregation `match` field.
|
||||
|
||||
### Features
|
||||
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): support [multiple if conditions](https://docs.victoriametrics.com/vmagent.html#relabeling:~:text=the%20if%20option%20may%20contain%20more%20than%20one%20filter) for relabeling. See [this issue](https://github.com/VictoriaMetrics/operator/issues/730) for details.
|
||||
- [vmagent](./api.md#vmagent): support [multiple if conditions](https://docs.victoriametrics.com/vmagent.html#relabeling:~:text=the%20if%20option%20may%20contain%20more%20than%20one%20filter) for relabeling. See [this issue](https://github.com/VictoriaMetrics/operator/issues/730) for details.
|
||||
|
||||
<a name="v0.36.1"></a>
|
||||
## [v0.36.1](https://github.com/VictoriaMetrics/operator/releases/tag/v0.36.0) - 25 Aug 2023
|
||||
|
||||
### Fixes
|
||||
|
||||
- [vmselect](https://docs.victoriametrics.com/operator/api.html#vmcluster): fix cache directory when `cacheDataPath` not specified, before it will use `/tmp` which is protect by default strict securityContext.
|
||||
- [vmselect](./api.md#vmcluster): fix cache directory when `cacheDataPath` not specified, before it will use `/tmp` which is protect by default strict securityContext.
|
||||
|
||||
### Features
|
||||
|
||||
|
|
@ -59,7 +72,7 @@
|
|||
|
||||
### Breaking changes
|
||||
|
||||
- **[vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): Field `OAuth2` was renamed to `oauth2` due to compatibility issue. If you defined `OAuth2` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `oauth2` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/689) for details.**
|
||||
- **[vmalert](./api.md#vmalert): Field `OAuth2` was renamed to `oauth2` due to compatibility issue. If you defined `OAuth2` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `oauth2` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/689) for details.**
|
||||
- **Affected fields:**
|
||||
- **`VMAlert.spec.datasource.OAuth2` -> `VMAlert.spec.datasource.oauth2`,**
|
||||
- **`VMAlert.spec.notifier.OAuth2` -> `VMAlert.spec.notifier.oauth2`,**
|
||||
|
|
@ -67,7 +80,7 @@
|
|||
- **`VMAlert.spec.remoteRead.OAuth2` -> `VMAlert.spec.remoteRead.oauth2`,**
|
||||
- **`VMAlert.spec.remoteWrite.OAuth2` -> `VMAlert.spec.remoteWrite.oauth2`,**
|
||||
|
||||
- **[vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): Field `bearerTokenFilePath` was renamed to `bearerTokenFile` due to compatibility issue. If you defined `bearerTokenFilePath` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `bearerTokenFile` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/688/) for details.**
|
||||
- **[vmalert](./api.md#vmalert): Field `bearerTokenFilePath` was renamed to `bearerTokenFile` due to compatibility issue. If you defined `bearerTokenFilePath` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `bearerTokenFile` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/688/) for details.**
|
||||
- **Affected fields:**
|
||||
- **`VMAlert.spec.datasource.bearerTokenFilePath` --> `VMAlert.spec.datasource.bearerTokenFile`,**
|
||||
- **`VMAlert.spec.notifier.bearerTokenFilePath` --> `VMAlert.spec.notifier.bearerTokenFile`,**
|
||||
|
|
@ -80,22 +93,22 @@
|
|||
- operator set resource requests for config-reloader container by default. See [this PR](https://github.com/VictoriaMetrics/operator/pull/695/) for details.
|
||||
- fix `attachMetadata` value miscovert for scrape objects. See [this issue](https://github.com/VictoriaMetrics/operator/issues/697) and [this PR](https://github.com/VictoriaMetrics/operator/pull/698) for details.
|
||||
- fix volumeClaimTemplates change check for objects that generate statefulset, like vmstorage, vmselect. Before, the statefulset won't be recreated if additional `claimTemplates` object changed. See [this issue](https://github.com/VictoriaMetrics/operator/issues/507) and [this PR](https://github.com/VictoriaMetrics/operator/pull/719) for details.
|
||||
- [vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): fix `tlsCAFile` argument value generation when using secret or configMap. See [this issue](https://github.com/VictoriaMetrics/operator/issues/699) and [this PR](https://github.com/VictoriaMetrics/operator/issues/699) for details.
|
||||
- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): fix default request memory and apply default resources if not set. See [this issue](https://github.com/VictoriaMetrics/operator/issues/706) and [this PR](https://github.com/VictoriaMetrics/operator/pull/710) for details.
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fix missing additional VolumeClaimTemplates when using `ClaimTemplates` under StatefulMode.
|
||||
- [vmalert](./api.md#vmalert): fix `tlsCAFile` argument value generation when using secret or configMap. See [this issue](https://github.com/VictoriaMetrics/operator/issues/699) and [this PR](https://github.com/VictoriaMetrics/operator/issues/699) for details.
|
||||
- [vmalertmanager](./api.md#vmalertmanager): fix default request memory and apply default resources if not set. See [this issue](https://github.com/VictoriaMetrics/operator/issues/706) and [this PR](https://github.com/VictoriaMetrics/operator/pull/710) for details.
|
||||
- [vmagent](./api.md#vmagent): fix missing additional VolumeClaimTemplates when using `ClaimTemplates` under StatefulMode.
|
||||
|
||||
### Features
|
||||
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmagent_stateful_with_sharding.yaml) for vmagent statefulmode.
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent)/[vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): adapt new features in streaming aggregation:
|
||||
- [vmagent](./api.md#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmagent_stateful_with_sharding.yaml) for vmagent statefulmode.
|
||||
- [vmagent](./api.md#vmagent)/[vmsingle](./api.md#vmsingle): adapt new features in streaming aggregation:
|
||||
- support `streamAggr.dropInput`, see [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4243) for details;
|
||||
- support list for `match` parameter, see [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635) for details;
|
||||
- support `staleness_interval`, see [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4667) for details.
|
||||
- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmcluster_with_additional_claim.yaml) for cluster with custom storage claims.
|
||||
- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): support `update_entries_limit` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/691) for details.
|
||||
- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): support `keep_firing_for` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/711) for details.
|
||||
- [vmoperator parameters](https://docs.victoriametrics.com/operator/vars.html): Add option `VM_ENABLESTRICTSECURITY` and enable strict security context by default. See [this issue](https://github.com/VictoriaMetrics/operator/issues/637), [this](https://github.com/VictoriaMetrics/operator/pull/692/) and [this](https://github.com/VictoriaMetrics/operator/pull/712) PR for details.
|
||||
- [vmoperator parameters](https://docs.victoriametrics.com/operator/vars.html): change option `VM_PSPAUTOCREATEENABLED` default value from `true` to `false` cause PodSecurityPolicy already got deprecated since [kubernetes v1.25](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#psp-v125). See [this pr](https://github.com/VictoriaMetrics/operator/pull/726) for details.
|
||||
- [vmcluster](./api.md#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmcluster_with_additional_claim.yaml) for cluster with custom storage claims.
|
||||
- [vmrule](./api.md#vmrule): support `update_entries_limit` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/691) for details.
|
||||
- [vmrule](./api.md#vmrule): support `keep_firing_for` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/711) for details.
|
||||
- [vmoperator parameters](./vars.md): Add option `VM_ENABLESTRICTSECURITY` and enable strict security context by default. See [this issue](https://github.com/VictoriaMetrics/operator/issues/637), [this](https://github.com/VictoriaMetrics/operator/pull/692/) and [this](https://github.com/VictoriaMetrics/operator/pull/712) PR for details.
|
||||
- [vmoperator parameters](./vars.md): change option `VM_PSPAUTOCREATEENABLED` default value from `true` to `false` cause PodSecurityPolicy already got deprecated since [kubernetes v1.25](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#psp-v125). See [this pr](https://github.com/VictoriaMetrics/operator/pull/726) for details.
|
||||
|
||||
[Changes][v0.36.0]
|
||||
|
||||
|
|
@ -104,7 +117,7 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fixes regression with remoteWrite authorization (basicAuth/token). When `UseCustomConfigReloader` option was set, operator incorrectly rendered mounts for `vmagent` container. https://github.com/VictoriaMetrics/operator/commit/f2b8cf701a33f91cef19848c857fd6efb7db59dd
|
||||
- [vmagent](./api.md#vmagent): fixes regression with remoteWrite authorization (basicAuth/token). When `UseCustomConfigReloader` option was set, operator incorrectly rendered mounts for `vmagent` container. https://github.com/VictoriaMetrics/operator/commit/f2b8cf701a33f91cef19848c857fd6efb7db59dd
|
||||
|
||||
[Changes][v0.35.1]
|
||||
|
||||
|
|
@ -114,19 +127,19 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
* [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): fix vmselect url_map in vmuser. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/655). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): correctly set default port for vmauth components discovery. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/658). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): remove rate limit on delete. In https://github.com/VictoriaMetrics/operator/pull/672. Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): fix spec change check. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmuser](./api.md#vmuser): fix vmselect url_map in vmuser. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/655). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmalert](./api.md#vmalert): correctly set default port for vmauth components discovery. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/658). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmuser](./api.md#vmuser): remove rate limit on delete. In https://github.com/VictoriaMetrics/operator/pull/672. Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmcluster](./api.md#vmcluster): fix spec change check. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* Correctly publish multi-arch release at https://github.com/VictoriaMetrics/operator/pull/681. Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
|
||||
### Features
|
||||
|
||||
* [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): add validation when generate static scrape config. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): add validation for slack receiver url. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/661). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth)/[vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): implement configuration initiation for custom config reloader. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/619). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmagent](./api.md#vmagent): add validation when generate static scrape config. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): add validation for slack receiver url. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/661). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmauth](./api.md#vmauth)/[vmagent](./api.md#vmagent): implement configuration initiation for custom config reloader. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/619). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* add more generators Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/668
|
||||
* [vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): add status field. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/670). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
* [vmsingle](./api.md#vmsingle): add status field. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/670). Thanks [@Haleygo](https://github.com/Haleygo)
|
||||
|
||||
[Changes][v0.35.0]
|
||||
|
||||
|
|
@ -136,9 +149,9 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): fail fast on misconfigured or missing kubernetes pods. It should prevent rare bug with cascade pod deletion. See this [issue](https://github.com/VictoriaMetrics/operator/issues/643) for details
|
||||
- [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth)/[vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): correctly renders initConfig image with global container registry domain. See this [issue](https://github.com/VictoriaMetrics/operator/issues/654) for details.
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): correctly set RBAC permissions for single namespace mode and custom config reloader image. See this [issue](https://github.com/VictoriaMetrics/operator/issues/653) for details.
|
||||
- [vmcluster](./api.md#vmcluster): fail fast on misconfigured or missing kubernetes pods. It should prevent rare bug with cascade pod deletion. See this [issue](https://github.com/VictoriaMetrics/operator/issues/643) for details
|
||||
- [vmauth](./api.md#vmauth)/[vmagent](./api.md#vmagent): correctly renders initConfig image with global container registry domain. See this [issue](https://github.com/VictoriaMetrics/operator/issues/654) for details.
|
||||
- [vmagent](./api.md#vmagent): correctly set RBAC permissions for single namespace mode and custom config reloader image. See this [issue](https://github.com/VictoriaMetrics/operator/issues/653) for details.
|
||||
|
||||
[Changes][v0.34.1]
|
||||
|
||||
|
|
@ -152,18 +165,18 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmnodescrape](https://docs.victoriametrics.com/operator/api.html#vmnodescrape): fixed selectors for Exists and NotExists operators with empty label Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/646
|
||||
- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): Add config for vmrule in validating webhook Thanks in https://github.com/VictoriaMetrics/operator/pull/650
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): skips misconfigured objects with missed secret references: https://github.com/VictoriaMetrics/operator/issues/648
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): correctly renders initContainer for configuration download: https://github.com/VictoriaMetrics/operator/issues/649
|
||||
- [vmnodescrape](./api.md#vmnodescrape): fixed selectors for Exists and NotExists operators with empty label Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/646
|
||||
- [vmrule](./api.md#vmrule): Add config for vmrule in validating webhook Thanks in https://github.com/VictoriaMetrics/operator/pull/650
|
||||
- [vmagent](./api.md#vmagent): skips misconfigured objects with missed secret references: https://github.com/VictoriaMetrics/operator/issues/648
|
||||
- [vmagent](./api.md#vmagent): correctly renders initContainer for configuration download: https://github.com/VictoriaMetrics/operator/issues/649
|
||||
|
||||
### Features
|
||||
|
||||
- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): Bump alertmanager to v0.25.0 Thanks [@tamcore](https://github.com/tamcore) in https://github.com/VictoriaMetrics/operator/pull/636
|
||||
- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): added `clusterNativePort` field to VMSelect/VMInsert for multi-level cluster setup ([#634](https://github.com/VictoriaMetrics/operator/issues/634)) Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/639
|
||||
- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): add notifierHeader field in vmrule spec Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/622
|
||||
- [vmpodscrape](https://docs.victoriametrics.com/operator/api.html#vmpodscrape): adds FilterRunning option as prometheus does in https://github.com/VictoriaMetrics/operator/pull/640
|
||||
- [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth): adds latest features in https://github.com/VictoriaMetrics/operator/pull/642
|
||||
- [vmalertmanager](./api.md#vmalertmanager): Bump alertmanager to v0.25.0 Thanks [@tamcore](https://github.com/tamcore) in https://github.com/VictoriaMetrics/operator/pull/636
|
||||
- [vmcluster](./api.md#vmcluster): added `clusterNativePort` field to VMSelect/VMInsert for multi-level cluster setup ([#634](https://github.com/VictoriaMetrics/operator/issues/634)) Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/639
|
||||
- [vmrule](./api.md#vmrule): add notifierHeader field in vmrule spec Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/622
|
||||
- [vmpodscrape](./api.md#vmpodscrape): adds FilterRunning option as prometheus does in https://github.com/VictoriaMetrics/operator/pull/640
|
||||
- [vmauth](./api.md#vmauth): adds latest features in https://github.com/VictoriaMetrics/operator/pull/642
|
||||
|
||||
[Changes][v0.34.0]
|
||||
|
||||
|
|
@ -173,22 +186,22 @@
|
|||
|
||||
### Fixes
|
||||
|
||||
- [vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): skip bad rules and improve logging for rules exceed max configmap size https://github.com/VictoriaMetrics/operator/commit/bb754d5c20bb371a197cd6ff5afac1ba86a4d92b
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): fixed error with headers in VMAlertmanagerConfig.Receivers.EmailConfigs.Headers unmarshalling. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/610
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fixed keepInput setting for streaming aggregation. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/618
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): fix webhook config maxAlerts not work. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/625
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): Remove single quotes from remote write headers. Thanks [@axelsccp](https://github.com/axelsccp) in https://github.com/VictoriaMetrics/operator/pull/613
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): fix parse route error and some comments. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/630
|
||||
- [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): properly removes finalizers for objects https://github.com/VictoriaMetrics/operator/commit/8f10113920a353f21fbcc8637076905f2e57bb34
|
||||
- [vmalert](./api.md#vmalert): skip bad rules and improve logging for rules exceed max configmap size https://github.com/VictoriaMetrics/operator/commit/bb754d5c20bb371a197cd6ff5afac1ba86a4d92b
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): fixed error with headers in VMAlertmanagerConfig.Receivers.EmailConfigs.Headers unmarshalling. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/610
|
||||
- [vmagent](./api.md#vmagent): fixed keepInput setting for streaming aggregation. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/618
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): fix webhook config maxAlerts not work. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/625
|
||||
- [vmagent](./api.md#vmagent): Remove single quotes from remote write headers. Thanks [@axelsccp](https://github.com/axelsccp) in https://github.com/VictoriaMetrics/operator/pull/613
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): fix parse route error and some comments. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/630
|
||||
- [vmuser](./api.md#vmuser): properly removes finalizers for objects https://github.com/VictoriaMetrics/operator/commit/8f10113920a353f21fbcc8637076905f2e57bb34
|
||||
|
||||
### Features
|
||||
|
||||
- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): add option to disable route continue enforce. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/621
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): support set require_tls to false. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/624
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): add sanity check. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/627
|
||||
- [vmalertmanager](./api.md#vmalertmanager): add option to disable route continue enforce. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/621
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): support set require_tls to false. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/624
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): add sanity check. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/627
|
||||
- Makefile: bump Alpine base image to latest v3.17.3. Thanks [@denisgolius](https://github.com/denisgolius) in https://github.com/VictoriaMetrics/operator/pull/628
|
||||
- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): support sound field in pushover config. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/631
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent)/[vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth): download initial config with initContainer https://github.com/VictoriaMetrics/operator/commit/612e7c8f40659731e7938ef9556eb088c67eb4b7
|
||||
- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): support sound field in pushover config. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/631
|
||||
- [vmagent](./api.md#vmagent)/[vmauth](./api.md#vmauth): download initial config with initContainer https://github.com/VictoriaMetrics/operator/commit/612e7c8f40659731e7938ef9556eb088c67eb4b7
|
||||
|
||||
[Changes][v0.33.0]
|
||||
|
||||
|
|
@ -199,7 +212,7 @@
|
|||
### Fixes
|
||||
|
||||
- config: fixes typo at default vm apps version https://github.com/VictoriaMetrics/operator/issues/608
|
||||
- [vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): conditionally adds stream aggregation config https://github.com/VictoriaMetrics/operator/commit/4a0ca54113afcde439ca4c77e22d3ef1c0d36241
|
||||
- [vmsingle](./api.md#vmsingle): conditionally adds stream aggregation config https://github.com/VictoriaMetrics/operator/commit/4a0ca54113afcde439ca4c77e22d3ef1c0d36241
|
||||
|
||||
[Changes][v0.32.1]
|
||||
|
||||
|
|
@ -213,10 +226,10 @@
|
|||
|
||||
### Features
|
||||
|
||||
- [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth): automatically configures `proxy-protocol` client and `reloadAuthKey` for `config-reloader` container. https://github.com/VictoriaMetrics/operator/commit/611819233bf595a4dbd04b07d7be24b7e994379c
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): adds `scrapeTimeout` global configuration for `VMAgent` https://github.com/VictoriaMetrics/operator/commit/d1d5024c6befa0961f8d56c82a0554935a4b1878
|
||||
- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) for `remoteWrite` targets https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
|
||||
- [vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) as global configuration for database https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
|
||||
- [vmauth](./api.md#vmauth): automatically configures `proxy-protocol` client and `reloadAuthKey` for `config-reloader` container. https://github.com/VictoriaMetrics/operator/commit/611819233bf595a4dbd04b07d7be24b7e994379c
|
||||
- [vmagent](./api.md#vmagent): adds `scrapeTimeout` global configuration for `VMAgent` https://github.com/VictoriaMetrics/operator/commit/d1d5024c6befa0961f8d56c82a0554935a4b1878
|
||||
- [vmagent](./api.md#vmagent): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) for `remoteWrite` targets https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
|
||||
- [vmsingle](./api.md#vmsingle): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) as global configuration for database https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
|
||||
|
||||
[Changes][v0.32.0]
|
||||
|
||||
|
|
@ -231,8 +244,8 @@
|
|||
|
||||
### Features
|
||||
|
||||
- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): Add support of vmalertmanager.spec.templates and autoreload dirs for templates and configmaps thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/590 https://github.com/VictoriaMetrics/operator/issues/592
|
||||
- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): Add support "%SHARD_NUM%" placeholder for vmagent sts/deployment Thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/508
|
||||
- [vmalertmanager](./api.md#vmalertmanager): Add support of vmalertmanager.spec.templates and autoreload dirs for templates and configmaps thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/590 https://github.com/VictoriaMetrics/operator/issues/592
|
||||
- [vmalertmanager](./api.md#vmalertmanager): Add support "%SHARD_NUM%" placeholder for vmagent sts/deployment Thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/508
|
||||
|
||||
[Changes][v0.31.0]
|
||||
|
||||
|
|
|
|||
|
|
@ -1,17 +1,19 @@
|
|||
---
|
||||
sort: 15
|
||||
weight: 15
|
||||
sort: 9
|
||||
weight: 9
|
||||
title: FAQ
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 15
|
||||
identifier: "faq-operator"
|
||||
aliases:
|
||||
- /operator/FAQ.html
|
||||
weight: 9
|
||||
identifier: "operator-faq"
|
||||
---
|
||||
|
||||
# FAQ
|
||||
# FAQ (Frequency Asked Questions)
|
||||
|
||||
## How do you monitor the operator itself?
|
||||
|
||||
You can read about vmoperator monitoring in [this document](./monitoring.md).
|
||||
|
||||
## How to change VMStorage PVC storage class
|
||||
|
||||
|
|
@ -29,3 +31,53 @@ With Operator deployment:
|
|||
1. Run `kubectl delete statefulset --cascade=orphan {vmstorage-sts}`
|
||||
1. Update VMCluster spec to use new storage class
|
||||
1. Apply cluster configuration
|
||||
|
||||
## How to override image registry
|
||||
|
||||
You can use `VM_CONTAINERREGISTRY` parameter for operator:
|
||||
|
||||
- See details about tuning [operator settings here](./setup.md#settings).
|
||||
- See [available operator settings](./vars.md) here.
|
||||
|
||||
## How to set up automatic backups?
|
||||
|
||||
You can read about backups:
|
||||
|
||||
- for `VMSingle`: [Backup automation](./resources/vmsingle.md#backup-automation)
|
||||
- for `VMCluster`: [Backup automation](./resources/vmcluster.md#backup-automation)
|
||||
|
||||
## How to migrate from Prometheus-operator to VictoriaMetrics operator?
|
||||
|
||||
You can read about migration from prometheus operator on [this page](./migration.md).
|
||||
|
||||
## How to turn off conversion for prometheus resources
|
||||
|
||||
You can read about it on [this page](./migration.md#objects-convesion).
|
||||
|
||||
## My VM objects are not deleted/changed when I delete/change Prometheus objects
|
||||
|
||||
You can read about it in following sections of "Migration from prometheus-operator" docs:
|
||||
|
||||
- [Deletion synchronization](./migration.md#deletion-synchronization)
|
||||
- [Update synchronization](./migration.md#update-synchronization)
|
||||
- [Labels synchronization](./migration.md#labels-synchronization)
|
||||
|
||||
## What permissions does an operator need to run in a cluster?
|
||||
|
||||
You can read about needed permissions for operator in [this document](./security.md#roles).
|
||||
|
||||
## How to know the version of VM components in the operator?
|
||||
|
||||
See [printDefaults mode](./configuration.md).
|
||||
|
||||
In addition, you can use [Release notes](https://github.com/VictoriaMetrics/operator/releases)
|
||||
or [CHANGELOG](https://github.com/VictoriaMetrics/operator/blob/master/docs/CHANGELOG.md).
|
||||
- that's where we describe default version of VictoriaMetrics components.
|
||||
|
||||
## How to run VictoriaMetrics operator with permissions for one namespace only?
|
||||
|
||||
See this document for details: [Configuration -> Namespaced mode](./configuration.md#namespaced-mode).
|
||||
|
||||
## What versions of Kubernetes is the operator compatible with?
|
||||
|
||||
Operator tested at kubernetes versions from 1.16 to 1.27.
|
||||
|
|
|
|||
|
|
@ -1,21 +1,95 @@
|
|||
---
|
||||
sort: 27
|
||||
sort: 0
|
||||
weight: 0
|
||||
title: VictoriaMetrics Operator
|
||||
disableToc: true
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 0
|
||||
---
|
||||
|
||||
# VictoriaMetrics Operator
|
||||
|
||||
1. [VictoriaMetrics Operator](VictoriaMetrics-Operator.html)
|
||||
1. [Additional Scrape Configuration](additional-scrape.html)
|
||||
1. [API Docs](api.html)
|
||||
1. [Authorization and exposing components](auth.html)
|
||||
1. [vmbackupmanager](backups.html)
|
||||
1. [Design](design.html)
|
||||
1. [High Availability](high-availability.html)
|
||||
1. [VMAlert, VMAgent, VMAlertmanager, VMSingle version](managing-versions.html)
|
||||
1. [Victoria Metrics Operator Quick Start](quick-start.html)
|
||||
1. [VMAgent relabel](relabeling.html)
|
||||
1. [CRD Validation](resources-validation.html)
|
||||
1. [Security](security.html)
|
||||
1. [Auto Generated vars for package config](vars.html)
|
||||
Operator serves to make running VictoriaMetrics applications on top of Kubernetes as easy as possible while preserving Kubernetes-native configuration options.
|
||||
|
||||
VictoriaMetrics Operator (`vmoperator`) is the classic kubernetes-operator for VictoriaMetrics with many [great features](#features).
|
||||
It allows you to manage Victoria Metrics components in Kubernetes or OpenShift clusters
|
||||
in a declarative style according to [GitOps](https://www.redhat.com/en/topics/devops/what-is-gitops)
|
||||
and [IaC](https://en.wikipedia.org/wiki/Infrastructure_as_code) concepts.
|
||||
|
||||
VictoriaMetrics also provides [helm charts](https://github.com/VictoriaMetrics/helm-charts) without operator.
|
||||
Operator makes the same, simplifies it and provides [advanced features](#features).
|
||||
|
||||
Learn more about [key concepts](#key-concepts) of `vmoperator` and follow the **[quick start guide](./quick-start.md)** for a better experience.
|
||||
|
||||
## Features of vmoperator
|
||||
|
||||
- Deployment and management in a kubernetes clusters of any number of VictoriaMetrics applications (like vmsingle/vmcluster instances and another components like vmauth, vmagent, vmalert, etc...)
|
||||
- Seamless [migration from prometheus-operator](./migration.md) with auto-conversion of prometheus [custom resources](#custom-resources)
|
||||
- Simple VictoriaMetrics cluster installation, configuring, upgrading and managing with [crd-objects](./resources/README.md).
|
||||
- Ability to delegate the configuration (parts of configuration) of applications monitoring to the end-users and managing access to different configurations or configuration sections.
|
||||
- Integration with VictoriaMetrics [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager.html) - advanced tools for making backups. Check [Backup automation for VMSingle](./resources/vmsingle.md#backup-automation) or [Backup automation for VMCluster](./resources/vmcluster.md#backup-automation).
|
||||
- Everything you need for monitoring out of the box in [k8s-stack helm chart](https://victoriametrics.github.io/helm-charts/charts/victoria-metrics-k8s-stack/) with ready-made usecases and solutions.
|
||||
- Ability to template your own deployment scenarios.
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### Kubernetes-operators
|
||||
|
||||
[Kubernetes-operators](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) are software extensions
|
||||
for Kubernetes that make use of [custom resources](#custom-resources) to manage applications and their components.
|
||||
Operators follow Kubernetes principles, notably the control loop.
|
||||
It can be said that operators are custom controllers for Kubernetes that allow you to create business logic for custom resources.
|
||||
|
||||
Design and implementation of `vmoperator` inspired by [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator).
|
||||
|
||||
Useful links:
|
||||
- [Custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
|
||||
- [Custom resource definitions](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)
|
||||
- [Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/)
|
||||
- [Operator best practices](https://sdk.operatorframework.io/docs/best-practices/)
|
||||
|
||||
### Custom resources
|
||||
|
||||
Kubernetes-Operators use [custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
|
||||
for interaction. Custom resources are a mechanism built into Kubernetes that allows you to create your own extensions for Kubernetes,
|
||||
working on the same principles as those built into Kubernetes APIs. Custom resources make Kubernetes so modular and extensible.
|
||||
|
||||
In addition, thanks to CRD ([Custom Resource Definitions](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)),
|
||||
the mechanism of custom resources allows you to declare an API in the format of the OpenAPI specification and verify that the resources correspond to this API.
|
||||
|
||||
### Reconciliation cycle
|
||||
|
||||
The main task of the operator is to bring the state of the cluster in line with what is declared by the user in the custom resources.
|
||||
This process of constant monitoring and adjustment is called the "Reconciliation cycle" - it is the operator's workflow.
|
||||
|
||||
The basic workflow of working with the operator can be simplified as the following diagram:
|
||||
|
||||
<img src="README_operator-workflow.png" width="1200">
|
||||
|
||||
- Operator declares and owns [resources of Victoria Metrics](./resources/README.md).
|
||||
- Kubernetes validates of the resource according to the specification from CRD (see more in [custom resources](#custom-resources)).
|
||||
- Operator subscribed to change events (`create`, `update`, `delete`) for related resources.
|
||||
- When an event occurs, the operator reacts and updates the state of the objects in the cluster.
|
||||
- For some objects in the cluster the reconciliation cycle is performed at a given interval, even without the occurrence of change events (see `VM_FORCERESYNCINTERVAL`).
|
||||
|
||||
### Next steps
|
||||
|
||||
- [Quick Start Guide](./quick-start.md)
|
||||
- [Setup](./setup.md)
|
||||
- [Security](./security.md)
|
||||
- [Configuration](./configuration.md)
|
||||
- [Migration from Prometheus](./migration.md)
|
||||
- [Monitoring](./monitoring.md)
|
||||
- [Authorization and exposing components](./auth.md)
|
||||
- [High Availability](./high-availability.md)
|
||||
- [Enterprise](./enterprise.md)
|
||||
- [Custom resources](./resources/README.md)
|
||||
|
||||
If you have any questions, check out our [FAQ](./FAQ.md)
|
||||
and feel free to can ask them:
|
||||
- [VictoriaMetrics Slack](https://victoriametrics.slack.com/)
|
||||
- [VictoriaMetrics Telegram](https://t.me/VictoriaMetrics_en)
|
||||
|
||||
If you have any suggestions or find a bug, please create an issue
|
||||
on [GitHub](https://github.com/VictoriaMetrics/operator/issues/new).
|
||||
|
|
|
|||
BIN
docs/operator/README_operator-workflow.png
Normal file
BIN
docs/operator/README_operator-workflow.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 805 KiB |
|
|
@ -1,86 +1,9 @@
|
|||
---
|
||||
sort: 1
|
||||
weight: 1
|
||||
title: VictoriaMetrics Operator Overview
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 1
|
||||
sort: -1 # hide page from menu
|
||||
aliases:
|
||||
- /operator/VictoriaMetrics-Operator.html
|
||||
---
|
||||
|
||||
# VictoriaMetrics Operator Overview
|
||||
This page is deprecated.
|
||||
|
||||
## Overview
|
||||
|
||||
Design and implementation inspired by [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator). It's great a tool for managing monitoring configuration of your applications. VictoriaMetrics operator has api capability with it.
|
||||
So you can use familiar CRD objects: `ServiceMonitor`, `PodMonitor`, `PrometheusRule` and `Probe`. Or you can use VictoriaMetrics CRDs:
|
||||
- `VMServiceScrape` - defines scraping metrics configuration from pods backed by services.
|
||||
- `VMPodScrape` - defines scraping metrics configuration from pods.
|
||||
- `VMRule` - defines alerting or recording rules.
|
||||
- `VMProbe` - defines a probing configuration for targets with blackbox exporter.
|
||||
|
||||
Besides, operator allows you to manage VictoriaMetrics applications inside kubernetes cluster and simplifies this process [quick-start](https://docs.victoriametrics.com/operator/quick-start.html)
|
||||
With CRD (Custom Resource Definition) you can define application configuration and apply it to your cluster [crd-objects](https://docs.victoriametrics.com/operator/api.html).
|
||||
|
||||
Operator simplifies VictoriaMetrics cluster installation, upgrading and managing.
|
||||
|
||||
It has integration with VictoriaMetrics `vmbackupmanager` - advanced tools for making backups. Check backup [docs](https://docs.victoriametrics.com/operator/backups.html)
|
||||
|
||||
## Use cases
|
||||
|
||||
For kubernetes-cluster administrators, it simplifies installation, configuration and management for `VictoriaMetrics` application. The main feature of operator is its ability to delegate the configuration of applications monitoring to the end-users.
|
||||
|
||||
For applications developers, its great possibility for managing observability of applications. You can define metrics scraping and alerting configuration for your application and manage it with an application deployment process. Just define app_deployment.yaml, app_vmpodscrape.yaml and app_vmrule.yaml. That's it, you can apply it to a kubernetes cluster. Check [quick-start](/operator/quick-start.html) for an example.
|
||||
|
||||
## Operator vs helm-chart
|
||||
|
||||
VictoriaMetrics provides [helm charts](https://github.com/VictoriaMetrics/helm-charts). Operator makes the same, simplifies it and provides advanced features.
|
||||
|
||||
|
||||
## Configuration
|
||||
|
||||
Operator configured by env variables, list of it can be found at [link](https://docs.victoriametrics.com/operator/vars.html)
|
||||
|
||||
It defines default configuration options, like images for components, timeouts, features.
|
||||
|
||||
|
||||
## Kubernetes' compatibility versions
|
||||
|
||||
Operator tested at kubernetes versions from 1.16 to 1.22.
|
||||
|
||||
For clusters version below 1.16 you must use legacy CRDs from [path](https://github.com/VictoriaMetrics/operator/tree/master/config/crd/legacy)
|
||||
and disable CRD controller with flag: `--controller.disableCRDOwnership=true`
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- cannot apply crd at kubernetes 1.18 + version and kubectl reports error:
|
||||
```console
|
||||
Error from server (Invalid): error when creating "release/crds/crd.yaml": CustomResourceDefinition.apiextensions.k8s.io "vmalertmanagers.operator.victoriametrics.com" is invalid: [spec.validation.openAPIV3Schema.properties[spec].properties[initContainers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property]
|
||||
Error from server (Invalid): error when creating "release/crds/crd.yaml": CustomResourceDefinition.apiextensions.k8s.io "vmalerts.operator.victoriametrics.com" is invalid: [
|
||||
```
|
||||
upgrade to the latest release version. There is a bug with kubernetes objects at the early releases.
|
||||
|
||||
|
||||
## Development
|
||||
|
||||
- [operator-sdk](https://github.com/operator-framework/operator-sdk) version v1.0.0+
|
||||
- golang 1.15 +
|
||||
- minikube or kind
|
||||
|
||||
start:
|
||||
```console
|
||||
make run
|
||||
```
|
||||
|
||||
for test execution run:
|
||||
```console
|
||||
#unit tests
|
||||
|
||||
make test
|
||||
|
||||
# you need minikube or kind for e2e, do not run it on live cluster
|
||||
#e2e tests with local binary
|
||||
make e2e-local
|
||||
```
|
||||
Please see [Operator page](./README.md) instead.
|
||||
|
|
|
|||
|
|
@ -1,89 +1,10 @@
|
|||
---
|
||||
sort: 2
|
||||
weight: 2
|
||||
title: Additional Scrape Configuration
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 2
|
||||
sort: -1 # hide page from menu
|
||||
aliases:
|
||||
- /operator/additional-scrape.html
|
||||
---
|
||||
|
||||
# Additional Scrape Configuration
|
||||
|
||||
AdditionalScrapeConfigs is an additional way to add scrape targets in VMAgent CRD.
|
||||
There are two options for adding targets into VMAgent: inline configuration into CRD or defining it as a Kubernetes Secret.
|
||||
|
||||
No validation happens during the creation of configuration. However, you must validate job specs, and it must follow job spec configuration.
|
||||
Please check [scrape_configs documentation](https://docs.victoriametrics.com/sd_configs.html#scrape_configs) as references.
|
||||
|
||||
## Inline Additional Scrape Configuration in VMAgent CRD
|
||||
|
||||
You need to add scrape configuration directly to the vmagent spec.inlineScrapeConfig. It is raw text in YAML format.
|
||||
See example below
|
||||
|
||||
```yaml
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: example-vmagent
|
||||
spec:
|
||||
serviceScrapeSelector: {}
|
||||
replicas: 1
|
||||
serviceAccountName: vmagent
|
||||
inlineScrapeConfig: |
|
||||
- job_name: "prometheus"
|
||||
static_configs:
|
||||
- targets: ["localhost:9090"]
|
||||
remoteWrite:
|
||||
- url: "http://vmagent-example-vmsingle.default.svc:8429/api/v1/write"
|
||||
EOF
|
||||
```
|
||||
|
||||
**Note**: Do not use passwords and tokens with inlineScrapeConfig use Secret instead of
|
||||
|
||||
|
||||
## Define Additional Scrape Configuration as a Kubernetes Secret
|
||||
|
||||
You need to define Kubernetes Secret with a key.
|
||||
|
||||
The key is `prometheus-additional.yaml` in the example below
|
||||
|
||||
```yaml
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: additional-scrape-configs
|
||||
stringData:
|
||||
prometheus-additional.yaml: |
|
||||
- job_name: "prometheus"
|
||||
static_configs:
|
||||
- targets: ["localhost:9090"]
|
||||
EOF
|
||||
```
|
||||
|
||||
After that, you need to specify the secret's name and key in VMAgent CRD in `additionalScrapeConfigs` section
|
||||
|
||||
```yaml
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAgent
|
||||
metadata:
|
||||
name: example-vmagent
|
||||
spec:
|
||||
serviceScrapeSelector: {}
|
||||
replicas: 1
|
||||
serviceAccountName: vmagent
|
||||
additionalScrapeConfigs:
|
||||
name: additional-scrape-configs
|
||||
key: prometheus-additional.yaml
|
||||
remoteWrite:
|
||||
- url: "http://vmagent-example-vmsingle.default.svc:8429/api/v1/write"
|
||||
EOF
|
||||
```
|
||||
|
||||
**Note**: You can specify only one Secret in the VMAgent CRD configuration so use it for all additional scrape configurations.
|
||||
This page is deprecated.
|
||||
|
||||
Please see [Additional scrape configuration](./resources/vmagent.md#additional-scrape-configuration)
|
||||
section in VMAgent docs instead.
|
||||
|
|
|
|||
|
|
@ -1,15 +1,15 @@
|
|||
---
|
||||
sort: 16
|
||||
sort: 12
|
||||
weight: 12
|
||||
title: API Docs
|
||||
weight: 16
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 16
|
||||
aliases:
|
||||
- /operator/api.html
|
||||
weight: 12
|
||||
---
|
||||
|
||||
<!-- this doc autogenerated - don't edit it manually -->
|
||||
|
||||
# API Docs
|
||||
|
||||
This Document documents the types introduced by the VictoriaMetrics to be consumed by users.
|
||||
|
|
|
|||
|
|
@ -1,27 +1,27 @@
|
|||
---
|
||||
sort: 4
|
||||
weight: 4
|
||||
sort: 7
|
||||
weight: 7
|
||||
title: Authorization and exposing components
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 4
|
||||
aliases:
|
||||
- /operator/auth.html
|
||||
weight: 7
|
||||
---
|
||||
|
||||
# Authorization and exposing components
|
||||
|
||||
## Exposing components
|
||||
|
||||
CRD objects doesn't have `ingress` configuration.
|
||||
Instead, you can use [VMAuth](./resources/vmauth.md) as proxy between ingress-controller and VictoriaMetrics components.
|
||||
|
||||
CRD objects doesn't have `ingress` configuration. Instead, you can use `VMAuth` as proxy between ingress-controller and VM app components.
|
||||
It adds missing authorization and access control features and enforces it.
|
||||
It adds missing authorization and access control features and enforces it.
|
||||
|
||||
Access can be given with `VMUser` definition. It supports basic auth and bearer token authentication.
|
||||
Access can be given with [VMUser](./resources/vmuser.md) definition.
|
||||
|
||||
It supports basic auth and bearer token authentication:
|
||||
|
||||
```yaml
|
||||
cat << EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
|
|
@ -30,45 +30,43 @@ spec:
|
|||
userNamespaceSelector: {}
|
||||
userSelector: {}
|
||||
ingress: {}
|
||||
EOF
|
||||
unauthorizedAccessConfig: []
|
||||
```
|
||||
|
||||
Advanced configuration with cert-manager annotations:
|
||||
Advanced configuration with cert-manager annotations:
|
||||
|
||||
```yaml
|
||||
cat << EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMAuth
|
||||
metadata:
|
||||
name: router-main
|
||||
name: router-main
|
||||
spec:
|
||||
podMetadata:
|
||||
labels:
|
||||
component: vmauth
|
||||
userSelector: {}
|
||||
userNamespaceSelector: {}
|
||||
replicaCount: 2
|
||||
resources:
|
||||
requests:
|
||||
cpu: "250m"
|
||||
memory: "350Mi"
|
||||
limits:
|
||||
cpu: "500m"
|
||||
memory: "850Mi"
|
||||
ingress:
|
||||
tlsSecretName: vmauth-tls
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: base
|
||||
class_name: nginx
|
||||
tlsHosts:
|
||||
- vm-access.example.com
|
||||
EOF
|
||||
podMetadata:
|
||||
labels:
|
||||
component: vmauth
|
||||
userSelector: {}
|
||||
userNamespaceSelector: {}
|
||||
replicaCount: 2
|
||||
resources:
|
||||
requests:
|
||||
cpu: "250m"
|
||||
memory: "350Mi"
|
||||
limits:
|
||||
cpu: "500m"
|
||||
memory: "850Mi"
|
||||
ingress:
|
||||
tlsSecretName: vmauth-tls
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: base
|
||||
class_name: nginx
|
||||
tlsHosts:
|
||||
- vm-access.example.com
|
||||
```
|
||||
|
||||
|
||||
simple static routing with read-only access to vmagent for username - `user-1` with password `Asafs124142`
|
||||
Simple static routing with read-only access to vmagent for username - `user-1` with password `Asafs124142`:
|
||||
|
||||
```yaml
|
||||
# curl vmauth:8427/metrics -u 'user-1:Asafs124142'
|
||||
cat << EOF | kubectl apply -f
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMUser
|
||||
metadata:
|
||||
|
|
@ -79,14 +77,12 @@ spec:
|
|||
- static:
|
||||
url: http://vmagent-base.default.svc:8429
|
||||
paths: ["/targets/api/v1","/targets","/metrics"]
|
||||
EOF
|
||||
```
|
||||
|
||||
With bearer token access:
|
||||
With bearer token access:
|
||||
|
||||
```yaml
|
||||
# curl vmauth:8427/metrics -H 'Authorization: Bearer Asafs124142'
|
||||
cat << EOF | kubectl apply -f
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMUser
|
||||
metadata:
|
||||
|
|
@ -97,13 +93,12 @@ spec:
|
|||
- static:
|
||||
url: http://vmagent-base.default.svc:8429
|
||||
paths: ["/targets/api/v1","/targets","/metrics"]
|
||||
EOF
|
||||
```
|
||||
|
||||
It's also possible to use service discovery for objects:
|
||||
It's also possible to use service discovery for objects:
|
||||
|
||||
```yaml
|
||||
# curl vmauth:8427/metrics -H 'Authorization: Bearer Asafs124142'
|
||||
cat << EOF | kubectl apply -f
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMUser
|
||||
metadata:
|
||||
|
|
@ -116,12 +111,11 @@ spec:
|
|||
name: base
|
||||
namespace: default
|
||||
paths: ["/targets/api/v1","/targets","/metrics"]
|
||||
EOF
|
||||
```
|
||||
|
||||
Cluster components supports auto path generation for single tenant view:
|
||||
Cluster components supports auto path generation for single tenant view:
|
||||
|
||||
```yaml
|
||||
cat << EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMUser
|
||||
metadata:
|
||||
|
|
@ -143,17 +137,15 @@ spec:
|
|||
url: http://vmselect-test-persistent.default.svc:8481/
|
||||
paths:
|
||||
- /internal/resetRollupResultCache
|
||||
EOF
|
||||
```
|
||||
|
||||
For each `VMUser` operator generates corresponding secret with username/password or bearer token at the same namespace as `VMUser`.
|
||||
For each `VMUser` operator generates corresponding secret with username/password or bearer token at the same namespace as `VMUser`.
|
||||
|
||||
## Basic auth for targets
|
||||
|
||||
To authenticate a `VMServiceScrape`s over a metrics endpoint use [`basicAuth`](https://docs.victoriametrics.com/operator/api.html#basicauth)
|
||||
To authenticate a `VMServiceScrape`s over a metrics endpoint use [`basicAuth`](./api.md#basicauth):
|
||||
|
||||
```yaml
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMServiceScrape
|
||||
metadata:
|
||||
|
|
@ -173,11 +165,9 @@ spec:
|
|||
selector:
|
||||
matchLabels:
|
||||
app: myapp
|
||||
EOF
|
||||
```
|
||||
|
||||
```yaml
|
||||
cat <<EOF | kubectl apply -f -
|
||||
---
|
||||
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
|
|
@ -186,5 +176,14 @@ data:
|
|||
password: dG9vcg== # toor
|
||||
user: YWRtaW4= # admin
|
||||
type: Opaque
|
||||
EOF
|
||||
```
|
||||
|
||||
## Unauthorized access
|
||||
|
||||
You can expose some routes without authorization with `unauthorizedAccessConfig`.
|
||||
|
||||
Check more details in [VMAuth docs -> Unauthorized access](./resources/vmauth.md#unauthorized-access).
|
||||
|
||||
More details about features of `VMAuth` and `VMUser` you can read in:
|
||||
- [VMAuth docs](./resources/vmauth.md),
|
||||
- [VMUser docs](./resources/vmuser.md).
|
||||
|
|
|
|||
|
|
@ -1,155 +1,10 @@
|
|||
---
|
||||
sort: 5
|
||||
weight: 5
|
||||
title: Backups
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 5
|
||||
sort: -1 # hide page from menu
|
||||
aliases:
|
||||
- /operator/backups.html
|
||||
---
|
||||
|
||||
# Backups
|
||||
This page is deprecated.
|
||||
|
||||
## vmbackupmanager
|
||||
|
||||
You can check vmbackupmanager [documentation](https://docs.victoriametrics.com/vmbackupmanager.html). It contains a description of the service and its features. This documentation covers vmbackumanager integration in vmoperator
|
||||
|
||||
### vmbackupmanager is a part of VictoriaMetrics Enterprise offer
|
||||
|
||||
*Before using it, you must have signed contract and accept ESA https://victoriametrics.com/legal/esa/*
|
||||
|
||||
## Usage examples
|
||||
|
||||
`VMSingle` and `VMCluster` has built-in backup configuration, it uses `vmbackupmanager` - proprietary tool for backups.
|
||||
It supports incremental backups (hourly, daily, weekly, monthly) with popular object storages (aws s3, google cloud storage).
|
||||
|
||||
The configuration example is the following:
|
||||
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: remote-storage-keys
|
||||
type: Opaque
|
||||
stringData:
|
||||
credentials: |-
|
||||
[default]
|
||||
aws_access_key_id = your_access_key_id
|
||||
aws_secret_access_key = your_secret_access_key
|
||||
---
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMSingle
|
||||
metadata:
|
||||
name: example-vmsingle
|
||||
spec:
|
||||
# Add fields here
|
||||
retentionPeriod: "1"
|
||||
vmBackup:
|
||||
# This is Enterprise Package feature you need to have signed contract to use it
|
||||
# and accept the ESA https://victoriametrics.com/legal/esa/
|
||||
acceptEULA: true
|
||||
destination: "s3://your_bucket/folder"
|
||||
credentialsSecret:
|
||||
name: remote-storage-keys
|
||||
key: credentials
|
||||
---
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMCluster
|
||||
metadata:
|
||||
name: example-vmcluster-persistent
|
||||
spec:
|
||||
retentionPeriod: "4"
|
||||
replicationFactor: 2
|
||||
vmstorage:
|
||||
replicaCount: 2
|
||||
vmBackup:
|
||||
# This is Enterprise Package feature you need to have signed contract to use it
|
||||
# and accept the ESA https://victoriametrics.com/legal/esa/
|
||||
acceptEULA: true
|
||||
destination: "s3://your_bucket/folder"
|
||||
credentialsSecret:
|
||||
name: remote-storage-keys
|
||||
key: credentials
|
||||
```
|
||||
|
||||
|
||||
NOTE: for cluster version operator adds suffix for destination: `"s3://your_bucket/folder"`, it becomes `"s3://your_bucket/folder/$(POD_NAME)"`.
|
||||
It's needed to make consistent backups for each storage node.
|
||||
|
||||
You can read more about backup configuration options and mechanics [here](https://docs.victoriametrics.com/vmbackup.html)
|
||||
|
||||
Possible configuration options for backup crd can be found at [link](https://docs.victoriametrics.com/operator/api.html#vmbackup)
|
||||
|
||||
|
||||
## Restoring backups
|
||||
|
||||
There are several ways to restore with [vmrestore](https://docs.victoriametrics.com/vmrestore.html) or [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager.html).
|
||||
|
||||
|
||||
### Manually mounting disk
|
||||
You have to stop `VMSingle` by scaling it replicas to zero and manually restore data to the database directory.
|
||||
|
||||
Steps:
|
||||
1. edit `VMSingle` CRD, set replicaCount: 0
|
||||
1. wait until database stops
|
||||
1. ssh to some server, where you can mount `VMSingle` disk and mount it manually
|
||||
1. restore files with `vmrestore`
|
||||
1. umount disk
|
||||
1. edit `VMSingle` CRD, set replicaCount: 1
|
||||
1. wait database start
|
||||
|
||||
### Using VMRestore init container
|
||||
|
||||
1. add init container with vmrestore command to `VMSingle` CRD, example:
|
||||
|
||||
```yaml
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMSingle
|
||||
metadata:
|
||||
name: vmsingle-restored
|
||||
namespace: monitoring-system
|
||||
spec:
|
||||
initContainers:
|
||||
- name: vmrestore
|
||||
image: victoriametrics/vmrestore:latest
|
||||
volumeMounts:
|
||||
- mountPath: /victoria-metrics-data
|
||||
name: data
|
||||
- mountPath: /etc/vm/creds
|
||||
name: secret-remote-storage-keys
|
||||
readOnly: true
|
||||
args:
|
||||
- -storageDataPath=/victoria-metrics-data
|
||||
- -src=s3://your_bucket/folder/latest
|
||||
- -credsFilePath=/etc/vm/creds/credentials
|
||||
vmBackup:
|
||||
# This is Enterprise Package feature you need to have signed contract to use it
|
||||
# and accept the ESA https://victoriametrics.com/legal/esa/
|
||||
acceptEULA: true
|
||||
destination: "s3://your_bucket/folder"
|
||||
extraArgs:
|
||||
runOnStart: "true"
|
||||
image:
|
||||
repository: victoriametrics/vmbackupmanager
|
||||
tag: v1.83.0-enterprise
|
||||
credentialsSecret:
|
||||
name: remote-storage-keys
|
||||
key: credentials
|
||||
```
|
||||
1. apply it, and db will be restored from s3
|
||||
|
||||
1. remove initContainers and apply crd.
|
||||
|
||||
Note that using `VMRestore` will require adjusting `src` for each pod because restore will be handled per-pod.
|
||||
|
||||
### Using VMBackupmanager init container
|
||||
|
||||
Using VMBackupmanager restore in Kubernetes environment is described [here](https://docs.victoriametrics.com/vmbackupmanager.html#how-to-restore-in-kubernetes).
|
||||
|
||||
Advantages of using `VMBackupmanager` include:
|
||||
- automatic adjustment of `src` for each pod when backup is requested
|
||||
- graceful handling of case when no restore is required - `VMBackupmanager` will exit with successful status code and won't prevent pod from starting
|
||||
Please see [Backup automation for VMSingle](./resources/vmsingle.md#backup-automation) or
|
||||
[Backup automation for VMCluster](./resources/vmcluster.md#backup-automation) instead.
|
||||
|
|
|
|||
264
docs/operator/configuration.md
Normal file
264
docs/operator/configuration.md
Normal file
|
|
@ -0,0 +1,264 @@
|
|||
---
|
||||
sort: 4
|
||||
weight: 4
|
||||
title: Configuration
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 4
|
||||
---
|
||||
|
||||
# Configuration
|
||||
|
||||
Operator configured by env variables, list of it can be found
|
||||
on [Variables](./vars.md) page.
|
||||
|
||||
It defines default configuration options, like images for components, timeouts, features.
|
||||
|
||||
In addition, the operator has a special startup mode for outputting all variables, their types and default values.
|
||||
For instance, with this mode you can know versions of VM components, which are used by default:
|
||||
|
||||
```console
|
||||
./operator --printDefaults
|
||||
|
||||
# This application is configured via the environment. The following environment variables can be used:
|
||||
#
|
||||
# KEY TYPE DEFAULT REQUIRED DESCRIPTION
|
||||
# VM_USECUSTOMCONFIGRELOADER True or False false
|
||||
# VM_CUSTOMCONFIGRELOADERIMAGE String victoriametrics/operator:config-reloader-v0.32.0
|
||||
# VM_VMALERTDEFAULT_IMAGE String victoriametrics/vmalert
|
||||
# VM_VMALERTDEFAULT_VERSION String v1.93.3
|
||||
# VM_VMALERTDEFAULT_USEDEFAULTRESOURCES True or False true
|
||||
# VM_VMALERTDEFAULT_RESOURCE_LIMIT_MEM String 500Mi
|
||||
# VM_VMALERTDEFAULT_RESOURCE_LIMIT_CPU String 200m
|
||||
# ...
|
||||
```
|
||||
|
||||
You can choose output format for variables with `--printFormat` flag, possible values: `json`, `yaml`, `list` and `table` (default):
|
||||
|
||||
```console
|
||||
.operator --printDefaults --printFormat=json
|
||||
|
||||
# {
|
||||
# 'VM_USECUSTOMCONFIGRELOADER': 'false',
|
||||
# 'VM_CUSTOMCONFIGRELOADERIMAGE': 'victoriametrics/operator:config-reloader-v0.32.0',
|
||||
# 'VM_VMALERTDEFAULT_IMAGE': 'victoriametrics/vmalert',
|
||||
# 'VM_VMALERTDEFAULT_VERSION': 'v1.93.3',
|
||||
# ...
|
||||
# 'VM_FORCERESYNCINTERVAL': '60s',
|
||||
# 'VM_ENABLESTRICTSECURITY': 'true'
|
||||
# }
|
||||
```
|
||||
|
||||
## Conversion of prometheus-operator objects
|
||||
|
||||
You can read detailed instructions about configuring prometheus-objects conversion in [this document](./migration.md).
|
||||
|
||||
## Helm-charts
|
||||
|
||||
In [helm-charts](https://github.com/VictoriaMetrics/helm-charts) some important configuration parameters are implemented as separate flags in `values.yaml`:
|
||||
|
||||
### victoria-metrics-k8s-stack
|
||||
|
||||
For possible values refer to [parameters](https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-k8s-stack#parameters).
|
||||
|
||||
Also, checkout [here possible ENV variables](./vars.md) to configure operator behaviour.
|
||||
ENV variables can be set in the `victoria-metrics-operator.env` section.
|
||||
|
||||
```yaml
|
||||
# values.yaml
|
||||
|
||||
victoria-metrics-operator:
|
||||
image:
|
||||
# -- Image repository
|
||||
repository: victoriametrics/operator
|
||||
# -- Image tag
|
||||
tag: v0.35.0
|
||||
# -- Image pull policy
|
||||
pullPolicy: IfNotPresent
|
||||
|
||||
# -- Tells helm to remove CRD after chart remove
|
||||
cleanupCRD: true
|
||||
cleanupImage:
|
||||
repository: gcr.io/google_containers/hyperkube
|
||||
tag: v1.18.0
|
||||
pullPolicy: IfNotPresent
|
||||
|
||||
operator:
|
||||
# -- By default, operator converts prometheus-operator objects.
|
||||
disable_prometheus_converter: false
|
||||
# -- Compare-options and sync-options for prometheus objects converted by operator for properly use with ArgoCD
|
||||
prometheus_converter_add_argocd_ignore_annotations: false
|
||||
# -- Enables ownership reference for converted prometheus-operator objects,
|
||||
# it will remove corresponding victoria-metrics objects in case of deletion prometheus one.
|
||||
enable_converter_ownership: false
|
||||
# -- By default, operator creates psp for its objects.
|
||||
psp_auto_creation_enabled: true
|
||||
# -- Enables custom config-reloader, bundled with operator.
|
||||
# It should reduce vmagent and vmauth config sync-time and make it predictable.
|
||||
useCustomConfigReloader: false
|
||||
|
||||
# -- extra settings for the operator deployment. full list Ref: [https://github.com/VictoriaMetrics/operator/blob/master/vars.md](https://github.com/VictoriaMetrics/operator/blob/master/vars.md)
|
||||
env:
|
||||
# -- default version for vmsingle
|
||||
- name: VM_VMSINGLEDEFAULT_VERSION
|
||||
value: v1.43.0
|
||||
# -- container registry name prefix, e.g. docker.io
|
||||
- name: VM_CONTAINERREGISTRY
|
||||
value: ""
|
||||
# -- image for custom reloader (see the useCustomConfigReloader parameter)
|
||||
- name: VM_CUSTOMCONFIGRELOADERIMAGE
|
||||
value: victoriametrics/operator:config-reloader-v0.32.0
|
||||
|
||||
# By default, the operator will watch all the namespaces
|
||||
# If you want to override this behavior, specify the namespace it needs to watch separated by a comma.
|
||||
# Ex: my_namespace1,my_namespace2
|
||||
watchNamespace: ""
|
||||
|
||||
# Count of operator instances (can be increased for HA mode)
|
||||
replicaCount: 1
|
||||
|
||||
# -- VM operator log level
|
||||
# -- possible values: info and error.
|
||||
logLevel: "info"
|
||||
|
||||
# -- Resource object
|
||||
resources:
|
||||
{}
|
||||
# limits:
|
||||
# cpu: 120m
|
||||
# memory: 320Mi
|
||||
# requests:
|
||||
# cpu: 80m
|
||||
# memory: 120Mi
|
||||
```
|
||||
|
||||
### victoria-metrics-operator
|
||||
|
||||
For possible values refer to [parameters](https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-operator#parameters).
|
||||
|
||||
Also, checkout [here possible ENV variables](./vars.md) to configure operator behaviour.
|
||||
ENV variables can be set in the `env` section.
|
||||
|
||||
```yaml
|
||||
# values.yaml
|
||||
|
||||
image:
|
||||
# -- Image repository
|
||||
repository: victoriametrics/operator
|
||||
# -- Image tag
|
||||
tag: v0.35.0
|
||||
# -- Image pull policy
|
||||
pullPolicy: IfNotPresent
|
||||
|
||||
operator:
|
||||
# -- By default, operator converts prometheus-operator objects.
|
||||
disable_prometheus_converter: false
|
||||
# -- Compare-options and sync-options for prometheus objects converted by operator for properly use with ArgoCD
|
||||
prometheus_converter_add_argocd_ignore_annotations: false
|
||||
# -- Enables ownership reference for converted prometheus-operator objects,
|
||||
# it will remove corresponding victoria-metrics objects in case of deletion prometheus one.
|
||||
enable_converter_ownership: false
|
||||
# -- By default, operator creates psp for its objects.
|
||||
psp_auto_creation_enabled: true
|
||||
# -- Enables custom config-reloader, bundled with operator.
|
||||
# It should reduce vmagent and vmauth config sync-time and make it predictable.
|
||||
useCustomConfigReloader: false
|
||||
|
||||
# -- extra settings for the operator deployment. full list Ref: [https://github.com/VictoriaMetrics/operator/blob/master/vars.md](https://github.com/VictoriaMetrics/operator/blob/master/vars.md)
|
||||
env:
|
||||
# -- default version for vmsingle
|
||||
- name: VM_VMSINGLEDEFAULT_VERSION
|
||||
value: v1.43.0
|
||||
# -- container registry name prefix, e.g. docker.io
|
||||
- name: VM_CONTAINERREGISTRY
|
||||
value: ""
|
||||
# -- image for custom reloader (see the useCustomConfigReloader parameter)
|
||||
- name: VM_CUSTOMCONFIGRELOADERIMAGE
|
||||
value: victoriametrics/operator:config-reloader-v0.32.0
|
||||
|
||||
# By default, the operator will watch all the namespaces
|
||||
# If you want to override this behavior, specify the namespace it needs to watch separated by a comma.
|
||||
# Ex: my_namespace1,my_namespace2
|
||||
watchNamespace: ""
|
||||
|
||||
# Count of operator instances (can be increased for HA mode)
|
||||
replicaCount: 1
|
||||
|
||||
# -- VM operator log level
|
||||
# -- possible values: info and error.
|
||||
logLevel: "info"
|
||||
|
||||
# -- Resource object
|
||||
resources:
|
||||
{}
|
||||
# limits:
|
||||
# cpu: 120m
|
||||
# memory: 320Mi
|
||||
# requests:
|
||||
# cpu: 80m
|
||||
# memory: 120Mi
|
||||
```
|
||||
|
||||
## Namespaced mode
|
||||
|
||||
By default, the operator will watch all namespaces, but it can be configured to watch only specific namespace.
|
||||
|
||||
If you want to override this behavior, specify the namespace:
|
||||
|
||||
- in the `WATCH_NAMESPACE` environment variable.
|
||||
- in the `watchNamespace` field in the `values.yaml` file of helm-charts.
|
||||
|
||||
The operator supports only single namespace for watching.
|
||||
|
||||
You can find example of RBAC manifests for single-namespace mode in
|
||||
[this file](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/operator_rbac_for_single_namespace.yaml).
|
||||
|
||||
## Monitoring of cluster components
|
||||
|
||||
By default, operator creates [VMServiceScrape](./resources/vmservicescrape.md)
|
||||
object for each component that it manages.
|
||||
|
||||
You can disable this behaviour with `VM_DASABLESELFSERVICASCRAPECREATION` environment variable:
|
||||
|
||||
```shell
|
||||
VM_DASABLESELFSERVICASCRAPECREATION=false
|
||||
```
|
||||
|
||||
Also, you can override default configuration for self-scraping with `ServiceScrapeSpec` field in each deployable resource
|
||||
(`vmcluster/select`, `vmcluster/insert`, `vmcluster/storage`, `vmagent`, `vmalert`, `vmalertmanager`, `vmauth`, `vmsingle`):
|
||||
|
||||
## CRD Validation
|
||||
|
||||
Operator supports validation admission webhook [docs](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/)
|
||||
|
||||
It checks resources configuration and returns errors to caller before resource will be created at kubernetes api.
|
||||
This should reduce errors and simplify debugging.
|
||||
|
||||
Validation hooks at operator side must be enabled with flags:
|
||||
|
||||
```console
|
||||
./operator
|
||||
--webhook.enable
|
||||
# optional configuration for certDir and tls names.
|
||||
--webhook.certDir=/tmp/k8s-webhook-server/serving-certs/
|
||||
--webhook.keyName=tls.key
|
||||
--webhook.certName=tls.crt
|
||||
```
|
||||
|
||||
You have to mount correct certificates at give directory.
|
||||
It can be simplified with cert-manager and kustomize command:
|
||||
|
||||
```console
|
||||
kustomize build config/deployments/webhook/
|
||||
```
|
||||
|
||||
### Requirements
|
||||
|
||||
- Valid certificate with key must be provided to operator
|
||||
- Valid CABundle must be added to the `ValidatingWebhookConfiguration`
|
||||
|
||||
### Useful links
|
||||
|
||||
- [k8s admission webhooks](https://banzaicloud.com/blog/k8s-admission-webhooks/)
|
||||
- [olm webhooks](https://docs.openshift.com/container-platform/4.5/operators/user/olm-webhooks.html)
|
||||
|
|
@ -1,48 +1,10 @@
|
|||
---
|
||||
sort: 14
|
||||
weight: 14
|
||||
title: Configuration synchronization
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 14
|
||||
sort: -1 # hide page from menu
|
||||
aliases:
|
||||
- /operator/configuration_syncronization.html
|
||||
---
|
||||
|
||||
# Configuration synchronization
|
||||
This page is deprecated.
|
||||
|
||||
## Basic concepts
|
||||
VictoriaMetrics applications, like many other applications with configuration file deployed at Kubernetes, uses `ConfigMaps` and `Secrets` for configuration files.
|
||||
Usually, it's out of application scope to watch for configuration on-disk changes.
|
||||
Applications reload their configuration by a signal from a user or some other tool, that knows how to watch for updates.
|
||||
At Kubernetes, the most popular design for this case is a sidecar container, that watches for configuration file changes and sends an HTTP request to the application.
|
||||
|
||||
`Configmap` or `Secret` that mounted at `Pod` holds a copy of its content.
|
||||
Kubernetes component `kubelet` is responsible for content synchronization between an object at Kubernetes API and a file served on disk.
|
||||
It's not efficient to sync its content immediately, and `kubelet` eventually synchronizes it. There is a configuration option, that controls this period.
|
||||
|
||||
That's why, `VMAgent`, `VMAlert` and other applications managed by operator don't receive changes immediately. It usually takes 1-2 min, before content will be updated.
|
||||
|
||||
It may trigger errors when an application was deleted, but `VMAgent` still tries to scrape it.
|
||||
|
||||
## Possible mitigations
|
||||
|
||||
The naive solution for this case decrease the synchronization period. But it configures globally and may be hard for operator users.
|
||||
|
||||
That's why operator uses a few hacks.
|
||||
|
||||
For `ConfigMap` updates, operator changes annotation with a time of `Configmap` content update. It triggers `ConfigMap`'s content synchronization by kubelet immediately.
|
||||
It's the case for `VMAlert`, it uses `ConfigMap` as a configuration source.
|
||||
|
||||
For `Secret` it doesn't work. And operator offers its implementation for side-car container. It can be configured with env variable for operator:
|
||||
```
|
||||
- name: VM_USECUSTOMCONFIGRELOADER
|
||||
value: "true"
|
||||
```
|
||||
If it's defined, operator uses own [config-reloader](https://github.com/VictoriaMetrics/operator/tree/master/internal/config-reloader) instead of [prometheus-config-reload](https://github.com/prometheus-operator/prometheus-operator/tree/main/cmd/prometheus-config-reloader).
|
||||
|
||||
It watches corresponding `Secret` for changes with Kubernetes API watch call and writes content into emptyDir.
|
||||
This emptyDir shared with the application.
|
||||
In case of content changes, `config-reloader` sends HTTP requests to the application.
|
||||
It greatly reduces the time for configuration synchronization.
|
||||
Please see [Configuration synchronization](./resources/README.md#configuration-synchronization)
|
||||
in Resources section instead.
|
||||
|
|
|
|||
|
|
@ -1,241 +1,9 @@
|
|||
---
|
||||
sort: 6
|
||||
weight: 6
|
||||
title: Design
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 6
|
||||
sort: -1 # hide page from menu
|
||||
aliases:
|
||||
- /operator/design.html
|
||||
---
|
||||
|
||||
# Design
|
||||
This page is deprecated.
|
||||
|
||||
This document describes the design and interaction between the custom resource definitions (CRD) that the Victoria
|
||||
Metrics Operator introduces.
|
||||
|
||||
Operator introduces the following custom resources:
|
||||
|
||||
* [VMSingle](#vmsingle)
|
||||
* [VMCluster](#vmcluster)
|
||||
* [VMAgent](#vmagent)
|
||||
* [VMAlert](#vmalert)
|
||||
* [VMServiceScrape](#vmservicescrape)
|
||||
* [VMPodScrape](#vmpodscrape)
|
||||
* [VMAlertmanager](#vmalertmanager)
|
||||
* [VMAlertmanagerConfig](#vmalertmanagerconfig)
|
||||
* [VMRule](#vmrule)
|
||||
* [VMPrometheusConverter](#vmprometheusconverter)
|
||||
* [VMProbe](#vmprobe)
|
||||
* [VMNodeScrape](#vmnodescrape)
|
||||
* [VMStaticScrape](#vmstaticscrape)
|
||||
* [VMAuth](#vmauth)
|
||||
* [VMUser](#vmuser)
|
||||
|
||||
## VMSingle
|
||||
|
||||
The `VMSingle` CRD declaratively defines a [single-node VM](https://github.com/VictoriaMetrics/VictoriaMetrics)
|
||||
installation to run in a Kubernetes cluster.
|
||||
|
||||
For each `VMSingle` resource, the Operator deploys a properly configured `Deployment` in the same namespace.
|
||||
The VMSingle `Pod`s are configured to mount an empty dir or `PersistentVolumeClaimSpec` for storing data.
|
||||
Deployment update strategy set to [recreate](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#recreate-deployment).
|
||||
No more than one replica allowed.
|
||||
|
||||
For each `VMSingle` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
|
||||
name `<VMSingle-name>`.
|
||||
|
||||
## VMCluster
|
||||
|
||||
The `VMCluster` CRD defines a [cluster version VM](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster).
|
||||
|
||||
For each `VMCluster` resource, the Operator creates `VMStorage` as `StatefulSet`, `VMSelect` as `StatefulSet` and `VMInsert`
|
||||
as deployment. For `VMStorage` and `VMSelect` headless services are created. `VMInsert` is created as service with clusterIP.
|
||||
|
||||
There is a strict order for these objects creation and reconciliation:
|
||||
1. `VMStorage` is synced - the Operator waits until all its pods are ready;
|
||||
1. Then it syncs `VMSelect` with the same manner;
|
||||
1. `VMInsert` is the last object to sync.
|
||||
|
||||
All statefulsets are created with [OnDelete](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#on-delete)
|
||||
update type. It allows to manually manage the rolling update process for Operator by deleting pods one by one and waiting
|
||||
for the ready status.
|
||||
|
||||
Rolling update process may be configured by the operator env variables.
|
||||
The most important is `VM_PODWAITREADYTIMEOUT=80s` - it controls how long to wait for pod's ready status.
|
||||
|
||||
## VMAgent
|
||||
|
||||
The `VMAgent` CRD declaratively defines a desired [VMAgent](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmagent)
|
||||
setup to run in a Kubernetes cluster.
|
||||
|
||||
For each `VMAgent` resource Operator deploys a properly configured `Deployment` in the same namespace.
|
||||
The VMAgent `Pod`s are configured to mount a `Secret` prefixed with `<VMAgent-name>` containing the configuration
|
||||
for VMAgent.
|
||||
|
||||
For each `VMAgent` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
|
||||
name `<VMAgent-name>`.
|
||||
|
||||
The CRD specifies which `VMServiceScrape` should be covered by the deployed VMAgent instances based on label selection.
|
||||
The Operator then generates a configuration based on the included `VMServiceScrape`s and updates the `Secret` which
|
||||
contains the configuration. It continuously does so for all changes that are made to the `VMServiceScrape`s or the
|
||||
`VMAgent` resource itself.
|
||||
|
||||
If no selection of `VMServiceScrape`s is provided - Operator leaves management of the `Secret` to the user,
|
||||
so user can set custom configuration while still benefiting from the Operator's capabilities of managing VMAgent setups.
|
||||
|
||||
## VMAlert
|
||||
|
||||
The `VMAlert` CRD declaratively defines a desired [VMAlert](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmalert)
|
||||
setup to run in a Kubernetes cluster.
|
||||
|
||||
For each `VMAlert` resource, the Operator deploys a properly configured `Deployment` in the same namespace.
|
||||
The VMAlert `Pod`s are configured to mount a list of `Configmaps` prefixed with `<VMAlert-name>-number` containing
|
||||
the configuration for alerting rules.
|
||||
|
||||
For each `VMAlert` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
|
||||
name `<VMAlert-name>`.
|
||||
|
||||
The CRD specifies which `VMRule`s should be covered by the deployed VMAlert instances based on label selection.
|
||||
The Operator then generates a configuration based on the included `VMRule`s and updates the `Configmaps` containing
|
||||
the configuration. It continuously does so for all changes that are made to `VMRule`s or to the `VMAlert` resource itself.
|
||||
|
||||
Alerting rules are filtered by selector `ruleNamespaceSelector` in `VMAlert` CRD definition. For selecting rules from all
|
||||
namespaces you must specify it to empty value:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
ruleNamespaceSelector: {}
|
||||
```
|
||||
|
||||
|
||||
## VMServiceScrape
|
||||
|
||||
The `VMServiceScrape` CRD allows to define a dynamic set of services for monitoring. Services
|
||||
and scraping configurations can be matched via label selections. This allows an organization to introduce conventions
|
||||
for how metrics should be exposed. Following these conventions new services will be discovered automatically without
|
||||
need to reconfigure.
|
||||
|
||||
Monitoring configuration based on `discoveryRole` setting. By default, `endpoints` is used to get objects from kubernetes api.
|
||||
It's also possible to use `discoveryRole: service` or `discoveryRole: endpointslices`
|
||||
|
||||
`Endpoints` objects are essentially lists of IP addresses.
|
||||
Typically, `Endpoints` objects are populated by `Service` object. `Service` object discovers `Pod`s by a label
|
||||
selector and adds those to the `Endpoints` object.
|
||||
|
||||
A `Service` may expose one or more service ports backed by a list of one or multiple endpoints pointing to
|
||||
specific `Pod`s. The same reflected in the respective `Endpoints` object as well.
|
||||
|
||||
The `VMServiceScrape` object discovers `Endpoints` objects and configures VMAgent to monitor `Pod`s.
|
||||
|
||||
The `Endpoints` section of the `VMServiceScrapeSpec` is used to configure which `Endpoints` ports should be scraped.
|
||||
For advanced use cases, one may want to monitor ports of backing `Pod`s, which are not a part of the service endpoints.
|
||||
Therefore, when specifying an endpoint in the `endpoints` section, they are strictly used.
|
||||
|
||||
> Note: `endpoints` (lowercase) is the field in the `VMServiceScrape` CRD, while `Endpoints` (capitalized) is the Kubernetes object kind.
|
||||
|
||||
Both `VMServiceScrape` and discovered targets may belong to any namespace. It is important for cross-namespace monitoring
|
||||
use cases, e.g. for meta-monitoring. Using the `serviceScrapeSelector` of the `VMAgentSpec`
|
||||
one can restrict the namespaces from which `VMServiceScrape`s are selected from by the respective VMAgent server.
|
||||
Using the `namespaceSelector` of the `VMServiceScrape` one can restrict the namespaces from which `Endpoints` can be
|
||||
discovered from. To discover targets in all namespaces the `namespaceSelector` has to be empty:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
namespaceSelector: {}
|
||||
```
|
||||
|
||||
## VMPodScrape
|
||||
|
||||
The `VMPodScrape` CRD allows to declaratively define how a dynamic set of pods should be monitored.
|
||||
Use label selections to match pods for scraping. This allows an organization to introduce conventions
|
||||
for how metrics should be exposed. Following these conventions new services will be discovered automatically without
|
||||
need to reconfigure.
|
||||
|
||||
A `Pod` is a collection of one or more containers which can expose Prometheus metrics on a number of ports.
|
||||
|
||||
The `VMPodScrape` object discovers pods and generates the relevant scraping configuration.
|
||||
|
||||
The `PodMetricsEndpoints` section of the `VMPodScrapeSpec` is used to configure which ports of a pod are going to be
|
||||
scraped for metrics and with which parameters.
|
||||
|
||||
Both `VMPodScrapes` and discovered targets may belong to any namespace. It is important for cross-namespace monitoring
|
||||
use cases, e.g. for meta-monitoring. Using the `namespaceSelector` of the `VMPodScrapeSpec` one can restrict the
|
||||
namespaces from which `Pods` are discovered from. To discover targets in all namespaces the `namespaceSelector` has to
|
||||
be empty:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
namespaceSelector:
|
||||
any: true
|
||||
```
|
||||
|
||||
## VMAlertmanager
|
||||
|
||||
The `VMAlertmanager` CRD declaratively defines a desired Alertmanager setup to run in a Kubernetes cluster.
|
||||
It provides options to configure replication and persistent storage.
|
||||
|
||||
For each `Alertmanager` resource, the Operator deploys a properly configured `StatefulSet` in the same namespace.
|
||||
The Alertmanager pods are configured to include a `Secret` called `<alertmanager-name>` which holds the used
|
||||
configuration file in the key `alertmanager.yaml`.
|
||||
|
||||
When there are two or more configured replicas the Operator runs the Alertmanager instances in high availability mode.
|
||||
|
||||
## VMAlertmanagerConfig
|
||||
|
||||
The `VMAlertmanagerConfig` provides way to configure `VMAlertmanager` configuration with CRD. It allows to define different configuration parts,
|
||||
which will be merged by operator into config. It behaves like other config parts - `VMServiceScrape` and etc.
|
||||
|
||||
## VMRule
|
||||
|
||||
The `VMRule` CRD declaratively defines a desired Prometheus rule to be consumed by one or more VMAlert instances.
|
||||
|
||||
Alerts and recording rules can be saved and applied as YAML files, and dynamically loaded without requiring any restart.
|
||||
|
||||
|
||||
## VMPrometheusConverter
|
||||
|
||||
By default, the Operator converts and updates existing prometheus-operator API objects:
|
||||
|
||||
`ServiceMonitor` into `VMServiceScrape`
|
||||
`PodMonitor` into `VMPodScrape`
|
||||
`PrometheusRule` into `VMRule`
|
||||
`Probe` into `VMProbe`
|
||||
Removing prometheus-operator API objects wouldn't delete any converted objects. So you can safely migrate or run
|
||||
two operators at the same time.
|
||||
|
||||
|
||||
## VMProbe
|
||||
|
||||
The `VMProbe` CRD provides probing target ability with a prober. The most common prober is [blackbox exporter](https://github.com/prometheus/blackbox_exporter).
|
||||
By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. It's possible to use static targets
|
||||
or use standard k8s discovery mechanism with `Ingress`.
|
||||
You have to configure blackbox exporter before you can use this feature. The second requirement is `VMAgent` selectors,
|
||||
it must match your `VMProbe` by label or namespace selector. `VMAgent` probeSelector must match `VMProbe` labels.
|
||||
See more details about selectors [here](https://docs.victoriametrics.com/operator/quick-start.html#object-selectors).
|
||||
|
||||
## VMNodeScrape
|
||||
|
||||
The `VMNodeScrape` CRD provides discovery mechanism for scraping metrics kubernetes nodes.
|
||||
By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. It's useful for cadvisor scraping,
|
||||
node-exporter or other node-based exporters. `VMAgent` nodeScrapeSelector must match `VMNodeScrape` labels.
|
||||
See more details about selectors [here](https://docs.victoriametrics.com/operator/quick-start.html#object-selectors).
|
||||
|
||||
## VMStaticScrape
|
||||
|
||||
The `VMStaticScrape` CRD provides mechanism for scraping metrics from static targets, configured by CRD targets.
|
||||
By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. It's useful for external targets management,
|
||||
when service-discovery is not available. `VMAgent` staticScrapeSelector must match `VMStaticScrape` labels.
|
||||
See more details about selectors [here](https://docs.victoriametrics.com/operator/quick-start.html#object-selectors).
|
||||
|
||||
## VMAuth
|
||||
|
||||
The `VMAuth` CRD provides mechanism for exposing application with authorization to outside world or to other applications inside kubernetes cluster.
|
||||
For first case, user can configure `ingress` setting at `VMAuth` CRD. For second one, operator will create secret with `username` and `password` at `VMUser` CRD name.
|
||||
So it will be possible to access these credentials from any application by targeting corresponding kubernetes secret.
|
||||
|
||||
## VMUser
|
||||
|
||||
The `VMUser` CRD describes user configuration, its authentication methods `basic auth` or `Authorization` header. User access permissions, with possible routing information.
|
||||
User can define routing target with `static` config, by entering target `url`, or with `CRDRef`, in this case, operator queries kubernetes API, retrieves information about CRD and builds proper url.
|
||||
Please see [Custom Resources](./resources/README.md) instead.
|
||||
|
|
|
|||
40
docs/operator/enterprise.md
Normal file
40
docs/operator/enterprise.md
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
sort: 13
|
||||
weight: 13
|
||||
title: Enterprise features
|
||||
menu:
|
||||
docs:
|
||||
parent: "operator"
|
||||
weight: 13
|
||||
---
|
||||
|
||||
# Using operator with enterprise features
|
||||
|
||||
Operator doesn't have enterprise version for itself, but it supports
|
||||
[enterprise features for VictoriaMetrics components](https://docs.victoriametrics.com/enterprise.html):
|
||||
|
||||
- [VMAgent Enterprise features](./resources/vmagent.md#enterprise-features):
|
||||
- [Reading metrics from kafka](./resources/vmagent.md#reading-metrics-from-kafka)
|
||||
- [Writing metrics to kafka](./resources/vmagent.md#writing-metrics-to-kafka)
|
||||
- [VMAlert Enterprise features](./resources/vmalert.md#enterprise-features):
|
||||
- [Reading rules from object storage](./resources/vmalert.md#reading-rules-from-object-storage)
|
||||
- [Multitenancy](./resources/vmalert.md#multitenancy)
|
||||
- [VMAuth Enterprise features](./resources/vmauth.md#enterprise-features)
|
||||
- [IP Filters](./resources/vmauth.md#ip-filters)
|
||||
- [VMCluster Enterprise features](./resources/vmcluster.md#enterprise-features)
|
||||
- [Downsampling](./resources/vmcluster.md#downsampling)
|
||||
- [Multiple retentions / Retention filters](./resources/vmcluster.md#retention-filters)
|
||||
- [Advanced per-tenant statistic](./resources/vmcluster.md#advanced-per-tenant-statistic)
|
||||
- [mTLS protection](./resources/vmcluster.md#mtls-protection)
|
||||
- [Backup atomation](./resources/vmcluster.md#backup-atomation)
|
||||
- [VMRule Enterprise features](./resources/vmrule.md#enterprise-features)
|
||||
- [Multitenancy](./resources/vmrule.md#multitenancy)
|
||||
- [VMSingle Enterprise features](./resources/vmsingle.md#enterprise-features)
|
||||
- [Downsampling](./resources/vmsingle.md#downsampling)
|
||||
- [Retention filters](./resources/vmsingle.md#retention-filters)
|
||||
- [Backup atomation](./resources/vmsingle.md#backup-atomation)
|
||||
- [VMUser Enterprise features](./resources/vmuser.md#enterprise-features)
|
||||
- [IP Filters](./resources/vmuser.md#ip-filters)
|
||||
|
||||
More information about enterprise features you can read
|
||||
on [VictoriaMetrics Enterprise page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue