/metrics` page. Consider limiting user's access
-to the web UI or `/metrics` page if this information is sensitive.
-
-[Alerts state](#alerts-state) page or [debug mode](#debug-mode) could emit additional information about configured
-datasource URL, GET params and headers. Sensitive information such as passwords or auth tokens is stripped by default.
-To disable stripping of such info pass `-datasource.showURL` cmd-line flag to vmalert.
-
-
-## Profiling
-
-`vmalert` provides handlers for collecting the following [Go profiles](https://blog.golang.org/profiling-go-programs):
-
-* Memory profile. It can be collected with the following command (replace `0.0.0.0` with hostname if needed):
-
-
-
-```console
-curl http://0.0.0.0:8880/debug/pprof/heap > mem.pprof
-```
-
-
-
-* CPU profile. It can be collected with the following command (replace `0.0.0.0` with hostname if needed):
-
-
-
-```console
-curl http://0.0.0.0:8880/debug/pprof/profile > cpu.pprof
-```
-
-
-
-The command for collecting CPU profile waits for 30 seconds before returning.
-
-The collected profiles may be analyzed with [go tool pprof](https://github.com/google/pprof).
-It is safe sharing the collected profiles from security point of view, since they do not contain sensitive information.
-
-## Configuration
-
-### Flags
-
-Pass `-help` to `vmalert` in order to see the full list of supported
-command-line flags with their descriptions.
-
-The shortlist of configuration flags is the following:
-{% raw %}
-```console
- -clusterMode
- If clusterMode is enabled, then vmalert automatically adds the tenant specified in config groups to -datasource.url, -remoteWrite.url and -remoteRead.url. See https://docs.victoriametrics.com/vmalert.html#multitenancy . This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -configCheckInterval duration
- Interval for checking for changes in '-rule' or '-notifier.config' files. By default, the checking is disabled. Send SIGHUP signal in order to force config check for changes.
- -datasource.appendTypePrefix
- Whether to add type prefix to -datasource.url based on the query type. Set to true if sending different query types to the vmselect URL.
- -datasource.basicAuth.password string
- Optional basic auth password for -datasource.url
- -datasource.basicAuth.passwordFile string
- Optional path to basic auth password to use for -datasource.url
- -datasource.basicAuth.username string
- Optional basic auth username for -datasource.url
- -datasource.bearerToken string
- Optional bearer auth token to use for -datasource.url.
- -datasource.bearerTokenFile string
- Optional path to bearer token file to use for -datasource.url.
- -datasource.disableKeepAlive
- Whether to disable long-lived connections to the datasource. If true, disables HTTP keep-alives and will only use the connection to the server for a single HTTP request.
- -datasource.disableStepParam
- Whether to disable adding 'step' param to the issued instant queries. This might be useful when using vmalert with datasources that do not support 'step' param for instant queries, like Google Managed Prometheus. It is not recommended to enable this flag if you use vmalert with VictoriaMetrics.
- -datasource.headers string
- Optional HTTP extraHeaders to send with each request to the corresponding -datasource.url. For example, -datasource.headers='My-Auth:foobar' would send 'My-Auth: foobar' HTTP header with every request to the corresponding -datasource.url. Multiple headers must be delimited by '^^': -datasource.headers='header1:value1^^header2:value2'
- -datasource.lookback duration
- Will be deprecated soon, please adjust "-search.latencyOffset" at datasource side or specify "latency_offset" in rule group's params. Lookback defines how far into the past to look when evaluating queries. For example, if the datasource.lookback=5m then param "time" with value now()-5m will be added to every query.
- -datasource.maxIdleConnections int
- Defines the number of idle (keep-alive connections) to each configured datasource. Consider setting this value equal to the value: groups_total * group.concurrency. Too low a value may result in a high number of sockets in TIME_WAIT state. (default 100)
- -datasource.oauth2.clientID string
- Optional OAuth2 clientID to use for -datasource.url.
- -datasource.oauth2.clientSecret string
- Optional OAuth2 clientSecret to use for -datasource.url.
- -datasource.oauth2.clientSecretFile string
- Optional OAuth2 clientSecretFile to use for -datasource.url.
- -datasource.oauth2.scopes string
- Optional OAuth2 scopes to use for -datasource.url. Scopes must be delimited by ';'
- -datasource.oauth2.tokenUrl string
- Optional OAuth2 tokenURL to use for -datasource.url.
- -datasource.queryStep duration
- How far a value can fallback to when evaluating queries. For example, if -datasource.queryStep=15s then param "step" with value "15s" will be added to every query. If set to 0, rule's evaluation interval will be used instead. (default 5m0s)
- -datasource.queryTimeAlignment
- Deprecated: please use "eval_alignment" in rule group instead. Whether to align "time" parameter with evaluation interval. Alignment supposed to produce deterministic results despite number of vmalert replicas or time they were started. See more details at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1257 (default true)
- -datasource.roundDigits int
- Adds "round_digits" GET param to datasource requests. In VM "round_digits" limits the number of digits after the decimal point in response values.
- -datasource.showURL
- Whether to avoid stripping sensitive information such as auth headers or passwords from URLs in log messages or UI and exported metrics. It is hidden by default, since it can contain sensitive info such as auth key
- -datasource.tlsCAFile string
- Optional path to TLS CA file to use for verifying connections to -datasource.url. By default, system CA is used
- -datasource.tlsCertFile string
- Optional path to client-side TLS certificate file to use when connecting to -datasource.url
- -datasource.tlsInsecureSkipVerify
- Whether to skip tls verification when connecting to -datasource.url
- -datasource.tlsKeyFile string
- Optional path to client-side TLS certificate key to use when connecting to -datasource.url
- -datasource.tlsServerName string
- Optional TLS server name to use for connections to -datasource.url. By default, the server name from -datasource.url is used
- -datasource.url string
- Datasource compatible with Prometheus HTTP API. It can be single node VictoriaMetrics or vmselect URL. Required parameter. E.g. http://127.0.0.1:8428 . See also -remoteRead.disablePathAppend and -datasource.showURL
- -defaultTenant.graphite string
- Default tenant for Graphite alerting groups. See https://docs.victoriametrics.com/vmalert.html#multitenancy .This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -defaultTenant.prometheus string
- Default tenant for Prometheus alerting groups. See https://docs.victoriametrics.com/vmalert.html#multitenancy . This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -disableAlertgroupLabel
- Whether to disable adding group's Name as label to generated alerts and time series.
- -dryRun
- Whether to check only config files without running vmalert. The rules file are validated. The -rule flag must be specified.
- -enableTCP6
- Whether to enable IPv6 for listening and dialing. By default, only IPv4 TCP and UDP are used
- -envflag.enable
- Whether to enable reading flags from environment variables in addition to the command line. Command line flag values have priority over values from environment vars. Flags are read only from the command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
- -envflag.prefix string
- Prefix for environment variables if -envflag.enable is set
- -eula
- Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -evaluationInterval duration
- How often to evaluate the rules (default 1m0s)
- -external.alert.source string
- External Alert Source allows to override the Source link for alerts sent to AlertManager for cases where you want to build a custom link to Grafana, Prometheus or any other service. Supports templating - see https://docs.victoriametrics.com/vmalert.html#templating . For example, link to Grafana: -external.alert.source='explore?orgId=1&left={"datasource":"VictoriaMetrics","queries":[{"expr":{{$expr|jsonEscape|queryEscape}},"refId":"A"}],"range":{"from":"now-1h","to":"now"}}'. Link to VMUI: -external.alert.source='vmui/#/?g0.expr={{.Expr|queryEscape}}'. If empty 'vmalert/alert?group_id={{.GroupID}}&alert_id={{.AlertID}}' is used.
- -external.label array
- Optional label in the form 'Name=value' to add to all generated recording rules and alerts. Pass multiple -label flags in order to add multiple label sets.
- Supports an array of values separated by comma or specified via multiple flags.
- -external.url string
- External URL is used as alert's source for sent alerts to the notifier. By default, hostname is used as address.
- -filestream.disableFadvise
- Whether to disable fadvise() syscall when reading large data files. The fadvise() syscall prevents from eviction of recently accessed data from OS page cache during background merges and backups. In some rare cases it is better to disable the syscall if it uses too much CPU
- -flagsAuthKey string
- Auth key for /flags endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -fs.disableMmap
- Whether to use pread() instead of mmap() for reading data files. By default, mmap() is used for 64-bit arches and pread() is used for 32-bit arches, since they cannot read data files bigger than 2^32 bytes in memory. mmap() is usually faster for reading small data chunks than pread()
- -http.connTimeout duration
- Incoming http connections are closed after the configured timeout. This may help to spread the incoming load among a cluster of services behind a load balancer. Please note that the real timeout may be bigger by up to 10% as a protection against the thundering herd problem (default 2m0s)
- -http.disableResponseCompression
- Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
- -http.header.csp string
- Value for 'Content-Security-Policy' header
- -http.header.frameOptions string
- Value for 'X-Frame-Options' header
- -http.header.hsts string
- Value for 'Strict-Transport-Security' header
- -http.idleConnTimeout duration
- Timeout for incoming idle http connections (default 1m0s)
- -http.maxGracefulShutdownDuration duration
- The maximum duration for a graceful shutdown of the HTTP server. A highly loaded server may require increased value for a graceful shutdown (default 7s)
- -http.pathPrefix string
- An optional prefix to add to all the paths handled by http server. For example, if '-http.pathPrefix=/foo/bar' is set, then all the http requests will be handled on '/foo/bar/*' paths. This may be useful for proxied requests. See https://www.robustperception.io/using-external-urls-and-proxies-with-prometheus
- -http.shutdownDelay duration
- Optional delay before http server shutdown. During this delay, the server returns non-OK responses from /health page, so load balancers can route new requests to other servers
- -httpAuth.password string
- Password for HTTP server's Basic Auth. The authentication is disabled if -httpAuth.username is empty
- -httpAuth.username string
- Username for HTTP server's Basic Auth. The authentication is disabled if empty. See also -httpAuth.password
- -httpListenAddr string
- Address to listen for http connections. See also -httpListenAddr.useProxyProtocol (default ":8880")
- -httpListenAddr.useProxyProtocol
- Whether to use proxy protocol for connections accepted at -httpListenAddr . See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt . With enabled proxy protocol http server cannot serve regular /metrics endpoint. Use -pushmetrics.url for metrics pushing
- -internStringCacheExpireDuration duration
- The expiry duration for caches for interned strings. See https://en.wikipedia.org/wiki/String_interning . See also -internStringMaxLen and -internStringDisableCache (default 6m0s)
- -internStringDisableCache
- Whether to disable caches for interned strings. This may reduce memory usage at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringCacheExpireDuration and -internStringMaxLen
- -internStringMaxLen int
- The maximum length for strings to intern. A lower limit may save memory at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringDisableCache and -internStringCacheExpireDuration (default 500)
- -license string
- Lisense key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed via file specified by -licenseFile command-line flag
- -license.forceOffline
- Whether to enable offline verification for VictoriaMetrics Enterprise license key, which has been passed either via -license or via -licenseFile command-line flag. The issued license key must support offline verification feature. Contact info@victoriametrics.com if you need offline license verification. This flag is avilable only in Enterprise binaries
- -licenseFile string
- Path to file with license key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed inline via -license command-line flag
- -loggerDisableTimestamps
- Whether to disable writing timestamps in logs
- -loggerErrorsPerSecondLimit int
- Per-second limit on the number of ERROR messages. If more than the given number of errors are emitted per second, the remaining errors are suppressed. Zero values disable the rate limit
- -loggerFormat string
- Format for logs. Possible values: default, json (default "default")
- -loggerJSONFields string
- Allows renaming fields in JSON formatted logs. Example: "ts:timestamp,msg:message" renames "ts" to "timestamp" and "msg" to "message". Supported fields: ts, level, caller, msg
- -loggerLevel string
- Minimum level of errors to log. Possible values: INFO, WARN, ERROR, FATAL, PANIC (default "INFO")
- -loggerOutput string
- Output for the logs. Supported values: stderr, stdout (default "stderr")
- -loggerTimezone string
- Timezone to use for timestamps in logs. Timezone must be a valid IANA Time Zone. For example: America/New_York, Europe/Berlin, Etc/GMT+3 or Local (default "UTC")
- -loggerWarnsPerSecondLimit int
- Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
- -memory.allowedBytes size
- Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
- Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
- -memory.allowedPercent float
- Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)
- -metricsAuthKey string
- Auth key for /metrics endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -notifier.basicAuth.password array
- Optional basic auth password for -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.basicAuth.passwordFile array
- Optional path to basic auth password file for -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.basicAuth.username array
- Optional basic auth username for -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.bearerToken array
- Optional bearer token for -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.bearerTokenFile array
- Optional path to bearer token file for -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.blackhole bool
- Whether to blackhole alerting notifications. Enable this flag if you want vmalert to evaluate alerting rules without sending any notifications to external receivers (eg. alertmanager). -notifier.url, `-notifier.config` and `-notifier.blackhole` are mutually exclusive.
- -notifier.config string
- Path to configuration file for notifiers
- -notifier.oauth2.clientID array
- Optional OAuth2 clientID to use for -notifier.url. If multiple args are set, then they are applied independently for the corresponding -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.oauth2.clientSecret array
- Optional OAuth2 clientSecret to use for -notifier.url. If multiple args are set, then they are applied independently for the corresponding -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.oauth2.clientSecretFile array
- Optional OAuth2 clientSecretFile to use for -notifier.url. If multiple args are set, then they are applied independently for the corresponding -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.oauth2.scopes array
- Optional OAuth2 scopes to use for -notifier.url. Scopes must be delimited by ';'. If multiple args are set, then they are applied independently for the corresponding -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.oauth2.tokenUrl array
- Optional OAuth2 tokenURL to use for -notifier.url. If multiple args are set, then they are applied independently for the corresponding -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.showURL
- Whether to avoid stripping sensitive information such as passwords from URL in log messages or UI for -notifier.url. It is hidden by default, since it can contain sensitive info such as auth key
- -notifier.suppressDuplicateTargetErrors
- Whether to suppress 'duplicate target' errors during discovery
- -notifier.tlsCAFile array
- Optional path to TLS CA file to use for verifying connections to -notifier.url. By default, system CA is used
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.tlsCertFile array
- Optional path to client-side TLS certificate file to use when connecting to -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.tlsInsecureSkipVerify array
- Whether to skip tls verification when connecting to -notifier.url
- Supports array of values separated by comma or specified via multiple flags.
- -notifier.tlsKeyFile array
- Optional path to client-side TLS certificate key to use when connecting to -notifier.url
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.tlsServerName array
- Optional TLS server name to use for connections to -notifier.url. By default, the server name from -notifier.url is used
- Supports an array of values separated by comma or specified via multiple flags.
- -notifier.url array
- Prometheus Alertmanager URL, e.g. http://127.0.0.1:9093. List all Alertmanager URLs if it runs in the cluster mode to ensure high availability.
- Supports an array of values separated by comma or specified via multiple flags.
- -pprofAuthKey string
- Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -promscrape.consul.waitTime duration
- Wait time used by Consul service discovery. Default value is used if not set
- -promscrape.consulSDCheckInterval duration
- Interval for checking for changes in Consul. This works only if consul_sd_configs is configured in '-promscrape.config' file. See https://docs.victoriametrics.com/sd_configs.html#consul_sd_configs for details (default 30s)
- -promscrape.discovery.concurrency int
- The maximum number of concurrent requests to Prometheus autodiscovery API (Consul, Kubernetes, etc.) (default 100)
- -promscrape.discovery.concurrentWaitTime duration
- The maximum duration for waiting to perform API requests if more than -promscrape.discovery.concurrency requests are simultaneously performed (default 1m0s)
- -promscrape.dnsSDCheckInterval duration
- Interval for checking for changes in dns. This works only if dns_sd_configs is configured in '-promscrape.config' file. See https://docs.victoriametrics.com/sd_configs.html#dns_sd_configs for details (default 30s)
- -pushmetrics.extraLabel array
- Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
- Supports an array of values separated by comma or specified via multiple flags.
- -pushmetrics.interval duration
- Interval for pushing metrics to -pushmetrics.url (default 10s)
- -pushmetrics.url array
- Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
- Supports an array of values separated by comma or specified via multiple flags.
- -remoteRead.basicAuth.password string
- Optional basic auth password for -remoteRead.url
- -remoteRead.basicAuth.passwordFile string
- Optional path to basic auth password to use for -remoteRead.url
- -remoteRead.basicAuth.username string
- Optional basic auth username for -remoteRead.url
- -remoteRead.bearerToken string
- Optional bearer auth token to use for -remoteRead.url.
- -remoteRead.bearerTokenFile string
- Optional path to bearer token file to use for -remoteRead.url.
- -remoteRead.disablePathAppend
- Whether to disable automatic appending of '/api/v1/query' path to the configured -datasource.url and -remoteRead.url
- -remoteRead.headers string
- Optional HTTP headers to send with each request to the corresponding -remoteRead.url. For example, -remoteRead.headers='My-Auth:foobar' would send 'My-Auth: foobar' HTTP header with every request to the corresponding -remoteRead.url. Multiple headers must be delimited by '^^': -remoteRead.headers='header1:value1^^header2:value2'
- -remoteRead.ignoreRestoreErrors
- Whether to ignore errors from remote storage when restoring alerts state on startup. DEPRECATED - this flag has no effect and will be removed in the next releases. (default true)
- -remoteRead.lookback duration
- Lookback defines how far to look into past for alerts timeseries. For example, if lookback=1h then range from now() to now()-1h will be scanned. (default 1h0m0s)
- -remoteRead.oauth2.clientID string
- Optional OAuth2 clientID to use for -remoteRead.url.
- -remoteRead.oauth2.clientSecret string
- Optional OAuth2 clientSecret to use for -remoteRead.url.
- -remoteRead.oauth2.clientSecretFile string
- Optional OAuth2 clientSecretFile to use for -remoteRead.url.
- -remoteRead.oauth2.scopes string
- Optional OAuth2 scopes to use for -remoteRead.url. Scopes must be delimited by ';'.
- -remoteRead.oauth2.tokenUrl string
- Optional OAuth2 tokenURL to use for -remoteRead.url.
- -remoteRead.showURL
- Whether to show -remoteRead.url in the exported metrics. It is hidden by default, since it can contain sensitive info such as auth key
- -remoteRead.tlsCAFile string
- Optional path to TLS CA file to use for verifying connections to -remoteRead.url. By default, system CA is used
- -remoteRead.tlsCertFile string
- Optional path to client-side TLS certificate file to use when connecting to -remoteRead.url
- -remoteRead.tlsInsecureSkipVerify
- Whether to skip tls verification when connecting to -remoteRead.url
- -remoteRead.tlsKeyFile string
- Optional path to client-side TLS certificate key to use when connecting to -remoteRead.url
- -remoteRead.tlsServerName string
- Optional TLS server name to use for connections to -remoteRead.url. By default, the server name from -remoteRead.url is used
- -remoteRead.url vmalert
- Optional URL to datasource compatible with Prometheus HTTP API. It can be single node VictoriaMetrics or vmselect.Remote read is used to restore alerts state.This configuration makes sense only if vmalert was configured with `remoteWrite.url` before and has been successfully persisted its state. E.g. http://127.0.0.1:8428. See also '-remoteRead.disablePathAppend', '-remoteRead.showURL'.
- -remoteWrite.basicAuth.password string
- Optional basic auth password for -remoteWrite.url
- -remoteWrite.basicAuth.passwordFile string
- Optional path to basic auth password to use for -remoteWrite.url
- -remoteWrite.basicAuth.username string
- Optional basic auth username for -remoteWrite.url
- -remoteWrite.bearerToken string
- Optional bearer auth token to use for -remoteWrite.url.
- -remoteWrite.bearerTokenFile string
- Optional path to bearer token file to use for -remoteWrite.url.
- -remoteWrite.concurrency int
- Defines number of writers for concurrent writing into remote write endpoint (default 1)
- -remoteWrite.disablePathAppend
- Whether to disable automatic appending of '/api/v1/write' path to the configured -remoteWrite.url.
- -remoteWrite.flushInterval duration
- Defines interval of flushes to remote write endpoint (default 5s)
- -remoteWrite.headers string
- Optional HTTP headers to send with each request to the corresponding -remoteWrite.url. For example, -remoteWrite.headers='My-Auth:foobar' would send 'My-Auth: foobar' HTTP header with every request to the corresponding -remoteWrite.url. Multiple headers must be delimited by '^^': -remoteWrite.headers='header1:value1^^header2:value2'
- -remoteWrite.maxBatchSize int
- Defines max number of timeseries to be flushed at once (default 1000)
- -remoteWrite.maxQueueSize int
- Defines the max number of pending datapoints to remote write endpoint (default 100000)
- -remoteWrite.oauth2.clientID string
- Optional OAuth2 clientID to use for -remoteWrite.url.
- -remoteWrite.oauth2.clientSecret string
- Optional OAuth2 clientSecret to use for -remoteWrite.url.
- -remoteWrite.oauth2.clientSecretFile string
- Optional OAuth2 clientSecretFile to use for -remoteWrite.url.
- -remoteWrite.oauth2.scopes string
- Optional OAuth2 scopes to use for -notifier.url. Scopes must be delimited by ';'.
- -remoteWrite.oauth2.tokenUrl string
- Optional OAuth2 tokenURL to use for -notifier.url.
- -remoteWrite.retryMaxTime duration
- The max time spent on retry attempts for the failed remote-write request. Change this value if it is expected for remoteWrite.url to be unreachable for more than -remoteWrite.retryMaxTime. See also -remoteWrite.retryMinInterval (default 30s)
- -remoteWrite.retryMinInterval duration
- The minimum delay between retry attempts. Every next retry attempt will double the delay to prevent hammering of remote database. See also -remoteWrite.retryMaxInterval (default 1s)
- -remoteWrite.sendTimeout duration
- Timeout for sending data to the configured -remoteWrite.url. (default 30s)
- -remoteWrite.showURL
- Whether to show -remoteWrite.url in the exported metrics. It is hidden by default, since it can contain sensitive info such as auth key
- -remoteWrite.tlsCAFile string
- Optional path to TLS CA file to use for verifying connections to -remoteWrite.url. By default, system CA is used
- -remoteWrite.tlsCertFile string
- Optional path to client-side TLS certificate file to use when connecting to -remoteWrite.url
- -remoteWrite.tlsInsecureSkipVerify
- Whether to skip tls verification when connecting to -remoteWrite.url
- -remoteWrite.tlsKeyFile string
- Optional path to client-side TLS certificate key to use when connecting to -remoteWrite.url
- -remoteWrite.tlsServerName string
- Optional TLS server name to use for connections to -remoteWrite.url. By default, the server name from -remoteWrite.url is used
- -remoteWrite.url string
- Optional URL to VictoriaMetrics or vminsert where to persist alerts state and recording rules results in form of timeseries. For example, if -remoteWrite.url=http://127.0.0.1:8428 is specified, then the alerts state will be written to http://127.0.0.1:8428/api/v1/write . See also -remoteWrite.disablePathAppend, '-remoteWrite.showURL'.
- -replay.disableProgressBar
- Whether to disable rendering progress bars during the replay. Progress bar rendering might be verbose or break the logs parsing, so it is recommended to be disabled when not used in interactive mode.
- -replay.maxDatapointsPerQuery /query_range
- Max number of data points expected in one request. It affects the max time range for every /query_range request during the replay. The higher the value, the less requests will be made during replay. (default 1000)
- -replay.ruleRetryAttempts int
- Defines how many retries to make before giving up on rule if request for it returns an error. (default 5)
- -replay.rulesDelay duration
- Delay between rules evaluation within the group. Could be important if there are chained rules inside the group and processing need to wait for previous rule results to be persisted by remote storage before evaluating the next rule.Keep it equal or bigger than -remoteWrite.flushInterval. (default 1s)
- -replay.timeFrom string
- The time filter in RFC3339 format to select time series with timestamp equal or higher than provided value. E.g. '2020-01-01T20:07:00Z'
- -replay.timeTo string
- The time filter in RFC3339 format to select timeseries with timestamp equal or lower than provided value. E.g. '2020-01-01T20:07:00Z'
- -rule array
- Path to the files or http url with alerting and/or recording rules.
- Supports hierarchical patterns and regexpes.
- Examples:
- -rule="/path/to/file". Path to a single file with alerting rules.
- -rule="http:///path/to/rules". HTTP URL to a page with alerting rules.
- -rule="dir/*.yaml" -rule="/*.yaml" -rule="gcs://vmalert-rules/tenant_%{TENANT_ID}/prod".
- -rule="dir/**/*.yaml". Includes all the .yaml files in "dir" subfolders recursively.
- Rule files may contain %{ENV_VAR} placeholders, which are substituted by the corresponding env vars.
-
- Enterprise version of vmalert supports S3 and GCS paths to rules.
- For example: gs://bucket/path/to/rules, s3://bucket/path/to/rules
- S3 and GCS paths support only matching by prefix, e.g. s3://bucket/dir/rule_ matches
- all files with prefix rule_ in folder dir.
- See https://docs.victoriametrics.com/vmalert.html#reading-rules-from-object-storage
-
- Supports an array of values separated by comma or specified via multiple flags.
- -rule.evalDelay time
- Adjustment of the time parameter for rule evaluation requests to compensate intentional data delay from the datasource.Normally, should be equal to `-search.latencyOffset` (cmd-line flag configured for VictoriaMetrics single-node or vmselect). (default 30s)
- -rule.maxResolveDuration duration
- Limits the maxiMum duration for automatic alert expiration, which by default is 4 times evaluationInterval of the parent group
- -rule.resendDelay duration
- MiniMum amount of time to wait before resending an alert to notifier
- -rule.templates array
- Path or glob pattern to location with go template definitions for rules annotations templating. Flag can be specified multiple times.
- Examples:
- -rule.templates="/path/to/file". Path to a single file with go templates
- -rule.templates="dir/*.tpl" -rule.templates="/*.tpl". Relative path to all .tpl files in "dir" folder,
- absolute path to all .tpl files in root.
- -rule.templates="dir/**/*.tpl". Includes all the .tpl files in "dir" subfolders recursively.
-
- Supports an array of values separated by comma or specified via multiple flags.
- -rule.updateEntriesLimit int
- Defines the max number of rule's state updates stored in-memory. Rule's updates are available on rule's Details page and are used for debugging purposes. The number of stored updates can be overridden per rule via update_entries_limit param. (default 20)
- -rule.validateExpressions
- Whether to validate rules expressions via MetricsQL engine (default true)
- -rule.validateTemplates
- Whether to validate annotation and label templates (default true)
- -s3.configFilePath string
- Path to file with S3 configs. Configs are loaded from default location if not set.
- See https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html . This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -s3.configProfile string
- Profile name for S3 configs. If no set, the value of the environment variable will be loaded (AWS_PROFILE or AWS_DEFAULT_PROFILE), or if both not set, DefaultSharedConfigProfile is used. This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -s3.credsFilePath string
- Path to file with GCS or S3 credentials. Credentials are loaded from default locations if not set.
- See https://cloud.google.com/iam/docs/creating-managing-service-account-keys and https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html . This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -s3.customEndpoint string
- Custom S3 endpoint for use with S3-compatible storages (e.g. MinIO). S3 is used if not set. This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -s3.forcePathStyle
- Prefixing endpoint with bucket name when set false, true by default. This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html (default true)
- -tls
- Whether to enable TLS for incoming HTTP requests at -httpListenAddr (aka https). -tlsCertFile and -tlsKeyFile must be set if -tls is set
- -tlsCertFile string
- Path to file with TLS certificate if -tls is set. Prefer ECDSA certs instead of RSA certs as RSA certs are slower. The provided certificate file is automatically re-read every second, so it can be dynamically updated
- -tlsCipherSuites array
- Optional list of TLS cipher suites for incoming requests over HTTPS if -tls is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants
- Supports an array of values separated by comma or specified via multiple flags.
- -tlsKeyFile string
- Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
- -tlsMinVersion string
- Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
- -version
- Show VictoriaMetrics version
-```
-{% endraw %}
-
-### Hot config reload
-
-`vmalert` supports "hot" config reload via the following methods:
-
-* send SIGHUP signal to `vmalert` process;
-* send GET request to `/-/reload` endpoint;
-* configure `-configCheckInterval` flag for periodic reload
- on config change.
-
-### URL params
-
-To set additional URL params for `datasource.url`, `remoteWrite.url` or `remoteRead.url`
-just add them in address: `-datasource.url=http://localhost:8428?nocache=1`.
-
-To set additional URL params for specific [group of rules](#Groups) modify
-the `params` group:
-
-```yaml
-groups:
- - name: TestGroup
- params:
- denyPartialResponse: ["true"]
- extra_label: ["env=dev"]
-```
-
-Please note, `params` are used only for executing rules expressions (requests to `datasource.url`).
-If there would be a conflict between URL params set in `datasource.url` flag and params in group definition
-the latter will have higher priority.
-
-### Notifier configuration file
-
-Notifier also supports configuration via file specified with flag `notifier.config`:
-
-```
-./bin/vmalert -rule=app/vmalert/config/testdata/rules.good.rules \
- -datasource.url=http://localhost:8428 \
- -notifier.config=app/vmalert/notifier/testdata/consul.good.yaml
-```
-
-The configuration file allows to configure static notifiers, discover notifiers via
-[Consul](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#consul_sd_config)
-and [DNS](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#dns_sd_config):
-For example:
-
-```yaml
-static_configs:
- - targets:
- # support using full url
- - 'http://alertmanager:9093/test/api/v2/alerts'
- - 'https://alertmanager:9093/api/v2/alerts'
- # the following target with only host:port will be used as ://localhost:9093//api/v2/alerts
- - localhost:9093
-
-consul_sd_configs:
- - server: localhost:8500
- services:
- - alertmanager
-
-dns_sd_configs:
- - names:
- - my.domain.com
- type: 'A'
- port: 9093
-```
-
-The list of configured or discovered Notifiers can be explored via [UI](#web).
-If Alertmanager runs in cluster mode then all its URLs needs to be available during discovery
-to ensure [high availability](https://github.com/prometheus/alertmanager#high-availability).
-
-The configuration file [specification](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmalert/notifier/config.go)
-is the following:
-
-```yaml
-# Per-target Notifier timeout when pushing alerts.
-[ timeout: | default = 10s ]
-
-# Prefix for the HTTP path alerts are pushed to.
-[ path_prefix: | default = / ]
-
-# Configures the protocol scheme used for requests.
-[ scheme: | default = http ]
-
-# Sets the `Authorization` header on every request with the
-# configured username and password.
-# password and password_file are mutually exclusive.
-basic_auth:
- [ username: ]
- [ password: ]
- [ password_file: ]
-
-# Optional `Authorization` header configuration.
-authorization:
- # Sets the authentication type.
- [ type: | default: Bearer ]
- # Sets the credentials. It is mutually exclusive with
- # `credentials_file`.
- [ credentials: ]
- # Sets the credentials to the credentials read from the configured file.
- # It is mutually exclusive with `credentials`.
- [ credentials_file: ]
-
-# Configures the scrape request's TLS settings.
-# see https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config
-tls_config:
- [ ]
-
-# Configures Bearer authentication token via string
-bearer_token:
-# or by passing path to the file with token.
-bearer_token_file:
-
-# Configures OAuth 2.0 authentication
-# see https://prometheus.io/docs/prometheus/latest/configuration/configuration/#oauth2
-oauth2:
- [ ]
-
-# Optional list of HTTP headers in form `header-name: value`
-# applied for all requests to notifiers
-# For example:
-# headers:
-# - "CustomHeader: foo"
-# - "CustomHeader2: bar"
-headers:
- [ , ...]
-
-# List of labeled statically configured Notifiers.
-#
-# Each list of targets may be additionally instructed with
-# authorization params. Target's authorization params will
-# inherit params from global authorization params if there
-# are no conflicts.
-static_configs:
- [ - targets: ]
- [ - '' ]
- [ oauth2 ]
- [ basic_auth ]
- [ authorization ]
- [ tls_config ]
- [ bearer_token ]
- [ bearer_token_file ]
- [ headers ]
-
-# List of Consul service discovery configurations.
-# See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#consul_sd_config
-consul_sd_configs:
- [ - ... ]
-
-# List of DNS service discovery configurations.
-# See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#dns_sd_config
-dns_sd_configs:
- [ - ... ]
-
-# List of relabel configurations for entities discovered via service discovery.
-# Supports the same relabeling features as the rest of VictoriaMetrics components.
-# See https://docs.victoriametrics.com/vmagent.html#relabeling
-relabel_configs:
- [ - ... ]
-
-# List of relabel configurations for alert labels sent via Notifier.
-# Supports the same relabeling features as the rest of VictoriaMetrics components.
-# See https://docs.victoriametrics.com/vmagent.html#relabeling
-alert_relabel_configs:
- [ - ... ]
-```
-
-The configuration file can be [hot-reloaded](#hot-config-reload).
-
-## Contributing
-
-`vmalert` is mostly designed and built by VictoriaMetrics community.
-Feel free to share your experience and ideas for improving this
-software. Please keep simplicity as the main priority.
-
-## How to build from sources
-
-It is recommended using
-[binary releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest)
-
-* `vmalert` is located in `vmutils-*` archives there.
-
-### Docker image
-
-You can build `vmalert` docker image from source and push it to your own docker repository.
-Run the following commands from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics):
-```console
-make package-vmalert
-docker tag victoria-metrics/vmalert:version my-repo:my-version-name
-docker push my-repo:my-version-name
-```
-
-To run the built image in `victoria-metrics-k8s-stack` or `VMAlert` CR object apply the following config change:
-
-```yaml
-kind: VMAlert
-spec:
- image:
- repository: my-repo
- tag: my-version-name
-```
-
-### Development build
-
-1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.20.
-1. Run `make vmalert` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmalert` binary and puts it into the `bin` folder.
-
-### Production build
-
-1. [Install docker](https://docs.docker.com/install/).
-1. Run `make vmalert-prod` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmalert-prod` binary and puts it into the `bin` folder.
-
-### ARM build
-
-ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://blog.cloudflare.com/arm-takes-wing/).
-
-### Development ARM build
-
-1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.20.
-1. Run `make vmalert-linux-arm` or `make vmalert-linux-arm64` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmalert-linux-arm` or `vmalert-linux-arm64` binary respectively and puts it into the `bin` folder.
-
-### Production ARM build
-
-1. [Install docker](https://docs.docker.com/install/).
-1. Run `make vmalert-linux-arm-prod` or `make vmalert-linux-arm64-prod` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmalert-linux-arm-prod` or `vmalert-linux-arm64-prod` binary respectively and puts it into the `bin` folder.
+vmalert docs can be edited at [docs/vmalert.md](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/vmalert.md).
diff --git a/app/vmalert/vmalert_cluster.png b/app/vmalert/vmalert_cluster.png
deleted file mode 100644
index 184df339e..000000000
Binary files a/app/vmalert/vmalert_cluster.png and /dev/null differ
diff --git a/app/vmalert/vmalert_ha.png b/app/vmalert/vmalert_ha.png
deleted file mode 100644
index 76ac521ab..000000000
Binary files a/app/vmalert/vmalert_ha.png and /dev/null differ
diff --git a/app/vmalert/vmalert_multicluster.png b/app/vmalert/vmalert_multicluster.png
deleted file mode 100644
index 8b51f2546..000000000
Binary files a/app/vmalert/vmalert_multicluster.png and /dev/null differ
diff --git a/app/vmalert/vmalert_multiple_rw.png b/app/vmalert/vmalert_multiple_rw.png
deleted file mode 100644
index 0f44e4000..000000000
Binary files a/app/vmalert/vmalert_multiple_rw.png and /dev/null differ
diff --git a/app/vmalert/vmalert_single.png b/app/vmalert/vmalert_single.png
deleted file mode 100644
index 46f087449..000000000
Binary files a/app/vmalert/vmalert_single.png and /dev/null differ
diff --git a/app/vmalert/vmalert_state.png b/app/vmalert/vmalert_state.png
deleted file mode 100644
index 5bf656b01..000000000
Binary files a/app/vmalert/vmalert_state.png and /dev/null differ
diff --git a/app/vmalert/vmalert_ts_data_delay.gif b/app/vmalert/vmalert_ts_data_delay.gif
deleted file mode 100644
index 2da024b46..000000000
Binary files a/app/vmalert/vmalert_ts_data_delay.gif and /dev/null differ
diff --git a/app/vmalert/vmalert_ts_normal.gif b/app/vmalert/vmalert_ts_normal.gif
deleted file mode 100644
index a05c74061..000000000
Binary files a/app/vmalert/vmalert_ts_normal.gif and /dev/null differ
diff --git a/app/vmauth/README.md b/app/vmauth/README.md
index cc0640762..34491ac22 100644
--- a/app/vmauth/README.md
+++ b/app/vmauth/README.md
@@ -1,576 +1,3 @@
-# vmauth
+See vmauth docs [here](https://docs.victoriametrics.com/vmauth.html).
-`vmauth` is a simple auth proxy, router and [load balancer](#load-balancing) for [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics).
-It reads auth credentials from `Authorization` http header ([Basic Auth](https://en.wikipedia.org/wiki/Basic_access_authentication), `Bearer token` and [InfluxDB authorization](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1897) is supported),
-matches them against configs pointed by [-auth.config](#auth-config) command-line flag and proxies incoming HTTP requests to the configured per-user `url_prefix` on successful match.
-The `-auth.config` can point to either local file or to http url.
-
-## Quick start
-
-Just download `vmutils-*` archive from [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest), unpack it
-and pass the following flag to `vmauth` binary in order to start authorizing and routing requests:
-
-```console
-/path/to/vmauth -auth.config=/path/to/auth/config.yml
-```
-
-After that `vmauth` starts accepting HTTP requests on port `8427` and routing them according to the provided [-auth.config](#auth-config).
-The port can be modified via `-httpListenAddr` command-line flag.
-
-The auth config can be reloaded via the following ways:
-
-- By passing `SIGHUP` signal to `vmauth`.
-- By querying `/-/reload` http endpoint. This endpoint can be protected with `-reloadAuthKey` command-line flag. See [security docs](#security) for more details.
-- By specifying `-configCheckInterval` command-line flag to the interval between config re-reads. For example, `-configCheckInterval=5s` will re-read the config
- and apply new changes every 5 seconds.
-
-Docker images for `vmauth` are available [here](https://hub.docker.com/r/victoriametrics/vmauth/tags).
-See how `vmauth` used in [docker-compose env](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/README.md#victoriametrics-cluster).
-
-Pass `-help` to `vmauth` in order to see all the supported command-line flags with their descriptions.
-
-Feel free [contacting us](mailto:info@victoriametrics.com) if you need customized auth proxy for VictoriaMetrics with the support of LDAP, SSO, RBAC, SAML,
-accounting and rate limiting such as [vmgateway](https://docs.victoriametrics.com/vmgateway.html).
-
-## Dropping request path prefix
-
-By default `vmauth` doesn't drop the path prefix from the original request when proxying the request to the matching backend.
-Sometimes it is needed to drop path prefix before routing the request to the backend. This can be done by specifying the number of `/`-delimited
-prefix parts to drop from the request path via `drop_src_path_prefix_parts` option at `url_map` level or at `user` level.
-
-For example, if you need to serve requests to [vmalert](https://docs.victoriametrics.com/vmalert.html) at `/vmalert/` path prefix,
-while serving requests to [vmagent](https://docs.victoriametrics.com/vmagent.html) at `/vmagent/` path prefix for a particular user,
-then the following [-auth.config](#auth-config) can be used:
-
-```yml
-users:
-- username: foo
- url_map:
-
- # proxy all the requests, which start with `/vmagent/`, to vmagent backend
- - src_paths:
- - "/vmagent/.+"
-
- # drop /vmagent/ path prefix from the original request before proxying it to url_prefix.
- drop_src_path_prefix_parts: 1
- url_prefix: "http://vmagent-backend:8429/"
-
- # proxy all the requests, which start with `/vmalert`, to vmalert backend
- - src_paths:
- - "/vmalert/.+"
-
- # drop /vmalert/ path prefix from the original request before proxying it to url_prefix.
- drop_src_path_prefix_parts: 1
- url_prefix: "http://vmalert-backend:8880/"
-```
-
-## Load balancing
-
-Each `url_prefix` in the [-auth.config](#auth-config) may contain either a single url or a list of urls.
-In the latter case `vmauth` balances load among the configured urls in least-loaded round-robin manner.
-
-If the backend at the configured url isn't available, then `vmauth` tries sending the request to the remaining configured urls.
-
-It is possible to configure automatic retry of requests if the backend responds with status code from optional `retry_status_codes` list.
-
-Load balancing feature can be used in the following cases:
-
-- Balancing the load among multiple `vmselect` and/or `vminsert` nodes in [VictoriaMetrics cluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html).
- The following `-auth.config` file can be used for spreading incoming requests among 3 vmselect nodes and re-trying failed requests
- or requests with 500 and 502 response status codes:
-
- ```yml
- unauthorized_user:
- url_prefix:
- - http://vmselect1:8481/
- - http://vmselect2:8481/
- - http://vmselect3:8481/
- retry_status_codes: [500, 502]
- ```
-
-- Spreading select queries among multiple availability zones (AZs) with identical data. For example, the following config spreads select queries
- among 3 AZs. Requests are re-tried if some AZs are temporarily unavailable or if some `vmstorage` nodes in some AZs are temporarily unavailable.
- `vmauth` adds `deny_partial_response=1` query arg to all the queries in order to guarantee to get full response from every AZ.
- See [these docs](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#cluster-availability) for details.
-
- ```yml
- unauthorized_user:
- url_prefix:
- - https://vmselect-az1/?deny_partial_response=1
- - https://vmselect-az2/?deny_partial_response=1
- - https://vmselect-az3/?deny_partial_response=1
- retry_status_codes: [500, 502, 503]
- ```
-
-Load balancig can also be configured independently per each user and per each `url_map` entry.
-See [auth config docs](#auth-config) for more details.
-
-## Concurrency limiting
-
-`vmauth` limits the number of concurrent requests it can proxy according to the following command-line flags:
-
-- `-maxConcurrentRequests` limits the global number of concurrent requests `vmauth` can serve across all the configured users.
-- `-maxConcurrentPerUserRequests` limits the number of concurrent requests `vmauth` can serve per each configured user.
-
-It is also possible to set individual limits on the number of concurrent requests per each user
-with the `max_concurrent_requests` option - see [auth config example](#auth-config).
-
-`vmauth` responds with `429 Too Many Requests` HTTP error when the number of concurrent requests exceeds the configured limits.
-
-The following [metrics](#monitoring) related to concurrency limits are exposed by `vmauth`:
-
-- `vmauth_concurrent_requests_capacity` - the global limit on the number of concurrent requests `vmauth` can serve.
- It is set via `-maxConcurrentRequests` command-line flag.
-- `vmauth_concurrent_requests_current` - the current number of concurrent requests `vmauth` processes.
-- `vmauth_concurrent_requests_limit_reached_total` - the number of requests rejected with `429 Too Many Requests` error
- because of the global concurrency limit has been reached.
-- `vmauth_user_concurrent_requests_capacity{username="..."}` - the limit on the number of concurrent requests for the given `username`.
-- `vmauth_user_concurrent_requests_current{username="..."}` - the current number of concurrent requests for the given `username`.
-- `vmauth_user_concurrent_requests_limit_reached_total{username="foo"}` - the number of requests rejected with `429 Too Many Requests` error
- because of the concurrency limit has been reached for the given `username`.
-- `vmauth_unauthorized_user_concurrent_requests_capacity` - the limit on the number of concurrent requests for unauthorized users (if `unauthorized_user` section is used).
-- `vmauth_unauthorized_user_concurrent_requests_current` - the current number of concurrent requests for unauthorized users (if `unauthorized_user` section is used).
-- `vmauth_unauthorized_user_concurrent_requests_limit_reached_total` - the number of requests rejected with `429 Too Many Requests` error
- because of the concurrency limit has been reached for unauthorized users (if `unauthorized_user` section is used).
-
-## Backend TLS setup
-
-By default `vmauth` uses system settings when performing requests to HTTPS backends specified via `url_prefix` option
-in the [`-auth.config`](https://docs.victoriametrics.com/vmauth.html#auth-config). These settings can be overridden with the following command-line flags:
-
-- `-backend.tlsInsecureSkipVerify` allows skipping TLS verification when connecting to HTTPS backends.
- This global setting can be overridden at per-user level inside [`-auth.config`](https://docs.victoriametrics.com/vmauth.html#auth-config)
- via `tls_insecure_skip_verify` option. For example:
-
- ```yml
- - username: "foo"
- url_prefix: "https://localhost"
- tls_insecure_skip_verify: true
- ```
-
-- `-backend.tlsCAFile` allows specifying the path to TLS Root CA, which will be used for TLS verification when connecting to HTTPS backends.
- The `-backend.tlsCAFile` may point either to local file or to `http` / `https` url.
- This global setting can be overridden at per-user level inside [`-auth.config`](https://docs.victoriametrics.com/vmauth.html#auth-config)
- via `tls_ca_file` option. For example:
-
- ```yml
- - username: "foo"
- url_prefix: "https://localhost"
- tls_ca_file: "/path/to/tls/root/ca"
- ```
-
-## IP filters
-
-[Enterprise version](https://docs.victoriametrics.com/enterprise.html) of `vmauth` can be configured to allow / deny incoming requests via global and per-user IP filters.
-
-For example, the following config allows requests to `vmauth` from `10.0.0.0/24` network and from `1.2.3.4` IP address, while denying requests from `10.0.0.42` IP address:
-
-```yml
-users:
-# User configs here
-
-ip_filters:
- allow_list:
- - 10.0.0.0/24
- - 1.2.3.4
- deny_list: [10.0.0.42]
-```
-
-The following config allows requests for the user 'foobar' only from the IP `127.0.0.1`:
-
-```yml
-users:
-- username: "foobar"
- password: "***"
- url_prefix: "http://localhost:8428"
- ip_filters:
- allow_list: [127.0.0.1]
-```
-
-See config example of using IP filters [here](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmauth/example_config_ent.yml).
-
-## Auth config
-
-`-auth.config` is represented in the following simple `yml` format:
-
-```yml
-# Arbitrary number of usernames may be put here.
-# It is possible to set multiple identical usernames with different passwords.
-# Such usernames can be differentiated by `name` option.
-
-users:
- # Requests with the 'Authorization: Bearer XXXX' and 'Authorization: Token XXXX'
- # header are proxied to http://localhost:8428 .
- # For example, http://vmauth:8427/api/v1/query is proxied to http://localhost:8428/api/v1/query
- # Requests with the Basic Auth username=XXXX are proxied to http://localhost:8428 as well.
-- bearer_token: "XXXX"
- url_prefix: "http://localhost:8428"
-
- # Requests with the 'Authorization: Bearer YYY' header are proxied to http://localhost:8428 ,
- # The `X-Scope-OrgID: foobar` http header is added to every proxied request.
- # The `X-Server-Hostname` http header is removed from the proxied response.
- # For example, http://vmauth:8427/api/v1/query is proxied to http://localhost:8428/api/v1/query
-- bearer_token: "YYY"
- url_prefix: "http://localhost:8428"
- # extra headers to add to the request or remove from the request (if header value is empty)
- headers:
- - "X-Scope-OrgID: foobar"
- # extra headers to add to the response or remove from the response (if header value is empty)
- response_headers:
- - "X-Server-Hostname:" # empty value means the header will be removed from the response
-
- # All the requests to http://vmauth:8427 with the given Basic Auth (username:password)
- # are proxied to http://localhost:8428 .
- # For example, http://vmauth:8427/api/v1/query is proxied to http://localhost:8428/api/v1/query
- #
- # The given user can send maximum 10 concurrent requests according to the provided max_concurrent_requests.
- # Excess concurrent requests are rejected with 429 HTTP status code.
- # See also -maxConcurrentPerUserRequests and -maxConcurrentRequests command-line flags.
-- username: "local-single-node"
- password: "***"
- url_prefix: "http://localhost:8428"
- max_concurrent_requests: 10
-
- # All the requests to http://vmauth:8427 with the given Basic Auth (username:password)
- # are proxied to http://localhost:8428 with extra_label=team=dev query arg.
- # For example, http://vmauth:8427/api/v1/query is routed to http://localhost:8428/api/v1/query?extra_label=team=dev
-- username: "local-single-node2"
- password: "***"
- url_prefix: "http://localhost:8428?extra_label=team=dev"
-
- # All the requests to http://vmauth:8427 with the given Basic Auth (username:password)
- # are proxied to https://localhost:8428.
- # For example, http://vmauth:8427/api/v1/query is routed to https://localhost/api/v1/query
- # TLS verification is skipped for https://localhost.
-- username: "local-single-node-with-tls"
- password: "***"
- url_prefix: "https://localhost"
- tls_insecure_skip_verify: true
-
- # All the requests to http://vmauth:8427 with the given Basic Auth (username:password)
- # are load-balanced among http://vmselect1:8481/select/123/prometheus and http://vmselect2:8481/select/123/prometheus
- # For example, http://vmauth:8427/api/v1/query is proxied to the following urls in a round-robin manner:
- # - http://vmselect1:8481/select/123/prometheus/api/v1/select
- # - http://vmselect2:8481/select/123/prometheus/api/v1/select
-- username: "cluster-select-account-123"
- password: "***"
- url_prefix:
- - "http://vmselect1:8481/select/123/prometheus"
- - "http://vmselect2:8481/select/123/prometheus"
-
- # All the requests to http://vmauth:8427 with the given Basic Auth (username:password)
- # are load-balanced between http://vminsert1:8480/insert/42/prometheus and http://vminsert2:8480/insert/42/prometheus
- # For example, http://vmauth:8427/api/v1/write is proxied to the following urls in a round-robin manner:
- # - http://vminsert1:8480/insert/42/prometheus/api/v1/write
- # - http://vminsert2:8480/insert/42/prometheus/api/v1/write
-- username: "cluster-insert-account-42"
- password: "***"
- url_prefix:
- - "http://vminsert1:8480/insert/42/prometheus"
- - "http://vminsert2:8480/insert/42/prometheus"
-
- # A single user for querying and inserting data:
- #
- # - Requests to http://vmauth:8427/api/v1/query, http://vmauth:8427/api/v1/query_range
- # and http://vmauth:8427/api/v1/label//values are proxied to the following urls in a round-robin manner:
- # - http://vmselect1:8481/select/42/prometheus
- # - http://vmselect2:8481/select/42/prometheus
- # For example, http://vmauth:8427/api/v1/query is proxied to http://vmselect1:8480/select/42/prometheus/api/v1/query
- # or to http://vmselect2:8480/select/42/prometheus/api/v1/query .
- # Requests are re-tried at other url_prefix backends if response status codes match 500 or 502.
- #
- # - Requests to http://vmauth:8427/api/v1/write are proxied to http://vminsert:8480/insert/42/prometheus/api/v1/write .
- # The "X-Scope-OrgID: abc" http header is added to these requests.
- # The "X-Server-Hostname" http header is removed from the proxied response.
- #
- # Request which do not match `src_paths` from the `url_map` are proxied to the urls from `default_url`
- # in a round-robin manner. The original request path is passed in `request_path` query arg.
- # For example, request to http://vmauth:8427/non/existing/path are proxied:
- # - to http://default1:8888/unsupported_url_handler?request_path=/non/existing/path
- # - or http://default2:8888/unsupported_url_handler?request_path=/non/existing/path
- #
- # Regular expressions are allowed in `src_paths` entries.
-- username: "foobar"
- url_map:
- - src_paths:
- - "/api/v1/query"
- - "/api/v1/query_range"
- - "/api/v1/label/[^/]+/values"
- url_prefix:
- - "http://vmselect1:8481/select/42/prometheus"
- - "http://vmselect2:8481/select/42/prometheus"
- retry_status_codes: [500, 502]
- - src_paths: ["/api/v1/write"]
- url_prefix: "http://vminsert:8480/insert/42/prometheus"
- headers:
- - "X-Scope-OrgID: abc"
- response_headers:
- - "X-Server-Hostname:" # empty value means the header will be removed from the response
- ip_filters:
- deny_list: [127.0.0.1]
- default_url:
- - "http://default1:8888/unsupported_url_handler"
- - "http://default2:8888/unsupported_url_handler"
-
-# Requests without Authorization header are routed according to `unauthorized_user` section.
-# Requests are routed in round-robin fashion between `url_prefix` backends.
-# The deny_partial_response query arg is added to all the routed requests.
-# The requests are re-tried if url_prefix backends send 500 or 503 response status codes.
-# Note that the unauthorized_user section takes precedence when processing a route without credentials,
-# even if such a route also exists in the users section (see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5236).
-unauthorized_user:
- url_prefix:
- - http://vmselect-az1/?deny_partial_response=1
- - http://vmselect-az2/?deny_partial_response=1
- retry_status_codes: [503, 500]
-
-ip_filters:
- allow_list: ["1.2.3.0/24", "127.0.0.1"]
- deny_list:
- - 10.1.0.1
-```
-
-The config may contain `%{ENV_VAR}` placeholders, which are substituted by the corresponding `ENV_VAR` environment variable values.
-This may be useful for passing secrets to the config.
-
-Please note, vmauth doesn't follow redirects. If destination redirects request to a new location, make sure this
-location is supported in vmauth `url_map` config.
-
-## Security
-
-It is expected that all the backend services protected by `vmauth` are located in an isolated private network, so they can be accessed by external users only via `vmauth`.
-
-Do not transfer Basic Auth headers in plaintext over untrusted networks. Enable https. This can be done by passing the following `-tls*` command-line flags to `vmauth`:
-
-```console
- -tls
- Whether to enable TLS (aka HTTPS) for incoming requests. -tlsCertFile and -tlsKeyFile must be set if -tls is set
- -tlsCertFile string
- Path to file with TLS certificate. Used only if -tls is set. Prefer ECDSA certs instead of RSA certs, since RSA certs are slow
- -tlsKeyFile string
- Path to file with TLS key. Used only if -tls is set
-```
-
-Alternatively, [https termination proxy](https://en.wikipedia.org/wiki/TLS_termination_proxy) may be put in front of `vmauth`.
-
-It is recommended protecting the following endpoints with authKeys:
-* `/-/reload` with `-reloadAuthKey` command-line flag, so external users couldn't trigger config reload.
-* `/flags` with `-flagsAuthKey` command-line flag, so unauthorized users couldn't get application command-line flags.
-* `/metrics` with `-metricsAuthKey` command-line flag, so unauthorized users couldn't get access to [vmauth metrics](#monitoring).
-* `/debug/pprof` with `-pprofAuthKey` command-line flag, so unauthorized users couldn't get access to [profiling information](#profiling).
-
-`vmauth` also supports the ability to restrict access by IP - see [these docs](#ip-filters). See also [concurrency limiting docs](#concurrency-limiting).
-
-## Monitoring
-
-`vmauth` exports various metrics in Prometheus exposition format at `http://vmauth-host:8427/metrics` page. It is recommended setting up regular scraping of this page
-either via [vmagent](https://docs.victoriametrics.com/vmagent.html) or via Prometheus, so the exported metrics could be analyzed later.
-
-`vmauth` exports `vmauth_user_requests_total` [counter](https://docs.victoriametrics.com/keyConcepts.html#counter) metric
-and `vmauth_user_request_duration_seconds_*` [summary](https://docs.victoriametrics.com/keyConcepts.html#summary) metric
-with `username` label. The `username` label value equals to `username` field value set in the `-auth.config` file.
-It is possible to override or hide the value in the label by specifying `name` field.
-For example, the following config will result in `vmauth_user_requests_total{username="foobar"}`
-instead of `vmauth_user_requests_total{username="secret_user"}`:
-
-```yml
-users:
-- username: "secret_user"
- name: "foobar"
- # other config options here
-```
-
-For unauthorized users `vmauth` exports `vmauth_unauthorized_user_requests_total`
-[counter](https://docs.victoriametrics.com/keyConcepts.html#counter) metric and
-`vmauth_unauthorized_user_request_duration_seconds_*` [summary](https://docs.victoriametrics.com/keyConcepts.html#summary)
-metric without label (if `unauthorized_user` section of config is used).
-
-## How to build from sources
-
-It is recommended using [binary releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest) - `vmauth` is located in `vmutils-*` archives there.
-
-### Development build
-
-1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.20.
-1. Run `make vmauth` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmauth` binary and puts it into the `bin` folder.
-
-### Production build
-
-1. [Install docker](https://docs.docker.com/install/).
-1. Run `make vmauth-prod` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmauth-prod` binary and puts it into the `bin` folder.
-
-### Building docker images
-
-Run `make package-vmauth`. It builds `victoriametrics/vmauth:` docker image locally.
-`` is auto-generated image tag, which depends on source code in the repository.
-The `` may be manually set via `PKG_TAG=foobar make package-vmauth`.
-
-The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
-by setting it via `` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
-
-```console
-ROOT_IMAGE=scratch make package-vmauth
-```
-
-## Profiling
-
-`vmauth` provides handlers for collecting the following [Go profiles](https://blog.golang.org/profiling-go-programs):
-
-* Memory profile. It can be collected with the following command (replace `0.0.0.0` with hostname if needed):
-
-
-
-```console
-curl http://0.0.0.0:8427/debug/pprof/heap > mem.pprof
-```
-
-
-
-* CPU profile. It can be collected with the following command (replace `0.0.0.0` with hostname if needed):
-
-
-
-```console
-curl http://0.0.0.0:8427/debug/pprof/profile > cpu.pprof
-```
-
-
-
-The command for collecting CPU profile waits for 30 seconds before returning.
-
-The collected profiles may be analyzed with [go tool pprof](https://github.com/google/pprof).
-It is safe sharing the collected profiles from security point of view, since they do not contain sensitive information.
-
-## Advanced usage
-
-Pass `-help` command-line arg to `vmauth` in order to see all the configuration options:
-
-```console
-./vmauth -help
-
-vmauth authenticates and authorizes incoming requests and proxies them to VictoriaMetrics.
-
-See the docs at https://docs.victoriametrics.com/vmauth.html .
-
- -auth.config string
- Path to auth config. It can point either to local file or to http url. See https://docs.victoriametrics.com/vmauth.html for details on the format of this auth config
- -configCheckInterval duration
- interval for config file re-read. Zero value disables config re-reading. By default, refreshing is disabled, send SIGHUP for config refresh.
- -enableTCP6
- Whether to enable IPv6 for listening and dialing. By default, only IPv4 TCP and UDP are used
- -envflag.enable
- Whether to enable reading flags from environment variables in addition to the command line. Command line flag values have priority over values from environment vars. Flags are read only from the command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
- -envflag.prefix string
- Prefix for environment variables if -envflag.enable is set
- -eula
- Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -failTimeout duration
- Sets a delay period for load balancing to skip a malfunctioning backend (default 3s)
- -filestream.disableFadvise
- Whether to disable fadvise() syscall when reading large data files. The fadvise() syscall prevents from eviction of recently accessed data from OS page cache during background merges and backups. In some rare cases it is better to disable the syscall if it uses too much CPU
- -flagsAuthKey string
- Auth key for /flags endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -fs.disableMmap
- Whether to use pread() instead of mmap() for reading data files. By default, mmap() is used for 64-bit arches and pread() is used for 32-bit arches, since they cannot read data files bigger than 2^32 bytes in memory. mmap() is usually faster for reading small data chunks than pread()
- -http.connTimeout duration
- Incoming http connections are closed after the configured timeout. This may help to spread the incoming load among a cluster of services behind a load balancer. Please note that the real timeout may be bigger by up to 10% as a protection against the thundering herd problem (default 2m0s)
- -http.disableResponseCompression
- Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
- -http.header.csp string
- Value for 'Content-Security-Policy' header
- -http.header.frameOptions string
- Value for 'X-Frame-Options' header
- -http.header.hsts string
- Value for 'Strict-Transport-Security' header
- -http.idleConnTimeout duration
- Timeout for incoming idle http connections (default 1m0s)
- -http.maxGracefulShutdownDuration duration
- The maximum duration for a graceful shutdown of the HTTP server. A highly loaded server may require increased value for a graceful shutdown (default 7s)
- -http.pathPrefix string
- An optional prefix to add to all the paths handled by http server. For example, if '-http.pathPrefix=/foo/bar' is set, then all the http requests will be handled on '/foo/bar/*' paths. This may be useful for proxied requests. See https://www.robustperception.io/using-external-urls-and-proxies-with-prometheus
- -http.shutdownDelay duration
- Optional delay before http server shutdown. During this delay, the server returns non-OK responses from /health page, so load balancers can route new requests to other servers
- -httpAuth.password string
- Password for HTTP server's Basic Auth. The authentication is disabled if -httpAuth.username is empty
- -httpAuth.username string
- Username for HTTP server's Basic Auth. The authentication is disabled if empty. See also -httpAuth.password
- -httpListenAddr string
- TCP address to listen for http connections. See also -httpListenAddr.useProxyProtocol (default ":8427")
- -httpListenAddr.useProxyProtocol
- Whether to use proxy protocol for connections accepted at -httpListenAddr . See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt . With enabled proxy protocol http server cannot serve regular /metrics endpoint. Use -pushmetrics.url for metrics pushing
- -internStringCacheExpireDuration duration
- The expiry duration for caches for interned strings. See https://en.wikipedia.org/wiki/String_interning . See also -internStringMaxLen and -internStringDisableCache (default 6m0s)
- -internStringDisableCache
- Whether to disable caches for interned strings. This may reduce memory usage at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringCacheExpireDuration and -internStringMaxLen
- -internStringMaxLen int
- The maximum length for strings to intern. A lower limit may save memory at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringDisableCache and -internStringCacheExpireDuration (default 500)
- -license string
- Lisense key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed via file specified by -licenseFile command-line flag
- -license.forceOffline
- Whether to enable offline verification for VictoriaMetrics Enterprise license key, which has been passed either via -license or via -licenseFile command-line flag. The issued license key must support offline verification feature. Contact info@victoriametrics.com if you need offline license verification. This flag is avilable only in Enterprise binaries
- -licenseFile string
- Path to file with license key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed inline via -license command-line flag
- -logInvalidAuthTokens
- Whether to log requests with invalid auth tokens. Such requests are always counted at vmauth_http_request_errors_total{reason="invalid_auth_token"} metric, which is exposed at /metrics page
- -loggerDisableTimestamps
- Whether to disable writing timestamps in logs
- -loggerErrorsPerSecondLimit int
- Per-second limit on the number of ERROR messages. If more than the given number of errors are emitted per second, the remaining errors are suppressed. Zero values disable the rate limit
- -loggerFormat string
- Format for logs. Possible values: default, json (default "default")
- -loggerJSONFields string
- Allows renaming fields in JSON formatted logs. Example: "ts:timestamp,msg:message" renames "ts" to "timestamp" and "msg" to "message". Supported fields: ts, level, caller, msg
- -loggerLevel string
- Minimum level of errors to log. Possible values: INFO, WARN, ERROR, FATAL, PANIC (default "INFO")
- -loggerOutput string
- Output for the logs. Supported values: stderr, stdout (default "stderr")
- -loggerTimezone string
- Timezone to use for timestamps in logs. Timezone must be a valid IANA Time Zone. For example: America/New_York, Europe/Berlin, Etc/GMT+3 or Local (default "UTC")
- -loggerWarnsPerSecondLimit int
- Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
- -maxConcurrentPerUserRequests int
- The maximum number of concurrent requests vmauth can process per each configured user. Other requests are rejected with '429 Too Many Requests' http status code. See also -maxConcurrentRequests command-line option and max_concurrent_requests option in per-user config (default 300)
- -maxConcurrentRequests int
- The maximum number of concurrent requests vmauth can process. Other requests are rejected with '429 Too Many Requests' http status code. See also -maxConcurrentPerUserRequests and -maxIdleConnsPerBackend command-line options (default 1000)
- -maxIdleConnsPerBackend int
- The maximum number of idle connections vmauth can open per each backend host. See also -maxConcurrentRequests (default 100)
- -maxRequestBodySizeToRetry size
- The maximum request body size, which can be cached and re-tried at other backends. Bigger values may require more memory
- Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 16384)
- -memory.allowedBytes size
- Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
- Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
- -memory.allowedPercent float
- Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)
- -metricsAuthKey string
- Auth key for /metrics endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -pprofAuthKey string
- Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -pushmetrics.extraLabel array
- Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
- Supports an array of values separated by comma or specified via multiple flags.
- -pushmetrics.interval duration
- Interval for pushing metrics to -pushmetrics.url (default 10s)
- -pushmetrics.url array
- Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
- Supports an array of values separated by comma or specified via multiple flags.
- -reloadAuthKey string
- Auth key for /-/reload http endpoint. It must be passed as authKey=...
- -responseTimeout duration
- The timeout for receiving a response from backend (default 5m0s)
- -tls
- Whether to enable TLS for incoming HTTP requests at -httpListenAddr (aka https). -tlsCertFile and -tlsKeyFile must be set if -tls is set
- -tlsCertFile string
- Path to file with TLS certificate if -tls is set. Prefer ECDSA certs instead of RSA certs as RSA certs are slower. The provided certificate file is automatically re-read every second, so it can be dynamically updated
- -tlsCipherSuites array
- Optional list of TLS cipher suites for incoming requests over HTTPS if -tls is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants
- Supports an array of values separated by comma or specified via multiple flags.
- -tlsKeyFile string
- Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
- -tlsMinVersion string
- Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
- -version
- Show VictoriaMetrics version
-```
+vmauth docs can be edited at [docs/vmauth.md](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/vmauth.md).
diff --git a/app/vmbackup/README.md b/app/vmbackup/README.md
index 5eb2b824b..3a2d1f010 100644
--- a/app/vmbackup/README.md
+++ b/app/vmbackup/README.md
@@ -1,444 +1,3 @@
-# vmbackup
+See vmbackup docs [here](https://docs.victoriametrics.com/vmbackup.html).
-`vmbackup` creates VictoriaMetrics data backups from [instant snapshots](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-work-with-snapshots).
-
-`vmbackup` supports incremental and full backups. Incremental backups are created automatically if the destination path already contains data from the previous backup.
-Full backups can be sped up with `-origin` pointing to an already existing backup on the same remote storage. In this case `vmbackup` makes server-side copy for the shared
-data between the existing backup and new backup. It saves time and costs on data transfer.
-
-Backup process can be interrupted at any time. It is automatically resumed from the interruption point when restarting `vmbackup` with the same args.
-
-Backed up data can be restored with [vmrestore](https://docs.victoriametrics.com/vmrestore.html).
-
-See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-series-databases-533c1a927883) for more details.
-
-See also [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager.html) tool built on top of `vmbackup`. This tool simplifies
-creation of hourly, daily, weekly and monthly backups.
-
-## Supported storage types
-
-`vmbackup` supports the following `-dst` storage types:
-
-* [GCS](https://cloud.google.com/storage/). Example: `gs:///`
-* [S3](https://aws.amazon.com/s3/). Example: `s3:///`
-* [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs/). Example: `azblob:///`
-* Any S3-compatible storage such as [MinIO](https://github.com/minio/minio), [Ceph](https://docs.ceph.com/en/pacific/radosgw/s3/) or [Swift](https://platform.swiftstack.com/docs/admin/middleware/s3_middleware.html). See [these docs](#advanced-usage) for details.
-* Local filesystem. Example: `fs://`. Note that `vmbackup` prevents from storing the backup into the directory pointed by `-storageDataPath` command-line flag, since this directory should be managed solely by VictoriaMetrics or `vmstorage`.
-
-## Use cases
-
-### Regular backups
-
-Regular backup can be performed with the following command:
-
-```console
-./vmbackup -storageDataPath= -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs:///
-```
-
-* `` - path to VictoriaMetrics data pointed by `-storageDataPath` command-line flag in single-node VictoriaMetrics or in cluster `vmstorage`.
- There is no need to stop VictoriaMetrics for creating backups since they are performed from immutable [instant snapshots](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-work-with-snapshots).
-* `http://victoriametrics:8428/snapshot/create` is the url for creating snapshots according to [these docs](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-work-with-snapshots). `vmbackup` creates a snapshot by querying the provided `-snapshot.createURL`, then performs the backup and then automatically removes the created snapshot.
-* `` is an already existing name for [GCS bucket](https://cloud.google.com/storage/docs/creating-buckets).
-* `` is the destination path where new backup will be placed.
-
-### Regular backups with server-side copy from existing backup
-
-If the destination GCS bucket already contains the previous backup at `-origin` path, then new backup can be sped up
-with the following command:
-
-```console
-./vmbackup -storageDataPath= -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs:/// -origin=gs:///
-```
-
-It saves time and network bandwidth costs by performing server-side copy for the shared data from the `-origin` to `-dst`.
-
-### Incremental backups
-
-Incremental backups are performed if `-dst` points to an already existing backup. In this case only new data is uploaded to remote storage.
-It saves time and network bandwidth costs when working with big backups:
-
-```console
-./vmbackup -storageDataPath= -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs:///
-```
-
-### Smart backups
-
-Smart backups mean storing full daily backups into `YYYYMMDD` folders and creating incremental hourly backup into `latest` folder:
-
-* Run the following command every hour:
-
-```console
-./vmbackup -storageDataPath= -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs:///latest
-```
-
-Where `` is the latest [snapshot](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-work-with-snapshots).
-The command will upload only changed data to `gs:///latest`.
-
-* Run the following command once a day:
-
-```console
-./vmbackup -storageDataPath= -snapshot.createURL=http://localhost:8428/snapshot/create -dst=gs:/// -origin=gs:///latest
-```
-
-Where `` is the snapshot for the last day ``.
-
-This approach saves network bandwidth costs on hourly backups (since they are incremental) and allows recovering data from either the last hour (`latest` backup)
-or from any day (`YYYYMMDD` backups). Note that hourly backup shouldn't run when creating daily backup.
-
-Do not forget to remove old backups when they are no longer needed in order to save storage costs.
-
-See also [vmbackupmanager tool](https://docs.victoriametrics.com/vmbackupmanager.html) for automating smart backups.
-
-### Server-side copy of the existing backup
-
-Sometimes it is needed to make server-side copy of the existing backup. This can be done by specifying the source backup path via `-origin` command-line flag,
-while the destination path for backup copy must be specified via `-dst` command-line flag. For example, the following command copies backup
-from `gs://bucket/foo` to `gs://bucket/bar`:
-
-```console
-./vmbackup -origin=gs://bucket/foo -dst=gs://bucket/bar
-```
-
-The `-origin` and `-dst` must point to the same object storage bucket or to the same filesystem.
-
-The server-side backup copy is usually performed at much faster speed comparing to the usual backup, since backup data isn't transferred
-between the remote storage and locally running `vmbackup` tool.
-
-If the `-dst` already contains some data, then its' contents is synced with the `-origin` data. This allows making incremental server-side copies of backups.
-
-## How does it work?
-
-The backup algorithm is the following:
-
-1. Create a snapshot by querying the provided `-snapshot.createURL`
-1. Collect information about files in the created snapshot, in the `-dst` and in the `-origin`.
-1. Determine which files in `-dst` are missing in the created snapshot, and delete them. These are usually small files, which are already merged into bigger files in the snapshot.
-1. Determine which files in the created snapshot are missing in `-dst`. These are usually small new files and bigger merged files.
-1. Determine which files from step 3 exist in the `-origin`, and perform server-side copy of these files from `-origin` to `-dst`.
- These are usually the biggest and the oldest files, which are shared between backups.
-1. Upload the remaining files from step 3 from the created snapshot to `-dst`.
-1. Delete the created snapshot.
-
-The algorithm splits source files into 1 GiB chunks in the backup. Each chunk is stored as a separate file in the backup.
-Such splitting balances between the number of files in the backup and the amounts of data that needs to be re-transferred after temporary errors.
-
-`vmbackup` relies on [instant snapshot](https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282) properties:
-
-* All the files in the snapshot are immutable.
-* Old files are periodically merged into new files.
-* Smaller files have higher probability to be merged.
-* Consecutive snapshots share many identical files.
-
-These properties allow performing fast and cheap incremental backups and server-side copying from `-origin` paths.
-See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-series-databases-533c1a927883) for more details.
-`vmbackup` can work improperly or slowly when these properties are violated.
-
-## Troubleshooting
-
-* If the backup is slow, then try setting higher value for `-concurrency` flag. This will increase the number of concurrent workers that upload data to backup storage.
-* If `vmbackup` eats all the network bandwidth or CPU, then either decrease the `-concurrency` command-line flag value or set `-maxBytesPerSecond` command-line flag value to lower value.
-* If `vmbackup` consumes all the CPU on systems with big number of CPU cores, then try running it with `-filestream.disableFadvise` command-line flag.
-* If `vmbackup` has been interrupted due to temporary error, then just restart it with the same args. It will resume the backup process.
-* Backups created from [single-node VictoriaMetrics](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html) cannot be restored
- at [cluster VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html) and vice versa.
-
-## Advanced usage
-
-
-### Providing credentials as a file
-
-Obtaining credentials from a file.
-
-Add flag `-credsFilePath=/etc/credentials` with the following content:
-
-- for S3 (AWS, MinIO or other S3 compatible storages):
-
- ```console
- [default]
- aws_access_key_id=theaccesskey
- aws_secret_access_key=thesecretaccesskeyvalue
- ```
-
-- for GCP cloud storage:
-
- ```json
- {
- "type": "service_account",
- "project_id": "project-id",
- "private_key_id": "key-id",
- "private_key": "-----BEGIN PRIVATE KEY-----\nprivate-key\n-----END PRIVATE KEY-----\n",
- "client_email": "service-account-email",
- "client_id": "client-id",
- "auth_uri": "https://accounts.google.com/o/oauth2/auth",
- "token_uri": "https://accounts.google.com/o/oauth2/token",
- "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
- "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/service-account-email"
- }
- ```
-
-### Providing credentials via env variables
-
-Obtaining credentials from env variables.
-- For AWS S3 compatible storages set env variable `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`.
- Also you can set env variable `AWS_SHARED_CREDENTIALS_FILE` with path to credentials file.
-- For GCE cloud storage set env variable `GOOGLE_APPLICATION_CREDENTIALS` with path to credentials file.
-- For Azure storage either set env variables `AZURE_STORAGE_ACCOUNT_NAME` and `AZURE_STORAGE_ACCOUNT_KEY`, or `AZURE_STORAGE_ACCOUNT_CONNECTION_STRING`.
-
-Please, note that `vmbackup` will use credentials provided by cloud providers metadata service [when applicable](https://docs.victoriametrics.com/vmbackup.html#using-cloud-providers-metadata-service).
-
-### Using cloud providers metadata service
-
-`vmbackup` and `vmbackupmanager` will automatically use cloud providers metadata service in order to obtain credentials if they are running in cloud environment
-and credentials are not explicitly provided via flags or env variables.
-
-### Providing credentials in Kubernetes
-
-The simplest way to provide credentials in Kubernetes is to use [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/)
-and inject them into the pod as environment variables. For example, the following secret can be used for AWS S3 credentials:
-```yaml
-apiVersion: v1
-kind: Secret
-metadata:
- name: vmbackup-credentials
-data:
- access_key: key
- secret_key: secret
-```
-And then it can be injected into the pod as environment variables:
-```yaml
-...
-env:
-- name: AWS_ACCESS_KEY_ID
- valueFrom:
- secretKeyRef:
- key: access_key
- name: vmbackup-credentials
-- name: AWS_SECRET_ACCESS_KEY
- valueFrom:
- secretKeyRef:
- key: secret_key
- name: vmbackup-credentials
-...
-```
-
-A more secure way is to use IAM roles to provide tokens for pods instead of managing credentials manually.
-
-For AWS deployments it will be required to configure [IAM roles for service accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).
-In order to use IAM roles for service accounts with `vmbackup` or `vmbackupmanager` it is required to create ServiceAccount with IAM role mapping:
-```yaml
-apiVersion: v1
-kind: ServiceAccount
-metadata:
- name: monitoring-backups
- annotations:
- eks.amazonaws.com/role-arn: arn:aws:iam::{ACCOUNT_ID}:role/{ROLE_NAME}
-```
-And [configure pod to use service account](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/).
-After this `vmbackup` and `vmbackupmanager` will automatically use IAM role for service account in order to obtain credentials.
-
-For GCP deployments it will be required to configure [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity).
-In order to use Workload Identity with `vmbackup` or `vmbackupmanager` it is required to create ServiceAccount with Workload Identity annotation:
-```yaml
----
-apiVersion: v1
-kind: ServiceAccount
-metadata:
- name: monitoring-backups
- annotations:
- iam.gke.io/gcp-service-account: {sa_name}@{project_name}.iam.gserviceaccount.com
-```
-And [configure pod to use service account](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/).
-After this `vmbackup` and `vmbackupmanager` will automatically use Workload Identity for servicpe account in order to obtain credentials.
-
-### Using custom S3 endpoint
-
-Usage with s3 custom url endpoint. It is possible to use `vmbackup` with s3 compatible storages like minio, cloudian, etc.
-You have to add a custom url endpoint via flag:
-
-- for MinIO
- ```console
- -customS3Endpoint=http://localhost:9000
- ```
-
-- for aws gov region
- ```console
- -customS3Endpoint=https://s3-fips.us-gov-west-1.amazonaws.com
- ```
-
-### Permanent deletion of objects in S3-compatible storages
-
-`vmbackup` and [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager.html) use standard delete operation
-for S3-compatible object storage when pefrorming [incremental backups](#incremental-backups).
-This operation removes only the current version of the object. This works OK in most cases.
-
-Sometimes it is needed to remove all the versions of an object. In this case pass `-deleteAllObjectVersions` command-line flag to `vmbackup`.
-
-Alternatively, it is possible to use object storage lifecycle rules to remove non-current versions of objects automatically.
-Refer to the respective documentation for your object storage provider for more details.
-
-### Command-line flags
-
-Run `vmbackup -help` in order to see all the available options:
-
-```console
- -concurrency int
- The number of concurrent workers. Higher concurrency may reduce backup duration (default 10)
- -configFilePath string
- Path to file with S3 configs. Configs are loaded from default location if not set.
- See https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html
- -configProfile string
- Profile name for S3 configs. If no set, the value of the environment variable will be loaded (AWS_PROFILE or AWS_DEFAULT_PROFILE), or if both not set, DefaultSharedConfigProfile is used
- -credsFilePath string
- Path to file with GCS or S3 credentials. Credentials are loaded from default locations if not set.
- See https://cloud.google.com/iam/docs/creating-managing-service-account-keys and https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html
- -customS3Endpoint string
- Custom S3 endpoint for use with S3-compatible storages (e.g. MinIO). S3 is used if not set
- -deleteAllObjectVersions
- Whether to prune previous object versions when deleting an object. By default, when object storage has versioning enabled deleting the file removes only current version. This option forces removal of all previous versions. See: https://docs.victoriametrics.com/vmbackup.html#permanent-deletion-of-objects-in-s3-compatible-storages
- -dst string
- Where to put the backup on the remote storage. Example: gs://bucket/path/to/backup, s3://bucket/path/to/backup, azblob://container/path/to/backup or fs:///path/to/local/backup/dir
- -dst can point to the previous backup. In this case incremental backup is performed, i.e. only changed data is uploaded
- -enableTCP6
- Whether to enable IPv6 for listening and dialing. By default, only IPv4 TCP and UDP are used
- -envflag.enable
- Whether to enable reading flags from environment variables in addition to the command line. Command line flag values have priority over values from environment vars. Flags are read only from the command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
- -envflag.prefix string
- Prefix for environment variables if -envflag.enable is set
- -eula
- Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -filestream.disableFadvise
- Whether to disable fadvise() syscall when reading large data files. The fadvise() syscall prevents from eviction of recently accessed data from OS page cache during background merges and backups. In some rare cases it is better to disable the syscall if it uses too much CPU
- -flagsAuthKey string
- Auth key for /flags endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -fs.disableMmap
- Whether to use pread() instead of mmap() for reading data files. By default, mmap() is used for 64-bit arches and pread() is used for 32-bit arches, since they cannot read data files bigger than 2^32 bytes in memory. mmap() is usually faster for reading small data chunks than pread()
- -http.connTimeout duration
- Incoming http connections are closed after the configured timeout. This may help to spread the incoming load among a cluster of services behind a load balancer. Please note that the real timeout may be bigger by up to 10% as a protection against the thundering herd problem (default 2m0s)
- -http.disableResponseCompression
- Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
- -http.header.csp string
- Value for 'Content-Security-Policy' header
- -http.header.frameOptions string
- Value for 'X-Frame-Options' header
- -http.header.hsts string
- Value for 'Strict-Transport-Security' header
- -http.idleConnTimeout duration
- Timeout for incoming idle http connections (default 1m0s)
- -http.maxGracefulShutdownDuration duration
- The maximum duration for a graceful shutdown of the HTTP server. A highly loaded server may require increased value for a graceful shutdown (default 7s)
- -http.pathPrefix string
- An optional prefix to add to all the paths handled by http server. For example, if '-http.pathPrefix=/foo/bar' is set, then all the http requests will be handled on '/foo/bar/*' paths. This may be useful for proxied requests. See https://www.robustperception.io/using-external-urls-and-proxies-with-prometheus
- -http.shutdownDelay duration
- Optional delay before http server shutdown. During this delay, the server returns non-OK responses from /health page, so load balancers can route new requests to other servers
- -httpAuth.password string
- Password for HTTP server's Basic Auth. The authentication is disabled if -httpAuth.username is empty
- -httpAuth.username string
- Username for HTTP server's Basic Auth. The authentication is disabled if empty. See also -httpAuth.password
- -httpListenAddr string
- TCP address for exporting metrics at /metrics page (default ":8420")
- -internStringCacheExpireDuration duration
- The expiry duration for caches for interned strings. See https://en.wikipedia.org/wiki/String_interning . See also -internStringMaxLen and -internStringDisableCache (default 6m0s)
- -internStringDisableCache
- Whether to disable caches for interned strings. This may reduce memory usage at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringCacheExpireDuration and -internStringMaxLen
- -internStringMaxLen int
- The maximum length for strings to intern. A lower limit may save memory at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringDisableCache and -internStringCacheExpireDuration (default 500)
- -license string
- Lisense key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed via file specified by -licenseFile command-line flag
- -license.forceOffline
- Whether to enable offline verification for VictoriaMetrics Enterprise license key, which has been passed either via -license or via -licenseFile command-line flag. The issued license key must support offline verification feature. Contact info@victoriametrics.com if you need offline license verification. This flag is avilable only in Enterprise binaries
- -licenseFile string
- Path to file with license key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed inline via -license command-line flag
- -loggerDisableTimestamps
- Whether to disable writing timestamps in logs
- -loggerErrorsPerSecondLimit int
- Per-second limit on the number of ERROR messages. If more than the given number of errors are emitted per second, the remaining errors are suppressed. Zero values disable the rate limit
- -loggerFormat string
- Format for logs. Possible values: default, json (default "default")
- -loggerJSONFields string
- Allows renaming fields in JSON formatted logs. Example: "ts:timestamp,msg:message" renames "ts" to "timestamp" and "msg" to "message". Supported fields: ts, level, caller, msg
- -loggerLevel string
- Minimum level of errors to log. Possible values: INFO, WARN, ERROR, FATAL, PANIC (default "INFO")
- -loggerOutput string
- Output for the logs. Supported values: stderr, stdout (default "stderr")
- -loggerTimezone string
- Timezone to use for timestamps in logs. Timezone must be a valid IANA Time Zone. For example: America/New_York, Europe/Berlin, Etc/GMT+3 or Local (default "UTC")
- -loggerWarnsPerSecondLimit int
- Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
- -maxBytesPerSecond size
- The maximum upload speed. There is no limit if it is set to 0
- Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
- -memory.allowedBytes size
- Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
- Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
- -memory.allowedPercent float
- Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)
- -metricsAuthKey string
- Auth key for /metrics endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -origin string
- Optional origin directory on the remote storage with old backup for server-side copying when performing full backup. This speeds up full backups
- -pprofAuthKey string
- Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -pushmetrics.extraLabel array
- Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
- Supports an array of values separated by comma or specified via multiple flags.
- -pushmetrics.interval duration
- Interval for pushing metrics to -pushmetrics.url (default 10s)
- -pushmetrics.url array
- Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
- Supports an array of values separated by comma or specified via multiple flags.
- -s3ForcePathStyle
- Prefixing endpoint with bucket name when set false, true by default. (default true)
- -s3StorageClass string
- The Storage Class applied to objects uploaded to AWS S3. Supported values are: GLACIER, DEEP_ARCHIVE, GLACIER_IR, INTELLIGENT_TIERING, ONEZONE_IA, OUTPOSTS, REDUCED_REDUNDANCY, STANDARD, STANDARD_IA.
- See https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html
- -snapshot.createURL string
- VictoriaMetrics create snapshot url. When this is given a snapshot will automatically be created during backup. Example: http://victoriametrics:8428/snapshot/create . There is no need in setting -snapshotName if -snapshot.createURL is set
- -snapshot.deleteURL string
- VictoriaMetrics delete snapshot url. Optional. Will be generated from -snapshot.createURL if not provided. All created snapshots will be automatically deleted. Example: http://victoriametrics:8428/snapshot/delete
- -snapshotName string
- Name for the snapshot to backup. See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-work-with-snapshots. There is no need in setting -snapshotName if -snapshot.createURL is set
- -storageDataPath string
- Path to VictoriaMetrics data. Must match -storageDataPath from VictoriaMetrics or vmstorage (default "victoria-metrics-data")
- -tls
- Whether to enable TLS for incoming HTTP requests at -httpListenAddr (aka https). -tlsCertFile and -tlsKeyFile must be set if -tls is set
- -tlsCertFile string
- Path to file with TLS certificate if -tls is set. Prefer ECDSA certs instead of RSA certs as RSA certs are slower. The provided certificate file is automatically re-read every second, so it can be dynamically updated
- -tlsCipherSuites array
- Optional list of TLS cipher suites for incoming requests over HTTPS if -tls is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants
- Supports an array of values separated by comma or specified via multiple flags.
- -tlsKeyFile string
- Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
- -tlsMinVersion string
- Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
- -version
- Show VictoriaMetrics version
-```
-
-## How to build from sources
-
-It is recommended using [binary releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest) - see `vmutils-*` archives there.
-
-### Development build
-
-1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.20.
-1. Run `make vmbackup` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmbackup` binary and puts it into the `bin` folder.
-
-### Production build
-
-1. [Install docker](https://docs.docker.com/install/).
-1. Run `make vmbackup-prod` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmbackup-prod` binary and puts it into the `bin` folder.
-
-### Building docker images
-
-Run `make package-vmbackup`. It builds `victoriametrics/vmbackup:` docker image locally.
-`` is auto-generated image tag, which depends on source code in the repository.
-The `` may be manually set via `PKG_TAG=foobar make package-vmbackup`.
-
-The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
-by setting it via `` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
-
-```console
-ROOT_IMAGE=scratch make package-vmbackup
-```
+vmbackup docs can be edited at [docs/vmbackup.md](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/vmbackup.md).
diff --git a/app/vmbackupmanager/README.md b/app/vmbackupmanager/README.md
index 32d5914cc..418e63721 100644
--- a/app/vmbackupmanager/README.md
+++ b/app/vmbackupmanager/README.md
@@ -1,554 +1,3 @@
-# vmbackupmanager
+See vmbackupmanager docs [here](https://docs.victoriametrics.com/vmbackupmanager.html).
-***vmbackupmanager is a part of [enterprise package](https://docs.victoriametrics.com/enterprise.html).
-It is available for download and evaluation at [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
-See how to request a free trial license [here](https://victoriametrics.com/products/enterprise/trial/).***
-
-The VictoriaMetrics backup manager automates regular backup procedures. It supports the following backup intervals: **hourly**, **daily**, **weekly** and **monthly**.
-Multiple backup intervals may be configured simultaneously. I.e. the backup manager creates hourly backups every hour, while it creates daily backups every day, etc.
-Backup manager must have read access to the storage data, so best practice is to install it on the same machine (or as a sidecar) where the storage node is installed.
-The backup service makes a backup every hour and puts it to the latest folder and then copies data to the folders
-which represent the backup intervals (hourly, daily, weekly and monthly)
-
-The required flags for running the service are as follows:
-
-* -eula - should be true and means that you have the legal right to run a backup manager. That can either be a signed contract or an email
- with confirmation to run the service in a trial period.
-* -storageDataPath - path to VictoriaMetrics or vmstorage data path to make backup from.
-* -snapshot.createURL - VictoriaMetrics creates snapshot URL which will automatically be created during backup. Example:
-* -dst - backup destination at [the supported storage types](https://docs.victoriametrics.com/vmbackup.html#supported-storage-types).
-* -credsFilePath - path to file with GCS or S3 credentials. Credentials are loaded from default locations if not set.
- See [https://cloud.google.com/iam/docs/creating-managing-service-account-keys](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
- and [https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html](https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html).
-
-Backup schedule is controlled by the following flags:
-
-* -disableHourly - disable hourly run. Default false
-* -disableDaily - disable daily run. Default false
-* -disableWeekly - disable weekly run. Default false
-* -disableMonthly - disable monthly run. Default false
-
-By default, all flags are turned on and Backup Manager backups data every hour for every interval (hourly, daily, weekly and monthly).
-
-The backup manager creates the following directory hierarchy at **-dst**:
-
-* /latest/ - contains the latest backup
-* /hourly/ - contains hourly backups. Each backup is named as *YYYY-MM-DD:HH*
-* /daily/ - contains daily backups. Each backup is named as *YYYY-MM-DD*
-* /weekly/ - contains weekly backups. Each backup is named as *YYYY-WW*
-* /monthly/ - contains monthly backups. Each backup is named as *YYYY-MM*
-
-To get the full list of supported flags please run the following command:
-
-```console
-./vmbackupmanager --help
-```
-
-The service creates a **full** backup each run. This means that the system can be restored fully
-from any particular backup using [vmrestore](https://docs.victoriametrics.com/vmrestore.html).
-Backup manager uploads only the data that has been changed or created since the most recent backup (incremental backup).
-This reduces the consumed network traffic and the time needed for performing the backup.
-See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-series-databases-533c1a927883) for details.
-
-*Please take into account that the first backup upload could take a significant amount of time as it needs to upload all the data.*
-
-There are two flags which could help with performance tuning:
-
-* -maxBytesPerSecond - the maximum upload speed. There is no limit if it is set to 0
-* -concurrency - The number of concurrent workers. Higher concurrency may improve upload speed (default 10)
-
-## Example of Usage
-
-GCS and cluster version. You need to have a credentials file in json format with following structure:
-
-credentials.json
-
-```json
-{
- "type": "service_account",
- "project_id": "",
- "private_key_id": "",
- "private_key": "-----BEGIN PRIVATE KEY-----\-----END PRIVATE KEY-----\n",
- "client_email": "test@.iam.gserviceaccount.com",
- "client_id": "",
- "auth_uri": "https://accounts.google.com/o/oauth2/auth",
- "token_uri": "https://oauth2.googleapis.com/token",
- "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
- "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/test%40.iam.gserviceaccount.com"
-}
-
-```
-
-Backup manager launched with the following configuration:
-
-```console
-export NODE_IP=192.168.0.10
-export VMSTORAGE_ENDPOINT=http://127.0.0.1:8428
-./vmbackupmanager -dst=gs://vmstorage-data/$NODE_IP -credsFilePath=credentials.json -storageDataPath=/vmstorage-data -snapshot.createURL=$VMSTORAGE_ENDPOINT/snapshot/create -eula
-```
-
-Expected logs in vmbackupmanager:
-
-```console
-info lib/backup/actions/backup.go:131 server-side copied 81 out of 81 parts from GCS{bucket: "vmstorage-data", dir: "192.168.0.10//latest/"} to GCS{bucket: "vmstorage-data", dir: "192.168.0.10//weekly/2020-34/"} in 2.549833008s
-info lib/backup/actions/backup.go:169 backed up 853315 bytes in 2.882 seconds; deleted 0 bytes; server-side copied 853315 bytes; uploaded 0 bytes
-```
-
-Expected logs in vmstorage:
-
-```console
-info VictoriaMetrics/lib/storage/table.go:146 creating table snapshot of "/vmstorage-data/data"...
-info VictoriaMetrics/lib/storage/storage.go:311 deleting snapshot "/vmstorage-data/snapshots/20200818201959-162C760149895DDA"...
-info VictoriaMetrics/lib/storage/storage.go:319 deleted snapshot "/vmstorage-data/snapshots/20200818201959-162C760149895DDA" in 0.169 seconds
-```
-
-The result on the GCS bucket
-
-* The root folder
-
- 
-
-* The latest folder
-
- 
-
-Please, see [vmbackup docs](https://docs.victoriametrics.com/vmbackup.html#advanced-usage) for more examples of authentication with different
-storage types.
-
-## Backup Retention Policy
-
-Backup retention policy is controlled by:
-
-* -keepLastHourly - keep the last N hourly backups. Disabled by default
-* -keepLastDaily - keep the last N daily backups. Disabled by default
-* -keepLastWeekly - keep the last N weekly backups. Disabled by default
-* -keepLastMonthly - keep the last N monthly backups. Disabled by default
-
-> *Note*: 0 value in every keepLast flag results into deletion of ALL backups for particular type (hourly, daily, weekly and monthly)
-
-> *Note*: retention policy does not enforce removing previous versions of objects in object storages such if versioning is enabled. See [these docs](https://docs.victoriametrics.com/vmbackup.html#permanent-deletion-of-objects-in-s3-compatible-storages) for more details.
-
-Let’s assume we have a backup manager collecting daily backups for the past 10 days.
-
-
-
-We enable backup retention policy for backup manager by using following configuration:
-
-```console
-export NODE_IP=192.168.0.10
-export VMSTORAGE_ENDPOINT=http://127.0.0.1:8428
-./vmbackupmanager -dst=gs://vmstorage-data/$NODE_IP -credsFilePath=credentials.json -storageDataPath=/vmstorage-data -snapshot.createURL=$VMSTORAGE_ENDPOINT/snapshot/create
--keepLastDaily=3 -eula
-```
-
-Expected logs in backup manager on start:
-
-```console
-info lib/logger/flag.go:20 flag "keepLastDaily" = "3"
-```
-
-Expected logs in backup manager during retention cycle:
-
-```console
-info app/vmbackupmanager/retention.go:106 daily backups to delete [daily/2021-02-13 daily/2021-02-12 daily/2021-02-11 daily/2021-02-10 daily/2021-02-09 daily/2021-02-08 daily/2021-02-07]
-```
-
-The result on the GCS bucket. We see only 3 daily backups:
-
-
-
-### Protection backups against deletion by retention policy
-
-You can protect any backup against deletion by retention policy with the `vmbackupmanager backups lock` command.
-
-For instance:
-
-```console
-./vmbackupmanager backup lock daily/2021-02-13 -dst= -storageDataPath=/vmstorage-data -eula
-```
-
-After that the backup won't be deleted by retention policy.
-You can view the `locked` attribute in backup list:
-
-```console
-./vmbackupmanager backup list -dst= -storageDataPath=/vmstorage-data -eula
-```
-
-To remove protection, you can use the command `vmbackupmanager backups unlock`.
-
-For example:
-
-```console
-./vmbackupmanager backup unlock daily/2021-02-13 -dst= -storageDataPath=/vmstorage-data -eula
-```
-
-## API methods
-
-`vmbackupmanager` exposes the following API methods:
-
-* GET `/api/v1/backups` - returns list of backups in remote storage.
- Example output:
- ```json
- [{"name":"daily/2023-04-07","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:07+00:00"},{"name":"hourly/2023-04-07:11","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:06+00:00"},{"name":"latest","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:04+00:00"},{"name":"monthly/2023-04","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:10+00:00"},{"name":"weekly/2023-14","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:09+00:00"}]
- ```
- > Note: `created_at` field is in RFC3339 format.
-
-* GET `/api/v1/backups/` - returns backup info by name.
- Example output:
- ```json
- {"name":"daily/2023-04-07","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:07+00:00","locked":true}
- ```
-
-* PUT `/api/v1/backups/` - update "locked" attribute for backup by name.
- Example request body:
- ```json
- {"locked":true}
- ```
- Example response:
- ```json
- {"name":"daily/2023-04-07","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:07+00:00", "locked": true}
- ```
-
-* POST `/api/v1/restore` - saves backup name to restore when [performing restore](#restore-commands).
- Example request body:
- ```json
- {"backup":"daily/2022-10-06"}
- ```
-
-* GET `/api/v1/restore` - returns backup name from restore mark if it exists.
- Example response:
- ```json
- {"backup":"daily/2022-10-06"}
- ```
-
-* DELETE `/api/v1/restore` - delete restore mark.
-
-## CLI
-
-`vmbackupmanager` exposes CLI commands to work with [API methods](#api-methods) without external dependencies.
-
-Supported commands:
-```console
-vmbackupmanager backup
-
- vmbackupmanager backup list
- List backups in remote storage
-
- vmbackupmanager backup lock
- Locks backup in remote storage against deletion
-
- vmbackupmanager backup unlock
- Unlocks backup in remote storage for deletion
-
-vmbackupmanager restore
- Restore backup specified by restore mark if it exists
-
- vmbackupmanager restore get
- Get restore mark if it exists
-
- vmbackupmanager restore delete
- Delete restore mark if it exists
-
- vmbackupmanager restore create [backup_name]
- Create restore mark
-```
-
-By default, CLI commands are using `http://127.0.0.1:8300` endpoint to reach `vmbackupmanager` API.
-It can be changed by using flag:
-```
--apiURL string
- vmbackupmanager address to perform API requests (default "http://127.0.0.1:8300")
-```
-
-### Backup commands
-
-`vmbackupmanager backup list` lists backups in remote storage:
-```console
-$ ./vmbackupmanager backup list
-[{"name":"daily/2023-04-07","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:07+00:00"},{"name":"hourly/2023-04-07:11","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:06+00:00"},{"name":"latest","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:04+00:00"},{"name":"monthly/2023-04","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:10+00:00"},{"name":"weekly/2023-14","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:09+00:00"}]
-```
-
-### Restore commands
-
-Restore commands are used to create, get and delete restore mark.
-Restore mark is used by `vmbackupmanager` to store backup name to restore when running restore.
-
-
-Create restore mark:
-```console
-$ ./vmbackupmanager restore create daily/2022-10-06
-```
-
-Get restore mark if it exists:
-```console
-$ ./vmbackupmanager restore get
-{"backup":"daily/2022-10-06"}
-```
-
-Delete restore mark if it exists:
-```console
-$ ./vmbackupmanager restore delete
-```
-
-Perform restore:
-```console
-$ /vmbackupmanager-prod restore -dst=gs://vmstorage-data/$NODE_IP -credsFilePath=credentials.json -storageDataPath=/vmstorage-data
-```
-Note that `vmsingle` or `vmstorage` should be stopped before performing restore.
-
-If restore mark doesn't exist at `storageDataPath`(restore wasn't requested) `vmbackupmanager restore` will exit with successful status code.
-
-### How to restore backup via CLI
-
-1. Run `vmbackupmanager backup list` to get list of available backups:
- ```console
- $ /vmbackupmanager-prod backup list
- [{"name":"daily/2023-04-07","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:07+00:00"},{"name":"hourly/2023-04-07:11","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:06+00:00"},{"name":"latest","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:04+00:00"},{"name":"monthly/2023-04","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:10+00:00"},{"name":"weekly/2023-14","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:09+00:00"}]
- ```
-1. Run `vmbackupmanager restore create` to create restore mark:
- - Use relative path to backup to restore from currently used remote storage:
- ```console
- $ /vmbackupmanager-prod restore create daily/2023-04-07
- ```
- - Use full path to backup to restore from any remote storage:
- ```console
- $ /vmbackupmanager-prod restore create azblob://test1/vmbackupmanager/daily/2023-04-07
- ```
-1. Stop `vmstorage` or `vmsingle` node
-1. Run `vmbackupmanager restore` to restore backup:
- ```console
- $ /vmbackupmanager-prod restore -credsFilePath=credentials.json -storageDataPath=/vmstorage-data
- ```
-1. Start `vmstorage` or `vmsingle` node
-
-
-### How to restore in Kubernetes
-
-1. Ensure there is an init container with `vmbackupmanager restore` in `vmstorage` or `vmsingle` pod.
- For [VictoriaMetrics operator](https://docs.victoriametrics.com/operator/VictoriaMetrics-Operator.html) deployments it is required to add:
- ```yaml
- vmbackup:
- restore:
- onStart:
- enabled: "true"
- ```
- See operator `VMStorage` schema [here](https://docs.victoriametrics.com/operator/api.html#vmstorage) and `VMSingle` [here](https://docs.victoriametrics.com/operator/api.html#vmsinglespec).
-1. Enter container running `vmbackupmanager`
-1. Use `vmbackupmanager backup list` to get list of available backups:
- ```console
- $ /vmbackupmanager-prod backup list
- [{"name":"daily/2023-04-07","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:07+00:00"},{"name":"hourly/2023-04-07:11","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:06+00:00"},{"name":"latest","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:04+00:00"},{"name":"monthly/2023-04","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:10+00:00"},{"name":"weekly/2023-14","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:09+00:00"}]
- ```
-1. Use `vmbackupmanager restore create` to create restore mark:
- - Use relative path to backup to restore from currently used remote storage:
- ```console
- $ /vmbackupmanager-prod restore create daily/2023-04-07
- ```
- - Use full path to backup to restore from any remote storage:
- ```console
- $ /vmbackupmanager-prod restore create azblob://test1/vmbackupmanager/daily/2023-04-07
- ```
-1. Restart pod
-
-#### Restore cluster into another cluster
-
-These steps are assuming that [VictoriaMetrics operator](https://docs.victoriametrics.com/operator/VictoriaMetrics-Operator.html) is used to manage `VMCluster`.
-Clusters here are referred to as `source` and `destination`.
-
-1. Create a new cluster with access to *source* cluster `vmbackupmanager` storage and same number of storage nodes.
- Add the following section in order to enable restore on start (operator `VMStorage` schema can be found [here](https://docs.victoriametrics.com/operator/api.html#vmstorage):
- ```yaml
- vmbackup:
- restore:
- onStart:
- enabled: "true"
- ```
- Note: it is safe to leave this section in the cluster configuration, since it will be ignored if restore mark doesn't exist.
- > Important! Use different `-dst` for *destination* cluster to avoid overwriting backup data of the *source* cluster.
-1. Enter container running `vmbackupmanager` in *source* cluster
-1. Use `vmbackupmanager backup list` to get list of available backups:
- ```console
- $ /vmbackupmanager-prod backup list
- [{"name":"daily/2023-04-07","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:07+00:00"},{"name":"hourly/2023-04-07:11","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:06+00:00"},{"name":"latest","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:04+00:00"},{"name":"monthly/2023-04","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:10+00:00"},{"name":"weekly/2023-14","size_bytes":318837,"size":"311.4ki","created_at":"2023-04-07T16:15:09+00:00"}]
- ```
-1. Use `vmbackupmanager restore create` to create restore mark at each pod of the *destination* cluster.
- Each pod in *destination* cluster should be restored from backup of respective pod in *source* cluster.
- For example: `vmstorage-source-0` in *source* cluster should be restored from `vmstorage-destination-0` in *destination* cluster.
- ```console
- $ /vmbackupmanager-prod restore create s3://source_cluster/vmstorage-source-0/daily/2023-04-07
- ```
-1. Restart `vmstorage` pods of *destination* cluster. On pod start `vmbackupmanager` will restore data from the specified backup.
-
-## Monitoring
-
-`vmbackupmanager` exports various metrics in Prometheus exposition format at `http://vmbackupmanager:8300/metrics` page. It is recommended setting up regular scraping of this page
-either via [vmagent](https://docs.victoriametrics.com/vmagent.html) or via Prometheus, so the exported metrics could be analyzed later.
-
-Use the official [Grafana dashboard](https://grafana.com/grafana/dashboards/17798-victoriametrics-backupmanager/) for `vmbackupmanager` overview.
-Graphs on this dashboard contain useful hints - hover the `i` icon in the top left corner of each graph in order to read it.
-If you have suggestions for improvements or have found a bug - please open an issue on github or add
-a review to the dashboard.
-
-## Configuration
-
-### Flags
-
-Pass `-help` to `vmbackupmanager` in order to see the full list of supported
-command-line flags with their descriptions.
-
-The shortlist of configuration flags is the following:
-
-```
-vmbackupmanager performs regular backups according to the provided configs.
-
-subcommands:
- backup: provides auxiliary backup-related commands
- restore: restores backup specified by restore mark if it exists
-
-command-line flags:
- -apiURL string
- vmbackupmanager address to perform API requests (default "http://127.0.0.1:8300")
- -concurrency int
- The number of concurrent workers. Higher concurrency may reduce backup duration (default 10)
- -configFilePath string
- Path to file with S3 configs. Configs are loaded from default location if not set.
- See https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html
- -configProfile string
- Profile name for S3 configs. If no set, the value of the environment variable will be loaded (AWS_PROFILE or AWS_DEFAULT_PROFILE), or if both not set, DefaultSharedConfigProfile is used
- -credsFilePath string
- Path to file with GCS or S3 credentials. Credentials are loaded from default locations if not set.
- See https://cloud.google.com/iam/docs/creating-managing-service-account-keys and https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html
- -customS3Endpoint string
- Custom S3 endpoint for use with S3-compatible storages (e.g. MinIO). S3 is used if not set
- -deleteAllObjectVersions
- Whether to prune previous object versions when deleting an object. By default, when object storage has versioning enabled deleting the file removes only current version. This option forces removal of all previous versions. See: https://docs.victoriametrics.com/vmbackup.html#permanent-deletion-of-objects-in-s3-compatible-storages
- -disableDaily
- Disable daily run. Default false
- -disableHourly
- Disable hourly run. Default false
- -disableMonthly
- Disable monthly run. Default false
- -disableWeekly
- Disable weekly run. Default false
- -dst string
- The root folder of Victoria Metrics backups. Example: gs://bucket/path/to/backup/dir, s3://bucket/path/to/backup/dir or fs:///path/to/local/backup/dir
- -enableTCP6
- Whether to enable IPv6 for listening and dialing. By default, only IPv4 TCP and UDP are used
- -envflag.enable
- Whether to enable reading flags from environment variables in addition to the command line. Command line flag values have priority over values from environment vars. Flags are read only from the command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
- -envflag.prefix string
- Prefix for environment variables if -envflag.enable is set
- -eula
- Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -filestream.disableFadvise
- Whether to disable fadvise() syscall when reading large data files. The fadvise() syscall prevents from eviction of recently accessed data from OS page cache during background merges and backups. In some rare cases it is better to disable the syscall if it uses too much CPU
- -flagsAuthKey string
- Auth key for /flags endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -fs.disableMmap
- Whether to use pread() instead of mmap() for reading data files. By default, mmap() is used for 64-bit arches and pread() is used for 32-bit arches, since they cannot read data files bigger than 2^32 bytes in memory. mmap() is usually faster for reading small data chunks than pread()
- -http.connTimeout duration
- Incoming http connections are closed after the configured timeout. This may help to spread the incoming load among a cluster of services behind a load balancer. Please note that the real timeout may be bigger by up to 10% as a protection against the thundering herd problem (default 2m0s)
- -http.disableResponseCompression
- Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
- -http.header.csp string
- Value for 'Content-Security-Policy' header
- -http.header.frameOptions string
- Value for 'X-Frame-Options' header
- -http.header.hsts string
- Value for 'Strict-Transport-Security' header
- -http.idleConnTimeout duration
- Timeout for incoming idle http connections (default 1m0s)
- -http.maxGracefulShutdownDuration duration
- The maximum duration for a graceful shutdown of the HTTP server. A highly loaded server may require increased value for a graceful shutdown (default 7s)
- -http.pathPrefix string
- An optional prefix to add to all the paths handled by http server. For example, if '-http.pathPrefix=/foo/bar' is set, then all the http requests will be handled on '/foo/bar/*' paths. This may be useful for proxied requests. See https://www.robustperception.io/using-external-urls-and-proxies-with-prometheus
- -http.shutdownDelay duration
- Optional delay before http server shutdown. During this delay, the server returns non-OK responses from /health page, so load balancers can route new requests to other servers
- -httpAuth.password string
- Password for HTTP server's Basic Auth. The authentication is disabled if -httpAuth.username is empty
- -httpAuth.username string
- Username for HTTP server's Basic Auth. The authentication is disabled if empty. See also -httpAuth.password
- -httpListenAddr string
- Address to listen for http connections (default ":8300")
- -internStringCacheExpireDuration duration
- The expiry duration for caches for interned strings. See https://en.wikipedia.org/wiki/String_interning . See also -internStringMaxLen and -internStringDisableCache (default 6m0s)
- -internStringDisableCache
- Whether to disable caches for interned strings. This may reduce memory usage at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringCacheExpireDuration and -internStringMaxLen
- -internStringMaxLen int
- The maximum length for strings to intern. A lower limit may save memory at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringDisableCache and -internStringCacheExpireDuration (default 500)
- -keepLastDaily int
- Keep last N daily backups. If 0 is specified next retention cycle removes all backups for given time period. (default -1)
- -keepLastHourly int
- Keep last N hourly backups. If 0 is specified next retention cycle removes all backups for given time period. (default -1)
- -keepLastMonthly int
- Keep last N monthly backups. If 0 is specified next retention cycle removes all backups for given time period. (default -1)
- -keepLastWeekly int
- Keep last N weekly backups. If 0 is specified next retention cycle removes all backups for given time period. (default -1)
- -license string
- Lisense key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed via file specified by -licenseFile command-line flag
- -license.forceOffline
- Whether to enable offline verification for VictoriaMetrics Enterprise license key, which has been passed either via -license or via -licenseFile command-line flag. The issued license key must support offline verification feature. Contact info@victoriametrics.com if you need offline license verification. This flag is avilable only in Enterprise binaries
- -licenseFile string
- Path to file with license key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed inline via -license command-line flag
- -loggerDisableTimestamps
- Whether to disable writing timestamps in logs
- -loggerErrorsPerSecondLimit int
- Per-second limit on the number of ERROR messages. If more than the given number of errors are emitted per second, the remaining errors are suppressed. Zero values disable the rate limit
- -loggerFormat string
- Format for logs. Possible values: default, json (default "default")
- -loggerJSONFields string
- Allows renaming fields in JSON formatted logs. Example: "ts:timestamp,msg:message" renames "ts" to "timestamp" and "msg" to "message". Supported fields: ts, level, caller, msg
- -loggerLevel string
- Minimum level of errors to log. Possible values: INFO, WARN, ERROR, FATAL, PANIC (default "INFO")
- -loggerOutput string
- Output for the logs. Supported values: stderr, stdout (default "stderr")
- -loggerTimezone string
- Timezone to use for timestamps in logs. Timezone must be a valid IANA Time Zone. For example: America/New_York, Europe/Berlin, Etc/GMT+3 or Local (default "UTC")
- -loggerWarnsPerSecondLimit int
- Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
- -maxBytesPerSecond int
- The maximum upload speed. There is no limit if it is set to 0
- -memory.allowedBytes size
- Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
- Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
- -memory.allowedPercent float
- Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)
- -metricsAuthKey string
- Auth key for /metrics endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -pprofAuthKey string
- Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -pushmetrics.extraLabel array
- Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
- Supports an array of values separated by comma or specified via multiple flags.
- -pushmetrics.interval duration
- Interval for pushing metrics to -pushmetrics.url (default 10s)
- -pushmetrics.url array
- Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
- Supports an array of values separated by comma or specified via multiple flags.
- -runOnStart
- Upload backups immediately after start of the service. Otherwise the backup starts on new hour
- -s3ForcePathStyle
- Prefixing endpoint with bucket name when set false, true by default. (default true)
- -s3StorageClass string
- The Storage Class applied to objects uploaded to AWS S3. Supported values are: GLACIER, DEEP_ARCHIVE, GLACIER_IR, INTELLIGENT_TIERING, ONEZONE_IA, OUTPOSTS, REDUCED_REDUNDANCY, STANDARD, STANDARD_IA.
- See https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html
- -snapshot.createURL string
- VictoriaMetrics create snapshot url. When this is given a snapshot will automatically be created during backup.Example: http://victoriametrics:8428/snapshot/create
- -snapshot.deleteURL string
- VictoriaMetrics delete snapshot url. Optional. Will be generated from snapshot.createURL if not provided. All created snaphosts will be automatically deleted.Example: http://victoriametrics:8428/snapshot/delete
- -storageDataPath string
- Path to VictoriaMetrics data. Must match -storageDataPath from VictoriaMetrics or vmstorage (default "victoria-metrics-data")
- -tls
- Whether to enable TLS for incoming HTTP requests at -httpListenAddr (aka https). -tlsCertFile and -tlsKeyFile must be set if -tls is set
- -tlsCertFile string
- Path to file with TLS certificate if -tls is set. Prefer ECDSA certs instead of RSA certs as RSA certs are slower. The provided certificate file is automatically re-read every second, so it can be dynamically updated
- -tlsCipherSuites array
- Optional list of TLS cipher suites for incoming requests over HTTPS if -tls is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants
- Supports an array of values separated by comma or specified via multiple flags.
- -tlsKeyFile string
- Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
- -tlsMinVersion string
- Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
- -version
- Show VictoriaMetrics version
-```
+vmbackupmanager docs can be edited at [docs/vmbackupmanager.md](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/vmbackupmanager.md).
diff --git a/app/vmbackupmanager/vmbackupmanager_latest_folder.png b/app/vmbackupmanager/vmbackupmanager_latest_folder.png
deleted file mode 100644
index 23162c905..000000000
Binary files a/app/vmbackupmanager/vmbackupmanager_latest_folder.png and /dev/null differ
diff --git a/app/vmbackupmanager/vmbackupmanager_root_folder.png b/app/vmbackupmanager/vmbackupmanager_root_folder.png
deleted file mode 100644
index 5d64e5525..000000000
Binary files a/app/vmbackupmanager/vmbackupmanager_root_folder.png and /dev/null differ
diff --git a/app/vmbackupmanager/vmbackupmanager_rp_daily_1.png b/app/vmbackupmanager/vmbackupmanager_rp_daily_1.png
deleted file mode 100644
index f2f050aa7..000000000
Binary files a/app/vmbackupmanager/vmbackupmanager_rp_daily_1.png and /dev/null differ
diff --git a/app/vmbackupmanager/vmbackupmanager_rp_daily_2.png b/app/vmbackupmanager/vmbackupmanager_rp_daily_2.png
deleted file mode 100644
index e716c5d6a..000000000
Binary files a/app/vmbackupmanager/vmbackupmanager_rp_daily_2.png and /dev/null differ
diff --git a/app/vmctl/README.md b/app/vmctl/README.md
index 97fc16945..a45c529d1 100644
--- a/app/vmctl/README.md
+++ b/app/vmctl/README.md
@@ -1,1108 +1,3 @@
-# vmctl
+See vmctl docs [here](https://docs.victoriametrics.com/vmctl.html).
-VictoriaMetrics command-line tool
-
-vmctl provides various useful actions with VictoriaMetrics components.
-
-Features:
-- migrate data from [Prometheus](#migrating-data-from-prometheus) to VictoriaMetrics using snapshot API
-- migrate data from [Thanos](#migrating-data-from-thanos) to VictoriaMetrics
-- migrate data from [Cortex](#migrating-data-from-cortex) to VictoriaMetrics
-- migrate data from [Mimir](#migrating-data-from-mimir) to VictoriaMetrics
-- migrate data from [InfluxDB](#migrating-data-from-influxdb-1x) to VictoriaMetrics
-- migrate data from [OpenTSDB](#migrating-data-from-opentsdb) to VictoriaMetrics
-- migrate data from [Promscale](#migrating-data-from-promscale)
-- migrate data between [VictoriaMetrics](#migrating-data-from-victoriametrics) single or cluster version.
-- migrate data by [Prometheus remote read protocol](#migrating-data-by-remote-read-protocol) to VictoriaMetrics
-- [verify](#verifying-exported-blocks-from-victoriametrics) exported blocks from VictoriaMetrics single or cluster version.
-
-To see the full list of supported modes
-run the following command:
-
-```console
-$ ./vmctl --help
-NAME:
- vmctl - VictoriaMetrics command-line tool
-
-USAGE:
- vmctl [global options] command [command options] [arguments...]
-
-COMMANDS:
- opentsdb Migrate timeseries from OpenTSDB
- influx Migrate timeseries from InfluxDB
- prometheus Migrate timeseries from Prometheus
- vm-native Migrate time series between VictoriaMetrics installations via native binary format
- remote-read Migrate timeseries by Prometheus remote read protocol
- verify-block Verifies correctness of data blocks exported via VictoriaMetrics Native format. See https://docs.victoriametrics.com/#how-to-export-data-in-native-format
-```
-
-Each mode has its own unique set of flags specific (e.g. prefixed with `influx` for influx mode)
-to the data source and common list of flags for destination (prefixed with `vm` for VictoriaMetrics):
-
-```
-$ ./vmctl influx --help
-OPTIONS:
- --influx-addr value InfluxDB server addr (default: "http://localhost:8086")
- --influx-user value InfluxDB user [$INFLUX_USERNAME]
-...
- --vm-addr vmctl VictoriaMetrics address to perform import requests.
-Should be the same as --httpListenAddr value for single-node version or vminsert component.
-When importing into the clustered version do not forget to set additionally --vm-account-id flag.
-Please note, that vmctl performs initial readiness check for the given address by checking `/health` endpoint. (default: "http://localhost:8428")
- --vm-user value VictoriaMetrics username for basic auth [$VM_USERNAME]
- --vm-password value VictoriaMetrics password for basic auth [$VM_PASSWORD]
-```
-
-When doing a migration user needs to specify flags for source (where and how to fetch data) and for
-destination (where to migrate data). Every mode has additional details and nuances, please see
-them below in corresponding sections.
-
-For the destination flags see the full description by running the following command:
-
-```
-$ ./vmctl influx --help | grep vm-
-```
-
-Some flags like [--vm-extra-label](#adding-extra-labels) or [--vm-significant-figures](#significant-figures)
-has additional sections with description below. Details about tweaking and adjusting settings
-are explained in [Tuning](#tuning) section.
-
-Please note, that if you're going to import data into VictoriaMetrics cluster do not
-forget to specify the `--vm-account-id` flag. See more details for cluster version
-[here](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster).
-
-## Articles
-
-- [How to migrate data from Prometheus](https://medium.com/@romanhavronenko/victoriametrics-how-to-migrate-data-from-prometheus-d44a6728f043)
-- [How to migrate data from Prometheus. Filtering and modifying time series](https://medium.com/@romanhavronenko/victoriametrics-how-to-migrate-data-from-prometheus-filtering-and-modifying-time-series-6d40cea4bf21)
-
-## Migrating data from OpenTSDB
-
-`vmctl` supports the `opentsdb` mode to migrate data from OpenTSDB to VictoriaMetrics time-series database.
-
-See `./vmctl opentsdb --help` for details and full list of flags.
-
-**Important:** OpenTSDB migration is not possible without a functioning [meta](http://opentsdb.net/docs/build/html/user_guide/metadata.html) table to search for metrics/series. Check in OpenTSDB config that appropriate options are [activated]( https://github.com/OpenTSDB/opentsdb/issues/681#issuecomment-177359563) and HBase meta tables are present. W/o them migration won't work.
-
-OpenTSDB migration works like so:
-
-1. Find metrics based on selected filters (or the default filter set `['a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z']`):
-
- `curl -Ss "http://opentsdb:4242/api/suggest?type=metrics&q=sys"`
-
-1. Find series associated with each returned metric:
-
- `curl -Ss "http://opentsdb:4242/api/search/lookup?m=system.load5&limit=1000000"`
-
- Here `results` return field should not be empty. Otherwise, it means that meta tables are absent and needs to be turned on previously.
-
-1. Download data for each series in chunks defined in the CLI switches:
-
- `-retention=sum-1m-avg:1h:90d` means:
- - `curl -Ss "http://opentsdb:4242/api/query?start=1h-ago&end=now&m=sum:1m-avg-none:system.load5\{host=host1\}"`
- - `curl -Ss "http://opentsdb:4242/api/query?start=2h-ago&end=1h-ago&m=sum:1m-avg-none:system.load5\{host=host1\}"`
- - `curl -Ss "http://opentsdb:4242/api/query?start=3h-ago&end=2h-ago&m=sum:1m-avg-none:system.load5\{host=host1\}"`
- - ...
- - `curl -Ss "http://opentsdb:4242/api/query?start=2160h-ago&end=2159h-ago&m=sum:1m-avg-none:system.load5\{host=host1\}"`
-
-This means that we must stream data from OpenTSDB to VictoriaMetrics in chunks. This is where concurrency for OpenTSDB comes in. We can query multiple chunks at once, but we shouldn't perform too many chunks at a time to avoid overloading the OpenTSDB cluster.
-
-```
-$ ./vmctl opentsdb --otsdb-addr http://opentsdb:4242/ --otsdb-retentions sum-1m-avg:1h:1d --otsdb-filters system --otsdb-normalize --vm-addr http://victoria:8428/
-OpenTSDB import mode
-2021/04/09 11:52:50 Will collect data starting at TS 1617990770
-2021/04/09 11:52:50 Loading all metrics from OpenTSDB for filters: [system]
-Found 9 metrics to import. Continue? [Y/n]
-2021/04/09 11:52:51 Starting work on system.load1
-23 / 402200 [>____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________] 0.01% 2 p/s
-```
-Where `:8428` is Prometheus port of VictoriaMetrics.
-
-For clustered VictoriaMetrics setup `--vm-account-id` flag needs to be added, for example:
-
-```
-$ ./vmctl opentsdb --otsdb-addr http://opentsdb:4242/ --otsdb-retentions sum-1m-avg:1h:1d --otsdb-filters system --otsdb-normalize --vm-addr http://victoria:8480/ --vm-account-id 0
-```
-This time `:8480` port is vminsert/Prometheus input port.
-
-### Retention strings
-
-Starting with a relatively simple retention string (`sum-1m-avg:1h:30d`), let's describe how this is converted into actual queries.
-
-There are two essential parts of a retention string:
-
-1. [aggregation](#aggregation)
-1. [windows/time ranges](#windows)
-
-#### Aggregation
-
-Retention strings essentially define the two levels of aggregation for our collected series.
-
-`sum-1m-avg` would become:
-
-- First order: `sum`
-- Second order: `1m-avg-none`
-
-##### First Order Aggregations
-
-First-order aggregation addresses how to aggregate any un-mentioned tags.
-
-This is, conceptually, directly opposite to how PromQL deals with tags. In OpenTSDB, if a tag isn't explicitly mentioned, all values associated with that tag will be aggregated.
-
-It is recommended to use `sum` for the first aggregation because it is relatively quick and should not cause any changes to the incoming data (because we collect each individual series).
-
-##### Second Order Aggregations
-
-Second-order aggregation (`1m-avg` in our example) defines any windowing that should occur before returning the data
-
-It is recommended to match the stat collection interval, so we again avoid transforming incoming data.
-
-We do not allow for defining the "null value" portion of the rollup window (e.g. in the aggregation, `1m-avg-none`, the user cannot change `none`), as the goal of this tool is to avoid modifying incoming data.
-
-#### Windows
-
-There are two important windows we define in a retention string:
-
-1. the "chunk" range of each query
-1. The time range we will be querying on with that "chunk"
-
-From our example, our windows are `1h:30d`.
-
-##### Window "chunks"
-
-The window `1h` means that each individual query to OpenTSDB should only span 1 hour of time (e.g. `start=2h-ago&end=1h-ago`).
-
-It is important to ensure this window somewhat matches the row size in HBase to help improve query times.
-
-For example, if the query is hitting a rollup table with a 4-hour row size, we should set a chunk size of a multiple of 4 hours (e.g. `4h`, `8h`, etc.) to avoid requesting data across row boundaries. Landing on row boundaries allows for more consistent request times to HBase.
-
-The default table created in HBase for OpenTSDB has a 1-hour row size, so if you aren't sure on a correct row size to use, `1h` is a reasonable choice.
-
-##### Time range
-
-The time range `30d` simply means we are asking for the last 30 days of data. This time range can be written using `h`, `d`, `w`, or `y`. (We can't use `m` for month because it already means `minute` in time parsing).
-
-#### Results of retention string
-
-The resultant queries that will be created, based on our example retention string of `sum-1m-avg:1h:30d` look like this:
-
-```
-http://opentsdb:4242/api/query?start=1h-ago&end=now&m=sum:1m-avg-none:
-http://opentsdb:4242/api/query?start=2h-ago&end=1h-ago&m=sum:1m-avg-none:
-http://opentsdb:4242/api/query?start=3h-ago&end=2h-ago&m=sum:1m-avg-none:
-...
-http://opentsdb:4242/api/query?start=721h-ago&end=720h-ago&m=sum:1m-avg-none:
-```
-
-Chunking the data like this means each individual query returns faster, so we can start populating data into VictoriaMetrics quicker.
-
-### Restarting OpenTSDB migrations
-
-One important note for OpenTSDB migration: Queries/HBase scans can "get stuck" within OpenTSDB itself. This can cause instability and performance issues within an OpenTSDB cluster, so stopping the migrator to deal with it may be necessary. Because of this, we provide the timestamp we started collecting data from at the beginning of the run. You can stop and restart the importer using this "hard timestamp" to ensure you collect data from the same time range over multiple runs.
-
-## Migrating data from InfluxDB (1.x)
-
-`vmctl` supports the `influx` mode for [migrating data from InfluxDB to VictoriaMetrics](https://docs.victoriametrics.com/guides/migrate-from-influx.html)
-time-series database.
-
-See `./vmctl influx --help` for details and full list of flags.
-
-To use migration tool please specify the InfluxDB address `--influx-addr`, the database `--influx-database` and VictoriaMetrics address `--vm-addr`.
-Flag `--vm-addr` for single-node VM is usually equal to `--httpListenAddr`, and for cluster version
-is equal to `--httpListenAddr` flag of vminsert component. Please note, that vmctl performs initial readiness check for the given address
-by checking `/health` endpoint. For cluster version it is additionally required to specify the `--vm-account-id` flag.
-See more details for cluster version [here](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster).
-
-As soon as required flags are provided and all endpoints are accessible, `vmctl` will start the InfluxDB scheme exploration.
-Basically, it just fetches all fields and timeseries from the provided database and builds up registry of all available timeseries.
-Then `vmctl` sends fetch requests for each timeseries to InfluxDB one by one and pass results to VM importer.
-VM importer then accumulates received samples in batches and sends import requests to VM.
-
-The importing process example for local installation of InfluxDB(`http://localhost:8086`)
-and single-node VictoriaMetrics(`http://localhost:8428`):
-
-```
-./vmctl influx --influx-database benchmark
-InfluxDB import mode
-2020/01/18 20:47:11 Exploring scheme for database "benchmark"
-2020/01/18 20:47:11 fetching fields: command: "show field keys"; database: "benchmark"; retention: "autogen"
-2020/01/18 20:47:11 found 10 fields
-2020/01/18 20:47:11 fetching series: command: "show series "; database: "benchmark"; retention: "autogen"
-Found 40000 timeseries to import. Continue? [Y/n] y
-40000 / 40000 [-----------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 21 p/s
-2020/01/18 21:19:00 Import finished!
-2020/01/18 21:19:00 VictoriaMetrics importer stats:
- idle duration: 13m51.461434876s;
- time spent while importing: 17m56.923899847s;
- total samples: 345600000;
- samples/s: 320914.04;
- total bytes: 5.9 GB;
- bytes/s: 5.4 MB;
- import requests: 40001;
-2020/01/18 21:19:00 Total time: 31m48.467044016s
-```
-
-### Data mapping
-
-Vmctl maps InfluxDB data the same way as VictoriaMetrics does by using the following rules:
-
-- `influx-database` arg is mapped into `db` label value unless `db` tag exists in the InfluxDB line.
-If you want to skip this mapping just enable flag `influx-skip-database-label`.
-- Field names are mapped to time series names prefixed with {measurement}{separator} value,
-where {separator} equals to _ by default.
-It can be changed with `--influx-measurement-field-separator` command-line flag.
-- Field values are mapped to time series values.
-- Tags are mapped to Prometheus labels format as-is.
-
-For example, the following InfluxDB line:
-
-```
-foo,tag1=value1,tag2=value2 field1=12,field2=40
-```
-
-is converted into the following Prometheus format data points:
-
-```
-foo_field1{tag1="value1", tag2="value2"} 12
-foo_field2{tag1="value1", tag2="value2"} 40
-```
-
-### Configuration
-
-The configuration flags should contain self-explanatory descriptions.
-
-### Filtering
-
-The filtering consists of two parts: timeseries and time.
-The first step of application is to select all available timeseries
-for given database and retention. User may specify additional filtering
-condition via `--influx-filter-series` flag. For example:
-
-```
-./vmctl influx --influx-database benchmark \
- --influx-filter-series "on benchmark from cpu where hostname='host_1703'"
-InfluxDB import mode
-2020/01/26 14:23:29 Exploring scheme for database "benchmark"
-2020/01/26 14:23:29 fetching fields: command: "show field keys"; database: "benchmark"; retention: "autogen"
-2020/01/26 14:23:29 found 12 fields
-2020/01/26 14:23:29 fetching series: command: "show series on benchmark from cpu where hostname='host_1703'"; database: "benchmark"; retention: "autogen"
-Found 10 timeseries to import. Continue? [Y/n]
-```
-
-The timeseries select query would be following:
- `fetching series: command: "show series on benchmark from cpu where hostname='host_1703'"; database: "benchmark"; retention: "autogen"`
-
-The second step of filtering is a time filter and it applies when fetching the datapoints from Influx.
-Time filtering may be configured with two flags:
-
-- --influx-filter-time-start
-- --influx-filter-time-end
-Here's an example of importing timeseries for one day only:
-`./vmctl influx --influx-database benchmark --influx-filter-series "where hostname='host_1703'" --influx-filter-time-start "2020-01-01T10:07:00Z" --influx-filter-time-end "2020-01-01T15:07:00Z"`
-
-Please see more about time filtering [here](https://docs.influxdata.com/influxdb/v1.7/query_language/schema_exploration#filter-meta-queries-by-time).
-
-## Migrating data from InfluxDB (2.x)
-
-Migrating data from InfluxDB v2.x is not supported yet ([#32](https://github.com/VictoriaMetrics/vmctl/issues/32)).
-You may find useful a 3rd party solution for this - .
-
-## Migrating data from Promscale
-
-[Promscale](https://github.com/timescale/promscale) supports [Prometheus Remote Read API](https://prometheus.io/docs/prometheus/latest/querying/remote_read_api/).
-To migrate historical data from Promscale to VictoriaMetrics we recommend using `vmctl`
-in [remote-read](https://docs.victoriametrics.com/vmctl.html#migrating-data-by-remote-read-protocol) mode.
-
-See the example of migration command below:
-```console
-./vmctl remote-read --remote-read-src-addr=http://:9201/read \
- --remote-read-step-interval=day \
- --remote-read-use-stream=false \ # promscale doesn't support streaming
- --vm-addr=http://:8428 \
- --remote-read-filter-time-start=2023-08-21T00:00:00Z \
- --remote-read-disable-path-append=true # promscale has custom remote read API HTTP path
-Selected time range "2023-08-21 00:00:00 +0000 UTC" - "2023-08-21 14:11:41.561979 +0000 UTC" will be split into 1 ranges according to "day" step. Continue? [Y/n] y
-VM worker 0:↙ 82831 samples/s
-VM worker 1:↙ 54378 samples/s
-VM worker 2:↙ 121616 samples/s
-VM worker 3:↙ 59164 samples/s
-VM worker 4:↙ 59220 samples/s
-VM worker 5:↙ 102072 samples/s
-Processing ranges: 1 / 1 [██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
-2023/08/21 16:11:55 Import finished!
-2023/08/21 16:11:55 VictoriaMetrics importer stats:
- idle duration: 0s;
- time spent while importing: 14.047045459s;
- total samples: 262111;
- samples/s: 18659.51;
- total bytes: 5.3 MB;
- bytes/s: 376.4 kB;
- import requests: 6;
- import requests retries: 0;
-2023/08/21 16:11:55 Total time: 14.063458792s
-```
-
-Here we specify the full path to Promscale's Remote Read API via `--remote-read-src-addr`, and disable auto-path
-appending via `--remote-read-disable-path-append` cmd-line flags. This is necessary, as Promscale has a different to
-Prometheus API path. Promscale doesn't support stream mode for Remote Read API,
-so we disable it via `--remote-read-use-stream=false`.
-
-## Migrating data from Prometheus
-
-`vmctl` supports the `prometheus` mode for migrating data from Prometheus to VictoriaMetrics time-series database.
-Migration is based on reading Prometheus snapshot, which is basically a hard-link to Prometheus data files.
-
-See `./vmctl prometheus --help` for details and full list of flags. Also see Prometheus related articles [here](#articles).
-
-To use migration tool please specify the file path to Prometheus snapshot `--prom-snapshot` (see how to make a snapshot [here](https://www.robustperception.io/taking-snapshots-of-prometheus-data)) and VictoriaMetrics address `--vm-addr`.
-Please note, that `vmctl` *do not make a snapshot from Prometheus*, it uses an already prepared snapshot. More about Prometheus snapshots may be found [here](https://www.robustperception.io/taking-snapshots-of-prometheus-data) and [here](https://medium.com/@romanhavronenko/victoriametrics-how-to-migrate-data-from-prometheus-d44a6728f043).
-Flag `--vm-addr` for single-node VM is usually equal to `--httpListenAddr`, and for cluster version
-is equal to `--httpListenAddr` flag of vminsert component. Please note, that vmctl performs initial readiness check for the given address
-by checking `/health` endpoint. For cluster version it is additionally required to specify the `--vm-account-id` flag.
-See more details for cluster version [here](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster).
-
-As soon as required flags are provided and all endpoints are accessible, `vmctl` will start the Prometheus snapshot exploration.
-Basically, it just fetches all available blocks in provided snapshot and read the metadata. It also does initial filtering by time
-if flags `--prom-filter-time-start` or `--prom-filter-time-end` were set. The exploration procedure prints some stats from read blocks.
-Please note that stats are not taking into account timeseries or samples filtering. This will be done during importing process.
-
-The importing process takes the snapshot blocks revealed from Explore procedure and processes them one by one
-accumulating timeseries and samples. Please note, that `vmctl` relies on responses from InfluxDB on this stage,
-so ensure that Explore queries are executed without errors or limits. Please see this
-[issue](https://github.com/VictoriaMetrics/vmctl/issues/30) for details.
-The data processed in chunks and then sent to VM.
-
-The importing process example for local installation of Prometheus
-and single-node VictoriaMetrics(`http://localhost:8428`):
-
-```
-./vmctl prometheus --prom-snapshot=/path/to/snapshot \
- --vm-concurrency=1 \
- --vm-batch-size=200000 \
- --prom-concurrency=3
-Prometheus import mode
-Prometheus snapshot stats:
- blocks found: 14;
- blocks skipped: 0;
- min time: 1581288163058 (2020-02-09T22:42:43Z);
- max time: 1582409128139 (2020-02-22T22:05:28Z);
- samples: 32549106;
- series: 27289.
-Found 14 blocks to import. Continue? [Y/n] y
-14 / 14 [-------------------------------------------------------------------------------------------] 100.00% 0 p/s
-2020/02/23 15:50:03 Import finished!
-2020/02/23 15:50:03 VictoriaMetrics importer stats:
- idle duration: 6.152953029s;
- time spent while importing: 44.908522491s;
- total samples: 32549106;
- samples/s: 724786.84;
- total bytes: 669.1 MB;
- bytes/s: 14.9 MB;
- import requests: 323;
- import requests retries: 0;
-2020/02/23 15:50:03 Total time: 51.077451066s
-```
-
-### Data mapping
-
-VictoriaMetrics has very similar data model to Prometheus and supports [RemoteWrite integration](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage).
-So no data changes will be applied.
-
-### Configuration
-
-The configuration flags should contain self-explanatory descriptions.
-
-### Filtering
-
-The filtering consists of three parts: by timeseries and time.
-
-Filtering by time may be configured via flags `--prom-filter-time-start` and `--prom-filter-time-end`
-in RFC3339 format. This filter applied twice: to drop blocks out of range and to filter timeseries in blocks with
-overlapping time range.
-
-Example of applying time filter:
-
-```
-./vmctl prometheus --prom-snapshot=/path/to/snapshot \
- --prom-filter-time-start=2020-02-07T00:07:01Z \
- --prom-filter-time-end=2020-02-11T00:07:01Z
-Prometheus import mode
-Prometheus snapshot stats:
- blocks found: 2;
- blocks skipped: 12;
- min time: 1581288163058 (2020-02-09T22:42:43Z);
- max time: 1581328800000 (2020-02-10T10:00:00Z);
- samples: 1657698;
- series: 3930.
-Found 2 blocks to import. Continue? [Y/n] y
-```
-
-Please notice, that total amount of blocks in provided snapshot is 14, but only 2 of them were in provided
-time range. So other 12 blocks were marked as `skipped`. The amount of samples and series is not taken into account,
-since this is heavy operation and will be done during import process.
-
-Filtering by timeseries is configured with following flags:
-
-- `--prom-filter-label` - the label name, e.g. `__name__` or `instance`;
-- `--prom-filter-label-value` - the regular expression to filter the label value. By default, matches all `.*`
-
-For example:
-
-```
-./vmctl prometheus --prom-snapshot=/path/to/snapshot \
- --prom-filter-label="__name__" \
- --prom-filter-label-value="promhttp.*" \
- --prom-filter-time-start=2020-02-07T00:07:01Z \
- --prom-filter-time-end=2020-02-11T00:07:01Z
-Prometheus import mode
-Prometheus snapshot stats:
- blocks found: 2;
- blocks skipped: 12;
- min time: 1581288163058 (2020-02-09T22:42:43Z);
- max time: 1581328800000 (2020-02-10T10:00:00Z);
- samples: 1657698;
- series: 3930.
-Found 2 blocks to import. Continue? [Y/n] y
-14 / 14 [------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
-2020/02/23 15:51:07 Import finished!
-2020/02/23 15:51:07 VictoriaMetrics importer stats:
- idle duration: 0s;
- time spent while importing: 37.415461ms;
- total samples: 10128;
- samples/s: 270690.24;
- total bytes: 195.2 kB;
- bytes/s: 5.2 MB;
- import requests: 2;
- import requests retries: 0;
-2020/02/23 15:51:07 Total time: 7.153158218s
-```
-
-## Migrating data by remote read protocol
-
-`vmctl` provides the `remote-read` mode for migrating data from remote databases supporting
-[Prometheus remote read API](https://prometheus.io/docs/prometheus/latest/querying/remote_read_api/).
-Remote read API has two implementations of remote read API: default (`SAMPLES`) and
-[streamed](https://prometheus.io/blog/2019/10/10/remote-read-meets-streaming/) (`STREAMED_XOR_CHUNKS`).
-Streamed version is more efficient but has lower adoption (e.g. [Promscale](#migrating-data-from-promscale)
-doesn't support it).
-
-See `./vmctl remote-read --help` for details and full list of flags.
-
-To start the migration process configure the following flags:
-
-1. `--remote-read-src-addr` - data source address to read from;
-1. `--vm-addr` - VictoriaMetrics address to write to. For single-node VM is usually equal to `--httpListenAddr`,
- and for cluster version is equal to `--httpListenAddr` flag of vminsert component (for example `http://:8480/insert//prometheus`);
-1. `--remote-read-filter-time-start` - the time filter in RFC3339 format to select time series with timestamp equal or higher than provided value. E.g. '2020-01-01T20:07:00Z';
-1. `--remote-read-filter-time-end` - the time filter in RFC3339 format to select time series with timestamp equal or smaller than provided value. E.g. '2020-01-01T20:07:00Z'. Current time is used when omitted.;
-1. `--remote-read-step-interval` - split export data into chunks. Valid values are `month, day, hour, minute`;
-1. `--remote-read-use-stream` - defines whether to use `SAMPLES` or `STREAMED_XOR_CHUNKS` mode. By default, is uses `SAMPLES` mode.
-
-The importing process example for local installation of Prometheus
-and single-node VictoriaMetrics(`http://localhost:8428`):
-
-```
-./vmctl remote-read \
---remote-read-src-addr=http://:9091 \
---remote-read-filter-time-start=2021-10-18T00:00:00Z \
---remote-read-step-interval=hour \
---vm-addr=http://:8428 \
---vm-concurrency=6
-
-Split defined times into 8798 ranges to import. Continue? [Y/n]
-VM worker 0:↘ 127177 samples/s
-VM worker 1:↘ 140137 samples/s
-VM worker 2:↘ 151606 samples/s
-VM worker 3:↘ 130765 samples/s
-VM worker 4:↘ 131904 samples/s
-VM worker 5:↘ 132693 samples/s
-Processing ranges: 8798 / 8798 [█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
-2022/10/19 16:45:37 Import finished!
-2022/10/19 16:45:37 VictoriaMetrics importer stats:
- idle duration: 6m57.793987511s;
- time spent while importing: 1m18.463744801s;
- total samples: 25348208;
- samples/s: 323056.31;
- total bytes: 669.7 MB;
- bytes/s: 8.5 MB;
- import requests: 127;
- import requests retries: 0;
-2022/10/19 16:45:37 Total time: 1m19.406283424s
-```
-
-Migrating big volumes of data may result in remote read client reaching the timeout.
-Consider increasing the value of `--remote-read-http-timeout` (default `5m`) command-line flag when seeing
-timeouts or `context canceled` errors.
-
-### Filtering
-
-The filtering consists of two parts: by labels and time.
-
-Filtering by time can be configured via flags `--remote-read-filter-time-start` and `--remote-read-filter-time-end`
-in RFC3339 format.
-
-Filtering by labels can be configured via flags `--remote-read-filter-label` and `--remote-read-filter-label-value`.
-For example, `--remote-read-filter-label=tenant` and `--remote-read-filter-label-value="team-eu"` will select only series
-with `tenant="team-eu"` label-value pair.
-
-## Migrating data from Thanos
-
-Thanos uses the same storage engine as Prometheus and the data layout on-disk should be the same. That means
-`vmctl` in mode `prometheus` may be used for Thanos historical data migration as well.
-These instructions may vary based on the details of your Thanos configuration.
-Please read carefully and verify as you go. We assume you're using Thanos Sidecar on your Prometheus pods,
-and that you have a separate Thanos Store installation.
-
-### Current data
-
-1. For now, keep your Thanos Sidecar and Thanos-related Prometheus configuration, but add this to also stream
- metrics to VictoriaMetrics:
-
- ```
- remote_write:
- - url: http://victoria-metrics:8428/api/v1/write
- ```
-
-1. Make sure VM is running, of course. Now check the logs to make sure that Prometheus is sending and VM is receiving.
- In Prometheus, make sure there are no errors. On the VM side, you should see messages like this:
-
- ```
- 2020-04-27T18:38:46.474Z info VictoriaMetrics/lib/storage/partition.go:207 creating a partition "2020_04" with smallPartsPath="/victoria-metrics-data/data/small/2020_04", bigPartsPath="/victoria-metrics-data/data/big/2020_04"
- 2020-04-27T18:38:46.506Z info VictoriaMetrics/lib/storage/partition.go:222 partition "2020_04" has been created
- ```
-
-1. Now just wait. Within two hours, Prometheus should finish its current data file and hand it off to Thanos Store for long term
- storage.
-
-### Historical data
-
-Let's assume your data is stored on S3 served by minio. You first need to copy that out to a local filesystem,
-then import it into VM using `vmctl` in `prometheus` mode.
-
-1. Copy data from minio.
- 1. Run the `minio/mc` Docker container.
- 1. `mc config host add minio http://minio:9000 accessKey secretKey`, substituting appropriate values for the last 3 items.
- 1. `mc cp -r minio/prometheus thanos-data`
-
-1. Import using `vmctl`.
- 1. Follow the [instructions](#how-to-build) to compile `vmctl` on your machine.
- 1. Use [prometheus](#migrating-data-from-prometheus) mode to import data:
-
- ```
- vmctl prometheus --prom-snapshot thanos-data --vm-addr http://victoria-metrics:8428
- ```
-
-### Remote read protocol
-
-Currently, Thanos doesn't support streaming remote read protocol. It is [recommended](https://thanos.io/tip/thanos/integrations.md/#storeapi-as-prometheus-remote-read)
-to use [thanos-remote-read](https://github.com/G-Research/thanos-remote-read) a proxy, that allows exposing any Thanos
-service (or anything that exposes gRPC StoreAPI e.g. Querier) via Prometheus remote read protocol.
-
-If you want to migrate data, you should run [thanos-remote-read](https://github.com/G-Research/thanos-remote-read) proxy
-and define the Thanos store address `./thanos-remote-read -store 127.0.0.1:19194`.
-It is important to know that `store` flag is Thanos Store API gRPC endpoint.
-Also, it is important to know that thanos-remote-read proxy doesn't support stream mode.
-When you run thanos-remote-read proxy, it exposes port to serve HTTP on `10080 by default`.
-
-The importing process example for local installation of Thanos
-and single-node VictoriaMetrics(`http://localhost:8428`):
-
-```
-./vmctl remote-read \
---remote-read-src-addr=http://127.0.0.1:10080 \
---remote-read-filter-time-start=2021-10-18T00:00:00Z \
---remote-read-step-interval=hour \
---vm-addr=http://127.0.0.1:8428 \
---vm-concurrency=6
-```
-
-On the [thanos-remote-read](https://github.com/G-Research/thanos-remote-read) proxy side you will see logs like:
-```
-ts=2022-10-19T15:05:04.193916Z caller=main.go:278 level=info traceID=00000000000000000000000000000000 msg="thanos request" request="min_time:1666180800000 max_time:1666184399999 matchers: aggregates:RAW "
-ts=2022-10-19T15:05:04.468852Z caller=main.go:278 level=info traceID=00000000000000000000000000000000 msg="thanos request" request="min_time:1666184400000 max_time:1666187999999 matchers: aggregates:RAW "
-ts=2022-10-19T15:05:04.553914Z caller=main.go:278 level=info traceID=00000000000000000000000000000000 msg="thanos request" request="min_time:1666188000000 max_time:1666191364863 matchers: aggregates:RAW "
-```
-
-And when process will finish you will see:
-```
-Split defined times into 8799 ranges to import. Continue? [Y/n]
-VM worker 0:↓ 98183 samples/s
-VM worker 1:↓ 114640 samples/s
-VM worker 2:↓ 131710 samples/s
-VM worker 3:↓ 114256 samples/s
-VM worker 4:↓ 105671 samples/s
-VM worker 5:↓ 124000 samples/s
-Processing ranges: 8799 / 8799 [█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
-2022/10/19 18:05:07 Import finished!
-2022/10/19 18:05:07 VictoriaMetrics importer stats:
- idle duration: 52m13.987637229s;
- time spent while importing: 9m1.728983776s;
- total samples: 70836111;
- samples/s: 130759.32;
- total bytes: 2.2 GB;
- bytes/s: 4.0 MB;
- import requests: 356;
- import requests retries: 0;
-2022/10/19 18:05:07 Total time: 9m2.607521618s
-```
-
-## Migrating data from Cortex
-
-Cortex has an implementation of the Prometheus remote read protocol. That means
-`vmctl` in mode `remote-read` may also be used for Cortex historical data migration.
-These instructions may vary based on the details of your Cortex configuration.
-Please read carefully and verify as you go.
-
-### Remote read protocol
-
-If you want to migrate data, you should check your cortex configuration in the section
-```yaml
-api:
- prometheus_http_prefix:
-```
-
-If you defined some prometheus prefix, you should use it when you define flag `--remote-read-src-addr=http://127.0.0.1:9009/{prometheus_http_prefix}`.
-By default, Cortex uses the `prometheus` path prefix, so you should define the flag `--remote-read-src-addr=http://127.0.0.1:9009/prometheus`.
-
-It is important to know that Cortex doesn't support the stream mode.
-When you run Cortex, it exposes a port to serve HTTP on `9009 by default`.
-
-The importing process example for the local installation of Cortex
-and single-node VictoriaMetrics(`http://localhost:8428`):
-
-```
-./vmctl remote-read \
---remote-read-src-addr=http://127.0.0.1:9009/prometheus \
---remote-read-filter-time-start=2021-10-18T00:00:00Z \
---remote-read-step-interval=hour \
---vm-addr=http://127.0.0.1:8428 \
---vm-concurrency=6
-```
-And when the process finishes, you will see the following:
-
-```
-Split defined times into 8842 ranges to import. Continue? [Y/n]
-VM worker 0:↗ 3863 samples/s
-VM worker 1:↗ 2686 samples/s
-VM worker 2:↗ 2620 samples/s
-VM worker 3:↗ 2705 samples/s
-VM worker 4:↗ 2643 samples/s
-VM worker 5:↗ 2593 samples/s
-Processing ranges: 8842 / 8842 [█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
-2022/10/21 12:09:49 Import finished!
-2022/10/21 12:09:49 VictoriaMetrics importer stats:
- idle duration: 0s;
- time spent while importing: 3.82640757s;
- total samples: 160232;
- samples/s: 41875.31;
- total bytes: 11.3 MB;
- bytes/s: 3.0 MB;
- import requests: 6;
- import requests retries: 0;
-2022/10/21 12:09:49 Total time: 4.71824253s
-```
-It is important to know that if you run your Cortex installation in multi-tenant mode, remote read protocol
-requires an Authentication header like `X-Scope-OrgID`. You can define it via the flag `--remote-read-headers=X-Scope-OrgID:demo`
-
-## Migrating data from Mimir
-
-Mimir has similar implementation as Cortex and supports Prometheus remote read API. That means historical data
-from Mimir can be migrated via `vmctl` in mode `remote-read` mode.
-The instructions for data migration via vmctl vary based on the details of your Mimir configuration.
-Please read carefully and verify as you go.
-
-### Remote read protocol
-
-By default, Mimir uses the `prometheus` path prefix so specifying the source
-should be as simple as `--remote-read-src-addr=http://:9009/prometheus`.
-But if prefix was overriden via `prometheus_http_prefix`, then source address should be updated
-to `--remote-read-src-addr=http://:9009/{prometheus_http_prefix}`.
-
-Mimir supports [streamed remote read API](https://prometheus.io/blog/2019/10/10/remote-read-meets-streaming/),
-so it is recommended setting `--remote-read-use-stream=true` flag for better performance and resource usage.
-
-When you run Mimir, it exposes a port to serve HTTP on `8080 by default`.
-
-Next example of the local installation was in multi-tenant mode (3 instances of Mimir) with nginx as load balancer.
-Load balancer expose single port `:9090`.
-
-As you can see in the example we call `:9009` instead of `:8080` because of proxy.
-
-The importing process example for the local installation of Mimir
-and single-node VictoriaMetrics(`http://localhost:8428`):
-
-```
-./vmctl remote-read
---remote-read-src-addr=http://:9009/prometheus \
---remote-read-filter-time-start=2021-10-18T00:00:00Z \
---remote-read-step-interval=hour \
---remote-read-headers=X-Scope-OrgID:demo \
---remote-read-use-stream=true \
---vm-addr=http://:8428 \
-```
-
-And when the process finishes, you will see the following:
-
-```
-Split defined times into 8847 ranges to import. Continue? [Y/n]
-VM worker 0:→ 12176 samples/s
-VM worker 1:→ 11918 samples/s
-VM worker 2:→ 11261 samples/s
-VM worker 3:→ 12861 samples/s
-VM worker 4:→ 11096 samples/s
-VM worker 5:→ 11575 samples/s
-Processing ranges: 8847 / 8847 [█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
-2022/10/21 17:22:23 Import finished!
-2022/10/21 17:22:23 VictoriaMetrics importer stats:
- idle duration: 0s;
- time spent while importing: 15.379614356s;
- total samples: 81243;
- samples/s: 5282.51;
- total bytes: 6.1 MB;
- bytes/s: 397.8 kB;
- import requests: 6;
- import requests retries: 0;
-2022/10/21 17:22:23 Total time: 16.287405248s
-```
-
-It is important to know that if you run your Mimir installation in multi-tenant mode, remote read protocol
-requires an Authentication header like `X-Scope-OrgID`. You can define it via the flag `--remote-read-headers=X-Scope-OrgID:demo`
-
-## Migrating data from VictoriaMetrics
-
-The simplest way to migrate data between VM instances is to copy data.
-See more details [here](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#data-migration).
-
-vmctl uses [native binary protocol](https://docs.victoriametrics.com/#how-to-export-data-in-native-format)
-(available since [1.42.0 release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.42.0))
-o migrate data between VM instances: single to single, cluster to cluster, single to cluster and vice versa.
-
-See `./vmctl vm-native --help` for details and full list of flags.
-
-Migration in `vm-native` mode takes two steps:
-1. Explore the list of the metrics to migrate via `api/v1/label/__name__/values` API;
-1. Migrate explored metrics one-by-one.
-
-```
-./vmctl vm-native \
- --vm-native-src-addr=http://127.0.0.1:8481/select/0/prometheus \ # migrate from
- --vm-native-dst-addr=http://localhost:8428 \ # migrate to
- --vm-native-filter-time-start='2022-11-20T00:00:00Z' \ # starting from
- --vm-native-filter-match='{__name__=~"vm_cache_.*"}' # only metrics matching the selector
-VictoriaMetrics Native import mode
-
-2023/03/02 09:22:02 Initing import process from "http://127.0.0.1:8481/select/0/prometheus/api/v1/export/native"
- to "http://localhost:8428/api/v1/import/native" with filter
- filter: match[]={__name__=~"vm_cache_.*"}
- start: 2022-11-20T00:00:00Z
-2023/03/02 09:22:02 Exploring metrics...
-Found 9 metrics to import. Continue? [Y/n]
-2023/03/02 09:22:04 Requests to make: 9
-Requests to make: 9 / 9 [█████████████████████████████████████████████████████████████████████████████] 100.00%
-2023/03/02 09:22:06 Import finished!
-2023/03/02 09:22:06 VictoriaMetrics importer stats:
- time spent while importing: 3.632638875s;
- total bytes: 7.8 MB;
- bytes/s: 2.1 MB;
- requests: 9;
- requests retries: 0;
-2023/03/02 09:22:06 Total time: 3.633127625s
-```
-
-_To disable explore phase and switch to the old way of data migration via single connection use
-`--vm-native-disable-retries` cmd-line flag. Please note, in this mode vmctl won't be able to retry failed requests._
-
-Importing tips:
-
-1. Migrating big volumes of data may result in reaching the safety limits on `src` side.
- Please verify that `-search.maxExportDuration` and `-search.maxExportSeries` were set with
- proper values for `src`. If hitting the limits, follow the recommendations
- [here](https://docs.victoriametrics.com/#how-to-export-data-in-native-format).
- If hitting `the number of matching timeseries exceeds...` error, adjust filters to match less time series or
- update `-search.maxSeries` command-line flag on vmselect/vmsingle;
-1. Migrating all the metrics from one VM to another may collide with existing application metrics
- (prefixed with `vm_`) at destination and lead to confusion when using
- [official Grafana dashboards](https://grafana.com/orgs/victoriametrics/dashboards).
- To avoid such situation try to filter out VM process metrics via `--vm-native-filter-match='{__name__!~"vm_.*"}'` flag.
-1. Migrating data with overlapping time range or via unstable network can produce duplicates series at destination.
- To avoid duplicates set `-dedup.minScrapeInterval=1ms` for `vmselect`/`vmstorage` at the destination.
- This will instruct `vmselect`/`vmstorage` to ignore duplicates with identical timestamps.
-1. When migrating large volumes of data use `--vm-native-step-interval` flag to split migration [into steps](#using-time-based-chunking-of-migration).
-1. When migrating data from one VM cluster to another, consider using [cluster-to-cluster mode](#cluster-to-cluster-migration-mode).
- Or manually specify addresses according to [URL format](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#url-format):
- ```console
- # Migrating from cluster specific tenantID to single
- --vm-native-src-addr=http://:8481/select/0/prometheus
- --vm-native-dst-addr=http://:8428
-
- # Migrating from single to cluster specific tenantID
- --vm-native-src-addr=http://:8428
- --vm-native-src-addr=http://:8480/insert/0/prometheus
-
- # Migrating single to single
- --vm-native-src-addr=http://:8428
- --vm-native-dst-addr=http://:8428
-
- # Migrating cluster to cluster for specific tenant ID
- --vm-native-src-addr=http://:8481/select/0/prometheus
- --vm-native-dst-addr=http://:8480/insert/0/prometheus
- ```
-1. Migrating data from VM cluster which had replication (`-replicationFactor` > 1) enabled won't produce the same amount
- of data copies for the destination database, and will result only in creating duplicates. To remove duplicates,
- destination database need to be configured with `-dedup.minScrapeInterval=1ms`. To restore the replication factor
- the destination `vminsert` component need to be configured with the according `-replicationFactor` value.
- See more about replication [here](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#replication-and-data-safety).
-1. Migration speed can be adjusted via `--vm-concurrency` cmd-line flag, which controls the number of concurrent
- workers busy with processing. Please note, that each worker can load up to a single vCPU core on VictoriaMetrics.
- So try to set it according to allocated CPU resources of your VictoriaMetrics destination installation.
-1. Migration is a backfilling process, so it is recommended to read
- [Backfilling tips](https://github.com/VictoriaMetrics/VictoriaMetrics#backfilling) section.
-1. `vmctl` doesn't provide relabeling or other types of labels management.
- Instead, use [relabeling in VictoriaMetrics](https://github.com/VictoriaMetrics/vmctl/issues/4#issuecomment-683424375).
-1. `vmctl` supports `--vm-native-src-headers` and `--vm-native-dst-headers` to define headers sent with each request
- to the corresponding source address.
-1. `vmctl` supports `--vm-native-disable-http-keep-alive` to allow `vmctl` to use non-persistent HTTP connections to avoid
- error `use of closed network connection` when run a longer export.
-
-
-### Using time-based chunking of migration
-
-It is possible split migration process into set of smaller batches based on time. This is especially useful when
-migrating large volumes of data as this adds indication of progress and ability to restore process from certain point
-in case of failure.
-
-To use this you need to specify `--vm-native-step-interval` flag. Supported values are: `month`, `week`, `day`, `hour`, `minute`.
-Note that in order to use this it is required `--vm-native-filter-time-start` to be set to calculate time ranges for
-export process.
-
-Every range is being processed independently, which means that:
-- after range processing is finished all data within range is migrated
-- if process fails on one of stages it is guaranteed that data of prior stages is already written,
-so it is possible to restart process starting from failed range.
-
-It is recommended using the `month` step when migrating the data over multiple months,
-since the migration with `week`, `day` and `hour` steps may take longer time to complete because of additional overhead.
-
-Usage example:
-```console
-./vmctl vm-native \
- --vm-native-src-addr=http://127.0.0.1:8481/select/0/prometheus \
- --vm-native-dst-addr=http://localhost:8428 \
- --vm-native-filter-time-start='2022-11-20T00:00:00Z' \
- --vm-native-step-interval=month \
- --vm-native-filter-match='{__name__=~"vm_cache_.*"}'
-VictoriaMetrics Native import mode
-
-2023/03/02 09:18:05 Initing import process from "http://127.0.0.1:8481/select/0/prometheus/api/v1/export/native" to "http://localhost:8428/api/v1/import/native" with filter
- filter: match[]={__name__=~"vm_cache_.*"}
- start: 2022-11-20T00:00:00Z
-2023/03/02 09:18:05 Exploring metrics...
-Found 9 metrics to import. Continue? [Y/n]
-2023/03/02 09:18:07 Selected time range will be split into 5 ranges according to "month" step. Requests to make: 45.
-Requests to make: 45 / 45 [█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
-2023/03/02 09:18:12 Import finished!
-2023/03/02 09:18:12 VictoriaMetrics importer stats:
- time spent while importing: 7.111870667s;
- total bytes: 7.7 MB;
- bytes/s: 1.1 MB;
- requests: 45;
- requests retries: 0;
-2023/03/02 09:18:12 Total time: 7.112405875s
-```
-
-### Cluster-to-cluster migration mode
-
-Using cluster-to-cluster migration mode helps to migrate all tenants data in a single `vmctl` run.
-
-Cluster-to-cluster uses `/admin/tenants` endpoint (available starting from [v1.84.0](https://docs.victoriametrics.com/CHANGELOG.html#v1840)) to discover list of tenants from source cluster.
-
-To use this mode you need to set `--vm-intercluster` flag to `true`, `--vm-native-src-addr` flag to 'http://vmselect:8481/' and `--vm-native-dst-addr` value to http://vminsert:8480/:
-
-```console
- ./vmctl vm-native --vm-native-src-addr=http://127.0.0.1:8481/ \
- --vm-native-dst-addr=http://127.0.0.1:8480/ \
- --vm-native-filter-match='{__name__="vm_app_uptime_seconds"}' \
- --vm-native-filter-time-start='2023-02-01T00:00:00Z' \
- --vm-native-step-interval=day \
---vm-intercluster
-VictoriaMetrics Native import mode
-2023/02/28 10:41:42 Discovering tenants...
-2023/02/28 10:41:42 The following tenants were discovered: [0:0 1:0 2:0 3:0 4:0]
-2023/02/28 10:41:42 Initing import process from "http://127.0.0.1:8481/select/0:0/prometheus/api/v1/export/native" to "http://127.0.0.1:8480/insert/0:0/prometheus/api/v1/import/native" with filter
- filter: match[]={__name__="vm_app_uptime_seconds"}
- start: 2023-02-01T00:00:00Z for tenant 0:0
-2023/02/28 10:41:42 Exploring metrics...
-2023/02/28 10:41:42 Found 1 metrics to import
-2023/02/28 10:41:42 Selected time range will be split into 28 ranges according to "day" step.
-Requests to make for tenant 0:0: 28 / 28 [███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
-
-2023/02/28 10:41:45 Initing import process from "http://127.0.0.1:8481/select/1:0/prometheus/api/v1/export/native" to "http://127.0.0.1:8480/insert/1:0/prometheus/api/v1/import/native" with filter
- filter: match[]={__name__="vm_app_uptime_seconds"}
- start: 2023-02-01T00:00:00Z for tenant 1:0
-2023/02/28 10:41:45 Exploring metrics...
-2023/02/28 10:41:45 Found 1 metrics to import
-2023/02/28 10:41:45 Selected time range will be split into 28 ranges according to "day" step. Requests to make: 28
-Requests to make for tenant 1:0: 28 / 28 [████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
-
-...
-
-2023/02/28 10:42:49 Import finished!
-2023/02/28 10:42:49 VictoriaMetrics importer stats:
- time spent while importing: 1m6.714210417s;
- total bytes: 39.7 MB;
- bytes/s: 594.4 kB;
- requests: 140;
- requests retries: 0;
-2023/02/28 10:42:49 Total time: 1m7.147971417s
-```
-
-## Verifying exported blocks from VictoriaMetrics
-
-In this mode, `vmctl` allows verifying correctness and integrity of data exported via
-[native format](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-export-data-in-native-format)
-from VictoriaMetrics.
-You can verify exported data at disk before uploading it by `vmctl verify-block` command:
-
-```console
-# export blocks from VictoriaMetrics
-curl localhost:8428/api/v1/export/native -g -d 'match[]={__name__!=""}' -o exported_data_block
-# verify block content
-./vmctl verify-block exported_data_block
-2022/03/30 18:04:50 verifying block at path="exported_data_block"
-2022/03/30 18:04:50 successfully verified block at path="exported_data_block", blockCount=123786
-2022/03/30 18:04:50 Total time: 100.108ms
-```
-
-## Tuning
-
-### InfluxDB mode
-
-The flag `--influx-concurrency` controls how many concurrent requests may be sent to InfluxDB while fetching
-timeseries. Please set it wisely to avoid InfluxDB overwhelming.
-
-The flag `--influx-chunk-size` controls the max amount of datapoints to return in single chunk from fetch requests.
-Please see more details [here](https://docs.influxdata.com/influxdb/v1.7/guides/querying_data/#chunking).
-The chunk size is used to control InfluxDB memory usage, so it won't OOM on processing large timeseries with
-billions of datapoints.
-
-### Prometheus mode
-
-The flag `--prom-concurrency` controls how many concurrent readers will be reading the blocks in snapshot.
-Since snapshots are just files on disk it would be hard to overwhelm the system. Please go with value equal
-to number of free CPU cores.
-
-### VictoriaMetrics importer
-
-The flag `--vm-concurrency` controls the number of concurrent workers that process the input from InfluxDB query results.
-Please note that each import request can load up to a single vCPU core on VictoriaMetrics. So try to set it according
-to allocated CPU resources of your VictoriaMetrics installation.
-
-The flag `--vm-batch-size` controls max amount of samples collected before sending the import request.
-For example, if `--influx-chunk-size=500` and `--vm-batch-size=2000` then importer will process not more
-than 4 chunks before sending the request.
-
-### Importer stats
-
-After successful import `vmctl` prints some statistics for details.
-The important numbers to watch are following:
-
-- `idle duration` - shows time that importer spent while waiting for data from InfluxDB/Prometheus
-to fill up `--vm-batch-size` batch size. Value shows total duration across all workers configured
-via `--vm-concurrency`. High value may be a sign of too slow InfluxDB/Prometheus fetches or too
-high `--vm-concurrency` value. Try to improve it by increasing `---concurrency` value or
-decreasing `--vm-concurrency` value.
-- `import requests` - shows how many import requests were issued to VM server.
-The import request is issued once the batch size(`--vm-batch-size`) is full and ready to be sent.
-Please prefer big batch sizes (50k-500k) to improve performance.
-- `import requests retries` - shows number of unsuccessful import requests. Non-zero value may be
-a sign of network issues or VM being overloaded. See the logs during import for error messages.
-
-### Silent mode
-
-By default `vmctl` waits confirmation from user before starting the import. If this is unwanted
-behavior and no user interaction required - pass `-s` flag to enable "silence" mode:
-
-See below the example of `vm-native` migration process:
-```
- -s Whether to run in silent mode. If set to true no confirmation prompts will appear. (default: false)
-```
-
-### Significant figures
-
-`vmctl` allows to limit the number of [significant figures](https://en.wikipedia.org/wiki/Significant_figures)
-before importing. For example, the average value for response size is `102.342305` bytes and it has 9 significant figures.
-If you ask a human to pronounce this value then with high probability value will be rounded to first 4 or 5 figures
-because the rest aren't really that important to mention. In most cases, such a high precision is too much.
-Moreover, such values may be just a result of [floating point arithmetic](https://en.wikipedia.org/wiki/Floating-point_arithmetic),
-create a [false precision](https://en.wikipedia.org/wiki/False_precision) and result into bad compression ratio
-according to [information theory](https://en.wikipedia.org/wiki/Information_theory).
-
-`vmctl` provides the following flags for improving data compression:
-
-- `--vm-round-digits` flag for rounding processed values to the given number of decimal digits after the point.
- For example, `--vm-round-digits=2` would round `1.2345` to `1.23`. By default, the rounding is disabled.
-
-- `--vm-significant-figures` flag for limiting the number of significant figures in processed values. It takes no effect if set
- to 0 (by default), but set `--vm-significant-figures=5` and `102.342305` will be rounded to `102.34`.
-
-The most common case for using these flags is to improve data compression for time series storing aggregation
-results such as `average`, `rate`, etc.
-
-### Adding extra labels
-
- `vmctl` allows to add extra labels to all imported series. It can be achieved with flag `--vm-extra-label label=value`.
- If multiple labels needs to be added, set flag for each label, for example, `--vm-extra-label label1=value1 --vm-extra-label label2=value2`.
- If timeseries already have label, that must be added with `--vm-extra-label` flag, flag has priority and will override label value from timeseries.
-
-### Rate limiting
-
-Limiting the rate of data transfer could help to reduce pressure on disk or on destination database.
-The rate limit may be set in bytes-per-second via `--vm-rate-limit` flag.
-
-Please note, you can also use [vmagent](https://docs.victoriametrics.com/vmagent.html)
-as a proxy between `vmctl` and destination with `-remoteWrite.rateLimit` flag enabled.
-
-## How to build
-
-It is recommended using [binary releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest) - `vmctl` is located in `vmutils-*` archives there.
-
-### Development build
-
-1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.20.
-1. Run `make vmctl` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmctl` binary and puts it into the `bin` folder.
-
-### Production build
-
-1. [Install docker](https://docs.docker.com/install/).
-1. Run `make vmctl-prod` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmctl-prod` binary and puts it into the `bin` folder.
-
-### Building docker images
-
-Run `make package-vmctl`. It builds `victoriametrics/vmctl:` docker image locally.
-`` is auto-generated image tag, which depends on source code in the repository.
-The `` may be manually set via `PKG_TAG=foobar make package-vmctl`.
-
-The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
-by setting it via `` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
-
-```console
-ROOT_IMAGE=scratch make package-vmctl
-```
-
-### ARM build
-
-ARM build may run on Raspberry Pi or on [energy-efficient ARM servers](https://blog.cloudflare.com/arm-takes-wing/).
-
-#### Development ARM build
-
-1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.20.
-1. Run `make vmctl-linux-arm` or `make vmctl-linux-arm64` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmctl-linux-arm` or `vmctl-linux-arm64` binary respectively and puts it into the `bin` folder.
-
-#### Production ARM build
-
-1. [Install docker](https://docs.docker.com/install/).
-1. Run `make vmctl-linux-arm-prod` or `make vmctl-linux-arm64-prod` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmctl-linux-arm-prod` or `vmctl-linux-arm64-prod` binary respectively and puts it into the `bin` folder.
+vmctl docs can be edited at [docs/vmctl.md](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/vmctl.md).
diff --git a/app/vmgateway/README.md b/app/vmgateway/README.md
index 0862ef113..32a5da543 100644
--- a/app/vmgateway/README.md
+++ b/app/vmgateway/README.md
@@ -1,467 +1,3 @@
-# vmgateway
+See vmgateway docs [here](https://docs.victoriametrics.com/vmgateway.html).
-***vmgateway is a part of [enterprise package](https://docs.victoriametrics.com/enterprise.html).
-It is available for download and evaluation at [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
-See how to request a free trial license [here](https://victoriametrics.com/products/enterprise/trial/).***
-
-
-
-`vmgateway` is a proxy for the VictoriaMetrics Time Series Database (TSDB). It provides the following features:
-
-* Rate Limiter
- * Based on cluster tenant's utilization, it supports multiple time interval limits for both the ingestion and retrieval of metrics
-* Token Access Control
- * Supports additional per-label access control for both the Single and Cluster versions of the VictoriaMetrics TSDB
- * Provides access by tenantID in the Cluster version
- * Allows for separate write/read/admin access to data
-
-`vmgateway` is included in our [enterprise packages](https://docs.victoriametrics.com/enterprise.html).
-
-## Access Control
-
-
-
-`vmgateway` supports jwt based authentication. With jwt payload can be configured to give access to specific tenants and labels as well as to read/write.
-
-jwt token must be in following format:
-
-```json
-{
- "exp": 1617304574,
- "vm_access": {
- "tenant_id": {
- "account_id": 1,
- "project_id": 5
- },
- "extra_labels": {
- "team": "dev",
- "project": "mobile"
- },
- "extra_filters": ["{env=~\"prod|dev\",team!=\"test\"}"],
- "mode": 1
- }
-}
-```
-
-Where:
-
-* `exp` - required, expire time in unix_timestamp. If the token expires then `vmgateway` rejects the request.
-* `vm_access` - required, dict with claim info, minimum form: `{"vm_access": {"tenant_id": {}}`
-* `tenant_id` - optional, for cluster mode, routes requests to the corresponding tenant.
-* `extra_labels` - optional, key-value pairs for label filters added to the ingested or selected metrics. Multiple filters are added with `and` operation. If defined, `extra_label` from original request removed.
-* `extra_filters` - optional, [series selectors](https://prometheus.io/docs/prometheus/latest/querying/basics/#time-series-selectors) added to the select query requests. Multiple selectors are added with `or` operation. If defined, `extra_filter` from original request removed.
-* `mode` - optional, access mode for api - read, write, or full. Supported values: 0 - full (default value), 1 - read, 2 - write.
-
-## QuickStart
-
-Start the single version of VictoriaMetrics
-
-```console
-# single
-# start node
-./bin/victoria-metrics --selfScrapeInterval=10s
-```
-
-Start vmgateway
-
-```console
-./bin/vmgateway -eula -enable.auth -read.url http://localhost:8428 --write.url http://localhost:8428
-```
-
-Retrieve data from the database
-
-```console
-curl 'http://localhost:8431/api/v1/series/count' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ2bV9hY2Nlc3MiOnsidGVuYW50X2lkIjp7fSwicm9sZSI6MX0sImV4cCI6MTkzOTM0NjIxMH0.5WUxEfdcV9hKo4CtQdtuZYOGpGXWwaqM9VuVivMMrVg'
-```
-
-A request with an incorrect token or without any token will be rejected:
-
-```console
-curl 'http://localhost:8431/api/v1/series/count'
-
-curl 'http://localhost:8431/api/v1/series/count' -H 'Authorization: Bearer incorrect-token'
-```
-
-## Rate Limiter
-
-
-
-Limits incoming requests by given, pre-configured limits. It supports read and write limiting by tenant.
-
-`vmgateway` needs a datasource for rate limit queries. It can be either single-node or cluster version of `victoria-metrics`.
-The metrics that you want to rate limit must be scraped from the cluster.
-
-List of supported limit types:
-
-* `queries` - count of api requests made at tenant to read the api, such as `/api/v1/query`, `/api/v1/series` and others.
-* `active_series` - count of current active series at any given tenant.
-* `new_series` - count of created series; aka churn rate
-* `rows_inserted` - count of inserted rows per tenant.
-
-List of supported time windows:
-
-* `minute`
-* `hour`
-
-Limits can be specified per tenant or at a global level if you omit `project_id` and `account_id`.
-
-Example of configuration file:
-
-```yaml
-limits:
- - type: queries
- value: 1000
- resolution: minute
- - type: queries
- value: 10000
- resolution: hour
- - type: queries
- value: 10
- resolution: minute
- project_id: 5
- account_id: 1
-```
-
-## QuickStart
-
-cluster version of VictoriaMetrics is required for rate limiting.
-
-```console
-# start datasource for cluster metrics
-
-cat << EOF > cluster.yaml
-scrape_configs:
- - job_name: cluster
- scrape_interval: 5s
- static_configs:
- - targets: ['127.0.0.1:8481','127.0.0.1:8482','127.0.0.1:8480']
-EOF
-
-./bin/victoria-metrics --promscrape.config cluster.yaml
-
-# start cluster
-
-# start vmstorage, vmselect and vminsert
-./bin/vmstorage -eula
-./bin/vmselect -eula -storageNode 127.0.0.1:8401
-./bin/vminsert -eula -storageNode 127.0.0.1:8400
-
-# create base rate limiting config:
-cat << EOF > limit.yaml
-limits:
- - type: queries
- value: 100
- - type: rows_inserted
- value: 100000
- - type: new_series
- value: 1000
- - type: active_series
- value: 100000
- - type: queries
- value: 1
- account_id: 15
-EOF
-
-# start gateway with clusterMoe
-./bin/vmgateway -eula -enable.rateLimit -ratelimit.config limit.yaml -datasource.url http://localhost:8428 -enable.auth -clusterMode -write.url=http://localhost:8480 --read.url=http://localhost:8481
-
-# ingest simple metric to tenant 1:5
-curl 'http://localhost:8431/api/v1/import/prometheus' -X POST -d 'foo{bar="baz1"} 123' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MjAxNjIwMDAwMDAsInZtX2FjY2VzcyI6eyJ0ZW5hbnRfaWQiOnsiYWNjb3VudF9pZCI6MTV9fX0.PB1_KXDKPUp-40pxOGk6lt_jt9Yq80PIMpWVJqSForQ'
-# read metric from tenant 1:5
-curl 'http://localhost:8431/api/v1/labels' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MjAxNjIwMDAwMDAsInZtX2FjY2VzcyI6eyJ0ZW5hbnRfaWQiOnsiYWNjb3VudF9pZCI6MTV9fX0.PB1_KXDKPUp-40pxOGk6lt_jt9Yq80PIMpWVJqSForQ'
-
-# check rate limit
-```
-
-## JWT signature verification
-
-`vmgateway` supports JWT signature verification.
-
-Supported algorithms are `RS256`, `RS384`, `RS512`, `ES256`, `ES384`, `ES512`, `PS256`, `PS384`, `PS512`.
-Tokens with unsupported algorithms will be rejected.
-
-In order to enable JWT signature verification, you need to specify keys for signature verification.
-The following flags are used to specify keys:
-- `-auth.publicKeyFiles` - allows to pass file path to file with public key.
-- `-auth.publicKeys` - allows to pass public key directly.
-
-Note that both flags support passing multiple keys and also can be used together.
-
-Example usage:
-```console
-./bin/vmgateway -eula \
- -enable.auth \
- -write.url=http://localhost:8480 \
- -read.url=http://localhost:8481 \
- -auth.publicKeyFiles=public_key.pem \
- -auth.publicKeyFiles=public_key2.pem \
- -auth.publicKeys=`-----BEGIN PUBLIC KEY-----
-MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAu1SU1LfVLPHCozMxH2Mo
-4lgOEePzNm0tRgeLezV6ffAt0gunVTLw7onLRnrq0/IzW7yWR7QkrmBL7jTKEn5u
-+qKhbwKfBstIs+bMY2Zkp18gnTxKLxoS2tFczGkPLPgizskuemMghRniWaoLcyeh
-kd3qqGElvW/VDL5AaWTg0nLVkjRo9z+40RQzuVaE8AkAFmxZzow3x+VJYKdjykkJ
-0iT9wCS0DRTXu269V264Vf/3jvredZiKRkgwlL9xNAwxXFg0x/XFw005UWVRIkdg
-cKWTjpBP2dPwVZ4WWC+9aGVd+Gyn1o0CLelf4rEjGoXbAAEgAqeGUxrcIlbjXfbc
-mwIDAQAB
------END PUBLIC KEY-----
-`
-```
-This command will result in 3 keys loaded: 2 keys from files and 1 from command line.
-
-### Using OpenID discovery endpoint for JWT signature verification
-
-`vmgateway` supports using OpenID discovery endpoint for JWKS keys discovery.
-
-In order to enable [OpenID discovery](https://openid.net/specs/openid-connect-discovery-1_0.html) endpoint for JWT signature verification, you need to specify OpenID discovery endpoint URLs by using `auth.oidcDiscoveryEndpoints` flag.
-When `auth.oidcDiscoveryEndpoints` is specified `vmageteway` will fetch JWKS keys from the specified endpoint and use them for JWT signature verification.
-
-Example usage for tokens issued by Azure Active Directory:
-```console
-/bin/vmgateway -eula \
- -enable.auth \
- -write.url=http://localhost:8480 \
- -read.url=http://localhost:8481 \
- -auth.oidcDiscoveryEndpoints=https://login.microsoftonline.com/common/v2.0/.well-known/openid-configuration
-```
-
-Example usage for tokens issued by Google:
-```console
-/bin/vmgateway -eula \
- -enable.auth \
- -write.url=http://localhost:8480 \
- -read.url=http://localhost:8481 \
- -auth.oidcDiscoveryEndpoints=https://accounts.google.com/.well-known/openid-configuration
-```
-
-### Using JWKS endpoint for JWT signature verification
-
-`vmgateway` supports using JWKS endpoint for JWT signature verification.
-
-In order to enable JWKS endpoint for JWT signature verification, you need to specify JWKS endpoint URL by using `auth.jwksEndpoints` flag.
-When `auth.jwksEndpoints` is specified `vmageteway` will fetch public keys from the specified endpoint and use them for JWT signature verification.
-
-Example usage for tokens issued by Azure Active Directory:
-```console
-/bin/vmgateway -eula \
- -enable.auth \
- -write.url=http://localhost:8480 \
- -read.url=http://localhost:8481 \
- -auth.jwksEndpoints=https://login.microsoftonline.com/common/discovery/v2.0/keys
-```
-
-Example usage for tokens issued by Google:
-```console
-/bin/vmgateway -eula \
- -enable.auth \
- -write.url=http://localhost:8480 \
- -read.url=http://localhost:8481 \
- -auth.jwksEndpoints=https://www.googleapis.com/oauth2/v3/certs
-```
-
-## Configuration
-
-The shortlist of configuration flags include the following:
-
-```console
- -auth.httpHeader string
- HTTP header name to look for JWT authorization token (default "Authorization")
- -auth.jwksEndpoints array
- JWKS endpoints to fetch keys for JWT tokens signature verification
- Supports an array of values separated by comma or specified via multiple flags.
- -auth.oidcDiscoveryEndpoints array
- OpenID Connect discovery endpoints to fetch keys for JWT tokens signature verification
- Supports an array of values separated by comma or specified via multiple flags.
- -auth.publicKeyFiles array
- Path file with public key to verify JWT token signature
- Supports an array of values separated by comma or specified via multiple flags.
- -auth.publicKeys array
- Public keys to verify JWT token signature
- Supports an array of values separated by comma or specified via multiple flags.
- -clusterMode
- enable this for the cluster version
- -datasource.appendTypePrefix
- Whether to add type prefix to -datasource.url based on the query type. Set to true if sending different query types to the vmselect URL.
- -datasource.basicAuth.password string
- Optional basic auth password for -datasource.url
- -datasource.basicAuth.passwordFile string
- Optional path to basic auth password to use for -datasource.url
- -datasource.basicAuth.username string
- Optional basic auth username for -datasource.url
- -datasource.bearerToken string
- Optional bearer auth token to use for -datasource.url.
- -datasource.bearerTokenFile string
- Optional path to bearer token file to use for -datasource.url.
- -datasource.disableKeepAlive
- Whether to disable long-lived connections to the datasource. If true, disables HTTP keep-alives and will only use the connection to the server for a single HTTP request.
- -datasource.disableStepParam
- Whether to disable adding 'step' param to the issued instant queries. This might be useful when using vmalert with datasources that do not support 'step' param for instant queries, like Google Managed Prometheus. It is not recommended to enable this flag if you use vmalert with VictoriaMetrics.
- -datasource.headers string
- Optional HTTP extraHeaders to send with each request to the corresponding -datasource.url. For example, -datasource.headers='My-Auth:foobar' would send 'My-Auth: foobar' HTTP header with every request to the corresponding -datasource.url. Multiple headers must be delimited by '^^': -datasource.headers='header1:value1^^header2:value2'
- -datasource.lookback duration
- Lookback defines how far into the past to look when evaluating queries. For example, if the datasource.lookback=5m then param "time" with value now()-5m will be added to every query.
- -datasource.maxIdleConnections int
- Defines the number of idle (keep-alive connections) to each configured datasource. Consider setting this value equal to the value: groups_total * group.concurrency. Too low a value may result in a high number of sockets in TIME_WAIT state. (default 100)
- -datasource.oauth2.clientID string
- Optional OAuth2 clientID to use for -datasource.url.
- -datasource.oauth2.clientSecret string
- Optional OAuth2 clientSecret to use for -datasource.url.
- -datasource.oauth2.clientSecretFile string
- Optional OAuth2 clientSecretFile to use for -datasource.url.
- -datasource.oauth2.scopes string
- Optional OAuth2 scopes to use for -datasource.url. Scopes must be delimited by ';'
- -datasource.oauth2.tokenUrl string
- Optional OAuth2 tokenURL to use for -datasource.url.
- -datasource.queryStep duration
- How far a value can fallback to when evaluating queries. For example, if -datasource.queryStep=15s then param "step" with value "15s" will be added to every query. If set to 0, rule's evaluation interval will be used instead. (default 5m0s)
- -datasource.queryTimeAlignment
- Deprecated: please use "eval_alignment" in rule group instead. Whether to align "time" parameter with evaluation interval. Alignment supposed to produce deterministic results despite number of vmalert replicas or time they were started. See more details at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1257 (default true)
- -datasource.roundDigits int
- Adds "round_digits" GET param to datasource requests. In VM "round_digits" limits the number of digits after the decimal point in response values.
- -datasource.showURL
- Whether to avoid stripping sensitive information such as auth headers or passwords from URLs in log messages or UI and exported metrics. It is hidden by default, since it can contain sensitive info such as auth key
- -datasource.tlsCAFile string
- Optional path to TLS CA file to use for verifying connections to -datasource.url. By default, system CA is used
- -datasource.tlsCertFile string
- Optional path to client-side TLS certificate file to use when connecting to -datasource.url
- -datasource.tlsInsecureSkipVerify
- Whether to skip tls verification when connecting to -datasource.url
- -datasource.tlsKeyFile string
- Optional path to client-side TLS certificate key to use when connecting to -datasource.url
- -datasource.tlsServerName string
- Optional TLS server name to use for connections to -datasource.url. By default, the server name from -datasource.url is used
- -datasource.url string
- Datasource compatible with Prometheus HTTP API. It can be single node VictoriaMetrics or vmselect URL. Required parameter. E.g. http://127.0.0.1:8428 . See also -remoteRead.disablePathAppend and -datasource.showURL
- -enable.auth
- enables auth with jwt token
- -enable.rateLimit
- enables rate limiter
- -enableTCP6
- Whether to enable IPv6 for listening and dialing. By default, only IPv4 TCP and UDP are used
- -envflag.enable
- Whether to enable reading flags from environment variables in addition to the command line. Command line flag values have priority over values from environment vars. Flags are read only from the command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
- -envflag.prefix string
- Prefix for environment variables if -envflag.enable is set
- -eula
- Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -filestream.disableFadvise
- Whether to disable fadvise() syscall when reading large data files. The fadvise() syscall prevents from eviction of recently accessed data from OS page cache during background merges and backups. In some rare cases it is better to disable the syscall if it uses too much CPU
- -flagsAuthKey string
- Auth key for /flags endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -fs.disableMmap
- Whether to use pread() instead of mmap() for reading data files. By default, mmap() is used for 64-bit arches and pread() is used for 32-bit arches, since they cannot read data files bigger than 2^32 bytes in memory. mmap() is usually faster for reading small data chunks than pread()
- -http.connTimeout duration
- Incoming http connections are closed after the configured timeout. This may help to spread the incoming load among a cluster of services behind a load balancer. Please note that the real timeout may be bigger by up to 10% as a protection against the thundering herd problem (default 2m0s)
- -http.disableResponseCompression
- Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
- -http.header.csp string
- Value for 'Content-Security-Policy' header
- -http.header.frameOptions string
- Value for 'X-Frame-Options' header
- -http.header.hsts string
- Value for 'Strict-Transport-Security' header
- -http.idleConnTimeout duration
- Timeout for incoming idle http connections (default 1m0s)
- -http.maxGracefulShutdownDuration duration
- The maximum duration for a graceful shutdown of the HTTP server. A highly loaded server may require increased value for a graceful shutdown (default 7s)
- -http.pathPrefix string
- An optional prefix to add to all the paths handled by http server. For example, if '-http.pathPrefix=/foo/bar' is set, then all the http requests will be handled on '/foo/bar/*' paths. This may be useful for proxied requests. See https://www.robustperception.io/using-external-urls-and-proxies-with-prometheus
- -http.shutdownDelay duration
- Optional delay before http server shutdown. During this delay, the server returns non-OK responses from /health page, so load balancers can route new requests to other servers
- -httpAuth.password string
- Password for HTTP server's Basic Auth. The authentication is disabled if -httpAuth.username is empty
- -httpAuth.username string
- Username for HTTP server's Basic Auth. The authentication is disabled if empty. See also -httpAuth.password
- -httpListenAddr string
- TCP address to listen for http connections. See also -httpListenAddr.useProxyProtocol (default ":8431")
- -httpListenAddr.useProxyProtocol
- Whether to use proxy protocol for connections accepted at -httpListenAddr . See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt
- -internStringCacheExpireDuration duration
- The expiry duration for caches for interned strings. See https://en.wikipedia.org/wiki/String_interning . See also -internStringMaxLen and -internStringDisableCache (default 6m0s)
- -internStringDisableCache
- Whether to disable caches for interned strings. This may reduce memory usage at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringCacheExpireDuration and -internStringMaxLen
- -internStringMaxLen int
- The maximum length for strings to intern. A lower limit may save memory at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringDisableCache and -internStringCacheExpireDuration (default 500)
- -license string
- Lisense key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed via file specified by -licenseFile command-line flag
- -license.forceOffline
- Whether to enable offline verification for VictoriaMetrics Enterprise license key, which has been passed either via -license or via -licenseFile command-line flag. The issued license key must support offline verification feature. Contact info@victoriametrics.com if you need offline license verification. This flag is avilable only in Enterprise binaries
- -licenseFile string
- Path to file with license key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed inline via -license command-line flag
- -loggerDisableTimestamps
- Whether to disable writing timestamps in logs
- -loggerErrorsPerSecondLimit int
- Per-second limit on the number of ERROR messages. If more than the given number of errors are emitted per second, the remaining errors are suppressed. Zero values disable the rate limit
- -loggerFormat string
- Format for logs. Possible values: default, json (default "default")
- -loggerJSONFields string
- Allows renaming fields in JSON formatted logs. Example: "ts:timestamp,msg:message" renames "ts" to "timestamp" and "msg" to "message". Supported fields: ts, level, caller, msg
- -loggerLevel string
- Minimum level of errors to log. Possible values: INFO, WARN, ERROR, FATAL, PANIC (default "INFO")
- -loggerOutput string
- Output for the logs. Supported values: stderr, stdout (default "stderr")
- -loggerTimezone string
- Timezone to use for timestamps in logs. Timezone must be a valid IANA Time Zone. For example: America/New_York, Europe/Berlin, Etc/GMT+3 or Local (default "UTC")
- -loggerWarnsPerSecondLimit int
- Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
- -memory.allowedBytes size
- Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
- Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
- -memory.allowedPercent float
- Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)
- -metricsAuthKey string
- Auth key for /metrics endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -pprofAuthKey string
- Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -pushmetrics.extraLabel array
- Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
- Supports an array of values separated by comma or specified via multiple flags.
- -pushmetrics.interval duration
- Interval for pushing metrics to -pushmetrics.url (default 10s)
- -pushmetrics.url array
- Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
- Supports an array of values separated by comma or specified via multiple flags.
- -ratelimit.config string
- path for configuration file. Accepts url address
- -ratelimit.configCheckInterval duration
- interval for config file re-read. Zero value disables config re-reading. By default, refreshing is disabled, send SIGHUP for config refresh.
- -ratelimit.extraLabels array
- additional labels, that will be applied to fetchdata from datasource
- Supports an array of values separated by comma or specified via multiple flags.
- -ratelimit.refreshInterval duration
- (default 5s)
- -read.url string
- read access url address, example: http://vmselect:8481
- -remoteRead.disablePathAppend
- Whether to disable automatic appending of '/api/v1/query' path to the configured -datasource.url and -remoteRead.url
- -tls
- Whether to enable TLS for incoming HTTP requests at -httpListenAddr (aka https). -tlsCertFile and -tlsKeyFile must be set if -tls is set
- -tlsCertFile string
- Path to file with TLS certificate if -tls is set. Prefer ECDSA certs instead of RSA certs as RSA certs are slower. The provided certificate file is automatically re-read every second, so it can be dynamically updated
- -tlsCipherSuites array
- Optional list of TLS cipher suites for incoming requests over HTTPS if -tls is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants
- Supports an array of values separated by comma or specified via multiple flags.
- -tlsKeyFile string
- Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
- -tlsMinVersion string
- Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
- -version
- Show VictoriaMetrics version
- -write.url string
- write access url address, example: http://vminsert:8480
-```
-
-## TroubleShooting
-
-* Access control:
- * incorrect `jwt` format, try with our tokens
- * expired token, check `exp` field.
-* Rate Limiting:
- * `scrape_interval` at datasource, reduce it to apply limits faster.
-
-## Limitations
-
-* Access Control:
- * `jwt` token signature verification for `HMAC` algorithms is not supported.
-* RateLimiting:
- * limits applied based on queries to `datasource.url`
- * only cluster version can be rate-limited.
+vmgateway docs can be edited at [docs/vmgateway.md](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/vmgateway.md).
diff --git a/app/vmgateway/vmgateway.png b/app/vmgateway/vmgateway.png
deleted file mode 100644
index 79b8579b7..000000000
Binary files a/app/vmgateway/vmgateway.png and /dev/null differ
diff --git a/app/vmrestore/README.md b/app/vmrestore/README.md
index ec62df29c..a2ee4e72c 100644
--- a/app/vmrestore/README.md
+++ b/app/vmrestore/README.md
@@ -1,237 +1,3 @@
-# vmrestore
+See vmrestore docs [here](https://docs.victoriametrics.com/vmrestore.html).
-`vmrestore` restores data from backups created by [vmbackup](https://docs.victoriametrics.com/vmbackup.html).
-
-Restore process can be interrupted at any time. It is automatically resumed from the interruption point
-when restarting `vmrestore` with the same args.
-
-## Usage
-
-VictoriaMetrics must be stopped during the restore process.
-
-Run the following command to restore backup from the given `-src` into the given `-storageDataPath`:
-
-```console
-./vmrestore -src=:// -storageDataPath=
-```
-
-* `://` is the path to backup made with [vmbackup](https://docs.victoriametrics.com/vmbackup.html).
- `vmrestore` can restore backups from the following storage types:
- * [GCS](https://cloud.google.com/storage/). Example: `-src=gs:///`
- * [S3](https://aws.amazon.com/s3/). Example: `-src=s3:///`
- * [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs/). Example: `-src=azblob:///`
- * Any S3-compatible storage such as [MinIO](https://github.com/minio/minio), [Ceph](https://docs.ceph.com/en/pacific/radosgw/s3/)
- or [Swift](https://platform.swiftstack.com/docs/admin/middleware/s3_middleware.html). See [these docs](#advanced-usage) for details.
- * Local filesystem. Example: `-src=fs://`. Note that `vmbackup` prevents from storing the backup
- into the directory pointed by `-storageDataPath` command-line flag, since this directory should be managed solely by VictoriaMetrics or `vmstorage`.
-* `` is the path to folder where data will be restored. This folder must be passed
- to VictoriaMetrics in `-storageDataPath` command-line flag after the restore process is complete.
-
-The original `-storageDataPath` directory may contain old files. They will be substituted by the files from backup,
-i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/questions/476041/how-do-i-make-rsync-delete-files-that-have-been-deleted-from-the-source-folder).
-
-
-## Troubleshooting
-
-* If `vmrestore` eats all the network bandwidth, then set `-maxBytesPerSecond` to the desired value.
-* If `vmrestore` has been interrupted due to temporary error, then just restart it with the same args. It will resume the restore process.
-
-## Advanced usage
-
-* Obtaining credentials from a file.
-
- Add flag `-credsFilePath=/etc/credentials` with following content:
-
- for s3 (aws, minio or other s3 compatible storages):
-
- ```console
- [default]
- aws_access_key_id=theaccesskey
- aws_secret_access_key=thesecretaccesskeyvalue
- ```
-
- for gce cloud storage:
-
- ```json
- {
- "type": "service_account",
- "project_id": "project-id",
- "private_key_id": "key-id",
- "private_key": "-----BEGIN PRIVATE KEY-----\nprivate-key\n-----END PRIVATE KEY-----\n",
- "client_email": "service-account-email",
- "client_id": "client-id",
- "auth_uri": "https://accounts.google.com/o/oauth2/auth",
- "token_uri": "https://accounts.google.com/o/oauth2/token",
- "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
- "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/service-account-email"
- }
- ```
-
-* Usage with s3 custom url endpoint. It is possible to use `vmrestore` with s3 api compatible storages, like minio, cloudian and other.
- You have to add custom url endpoint with a flag:
-
-```console
- # for minio:
- -customS3Endpoint=http://localhost:9000
-
- # for aws gov region
- -customS3Endpoint=https://s3-fips.us-gov-west-1.amazonaws.com
-```
-
-* Run `vmrestore -help` in order to see all the available options:
-
-```console
- -concurrency int
- The number of concurrent workers. Higher concurrency may reduce restore duration (default 10)
- -configFilePath string
- Path to file with S3 configs. Configs are loaded from default location if not set.
- See https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html
- -configProfile string
- Profile name for S3 configs. If no set, the value of the environment variable will be loaded (AWS_PROFILE or AWS_DEFAULT_PROFILE), or if both not set, DefaultSharedConfigProfile is used
- -credsFilePath string
- Path to file with GCS or S3 credentials. Credentials are loaded from default locations if not set.
- See https://cloud.google.com/iam/docs/creating-managing-service-account-keys and https://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html
- -customS3Endpoint string
- Custom S3 endpoint for use with S3-compatible storages (e.g. MinIO). S3 is used if not set
- -deleteAllObjectVersions
- Whether to prune previous object versions when deleting an object. By default, when object storage has versioning enabled deleting the file removes only current version. This option forces removal of all previous versions. See: https://docs.victoriametrics.com/vmbackup.html#permanent-deletion-of-objects-in-s3-compatible-storages
- -enableTCP6
- Whether to enable IPv6 for listening and dialing. By default, only IPv4 TCP and UDP are used
- -envflag.enable
- Whether to enable reading flags from environment variables in addition to the command line. Command line flag values have priority over values from environment vars. Flags are read only from the command line if this flag isn't set. See https://docs.victoriametrics.com/#environment-variables for more details
- -envflag.prefix string
- Prefix for environment variables if -envflag.enable is set
- -eula
- Deprecated, please use -license or -licenseFile flags instead. By specifying this flag, you confirm that you have an enterprise license and accept the ESA https://victoriametrics.com/legal/esa/ . This flag is available only in Enterprise binaries. See https://docs.victoriametrics.com/enterprise.html
- -filestream.disableFadvise
- Whether to disable fadvise() syscall when reading large data files. The fadvise() syscall prevents from eviction of recently accessed data from OS page cache during background merges and backups. In some rare cases it is better to disable the syscall if it uses too much CPU
- -flagsAuthKey string
- Auth key for /flags endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -fs.disableMmap
- Whether to use pread() instead of mmap() for reading data files. By default, mmap() is used for 64-bit arches and pread() is used for 32-bit arches, since they cannot read data files bigger than 2^32 bytes in memory. mmap() is usually faster for reading small data chunks than pread()
- -http.connTimeout duration
- Incoming http connections are closed after the configured timeout. This may help to spread the incoming load among a cluster of services behind a load balancer. Please note that the real timeout may be bigger by up to 10% as a protection against the thundering herd problem (default 2m0s)
- -http.disableResponseCompression
- Disable compression of HTTP responses to save CPU resources. By default, compression is enabled to save network bandwidth
- -http.header.csp string
- Value for 'Content-Security-Policy' header
- -http.header.frameOptions string
- Value for 'X-Frame-Options' header
- -http.header.hsts string
- Value for 'Strict-Transport-Security' header
- -http.idleConnTimeout duration
- Timeout for incoming idle http connections (default 1m0s)
- -http.maxGracefulShutdownDuration duration
- The maximum duration for a graceful shutdown of the HTTP server. A highly loaded server may require increased value for a graceful shutdown (default 7s)
- -http.pathPrefix string
- An optional prefix to add to all the paths handled by http server. For example, if '-http.pathPrefix=/foo/bar' is set, then all the http requests will be handled on '/foo/bar/*' paths. This may be useful for proxied requests. See https://www.robustperception.io/using-external-urls-and-proxies-with-prometheus
- -http.shutdownDelay duration
- Optional delay before http server shutdown. During this delay, the server returns non-OK responses from /health page, so load balancers can route new requests to other servers
- -httpAuth.password string
- Password for HTTP server's Basic Auth. The authentication is disabled if -httpAuth.username is empty
- -httpAuth.username string
- Username for HTTP server's Basic Auth. The authentication is disabled if empty. See also -httpAuth.password
- -httpListenAddr string
- TCP address for exporting metrics at /metrics page (default ":8421")
- -internStringCacheExpireDuration duration
- The expiry duration for caches for interned strings. See https://en.wikipedia.org/wiki/String_interning . See also -internStringMaxLen and -internStringDisableCache (default 6m0s)
- -internStringDisableCache
- Whether to disable caches for interned strings. This may reduce memory usage at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringCacheExpireDuration and -internStringMaxLen
- -internStringMaxLen int
- The maximum length for strings to intern. A lower limit may save memory at the cost of higher CPU usage. See https://en.wikipedia.org/wiki/String_interning . See also -internStringDisableCache and -internStringCacheExpireDuration (default 500)
- -license string
- Lisense key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed via file specified by -licenseFile command-line flag
- -license.forceOffline
- Whether to enable offline verification for VictoriaMetrics Enterprise license key, which has been passed either via -license or via -licenseFile command-line flag. The issued license key must support offline verification feature. Contact info@victoriametrics.com if you need offline license verification. This flag is avilable only in Enterprise binaries
- -licenseFile string
- Path to file with license key for VictoriaMetrics Enterprise. See https://victoriametrics.com/products/enterprise/ . Trial Enterprise license can be obtained from https://victoriametrics.com/products/enterprise/trial/ . This flag is available only in Enterprise binaries. The license key can be also passed inline via -license command-line flag
- -loggerDisableTimestamps
- Whether to disable writing timestamps in logs
- -loggerErrorsPerSecondLimit int
- Per-second limit on the number of ERROR messages. If more than the given number of errors are emitted per second, the remaining errors are suppressed. Zero values disable the rate limit
- -loggerFormat string
- Format for logs. Possible values: default, json (default "default")
- -loggerJSONFields string
- Allows renaming fields in JSON formatted logs. Example: "ts:timestamp,msg:message" renames "ts" to "timestamp" and "msg" to "message". Supported fields: ts, level, caller, msg
- -loggerLevel string
- Minimum level of errors to log. Possible values: INFO, WARN, ERROR, FATAL, PANIC (default "INFO")
- -loggerOutput string
- Output for the logs. Supported values: stderr, stdout (default "stderr")
- -loggerTimezone string
- Timezone to use for timestamps in logs. Timezone must be a valid IANA Time Zone. For example: America/New_York, Europe/Berlin, Etc/GMT+3 or Local (default "UTC")
- -loggerWarnsPerSecondLimit int
- Per-second limit on the number of WARN messages. If more than the given number of warns are emitted per second, then the remaining warns are suppressed. Zero values disable the rate limit
- -maxBytesPerSecond size
- The maximum download speed. There is no limit if it is set to 0
- Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
- -memory.allowedBytes size
- Allowed size of system memory VictoriaMetrics caches may occupy. This option overrides -memory.allowedPercent if set to a non-zero value. Too low a value may increase the cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache resulting in higher disk IO usage
- Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 0)
- -memory.allowedPercent float
- Allowed percent of system memory VictoriaMetrics caches may occupy. See also -memory.allowedBytes. Too low a value may increase cache miss rate usually resulting in higher CPU and disk IO usage. Too high a value may evict too much data from the OS page cache which will result in higher disk IO usage (default 60)
- -metricsAuthKey string
- Auth key for /metrics endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -pprofAuthKey string
- Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
- -pushmetrics.extraLabel array
- Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
- Supports an array of values separated by comma or specified via multiple flags.
- -pushmetrics.interval duration
- Interval for pushing metrics to -pushmetrics.url (default 10s)
- -pushmetrics.url array
- Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
- Supports an array of values separated by comma or specified via multiple flags.
- -s3ForcePathStyle
- Prefixing endpoint with bucket name when set false, true by default. (default true)
- -s3StorageClass string
- The Storage Class applied to objects uploaded to AWS S3. Supported values are: GLACIER, DEEP_ARCHIVE, GLACIER_IR, INTELLIGENT_TIERING, ONEZONE_IA, OUTPOSTS, REDUCED_REDUNDANCY, STANDARD, STANDARD_IA.
- See https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html
- -skipBackupCompleteCheck
- Whether to skip checking for 'backup complete' file in -src. This may be useful for restoring from old backups, which were created without 'backup complete' file
- -src string
- Source path with backup on the remote storage. Example: gs://bucket/path/to/backup, s3://bucket/path/to/backup, azblob://container/path/to/backup or fs:///path/to/local/backup
- -storageDataPath string
- Destination path where backup must be restored. VictoriaMetrics must be stopped when restoring from backup. -storageDataPath dir can be non-empty. In this case the contents of -storageDataPath dir is synchronized with -src contents, i.e. it works like 'rsync --delete' (default "victoria-metrics-data")
- -tls
- Whether to enable TLS for incoming HTTP requests at -httpListenAddr (aka https). -tlsCertFile and -tlsKeyFile must be set if -tls is set
- -tlsCertFile string
- Path to file with TLS certificate if -tls is set. Prefer ECDSA certs instead of RSA certs as RSA certs are slower. The provided certificate file is automatically re-read every second, so it can be dynamically updated
- -tlsCipherSuites array
- Optional list of TLS cipher suites for incoming requests over HTTPS if -tls is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants
- Supports an array of values separated by comma or specified via multiple flags.
- -tlsKeyFile string
- Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
- -tlsMinVersion string
- Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
- -version
- Show VictoriaMetrics version
-```
-
-## How to build from sources
-
-It is recommended using [binary releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest) - see `vmutils-*` archives there.
-
-### Development build
-
-1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.20.
-1. Run `make vmrestore` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmrestore` binary and puts it into the `bin` folder.
-
-### Production build
-
-1. [Install docker](https://docs.docker.com/install/).
-1. Run `make vmrestore-prod` from the root folder of [the repository](https://github.com/VictoriaMetrics/VictoriaMetrics).
- It builds `vmrestore-prod` binary and puts it into the `bin` folder.
-
-### Building docker images
-
-Run `make package-vmrestore`. It builds `victoriametrics/vmrestore:` docker image locally.
-`` is auto-generated image tag, which depends on source code in the repository.
-The `` may be manually set via `PKG_TAG=foobar make package-vmrestore`.
-
-The base docker image is [alpine](https://hub.docker.com/_/alpine) but it is possible to use any other base image
-by setting it via `` environment variable. For example, the following command builds the image on top of [scratch](https://hub.docker.com/_/scratch) image:
-
-```console
-ROOT_IMAGE=scratch make package-vmrestore
-```
+vmrestore docs can be edited at [docs/vmrestore.md](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/vmrestore.md).
diff --git a/docs/Cluster-VictoriaMetrics.md b/docs/Cluster-VictoriaMetrics.md
index 82276c020..b8b7b987f 100644
--- a/docs/Cluster-VictoriaMetrics.md
+++ b/docs/Cluster-VictoriaMetrics.md
@@ -11,7 +11,7 @@ aliases:
---
# Cluster version
-
+
VictoriaMetrics is a fast, cost-effective and scalable time series database. It can be used as a long-term remote storage for Prometheus.
@@ -57,7 +57,7 @@ This is a [shared nothing architecture](https://en.wikipedia.org/wiki/Shared-not
It increases cluster availability, and simplifies cluster maintenance as well as cluster scaling.
- 
+ 
## Multitenancy
diff --git a/docs/Cluster-VictoriaMetrics_cluster-scheme.png b/docs/Cluster-VictoriaMetrics_cluster-scheme.png
deleted file mode 100644
index 3ebae6fec..000000000
Binary files a/docs/Cluster-VictoriaMetrics_cluster-scheme.png and /dev/null differ
diff --git a/docs/Cluster-VictoriaMetrics_cluster-scheme.webp b/docs/Cluster-VictoriaMetrics_cluster-scheme.webp
new file mode 100644
index 000000000..d41bf4ca7
Binary files /dev/null and b/docs/Cluster-VictoriaMetrics_cluster-scheme.webp differ
diff --git a/docs/Makefile b/docs/Makefile
index 6992f3292..6a9c42dba 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -1,34 +1,24 @@
-docs-install:
- gem install jekyll bundler
- bundle install --gemfile=Gemfile
-
-# To run locally you need to use ruby version =< 2.7.6, but not >=3.x , see https://bbs.archlinux.org/viewtopic.php?pid=1976408#p1976408
-#
-# run local server for documentation website
-# at http://127.0.0.1:4000/
-# On first use, please run `make docs-install`
+# run local server for documentation website at http://127.0.0.1:4000/
docs-up:
- JEKYLL_GITHUB_TOKEN=blank PAGES_API_URL=http://0.0.0.0 bundle exec \
- --gemfile=Gemfile \
- jekyll server --livereload
-
-docs-up-docker:
docker run --rm -it \
-e JEKYLL_GITHUB_TOKEN=blank \
-e PAGES_API_URL=http://0.0.0.0 \
-e PAGES_REPO_NWO=VictoriaMetrics/VictoriaMetrics \
-p 4000:4000 \
- -v $(PWD):/srv/jekyll \
- jekyll/jekyll:3.8 jekyll serve --livereload
+ -v $(shell pwd)/docs:/srv/jekyll \
+ jekyll/jekyll:3.8 jekyll serve --livereload --incremental
-# Converts $IMAGES_EXTENSION in the specified $PATH_TO_IMAGES
-# to webp format with $IMAGE_QUALITY (0:small..100:big)
+# Converts images at docs folder to webp format
# See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#images-in-documentation
docs-images-to-webp:
- docker run -it --rm -v \
- $(PWD)/$(PATH_TO_IMAGES):/data \
- elswork/cwebp \
- find ./ -type f -name '*.$(IMAGES_EXTENSION)' -exec sh -c 'cwebp -q ${IMAGE_QUALITY} -m 6 $$1 -o "$${1%.png}.webp"' _ {} \;
+ IMAGES_EXTENSION=jpg $(MAKE) docs-images-to-webp-by-extension
+ IMAGES_EXTENSION=jpeg $(MAKE) docs-images-to-webp-by-extension
+ IMAGES_EXTENSION=png $(MAKE) docs-images-to-webp-by-extension
-docs-remove-old-images:
- rm -f $(PWD)/$(PATH_TO_IMAGES)/*.$(IMAGES_EXTENSION)
+docs-images-to-webp-by-extension:
+ docker run --rm -it \
+ -v $(shell pwd)/docs:/docs \
+ elswork/cwebp \
+ sh -c 'find /docs/ -type f ! -path "/docs/operator/*" ! -path "/docs/_site/*" -name "*.$(IMAGES_EXTENSION)" -print0 | \
+ xargs -0 -P $(MAKE_CONCURRENCY) -I {} sh -c '"'"'cwebp -preset drawing -m 6 -o "$${1%.*}.webp" $$1'"'"' _ {}'
+ find docs/ -type f ! -path 'docs/operator/*' ! -path 'docs/_site/*' -name '*.$(IMAGES_EXTENSION)' -print0 | xargs -0 rm -f
diff --git a/docs/PerTenantStatistic-stats.jpg b/docs/PerTenantStatistic-stats.jpg
deleted file mode 100644
index 3ef4a93e0..000000000
Binary files a/docs/PerTenantStatistic-stats.jpg and /dev/null differ
diff --git a/docs/PerTenantStatistic-stats.webp b/docs/PerTenantStatistic-stats.webp
new file mode 100644
index 000000000..3add3b189
Binary files /dev/null and b/docs/PerTenantStatistic-stats.webp differ
diff --git a/docs/PerTenantStatistic.md b/docs/PerTenantStatistic.md
index 78f559ec9..c1549fd61 100644
--- a/docs/PerTenantStatistic.md
+++ b/docs/PerTenantStatistic.md
@@ -14,7 +14,7 @@ aliases:
***The per-tenant statistic is a part of [enterprise package](https://docs.victoriametrics.com/enterprise.html). It is available for download and evaluation at [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest). You need to request a [free trial license](https://victoriametrics.com/products/enterprise/trial/) for evaluation.***
-
+
VictoriaMetrics cluster for enterprise provides various metrics and statistics usage per tenant:
diff --git a/docs/README.md b/docs/README.md
index 8efae360a..2e56fcb28 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -12,7 +12,7 @@ title: VictoriaMetrics
[](https://github.com/VictoriaMetrics/VictoriaMetrics/actions)
[](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics)
-
+
VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
@@ -521,7 +521,7 @@ DataDog agent allows configuring destinations for metrics sending via ENV variab
or via [configuration file](https://docs.datadoghq.com/agent/guide/agent-configuration-files/) in section `dd_url`.
- 
+ 
To configure DataDog agent via ENV variable add the following prefix:
@@ -555,7 +555,7 @@ DataDog allows configuring [Dual Shipping](https://docs.datadoghq.com/agent/guid
sending via ENV variable `DD_ADDITIONAL_ENDPOINTS` or via configuration file `additional_endpoints`.
- 
+ 
Run DataDog using the following ENV variable with VictoriaMetrics as additional metrics receiver:
@@ -2502,19 +2502,14 @@ Report bugs and propose new features [here](https://github.com/VictoriaMetrics/V
Please, keep image size and number of images per single page low. Keep the docs page as lightweight as possible.
If the page needs to have many images, consider using WEB-optimized image format [webp](https://developers.google.com/speed/webp).
-When adding a new doc with many images use `webp` format right away. Or use a MAKEFILE command below to
-convert already existing images automatically:
-```console
-PATH_TO_IMAGES=path/to/images IMAGES_EXTENSION={png|jpg|jpeg} IMAGE_QUALITY=0..100 make docs-images-to-webp
-```
+When adding a new doc with many images use `webp` format right away. Or use a Makefile command below to
+convert already existing images at `docs` folder automatically to `web` format:
-_// For this command to work ensure you run it when in `docs` dir (`cd docs`) and have Docker up&running._
+```console
+make docs-images-to-webp
+```
Once conversion is done, update the path to images in your docs and verify everything is correct.
-When you're happy with result - remove the originals with the following command:
-```console
-PATH_TO_IMAGES=path/to/images IMAGES_EXTENSION={png|jpg|jpeg} make docs-remove-old-images
-```
## VictoriaMetrics Logo
diff --git a/docs/Release-Guide.md b/docs/Release-Guide.md
index e9979d11d..e410d92de 100644
--- a/docs/Release-Guide.md
+++ b/docs/Release-Guide.md
@@ -173,7 +173,7 @@ Once updated, run the following commands:
1. Commit and push changes to `master`.
1. Run "Release" action on Github:
- 
+ 
1. Merge new PRs *"Automatic update CHANGELOGs and READMEs"* and *"Synchronize docs"* after pipelines are complete.
## Ansible Roles
diff --git a/docs/Release-Guide_helm-release.png b/docs/Release-Guide_helm-release.png
deleted file mode 100644
index 12a4350b0..000000000
Binary files a/docs/Release-Guide_helm-release.png and /dev/null differ
diff --git a/docs/Release-Guide_helm-release.webp b/docs/Release-Guide_helm-release.webp
new file mode 100644
index 000000000..c2d95d507
Binary files /dev/null and b/docs/Release-Guide_helm-release.webp differ
diff --git a/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM.png b/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM.png
deleted file mode 100644
index abe4e5fed..000000000
Binary files a/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM.png and /dev/null differ
diff --git a/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM.webp b/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM.webp
new file mode 100644
index 000000000..5a562dc74
Binary files /dev/null and b/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM.webp differ
diff --git a/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM_and_DD.png b/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM_and_DD.png
deleted file mode 100644
index 28d2947ac..000000000
Binary files a/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM_and_DD.png and /dev/null differ
diff --git a/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM_and_DD.webp b/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM_and_DD.webp
new file mode 100644
index 000000000..bddce9ab2
Binary files /dev/null and b/docs/Single-server-VictoriaMetrics-sending_DD_metrics_to_VM_and_DD.webp differ
diff --git a/docs/Single-server-VictoriaMetrics.md b/docs/Single-server-VictoriaMetrics.md
index 80289300f..9c8f6a66a 100644
--- a/docs/Single-server-VictoriaMetrics.md
+++ b/docs/Single-server-VictoriaMetrics.md
@@ -20,7 +20,7 @@ aliases:
[](https://github.com/VictoriaMetrics/VictoriaMetrics/actions)
[](https://codecov.io/gh/VictoriaMetrics/VictoriaMetrics)
-
+
VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
@@ -529,7 +529,7 @@ DataDog agent allows configuring destinations for metrics sending via ENV variab
or via [configuration file](https://docs.datadoghq.com/agent/guide/agent-configuration-files/) in section `dd_url`.
-
+
To configure DataDog agent via ENV variable add the following prefix:
@@ -563,7 +563,7 @@ DataDog allows configuring [Dual Shipping](https://docs.datadoghq.com/agent/guid
sending via ENV variable `DD_ADDITIONAL_ENDPOINTS` or via configuration file `additional_endpoints`.
-
+
Run DataDog using the following ENV variable with VictoriaMetrics as additional metrics receiver:
@@ -2510,19 +2510,14 @@ Report bugs and propose new features [here](https://github.com/VictoriaMetrics/V
Please, keep image size and number of images per single page low. Keep the docs page as lightweight as possible.
If the page needs to have many images, consider using WEB-optimized image format [webp](https://developers.google.com/speed/webp).
-When adding a new doc with many images use `webp` format right away. Or use a MAKEFILE command below to
-convert already existing images automatically:
-```console
-PATH_TO_IMAGES=path/to/images IMAGES_EXTENSION={png|jpg|jpeg} IMAGE_QUALITY=0..100 make docs-images-to-webp
-```
+When adding a new doc with many images use `webp` format right away. Or use a Makefile command below to
+convert already existing images at `docs` folder automatically to `web` format:
-_// For this command to work ensure you run it when in `docs` dir (`cd docs`) and have Docker up&running._
+```console
+make docs-images-to-webp
+```
Once conversion is done, update the path to images in your docs and verify everything is correct.
-When you're happy with result - remove the originals with the following command:
-```console
-PATH_TO_IMAGES=path/to/images IMAGES_EXTENSION={png|jpg|jpeg} make docs-remove-old-images
-```
## VictoriaMetrics Logo
diff --git a/docs/VictoriaLogs/querying/README.md b/docs/VictoriaLogs/querying/README.md
index 4c692e531..854330fff 100644
--- a/docs/VictoriaLogs/querying/README.md
+++ b/docs/VictoriaLogs/querying/README.md
@@ -81,7 +81,7 @@ with `vl_http_requests_total{path="/select/logsql/query"}` metric.
VictoriaLogs provides a simple Web UI for logs [querying](https://docs.victoriametrics.com/VictoriaLogs/LogsQL.html) and exploration
at `http://localhost:9428/select/vmui`. The UI allows exploring query results:
-
+
There are three modes of displaying query results:
diff --git a/docs/VictoriaLogs/querying/vmui.png b/docs/VictoriaLogs/querying/vmui.png
deleted file mode 100644
index 0032bcc3e..000000000
Binary files a/docs/VictoriaLogs/querying/vmui.png and /dev/null differ
diff --git a/docs/VictoriaLogs/querying/vmui.webp b/docs/VictoriaLogs/querying/vmui.webp
new file mode 100644
index 000000000..47935271d
Binary files /dev/null and b/docs/VictoriaLogs/querying/vmui.webp differ
diff --git a/docs/grafana-datasource.md b/docs/grafana-datasource.md
index d573d79c7..1ce920549 100644
--- a/docs/grafana-datasource.md
+++ b/docs/grafana-datasource.md
@@ -109,7 +109,7 @@ Please find the example of provisioning Grafana instance with VictoriaMetrics da
1. Download the latest release:
- ``` bash
+ ```bash
ver=$(curl -s https://api.github.com/repos/VictoriaMetrics/grafana-datasource/releases/latest | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1)
curl -L https://github.com/VictoriaMetrics/grafana-datasource/releases/download/$ver/victoriametrics-datasource-$ver.tar.gz -o plugin.tar.gz
tar -xf plugin.tar.gz -C ./victoriametrics-datasource
@@ -143,7 +143,7 @@ docker-compose -f docker-compose.yaml up
When Grafana starts successfully datasources should be present on the datasources tab
- 
+ 
### Install in Kubernetes
@@ -248,7 +248,7 @@ This example uses init container to download and install plugin.
1. To download plugin build and move contents into Grafana plugins directory:
- ``` bash
+ ```bash
ver=$(curl -s https://api.github.com/repos/VictoriaMetrics/grafana-datasource/releases/latest | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1)
curl -L https://github.com/VictoriaMetrics/grafana-datasource/releases/download/$ver/victoriametrics-datasource-$ver.tar.gz -o /var/lib/grafana/plugins/plugin.tar.gz
tar -xf /var/lib/grafana/plugins/plugin.tar.gz -C /var/lib/grafana/plugins/
@@ -262,14 +262,19 @@ This example uses init container to download and install plugin.
### 1. Configure Grafana
Installing dev version of Grafana plugin requires to change `grafana.ini` config to allow loading unsigned plugins:
+
+{% raw %}
``` ini
# Directory where Grafana will automatically scan and look for plugins
plugins = {{path to directory with plugin}}
```
+{% endraw %}
+
``` ini
[plugins]
allow_loading_unsigned_plugins = victoriametrics-datasource
```
+
### 2. Run the plugin
In the project directory, you can run:
```
diff --git a/docs/grafana-datasource_provision_datasources.webp b/docs/grafana-datasource_provision_datasources.webp
new file mode 100644
index 000000000..f2938eeaf
Binary files /dev/null and b/docs/grafana-datasource_provision_datasources.webp differ
diff --git a/docs/guides/getting-started-with-vm-operator.md b/docs/guides/getting-started-with-vm-operator.md
index 2d859fa74..4ea49bef8 100644
--- a/docs/guides/getting-started-with-vm-operator.md
+++ b/docs/guides/getting-started-with-vm-operator.md
@@ -232,7 +232,7 @@ To check that `VMAgent` collects metrics from the k8s cluster open in the browse
You will see something like this:
- 
+ 
`VMAgent` connects to [kubernetes service discovery](https://kubernetes.io/docs/concepts/services-networking/service/) and gets targets which needs to be scraped. This service discovery is controlled by [VictoriaMetrics Operator](https://github.com/VictoriaMetrics/operator)
@@ -312,13 +312,13 @@ EOF
To check that [VictoriaMetrics](https://victoriametrics.com) collecting metrics from the k8s cluster open in your browser [http://127.0.0.1:3000/dashboards](http://127.0.0.1:3000/dashboards) and choose the `VictoriaMetrics - cluster` dashboard. Use `admin` for login and the `password` that you previously got from kubectl.
- 
+ 
The expected output is:
- 
+ 
## 6. Summary
diff --git a/docs/guides/getting-started-with-vm-operator_vmcluster-grafana1.webp b/docs/guides/getting-started-with-vm-operator_vmcluster-grafana1.webp
new file mode 100644
index 000000000..53ec2ffae
Binary files /dev/null and b/docs/guides/getting-started-with-vm-operator_vmcluster-grafana1.webp differ
diff --git a/docs/guides/getting-started-with-vm-operator_vmcluster-grafana2.webp b/docs/guides/getting-started-with-vm-operator_vmcluster-grafana2.webp
new file mode 100644
index 000000000..08421c0d8
Binary files /dev/null and b/docs/guides/getting-started-with-vm-operator_vmcluster-grafana2.webp differ
diff --git a/docs/guides/getting-started-with-vm-operator_vmcluster.webp b/docs/guides/getting-started-with-vm-operator_vmcluster.webp
new file mode 100644
index 000000000..c16e63112
Binary files /dev/null and b/docs/guides/getting-started-with-vm-operator_vmcluster.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration.md b/docs/guides/grafana-vmgateway-openid-configuration.md
index 63cc7206a..35a0579ac 100644
--- a/docs/guides/grafana-vmgateway-openid-configuration.md
+++ b/docs/guides/grafana-vmgateway-openid-configuration.md
@@ -48,21 +48,21 @@ See details about all supported options in the [vmgateway documentation](https:/
Use `OpenID Connect` as `Client Type`.
Specify `grafana` as `Client ID`.
Click `Next`.
- 
+ 
1. Enable `Client authentication`.
Enable `Authorization`.
- 
+ 
Click `Next`.
1. Add Grafana URL as `Root URL`. For example, `http://localhost:3000/`.
- 
+ 
Click `Save`.
1. Go to `Clients` -> `grafana` -> `Credentials`.
- 
+ 
Copy the value of `Client secret`. It will be used later in Grafana configuration.
1. Go to `Clients` -> `grafana` -> `Client scopes`.
Click at `grafana-dedicated` -> `Add mapper` -> `By configuration` -> `User attribute`.
- 
- 
+ 
+ 
Configure the mapper as follows
- `Name` as `vm_access`.
- `Token Claim Name` as `vm_access`.
@@ -70,7 +70,7 @@ See details about all supported options in the [vmgateway documentation](https:/
- `Claim JSON Type` as `JSON`.
Enable `Add to ID token` and `Add to access token`.
- 
+ 
Click `Save`.
1. Go to `Users` -> select user to configure claims -> `Attributes`.
Specify `vm_access` as `Key`.
@@ -78,7 +78,7 @@ See details about all supported options in the [vmgateway documentation](https:/
- for the first user we will specify `{"tenant_id" : {"account_id": 0, "project_id": 0 },"extra_labels":{ "team": "admin" }}` as `Value`.
- for the second user we will specify `{"tenant_id" : {"account_id": 0, "project_id": 1 },"extra_labels":{ "team": "dev" }}` as `Value`.
- 
+ 
Click `Save`.
## Configure grafana
@@ -201,7 +201,7 @@ You can also use VictoriaMetrics [Grafana datasource](https://github.com/Victori
See installation instructions [here](https://github.com/VictoriaMetrics/grafana-datasource#installation).
Enable `Forward OAuth identity` flag.
-
+
Now you can use Grafana to query metrics from the specified tenant.
Users with `vm_access` claim will be able to query metrics from the specified tenant.
@@ -324,22 +324,22 @@ vmagent will write data into VictoriaMetrics single-node and cluster(with tenant
Grafana datasources configuration will be the following:
-
+
Let's login as user with `team=dev` labels limitation set via claims.
Using `vmgateway-cluster` results into `No data` response as proxied request will go to tenant `0:1`.
Since vmagent is only configured to write to `0:0` `No data` is an expected response.
-
+
Switching to `vmgateway-single` does have data. Note that it is limited to metrics with `team=dev` label.
-
+
Now lets login as user with `team=admin`.
Both cluster and single node datasources now return metrics for `team=admin`.
-
-
+
+
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/admin-cluster-data.png b/docs/guides/grafana-vmgateway-openid-configuration/admin-cluster-data.png
deleted file mode 100644
index d19bbe1ae..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/admin-cluster-data.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/admin-cluster-data.webp b/docs/guides/grafana-vmgateway-openid-configuration/admin-cluster-data.webp
new file mode 100644
index 000000000..341354ec1
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/admin-cluster-data.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/admin-single-data.png b/docs/guides/grafana-vmgateway-openid-configuration/admin-single-data.png
deleted file mode 100644
index 8e6630392..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/admin-single-data.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/admin-single-data.webp b/docs/guides/grafana-vmgateway-openid-configuration/admin-single-data.webp
new file mode 100644
index 000000000..f970144a7
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/admin-single-data.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/client-secret.png b/docs/guides/grafana-vmgateway-openid-configuration/client-secret.png
deleted file mode 100644
index 76c1cb2fc..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/client-secret.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/client-secret.webp b/docs/guides/grafana-vmgateway-openid-configuration/client-secret.webp
new file mode 100644
index 000000000..d1a60bd56
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/client-secret.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-client-1.png b/docs/guides/grafana-vmgateway-openid-configuration/create-client-1.png
deleted file mode 100644
index f989a5cd1..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/create-client-1.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-client-1.webp b/docs/guides/grafana-vmgateway-openid-configuration/create-client-1.webp
new file mode 100644
index 000000000..a4d22e438
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/create-client-1.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-client-2.png b/docs/guides/grafana-vmgateway-openid-configuration/create-client-2.png
deleted file mode 100644
index ee1776be2..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/create-client-2.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-client-2.webp b/docs/guides/grafana-vmgateway-openid-configuration/create-client-2.webp
new file mode 100644
index 000000000..d41139aa6
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/create-client-2.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-client-3.png b/docs/guides/grafana-vmgateway-openid-configuration/create-client-3.png
deleted file mode 100644
index 1fbb11d00..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/create-client-3.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-client-3.webp b/docs/guides/grafana-vmgateway-openid-configuration/create-client-3.webp
new file mode 100644
index 000000000..f93708072
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/create-client-3.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-1.png b/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-1.png
deleted file mode 100644
index 52556b9ef..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-1.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-1.webp b/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-1.webp
new file mode 100644
index 000000000..29cf7ac84
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-1.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-2.png b/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-2.png
deleted file mode 100644
index c838dec6c..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-2.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-2.webp b/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-2.webp
new file mode 100644
index 000000000..605a97099
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-2.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-3.png b/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-3.png
deleted file mode 100644
index 838d86691..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-3.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-3.webp b/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-3.webp
new file mode 100644
index 000000000..43b54e697
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/create-mapper-3.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/dev-cluster-nodata.png b/docs/guides/grafana-vmgateway-openid-configuration/dev-cluster-nodata.png
deleted file mode 100644
index f3fb73610..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/dev-cluster-nodata.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/dev-cluster-nodata.webp b/docs/guides/grafana-vmgateway-openid-configuration/dev-cluster-nodata.webp
new file mode 100644
index 000000000..c4345a558
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/dev-cluster-nodata.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/dev-single-data.png b/docs/guides/grafana-vmgateway-openid-configuration/dev-single-data.png
deleted file mode 100644
index 5e7ab6753..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/dev-single-data.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/dev-single-data.webp b/docs/guides/grafana-vmgateway-openid-configuration/dev-single-data.webp
new file mode 100644
index 000000000..b02f4e527
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/dev-single-data.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/grafana-ds.png b/docs/guides/grafana-vmgateway-openid-configuration/grafana-ds.png
deleted file mode 100644
index 40f415d70..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/grafana-ds.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/grafana-ds.webp b/docs/guides/grafana-vmgateway-openid-configuration/grafana-ds.webp
new file mode 100644
index 000000000..302e92088
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/grafana-ds.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/grafana-test-datasources.png b/docs/guides/grafana-vmgateway-openid-configuration/grafana-test-datasources.png
deleted file mode 100644
index 7086b8435..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/grafana-test-datasources.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/grafana-test-datasources.webp b/docs/guides/grafana-vmgateway-openid-configuration/grafana-test-datasources.webp
new file mode 100644
index 000000000..befd9da6c
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/grafana-test-datasources.webp differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/user-attributes.png b/docs/guides/grafana-vmgateway-openid-configuration/user-attributes.png
deleted file mode 100644
index c86e222fd..000000000
Binary files a/docs/guides/grafana-vmgateway-openid-configuration/user-attributes.png and /dev/null differ
diff --git a/docs/guides/grafana-vmgateway-openid-configuration/user-attributes.webp b/docs/guides/grafana-vmgateway-openid-configuration/user-attributes.webp
new file mode 100644
index 000000000..674b6ea02
Binary files /dev/null and b/docs/guides/grafana-vmgateway-openid-configuration/user-attributes.webp differ
diff --git a/docs/guides/guide-vmanomaly-alert-rule.png b/docs/guides/guide-vmanomaly-alert-rule.png
deleted file mode 100644
index 171753e8c..000000000
Binary files a/docs/guides/guide-vmanomaly-alert-rule.png and /dev/null differ
diff --git a/docs/guides/guide-vmanomaly-alerts-firing.png b/docs/guides/guide-vmanomaly-alerts-firing.png
deleted file mode 100644
index b553b6ef1..000000000
Binary files a/docs/guides/guide-vmanomaly-alerts-firing.png and /dev/null differ
diff --git a/docs/guides/guide-vmanomaly-anomaly-score.png b/docs/guides/guide-vmanomaly-anomaly-score.png
deleted file mode 100644
index bbef33616..000000000
Binary files a/docs/guides/guide-vmanomaly-anomaly-score.png and /dev/null differ
diff --git a/docs/guides/guide-vmanomaly-docker-compose.png b/docs/guides/guide-vmanomaly-docker-compose.png
deleted file mode 100644
index fee0cde45..000000000
Binary files a/docs/guides/guide-vmanomaly-docker-compose.png and /dev/null differ
diff --git a/docs/guides/guide-vmanomaly-files.png b/docs/guides/guide-vmanomaly-files.png
deleted file mode 100644
index 949054898..000000000
Binary files a/docs/guides/guide-vmanomaly-files.png and /dev/null differ
diff --git a/docs/guides/guide-vmanomaly-node-cpu-rate-graph.png b/docs/guides/guide-vmanomaly-node-cpu-rate-graph.png
deleted file mode 100644
index 184a6e905..000000000
Binary files a/docs/guides/guide-vmanomaly-node-cpu-rate-graph.png and /dev/null differ
diff --git a/docs/guides/guide-vmanomaly-vmalert.md b/docs/guides/guide-vmanomaly-vmalert.md
index 98d526d6e..c551fc366 100644
--- a/docs/guides/guide-vmanomaly-vmalert.md
+++ b/docs/guides/guide-vmanomaly-vmalert.md
@@ -82,7 +82,7 @@ These modes are mutually exclusive. A high _iowait_ means that you are disk or n
The metric `node_cpu_seconds_total` is a [counter](https://docs.victoriametrics.com/keyConcepts.html#counter) type of metric. If we'd like to see how much time CPU spent in each of the nodes, we need to calculate the per-second values change via [rate function](https://docs.victoriametrics.com/MetricsQL.html#rate): `rate(node_cpu_seconds_total)`.
Here is how this query may look like in Grafana:
-
+
This query result will generate 8 time series per each cpu, and we will use them as an input for our VM Anomaly Detection. vmanomaly will start learning configured model type separately for each of the time series.
@@ -195,7 +195,7 @@ Now we are going to configure the `docker-compose.yml` file to run all needed se
Here are all services we are going to run:
- 
+ 
* victoriametrics - VictoriaMetrics Time Series Database
@@ -264,6 +264,7 @@ Let's wrap it all up together into the `docker-compose.yml` file.
+{% raw %}
``` yaml
services:
vmagent:
@@ -378,13 +379,14 @@ volumes:
networks:
vm_net:
```
+{% endraw %}
Before running our docker-compose make sure that your directory contains all required files:
- 
+ 
This docker-compose file will pull docker images, set up each service and run them all together with the command:
@@ -425,29 +427,29 @@ Each of these metrics will contain same labels our query `rate(node_cpu_seconds_
### Anomaly scores for each metric with its according labels.
Query: `anomaly_score`
-
+
Check out if the anomaly score is high for datapoints you think are anomalies. If not, you can try other parameters in the config file or try other model type.
As you may notice a lot of data shows anomaly score greater than 1. It is expected as we just started to scrape and store data and there are not enough datapoints to train on. Just wait for some more time for gathering more data to see how well this particular model can find anomalies. In our configs we put 2 days of data required.
### Actual value from input query with predicted `yhat` metric.
Query: `yhat`
-
+
Here we are using one particular set of metrics for visualization. Check out the difference between model prediction and actual values. If values are very different from prediction, it can be considered as anomalous.
### Lower and upper boundaries that model predicted.
Queries: `yhat_lower` and `yhat_upper`
-
+
Boundaries of 'normal' metric values according to model inference.
### Alerting
On the page `http://localhost:8880/vmalert/groups` you can find our configured Alerting rule:
-
+
According to the rule configured for vmalert we will see Alert when anomaly score exceed 1. You will see an alert on Alert tab. `http://localhost:8880/vmalert/alerts`
-
+
## 10. Conclusion
Now we know how to set up Victoria Metric Anomaly Detection tool and use it together with vmalert. We also discovered core vmanomaly generated metrics and behaviour.
diff --git a/docs/guides/guide-vmanomaly-vmalert_alert-rule.webp b/docs/guides/guide-vmanomaly-vmalert_alert-rule.webp
new file mode 100644
index 000000000..d4b9b820f
Binary files /dev/null and b/docs/guides/guide-vmanomaly-vmalert_alert-rule.webp differ
diff --git a/docs/guides/guide-vmanomaly-vmalert_alerts-firing.webp b/docs/guides/guide-vmanomaly-vmalert_alerts-firing.webp
new file mode 100644
index 000000000..9697c9ed4
Binary files /dev/null and b/docs/guides/guide-vmanomaly-vmalert_alerts-firing.webp differ
diff --git a/docs/guides/guide-vmanomaly-vmalert_anomaly-score.webp b/docs/guides/guide-vmanomaly-vmalert_anomaly-score.webp
new file mode 100644
index 000000000..2a51ac4f1
Binary files /dev/null and b/docs/guides/guide-vmanomaly-vmalert_anomaly-score.webp differ
diff --git a/docs/guides/guide-vmanomaly-vmalert_docker-compose.webp b/docs/guides/guide-vmanomaly-vmalert_docker-compose.webp
new file mode 100644
index 000000000..78bdd19d0
Binary files /dev/null and b/docs/guides/guide-vmanomaly-vmalert_docker-compose.webp differ
diff --git a/docs/guides/guide-vmanomaly-vmalert_files.webp b/docs/guides/guide-vmanomaly-vmalert_files.webp
new file mode 100644
index 000000000..52c07fc5f
Binary files /dev/null and b/docs/guides/guide-vmanomaly-vmalert_files.webp differ
diff --git a/docs/guides/guide-vmanomaly-vmalert_node-cpu-rate-graph.webp b/docs/guides/guide-vmanomaly-vmalert_node-cpu-rate-graph.webp
new file mode 100644
index 000000000..7888241f1
Binary files /dev/null and b/docs/guides/guide-vmanomaly-vmalert_node-cpu-rate-graph.webp differ
diff --git a/docs/guides/guide-vmanomaly-vmalert_yhat-lower-upper.webp b/docs/guides/guide-vmanomaly-vmalert_yhat-lower-upper.webp
new file mode 100644
index 000000000..ccbbd07d4
Binary files /dev/null and b/docs/guides/guide-vmanomaly-vmalert_yhat-lower-upper.webp differ
diff --git a/docs/guides/guide-vmanomaly-vmalert_yhat.webp b/docs/guides/guide-vmanomaly-vmalert_yhat.webp
new file mode 100644
index 000000000..186d2d7b8
Binary files /dev/null and b/docs/guides/guide-vmanomaly-vmalert_yhat.webp differ
diff --git a/docs/guides/guide-vmanomaly-yhat-lower-upper.png b/docs/guides/guide-vmanomaly-yhat-lower-upper.png
deleted file mode 100644
index 5bf7dd795..000000000
Binary files a/docs/guides/guide-vmanomaly-yhat-lower-upper.png and /dev/null differ
diff --git a/docs/guides/guide-vmanomaly-yhat.png b/docs/guides/guide-vmanomaly-yhat.png
deleted file mode 100644
index 6ed6dd3f1..000000000
Binary files a/docs/guides/guide-vmanomaly-yhat.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-dashes-agent.png b/docs/guides/guide-vmcluster-dashes-agent.png
deleted file mode 100644
index ab27aac81..000000000
Binary files a/docs/guides/guide-vmcluster-dashes-agent.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-grafana-dash.png b/docs/guides/guide-vmcluster-grafana-dash.png
deleted file mode 100644
index 1f285649e..000000000
Binary files a/docs/guides/guide-vmcluster-grafana-dash.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-k8s-dashboard.png b/docs/guides/guide-vmcluster-k8s-dashboard.png
deleted file mode 100644
index 4fb771cd4..000000000
Binary files a/docs/guides/guide-vmcluster-k8s-dashboard.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-k8s-ha-explore-count-up-graph.png b/docs/guides/guide-vmcluster-k8s-ha-explore-count-up-graph.png
deleted file mode 100644
index a1d165a6e..000000000
Binary files a/docs/guides/guide-vmcluster-k8s-ha-explore-count-up-graph.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-k8s-ha-explore-count-up-graph2.png b/docs/guides/guide-vmcluster-k8s-ha-explore-count-up-graph2.png
deleted file mode 100644
index cc7465151..000000000
Binary files a/docs/guides/guide-vmcluster-k8s-ha-explore-count-up-graph2.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-k8s-ha-explore-count-up.png b/docs/guides/guide-vmcluster-k8s-ha-explore-count-up.png
deleted file mode 100644
index 5380bd2da..000000000
Binary files a/docs/guides/guide-vmcluster-k8s-ha-explore-count-up.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-k8s-ha-explore.png b/docs/guides/guide-vmcluster-k8s-ha-explore.png
deleted file mode 100644
index 89d745f54..000000000
Binary files a/docs/guides/guide-vmcluster-k8s-ha-explore.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-k8s-scheme.png b/docs/guides/guide-vmcluster-k8s-scheme.png
deleted file mode 100644
index 65b3c743f..000000000
Binary files a/docs/guides/guide-vmcluster-k8s-scheme.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-k8s-via-vm-operator-grafana1.png b/docs/guides/guide-vmcluster-k8s-via-vm-operator-grafana1.png
deleted file mode 100644
index d1d91c7e6..000000000
Binary files a/docs/guides/guide-vmcluster-k8s-via-vm-operator-grafana1.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-k8s-via-vm-operator-grafana2.png b/docs/guides/guide-vmcluster-k8s-via-vm-operator-grafana2.png
deleted file mode 100644
index 3d7b9cb01..000000000
Binary files a/docs/guides/guide-vmcluster-k8s-via-vm-operator-grafana2.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-k8s-via-vm-operator.png b/docs/guides/guide-vmcluster-k8s-via-vm-operator.png
deleted file mode 100644
index a33ae2576..000000000
Binary files a/docs/guides/guide-vmcluster-k8s-via-vm-operator.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-multiple-retention-scheme.png b/docs/guides/guide-vmcluster-multiple-retention-scheme.png
deleted file mode 100644
index 46fa3ca70..000000000
Binary files a/docs/guides/guide-vmcluster-multiple-retention-scheme.png and /dev/null differ
diff --git a/docs/guides/guide-vmcluster-multiple-retention-setup.md b/docs/guides/guide-vmcluster-multiple-retention-setup.md
index a00565eb8..bf75903bd 100644
--- a/docs/guides/guide-vmcluster-multiple-retention-setup.md
+++ b/docs/guides/guide-vmcluster-multiple-retention-setup.md
@@ -34,7 +34,7 @@ Solution contains 3 groups of vmstorages + vminserts and one group of vmselects.
The diagram below shows a proposed solution
- 
+ 
**Implementation Details**
diff --git a/docs/guides/guide-vmcluster-multiple-retention-setup.webp b/docs/guides/guide-vmcluster-multiple-retention-setup.webp
new file mode 100644
index 000000000..89b310238
Binary files /dev/null and b/docs/guides/guide-vmcluster-multiple-retention-setup.webp differ
diff --git a/docs/guides/guide-vmcluster-vmagent-grafana-dash.png b/docs/guides/guide-vmcluster-vmagent-grafana-dash.png
deleted file mode 100644
index 5af6667e7..000000000
Binary files a/docs/guides/guide-vmcluster-vmagent-grafana-dash.png and /dev/null differ
diff --git a/docs/guides/guide-vmsingle-grafana-dashboards.png b/docs/guides/guide-vmsingle-grafana-dashboards.png
deleted file mode 100644
index 44f0b6442..000000000
Binary files a/docs/guides/guide-vmsingle-grafana-dashboards.png and /dev/null differ
diff --git a/docs/guides/guide-vmsingle-grafana-k8s-dashboard.png b/docs/guides/guide-vmsingle-grafana-k8s-dashboard.png
deleted file mode 100644
index fa1b9838f..000000000
Binary files a/docs/guides/guide-vmsingle-grafana-k8s-dashboard.png and /dev/null differ
diff --git a/docs/guides/guide-vmsingle-grafana.png b/docs/guides/guide-vmsingle-grafana.png
deleted file mode 100644
index f4c9329e0..000000000
Binary files a/docs/guides/guide-vmsingle-grafana.png and /dev/null differ
diff --git a/docs/guides/guide-vmsingle-k8s-scheme.png b/docs/guides/guide-vmsingle-k8s-scheme.png
deleted file mode 100644
index 81f31eac0..000000000
Binary files a/docs/guides/guide-vmsingle-k8s-scheme.png and /dev/null differ
diff --git a/docs/guides/k8s-ha-monitoring-via-vm-cluster.md b/docs/guides/k8s-ha-monitoring-via-vm-cluster.md
index 0c9f3587d..243fa9ce8 100644
--- a/docs/guides/k8s-ha-monitoring-via-vm-cluster.md
+++ b/docs/guides/k8s-ha-monitoring-via-vm-cluster.md
@@ -378,19 +378,19 @@ To test via Grafana, we need to install it first. [Install and connect Grafana t
- 
+ 
Choose `victoriametrics` from the list of datasources and enter `count(up{kubernetes_pod_name=~".*vmselect.*"})` to the **Metric browser** field as shown on the screenshot, then press **Run query** button:
- 
+ 
The expected output is:
- 
+ 
## 5. High Availability
@@ -424,7 +424,7 @@ Return to Grafana Explore and press the **Run query** button again.
The expected output is:
-
+
As you can see, after we scaled down the `vmstorage` replicas number from three to two pods, metrics are still available and correct. The response is not partial as it was before scaling. Also we see that query `count(up{kubernetes_pod_name=~".*vmselect.*"})` returns the same value as before.
@@ -432,7 +432,7 @@ As you can see, after we scaled down the `vmstorage` replicas number from three
To confirm that the number of `vmstorage` pods is equivalent to two, execute the following request in Grafana Explore:
- 
+ 
diff --git a/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore-count-up-graph.webp b/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore-count-up-graph.webp
new file mode 100644
index 000000000..0c7a4f856
Binary files /dev/null and b/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore-count-up-graph.webp differ
diff --git a/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore-count-up-graph2.webp b/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore-count-up-graph2.webp
new file mode 100644
index 000000000..54746caf4
Binary files /dev/null and b/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore-count-up-graph2.webp differ
diff --git a/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore-count-up.webp b/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore-count-up.webp
new file mode 100644
index 000000000..3817858d2
Binary files /dev/null and b/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore-count-up.webp differ
diff --git a/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore.webp b/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore.webp
new file mode 100644
index 000000000..b3489d59b
Binary files /dev/null and b/docs/guides/k8s-ha-monitoring-via-vm-cluster_explore.webp differ
diff --git a/docs/guides/k8s-monitoring-via-vm-cluster.md b/docs/guides/k8s-monitoring-via-vm-cluster.md
index 9179ef4a4..cedace4d3 100644
--- a/docs/guides/k8s-monitoring-via-vm-cluster.md
+++ b/docs/guides/k8s-monitoring-via-vm-cluster.md
@@ -27,7 +27,7 @@ We will use:
* [kubectl 1.21](https://kubernetes.io/docs/tasks/tools/install-kubectl)
- 
+ 
## 1. VictoriaMetrics Helm repository
@@ -536,22 +536,22 @@ kubectl --namespace default port-forward $POD_NAME 3000
To check that [VictoriaMetrics](https://victoriametrics.com) collects metrics from k8s cluster open in browser [http://127.0.0.1:3000/dashboards](http://127.0.0.1:3000/dashboards) and choose the `Kubernetes Cluster Monitoring (via Prometheus)` dashboard. Use `admin` for login and `password` that you previously got from kubectl.
- 
+ 
You will see something like this:
- 
+ 
The VictoriaMetrics dashboard is also available to use:
- 
+ 
vmagent has its own dashboard:
- 
+ 
## 6. Final thoughts
diff --git a/docs/guides/k8s-monitoring-via-vm-cluster_dashboard.webp b/docs/guides/k8s-monitoring-via-vm-cluster_dashboard.webp
new file mode 100644
index 000000000..3022b00c8
Binary files /dev/null and b/docs/guides/k8s-monitoring-via-vm-cluster_dashboard.webp differ
diff --git a/docs/guides/k8s-monitoring-via-vm-cluster_dashes-agent.webp b/docs/guides/k8s-monitoring-via-vm-cluster_dashes-agent.webp
new file mode 100644
index 000000000..f3f3183d5
Binary files /dev/null and b/docs/guides/k8s-monitoring-via-vm-cluster_dashes-agent.webp differ
diff --git a/docs/guides/k8s-monitoring-via-vm-cluster_grafana-dash.webp b/docs/guides/k8s-monitoring-via-vm-cluster_grafana-dash.webp
new file mode 100644
index 000000000..f1f75d42a
Binary files /dev/null and b/docs/guides/k8s-monitoring-via-vm-cluster_grafana-dash.webp differ
diff --git a/docs/guides/k8s-monitoring-via-vm-cluster_scheme.webp b/docs/guides/k8s-monitoring-via-vm-cluster_scheme.webp
new file mode 100644
index 000000000..9190ef20f
Binary files /dev/null and b/docs/guides/k8s-monitoring-via-vm-cluster_scheme.webp differ
diff --git a/docs/guides/k8s-monitoring-via-vm-cluster_vmagent-grafana-dash.webp b/docs/guides/k8s-monitoring-via-vm-cluster_vmagent-grafana-dash.webp
new file mode 100644
index 000000000..2a41fd017
Binary files /dev/null and b/docs/guides/k8s-monitoring-via-vm-cluster_vmagent-grafana-dash.webp differ
diff --git a/docs/guides/k8s-monitoring-via-vm-single.md b/docs/guides/k8s-monitoring-via-vm-single.md
index 8be446b7a..85e5cee91 100644
--- a/docs/guides/k8s-monitoring-via-vm-single.md
+++ b/docs/guides/k8s-monitoring-via-vm-single.md
@@ -27,7 +27,7 @@ We will use:
* [kubectl 1.21](https://kubernetes.io/docs/tasks/tools/install-kubectl)
- 
+ 
## 1. VictoriaMetrics Helm repository
@@ -341,17 +341,17 @@ Now Grafana should be accessible on the [http://127.0.0.1:3000](http://127.0.0.1
To check that VictoriaMetrics has collects metrics from the k8s cluster open in browser [http://127.0.0.1:3000/dashboards](http://127.0.0.1:3000/dashboards) and choose `Kubernetes Cluster Monitoring (via Prometheus)` dashboard. Use `admin` for login and `password` that you previously obtained from kubectl.
- 
+ 
You will see something like this:
- 
+ 
VictoriaMetrics dashboard also available to use:
- 
+ 
## 5. Final thoughts
diff --git a/docs/guides/k8s-monitoring-via-vm-single_grafana-dashboards.webp b/docs/guides/k8s-monitoring-via-vm-single_grafana-dashboards.webp
new file mode 100644
index 000000000..6a27f2208
Binary files /dev/null and b/docs/guides/k8s-monitoring-via-vm-single_grafana-dashboards.webp differ
diff --git a/docs/guides/k8s-monitoring-via-vm-single_grafana-k8s-dashboard.webp b/docs/guides/k8s-monitoring-via-vm-single_grafana-k8s-dashboard.webp
new file mode 100644
index 000000000..3870f18de
Binary files /dev/null and b/docs/guides/k8s-monitoring-via-vm-single_grafana-k8s-dashboard.webp differ
diff --git a/docs/guides/k8s-monitoring-via-vm-single_grafana.webp b/docs/guides/k8s-monitoring-via-vm-single_grafana.webp
new file mode 100644
index 000000000..474a34d52
Binary files /dev/null and b/docs/guides/k8s-monitoring-via-vm-single_grafana.webp differ
diff --git a/docs/guides/k8s-monitoring-via-vm-single_k8s-scheme.webp b/docs/guides/k8s-monitoring-via-vm-single_k8s-scheme.webp
new file mode 100644
index 000000000..55229ab99
Binary files /dev/null and b/docs/guides/k8s-monitoring-via-vm-single_k8s-scheme.webp differ
diff --git a/docs/guides/migrate-from-influx-data-sample-in-influx.png b/docs/guides/migrate-from-influx-data-sample-in-influx.png
deleted file mode 100644
index 112756bb0..000000000
Binary files a/docs/guides/migrate-from-influx-data-sample-in-influx.png and /dev/null differ
diff --git a/docs/guides/migrate-from-influx-data-sample-in-vm.png b/docs/guides/migrate-from-influx-data-sample-in-vm.png
deleted file mode 100644
index 5184b2a31..000000000
Binary files a/docs/guides/migrate-from-influx-data-sample-in-vm.png and /dev/null differ
diff --git a/docs/guides/migrate-from-influx-vmui.png b/docs/guides/migrate-from-influx-vmui.png
deleted file mode 100644
index 0dbe4b577..000000000
Binary files a/docs/guides/migrate-from-influx-vmui.png and /dev/null differ
diff --git a/docs/guides/migrate-from-influx.md b/docs/guides/migrate-from-influx.md
index ff7a0122a..387d7cf20 100644
--- a/docs/guides/migrate-from-influx.md
+++ b/docs/guides/migrate-from-influx.md
@@ -142,7 +142,7 @@ for serving read queries. This API is used in various integrations such as
by [VMUI](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui) - a graphical User Interface for
querying and visualizing metrics:
-
+
See more about [how to query data in VictoriaMetrics](https://docs.victoriametrics.com/keyConcepts.html#query-data).
@@ -171,7 +171,7 @@ The data sample consists data points for a measurement `foo`
and a field `bar` with additional tag `instance=localhost`. If we would like plot this data as a time series in Grafana
it might have the following look:
-
+
The query used for this panel is written in
[InfluxQL](https://docs.influxdata.com/influxdb/v1.8/query_language/):
@@ -206,7 +206,7 @@ InfluxQL query might be translated to MetricsQL let's break it into components f
In result, executing the `foo_bar{instance="localhost"}` MetricsQL expression with `step=1m` for the same set of data in
Grafana will have the following form:
-
+
Visualizations from both databases are a bit different - VictoriaMetrics shows some extra points
filling the gaps in the graph. This behavior is described in more
diff --git a/docs/guides/migrate-from-influx_data-sample-in-influx.webp b/docs/guides/migrate-from-influx_data-sample-in-influx.webp
new file mode 100644
index 000000000..4daa50311
Binary files /dev/null and b/docs/guides/migrate-from-influx_data-sample-in-influx.webp differ
diff --git a/docs/guides/migrate-from-influx_data-sample-in-vm.webp b/docs/guides/migrate-from-influx_data-sample-in-vm.webp
new file mode 100644
index 000000000..67be175ba
Binary files /dev/null and b/docs/guides/migrate-from-influx_data-sample-in-vm.webp differ
diff --git a/docs/guides/migrate-from-influx_vmui.webp b/docs/guides/migrate-from-influx_vmui.webp
new file mode 100644
index 000000000..75364dec1
Binary files /dev/null and b/docs/guides/migrate-from-influx_vmui.webp differ
diff --git a/docs/guides/multi-regional-setup-dedicated-regions.md b/docs/guides/multi-regional-setup-dedicated-regions.md
index e9f284b64..58627dde9 100644
--- a/docs/guides/multi-regional-setup-dedicated-regions.md
+++ b/docs/guides/multi-regional-setup-dedicated-regions.md
@@ -18,7 +18,7 @@ The monitoring setup is in the dedicated regions as shown below:
diff --git a/docs/guides/multi-regional-setup-dedicated-regions.png b/docs/guides/multi-regional-setup-dedicated-regions.png
deleted file mode 100644
index 83b41c227..000000000
Binary files a/docs/guides/multi-regional-setup-dedicated-regions.png and /dev/null differ
diff --git a/docs/guides/multi-regional-setup-dedicated-regions.webp b/docs/guides/multi-regional-setup-dedicated-regions.webp
new file mode 100644
index 000000000..c7a532b31
Binary files /dev/null and b/docs/guides/multi-regional-setup-dedicated-regions.webp differ
diff --git a/docs/keyConcepts.md b/docs/keyConcepts.md
index 585604672..520eb0c37 100644
--- a/docs/keyConcepts.md
+++ b/docs/keyConcepts.md
@@ -133,7 +133,7 @@ So, the `counter` metric shows the number of observed events since the service s
In programming, `counter` is a variable that you **increment** each time something happens.
-
+
`vm_http_requests_total` is a typical example of a counter. The interpretation of a graph
above is that time series `vm_http_requests_total{instance="localhost:8428", job="victoriametrics", path="api/v1/query_range"}`
@@ -159,7 +159,7 @@ by humans from other metric types.
Gauge is used for measuring a value that can go up and down:
-
+
The metric `process_resident_memory_anon_bytes` on the graph shows the memory usage of the application at every given time.
It is changing frequently, going up and down showing how the process allocates and frees the memory.
@@ -257,7 +257,7 @@ Such a combination of `counter` metrics allows
plotting [Heatmaps in Grafana](https://grafana.com/docs/grafana/latest/visualizations/heatmap/)
and calculating [quantiles](https://prometheus.io/docs/practices/histograms/#quantiles):
-
+
Grafana doesn't understand buckets with `vmrange` labels, so the [prometheus_buckets](https://docs.victoriametrics.com/MetricsQL.html#prometheus_buckets)
function must be used for converting buckets with `vmrange` labels to buckets with `le` labels before building heatmaps in Grafana.
@@ -299,7 +299,7 @@ go_gc_duration_seconds_count 83
The visualisation of summaries is pretty straightforward:
-
+
Such an approach makes summaries easier to use but also puts significant limitations compared to [histograms](#histogram):
@@ -370,7 +370,7 @@ VictoriaMetrics supports both models used in modern monitoring applications: [pu
Client regularly sends the collected metrics to the server in the push model:
-
+
The client (application) decides when and where to send its metrics. VictoriaMetrics supports the following protocols
for data ingestion (aka `push protocols`):
@@ -430,7 +430,7 @@ The cons of push protocol:
Pull model is an approach popularized by [Prometheus](https://prometheus.io/), where the monitoring system decides when
and where to pull metrics from:
-
+
In pull model, the monitoring system needs to be aware of all the applications it needs to monitor. The metrics are
scraped (pulled) from the known applications (aka `scrape targets`) via HTTP protocol on a regular basis (aka `scrape_interval`).
@@ -461,7 +461,7 @@ models for data collection. Many installations use exclusively one of these mode
The most common approach for data collection is using both models:
-
+
In this approach the additional component is used - [vmagent](https://docs.victoriametrics.com/vmagent.html). Vmagent is
a lightweight agent whose main purpose is to collect, filter, relabel and deliver metrics to VictoriaMetrics.
@@ -476,7 +476,7 @@ installation for querying collected data.
VictoriaMetrics components allow building more advanced topologies. For example, vmagents can push metrics from separate datacenters to the central VictoriaMetrics:
-
+
VictoriaMetrics in this example may be either [single-node VictoriaMetrics](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html)
or [VictoriaMetrics Cluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html). Vmagent also allows
@@ -534,8 +534,8 @@ The data sample contains a list of samples for `foo_bar` time series with time i
plot this data sample on the graph, it will have the following form:
-
- 
+
+ 
@@ -572,8 +572,8 @@ we'll see that there is no raw sample at `2022-05-10 10:03`. What happens here i
requested timestamp - VictoriaMetrics will try to locate the closest sample on the left to the requested timestamp:
-
- 
+
+ 
@@ -708,8 +708,8 @@ an [instant query](#instant-query) executed `1 + (start-end)/step` times on the
this request in VictoriaMetrics the graph will be shown as the following:
-
- 
+
+ 
@@ -752,12 +752,12 @@ This flag prevents from non-consistent results due to the fact that only part of
Here is an illustration of a potential problem when `-search.latencyOffset` is set to zero:
-
+
When this flag is set, the VM will return the last metric value collected before the `-search.latencyOffset`
duration throughout the `-search.latencyOffset` duration:
-
+
It can be overridden on per-query basis via `latency_offset` query arg.
@@ -946,7 +946,7 @@ VictoriaMetrics has a built-in graphical User Interface for querying and visuali
[VMUI](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui).
Open `http://victoriametrics:8428/vmui` page, type the query and see the results:
-
+
VictoriaMetrics supports [Prometheus HTTP API](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#prometheus-querying-api-usage)
which makes it possible to [query it with Grafana](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#grafana-setup)
diff --git a/docs/keyConcepts_counter.png b/docs/keyConcepts_counter.png
deleted file mode 100644
index 6222119ff..000000000
Binary files a/docs/keyConcepts_counter.png and /dev/null differ
diff --git a/docs/keyConcepts_counter.webp b/docs/keyConcepts_counter.webp
new file mode 100644
index 000000000..287a5b2f4
Binary files /dev/null and b/docs/keyConcepts_counter.webp differ
diff --git a/docs/keyConcepts_data_collection.png b/docs/keyConcepts_data_collection.png
deleted file mode 100644
index 8fa675efc..000000000
Binary files a/docs/keyConcepts_data_collection.png and /dev/null differ
diff --git a/docs/keyConcepts_data_collection.webp b/docs/keyConcepts_data_collection.webp
new file mode 100644
index 000000000..1e6b98ba6
Binary files /dev/null and b/docs/keyConcepts_data_collection.webp differ
diff --git a/docs/keyConcepts_data_samples.png b/docs/keyConcepts_data_samples.png
deleted file mode 100644
index a4aed1e93..000000000
Binary files a/docs/keyConcepts_data_samples.png and /dev/null differ
diff --git a/docs/keyConcepts_data_samples.webp b/docs/keyConcepts_data_samples.webp
new file mode 100644
index 000000000..4ba87b66a
Binary files /dev/null and b/docs/keyConcepts_data_samples.webp differ
diff --git a/docs/keyConcepts_gauge.png b/docs/keyConcepts_gauge.png
deleted file mode 100644
index 4d91eede3..000000000
Binary files a/docs/keyConcepts_gauge.png and /dev/null differ
diff --git a/docs/keyConcepts_gauge.webp b/docs/keyConcepts_gauge.webp
new file mode 100644
index 000000000..fff1020ce
Binary files /dev/null and b/docs/keyConcepts_gauge.webp differ
diff --git a/docs/keyConcepts_histogram.png b/docs/keyConcepts_histogram.png
deleted file mode 100644
index 285e7916e..000000000
Binary files a/docs/keyConcepts_histogram.png and /dev/null differ
diff --git a/docs/keyConcepts_histogram.webp b/docs/keyConcepts_histogram.webp
new file mode 100644
index 000000000..c111471e1
Binary files /dev/null and b/docs/keyConcepts_histogram.webp differ
diff --git a/docs/keyConcepts_instant_query.png b/docs/keyConcepts_instant_query.png
deleted file mode 100644
index 050fda0ff..000000000
Binary files a/docs/keyConcepts_instant_query.png and /dev/null differ
diff --git a/docs/keyConcepts_instant_query.webp b/docs/keyConcepts_instant_query.webp
new file mode 100644
index 000000000..bc66c3d5b
Binary files /dev/null and b/docs/keyConcepts_instant_query.webp differ
diff --git a/docs/keyConcepts_pull_model.png b/docs/keyConcepts_pull_model.png
deleted file mode 100644
index b737a067a..000000000
Binary files a/docs/keyConcepts_pull_model.png and /dev/null differ
diff --git a/docs/keyConcepts_pull_model.webp b/docs/keyConcepts_pull_model.webp
new file mode 100644
index 000000000..6cd11ead8
Binary files /dev/null and b/docs/keyConcepts_pull_model.webp differ
diff --git a/docs/keyConcepts_push_model.png b/docs/keyConcepts_push_model.png
deleted file mode 100644
index cade73865..000000000
Binary files a/docs/keyConcepts_push_model.png and /dev/null differ
diff --git a/docs/keyConcepts_push_model.webp b/docs/keyConcepts_push_model.webp
new file mode 100644
index 000000000..f1aa10dae
Binary files /dev/null and b/docs/keyConcepts_push_model.webp differ
diff --git a/docs/keyConcepts_range_query.png b/docs/keyConcepts_range_query.png
deleted file mode 100644
index bc5d9397b..000000000
Binary files a/docs/keyConcepts_range_query.png and /dev/null differ
diff --git a/docs/keyConcepts_range_query.webp b/docs/keyConcepts_range_query.webp
new file mode 100644
index 000000000..7ba33f5ad
Binary files /dev/null and b/docs/keyConcepts_range_query.webp differ
diff --git a/docs/keyConcepts_summary.png b/docs/keyConcepts_summary.png
deleted file mode 100644
index d621dba87..000000000
Binary files a/docs/keyConcepts_summary.png and /dev/null differ
diff --git a/docs/keyConcepts_summary.webp b/docs/keyConcepts_summary.webp
new file mode 100644
index 000000000..77a3c550e
Binary files /dev/null and b/docs/keyConcepts_summary.webp differ
diff --git a/docs/keyConcepts_two_dcs.png b/docs/keyConcepts_two_dcs.png
deleted file mode 100644
index 162e36f79..000000000
Binary files a/docs/keyConcepts_two_dcs.png and /dev/null differ
diff --git a/docs/keyConcepts_two_dcs.webp b/docs/keyConcepts_two_dcs.webp
new file mode 100644
index 000000000..8ee900168
Binary files /dev/null and b/docs/keyConcepts_two_dcs.webp differ
diff --git a/docs/keyConcepts_vmui.png b/docs/keyConcepts_vmui.png
deleted file mode 100644
index 2c10383f5..000000000
Binary files a/docs/keyConcepts_vmui.png and /dev/null differ
diff --git a/docs/keyConcepts_vmui.webp b/docs/keyConcepts_vmui.webp
new file mode 100644
index 000000000..816cd1079
Binary files /dev/null and b/docs/keyConcepts_vmui.webp differ
diff --git a/docs/keyConcepts_with_latencyOffset.png b/docs/keyConcepts_with_latencyOffset.png
deleted file mode 100644
index 8e1951d93..000000000
Binary files a/docs/keyConcepts_with_latencyOffset.png and /dev/null differ
diff --git a/docs/keyConcepts_with_latencyOffset.webp b/docs/keyConcepts_with_latencyOffset.webp
new file mode 100644
index 000000000..96c6858c9
Binary files /dev/null and b/docs/keyConcepts_with_latencyOffset.webp differ
diff --git a/docs/keyConcepts_without_latencyOffset.png b/docs/keyConcepts_without_latencyOffset.png
deleted file mode 100644
index 12f1a26b9..000000000
Binary files a/docs/keyConcepts_without_latencyOffset.png and /dev/null differ
diff --git a/docs/keyConcepts_without_latencyOffset.webp b/docs/keyConcepts_without_latencyOffset.webp
new file mode 100644
index 000000000..bfd1870ee
Binary files /dev/null and b/docs/keyConcepts_without_latencyOffset.webp differ
diff --git a/docs/logo.png b/docs/logo.png
deleted file mode 100644
index e3d54e820..000000000
Binary files a/docs/logo.png and /dev/null differ
diff --git a/docs/logo.webp b/docs/logo.webp
new file mode 100644
index 000000000..a63168809
Binary files /dev/null and b/docs/logo.webp differ
diff --git a/docs/managed-victoriametrics/alerting-vmalert-managed-victoria-metrics.md b/docs/managed-victoriametrics/alerting-vmalert-managed-victoria-metrics.md
index a6c15999d..9f0db3e58 100644
--- a/docs/managed-victoriametrics/alerting-vmalert-managed-victoria-metrics.md
+++ b/docs/managed-victoriametrics/alerting-vmalert-managed-victoria-metrics.md
@@ -33,6 +33,7 @@ An example file with one alerting rule.
alerts.yml
+{% raw %}
```yaml
groups:
- name: common
@@ -46,6 +47,7 @@ groups:
summary: "{{ $labels.job }} instance: {{$labels.instance }} is not up"
description: "Job {{ $labels.job }} instance: {{$labels.instance }} is not up for the last 1 minute"
```
+{% endraw %}
### Managed VictoriaMetrics access token and deployment endpoint
@@ -88,6 +90,7 @@ docker run -it -p 8080:8080 -v $(pwd)/alerts.yml:/etc/alerts/alerts.yml victoria
##### Helm Chart
+{% raw %}
```console
export TOKEN=81e8226e-****-****-****-************
export MANAGED_VM_URL=https://gw-c15-1c.cloud.victoriametrics.com
@@ -125,6 +128,7 @@ server:
description: "Job {{ $labels.job }} instance: {{$labels.instance }} is not up for the last 1 minute"
EOF
```
+{% endraw %}
##### VMalert CRD for vmoperator
@@ -203,6 +207,7 @@ docker run -it -p 8080:8080 -v $(pwd)/alerts.yml:/etc/alerts/alerts.yml victoria
##### Helm Chart
+{% raw %}
```console
export TOKEN=76bc5470-****-****-****-************
export MANAGED_VM_READ_URL=https://gw-c15-1a.cloud.victoriametrics.com/select/0/prometheus/
@@ -241,6 +246,7 @@ server:
description: "Job {{ $labels.job }} instance: {{$labels.instance }} is not up for the last 1 minute"
EOF
```
+{% endraw %}
##### VMalert CRD for vmoperator
diff --git a/docs/provision_datasources.png b/docs/provision_datasources.png
deleted file mode 100644
index 7c633f91a..000000000
Binary files a/docs/provision_datasources.png and /dev/null differ
diff --git a/docs/stream-aggregation-check-avg.png b/docs/stream-aggregation-check-avg.png
deleted file mode 100644
index 5fb2d5e78..000000000
Binary files a/docs/stream-aggregation-check-avg.png and /dev/null differ
diff --git a/docs/stream-aggregation-check-avg.webp b/docs/stream-aggregation-check-avg.webp
new file mode 100644
index 000000000..ea2106f1b
Binary files /dev/null and b/docs/stream-aggregation-check-avg.webp differ
diff --git a/docs/stream-aggregation-check-increase.png b/docs/stream-aggregation-check-increase.png
deleted file mode 100644
index 98b63b88f..000000000
Binary files a/docs/stream-aggregation-check-increase.png and /dev/null differ
diff --git a/docs/stream-aggregation-check-increase.webp b/docs/stream-aggregation-check-increase.webp
new file mode 100644
index 000000000..4d5157c65
Binary files /dev/null and b/docs/stream-aggregation-check-increase.webp differ
diff --git a/docs/stream-aggregation-check-max.png b/docs/stream-aggregation-check-max.png
deleted file mode 100644
index f4805a543..000000000
Binary files a/docs/stream-aggregation-check-max.png and /dev/null differ
diff --git a/docs/stream-aggregation-check-max.webp b/docs/stream-aggregation-check-max.webp
new file mode 100644
index 000000000..cd1086d21
Binary files /dev/null and b/docs/stream-aggregation-check-max.webp differ
diff --git a/docs/stream-aggregation-check-min.png b/docs/stream-aggregation-check-min.png
deleted file mode 100644
index 69665bed4..000000000
Binary files a/docs/stream-aggregation-check-min.png and /dev/null differ
diff --git a/docs/stream-aggregation-check-min.webp b/docs/stream-aggregation-check-min.webp
new file mode 100644
index 000000000..4dbddec3e
Binary files /dev/null and b/docs/stream-aggregation-check-min.webp differ
diff --git a/docs/stream-aggregation-check-stdvar.png b/docs/stream-aggregation-check-stdvar.png
deleted file mode 100644
index 217d8a5b4..000000000
Binary files a/docs/stream-aggregation-check-stdvar.png and /dev/null differ
diff --git a/docs/stream-aggregation-check-stdvar.webp b/docs/stream-aggregation-check-stdvar.webp
new file mode 100644
index 000000000..c60d4c6f6
Binary files /dev/null and b/docs/stream-aggregation-check-stdvar.webp differ
diff --git a/docs/stream-aggregation-check-sum-samples.png b/docs/stream-aggregation-check-sum-samples.png
deleted file mode 100644
index d37d9d362..000000000
Binary files a/docs/stream-aggregation-check-sum-samples.png and /dev/null differ
diff --git a/docs/stream-aggregation-check-sum-samples.webp b/docs/stream-aggregation-check-sum-samples.webp
new file mode 100644
index 000000000..991fb73ab
Binary files /dev/null and b/docs/stream-aggregation-check-sum-samples.webp differ
diff --git a/docs/stream-aggregation-check-total-reset.png b/docs/stream-aggregation-check-total-reset.png
deleted file mode 100644
index bc577f7c4..000000000
Binary files a/docs/stream-aggregation-check-total-reset.png and /dev/null differ
diff --git a/docs/stream-aggregation-check-total-reset.webp b/docs/stream-aggregation-check-total-reset.webp
new file mode 100644
index 000000000..e6df40551
Binary files /dev/null and b/docs/stream-aggregation-check-total-reset.webp differ
diff --git a/docs/stream-aggregation-check-total.png b/docs/stream-aggregation-check-total.png
deleted file mode 100644
index 76c1aa65c..000000000
Binary files a/docs/stream-aggregation-check-total.png and /dev/null differ
diff --git a/docs/stream-aggregation-check-total.webp b/docs/stream-aggregation-check-total.webp
new file mode 100644
index 000000000..024958fbc
Binary files /dev/null and b/docs/stream-aggregation-check-total.webp differ
diff --git a/docs/stream-aggregation.md b/docs/stream-aggregation.md
index 65a901972..bcafd97eb 100644
--- a/docs/stream-aggregation.md
+++ b/docs/stream-aggregation.md
@@ -425,7 +425,7 @@ The results of `total` is equal to the `sum(some_counter)` query.
For example, see below time series produced by config with aggregation interval `1m` and `by: ["instance"]` and the regular query:
-
+
`total` is not affected by [counter resets](https://docs.victoriametrics.com/keyConcepts.html#counter) -
it continues to increase monotonically with respect to the previous value.
@@ -433,7 +433,7 @@ The counters are most often reset when the application is restarted.
For example:
-
+
The same behavior will occur when creating or deleting new series in an aggregation group -
`total` will increase monotonically considering the values of the series set.
@@ -452,7 +452,7 @@ The results of `increase` with aggregation interval of `1m` is equal to the `inc
For example, see below time series produced by config with aggregation interval `1m` and `by: ["instance"]` and the regular query:
-
+
`increase` can be used as an alternative for [rate](https://docs.victoriametrics.com/MetricsQL.html#rate) function.
For example, if we have `increase` with `interval` of `5m` for a counter `some_counter`, then to get `rate` we should divide
@@ -484,7 +484,7 @@ The results of `sum_samples` with aggregation interval of `1m` is equal to the `
For example, see below time series produced by config with aggregation interval `1m` and the regular query:
-
+
### last
@@ -503,7 +503,7 @@ The results of `min` with aggregation interval of `1m` is equal to the `min_over
For example, see below time series produced by config with aggregation interval `1m` and the regular query:
-
+
### max
@@ -513,7 +513,7 @@ The results of `max` with aggregation interval of `1m` is equal to the `max_over
For example, see below time series produced by config with aggregation interval `1m` and the regular query:
-
+
### avg
@@ -523,7 +523,7 @@ The results of `avg` with aggregation interval of `1m` is equal to the `avg_over
For example, see below time series produced by config with aggregation interval `1m` and `by: ["instance"]` and the regular query:
-
+
### stddev
@@ -541,7 +541,7 @@ The results of `stdvar` with aggregation interval of `1m` is equal to the `stdva
For example, see below time series produced by config with aggregation interval `1m` and the regular query:
-
+
### histogram_bucket
diff --git a/docs/vmagent.md b/docs/vmagent.md
index fa519ec71..7a588e826 100644
--- a/docs/vmagent.md
+++ b/docs/vmagent.md
@@ -19,7 +19,7 @@ or via [VictoriaMetrics `remote_write` protocol](#victoriametrics-remote-write-p
See [Quick Start](#quick-start) for details.
-
+
## Motivation
diff --git a/docs/vmagent.png b/docs/vmagent.png
deleted file mode 100644
index 14bb49fd2..000000000
Binary files a/docs/vmagent.png and /dev/null differ
diff --git a/docs/vmagent.webp b/docs/vmagent.webp
new file mode 100644
index 000000000..4e7060556
Binary files /dev/null and b/docs/vmagent.webp differ
diff --git a/docs/vmalert-tool.md b/docs/vmalert-tool.md
index 1990b315b..83b681cc3 100644
--- a/docs/vmalert-tool.md
+++ b/docs/vmalert-tool.md
@@ -229,6 +229,7 @@ tests:
#### `alerts.yaml`
+{% raw %}
```yaml
# This is the rules file.
@@ -252,4 +253,5 @@ groups:
expr: sum without(instance) (count_over_time(test[1m]))
- record: suquery_interval_test
expr: count_over_time(up[5m:])
-```
\ No newline at end of file
+```
+{% endraw %}
diff --git a/docs/vmalert.md b/docs/vmalert.md
index d5dd7d7cd..316c0a0cf 100644
--- a/docs/vmalert.md
+++ b/docs/vmalert.md
@@ -505,7 +505,7 @@ rules execution, storing recording rules results and alerts state.
-notifier.url=http://alertmanager:9093 # AlertManager addr to send alerts when they trigger
```
-
+
#### Cluster VictoriaMetrics
@@ -525,7 +525,7 @@ Cluster mode could have multiple `vminsert` and `vmselect` components.
-notifier.url=http://alertmanager:9093 # AlertManager addr to send alerts when they trigger
```
-
+
In case when you want to spread the load on these components - add balancers before them and configure
`vmalert` with balancer addresses. Please, see more about VM's cluster architecture
@@ -549,7 +549,7 @@ Alertmanagers.
-notifier.url=http://alertmanagerN:9093 # The same alert will be sent to all configured notifiers
```
-
+
To avoid recording rules results and alerts state duplication in VictoriaMetrics server
don't forget to configure [deduplication](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#deduplication).
@@ -625,7 +625,7 @@ or reducing resolution) and push results to "cold" cluster.
-remoteWrite.url=http://aggregated-cluster-vminsert:8480/insert/0/prometheus # vminsert addr to persist recording rules results
```
-
+
Please note, [replay](#rules-backfilling) feature may be used for transforming historical data.
@@ -639,7 +639,7 @@ For persisting recording or alerting rule results `vmalert` requires `-remoteWri
But this flag supports only one destination. To persist rule results to multiple destinations
we recommend using [vmagent](https://docs.victoriametrics.com/vmagent.html) as fan-out proxy:
-
+
In this topology, `vmalert` is configured to persist rule results to `vmagent`. And `vmagent`
is configured to fan-out received data to two or more destinations.
@@ -854,7 +854,7 @@ state updates for each rule starting from [v1.86](https://docs.victoriametrics.c
To check updates, click on `Details` link next to rule's name on `/vmalert/groups` page
and check the `Last updates` section:
-
+
Rows in the section represent ordered rule evaluations and their results. The column `curl` contains an example of
HTTP request sent by vmalert to the `-datasource.url` during evaluation. If specific state shows that there were
diff --git a/app/vmalert/vmalert_cluster.excalidraw b/docs/vmalert_cluster.excalidraw
similarity index 100%
rename from app/vmalert/vmalert_cluster.excalidraw
rename to docs/vmalert_cluster.excalidraw
diff --git a/docs/vmalert_cluster.png b/docs/vmalert_cluster.png
deleted file mode 100644
index 184df339e..000000000
Binary files a/docs/vmalert_cluster.png and /dev/null differ
diff --git a/docs/vmalert_cluster.webp b/docs/vmalert_cluster.webp
new file mode 100644
index 000000000..ffcf572f6
Binary files /dev/null and b/docs/vmalert_cluster.webp differ
diff --git a/app/vmalert/vmalert_ha.excalidraw b/docs/vmalert_ha.excalidraw
similarity index 100%
rename from app/vmalert/vmalert_ha.excalidraw
rename to docs/vmalert_ha.excalidraw
diff --git a/docs/vmalert_ha.png b/docs/vmalert_ha.png
deleted file mode 100644
index 76ac521ab..000000000
Binary files a/docs/vmalert_ha.png and /dev/null differ
diff --git a/docs/vmalert_ha.webp b/docs/vmalert_ha.webp
new file mode 100644
index 000000000..a48270ac3
Binary files /dev/null and b/docs/vmalert_ha.webp differ
diff --git a/app/vmalert/vmalert_multicluster.excalidraw b/docs/vmalert_multicluster.excalidraw
similarity index 100%
rename from app/vmalert/vmalert_multicluster.excalidraw
rename to docs/vmalert_multicluster.excalidraw
diff --git a/docs/vmalert_multicluster.png b/docs/vmalert_multicluster.png
deleted file mode 100644
index 8b51f2546..000000000
Binary files a/docs/vmalert_multicluster.png and /dev/null differ
diff --git a/docs/vmalert_multicluster.webp b/docs/vmalert_multicluster.webp
new file mode 100644
index 000000000..035265303
Binary files /dev/null and b/docs/vmalert_multicluster.webp differ
diff --git a/app/vmalert/vmalert_multiple_rw.excalidraw b/docs/vmalert_multiple_rw.excalidraw
similarity index 100%
rename from app/vmalert/vmalert_multiple_rw.excalidraw
rename to docs/vmalert_multiple_rw.excalidraw
diff --git a/docs/vmalert_multiple_rw.png b/docs/vmalert_multiple_rw.png
deleted file mode 100644
index 0f44e4000..000000000
Binary files a/docs/vmalert_multiple_rw.png and /dev/null differ
diff --git a/docs/vmalert_multiple_rw.webp b/docs/vmalert_multiple_rw.webp
new file mode 100644
index 000000000..f6cbd6c28
Binary files /dev/null and b/docs/vmalert_multiple_rw.webp differ
diff --git a/app/vmalert/vmalert_single.excalidraw b/docs/vmalert_single.excalidraw
similarity index 100%
rename from app/vmalert/vmalert_single.excalidraw
rename to docs/vmalert_single.excalidraw
diff --git a/docs/vmalert_single.png b/docs/vmalert_single.png
deleted file mode 100644
index 46f087449..000000000
Binary files a/docs/vmalert_single.png and /dev/null differ
diff --git a/docs/vmalert_single.webp b/docs/vmalert_single.webp
new file mode 100644
index 000000000..ea2db09d5
Binary files /dev/null and b/docs/vmalert_single.webp differ
diff --git a/docs/vmalert_state.png b/docs/vmalert_state.png
deleted file mode 100644
index 5bf656b01..000000000
Binary files a/docs/vmalert_state.png and /dev/null differ
diff --git a/docs/vmalert_state.webp b/docs/vmalert_state.webp
new file mode 100644
index 000000000..23c3ccefa
Binary files /dev/null and b/docs/vmalert_state.webp differ
diff --git a/docs/vmanomaly-holtwinters-example.png b/docs/vmanomaly-holtwinters-example.png
deleted file mode 100644
index 17a4c7721..000000000
Binary files a/docs/vmanomaly-holtwinters-example.png and /dev/null differ
diff --git a/docs/vmanomaly-holtwinters-example.webp b/docs/vmanomaly-holtwinters-example.webp
new file mode 100644
index 000000000..3da8a9e44
Binary files /dev/null and b/docs/vmanomaly-holtwinters-example.webp differ
diff --git a/docs/vmanomaly-prophet-example.png b/docs/vmanomaly-prophet-example.png
deleted file mode 100644
index 84a7d76ff..000000000
Binary files a/docs/vmanomaly-prophet-example.png and /dev/null differ
diff --git a/docs/vmanomaly-prophet-example.webp b/docs/vmanomaly-prophet-example.webp
new file mode 100644
index 000000000..6c9c7870f
Binary files /dev/null and b/docs/vmanomaly-prophet-example.webp differ
diff --git a/docs/vmanomaly.md b/docs/vmanomaly.md
index cd1511b93..152d2b3ea 100644
--- a/docs/vmanomaly.md
+++ b/docs/vmanomaly.md
@@ -105,14 +105,14 @@ Currently, vmanomaly ships with a few common models:
For example, here’s how Prophet predictions could look like on a real-data example
(Prophet auto-detected seasonality interval):
-
+
And here’s what Holt-Winters predictions real-world data could look like (seasonality manually
set to 1 week). Notice that it predicts anomalies in
different places than Prophet because the model noticed there are usually spikes on Friday
morning, so it accounted for that:
-
+
## Process
Upon starting, vmanomaly queries the initial range of data, and trains its model (“fit” by convention).
diff --git a/docs/vmbackupmanager.md b/docs/vmbackupmanager.md
index 076fead16..519a53215 100644
--- a/docs/vmbackupmanager.md
+++ b/docs/vmbackupmanager.md
@@ -117,11 +117,11 @@ The result on the GCS bucket
* The root folder
-
+ 
* The latest folder
-
+ 
Please, see [vmbackup docs](https://docs.victoriametrics.com/vmbackup.html#advanced-usage) for more examples of authentication with different
storage types.
@@ -141,7 +141,7 @@ Backup retention policy is controlled by:
Let’s assume we have a backup manager collecting daily backups for the past 10 days.
-
+
We enable backup retention policy for backup manager by using following configuration:
@@ -166,7 +166,7 @@ info app/vmbackupmanager/retention.go:106 daily backups to delete [daily/2
The result on the GCS bucket. We see only 3 daily backups:
-
+
### Protection backups against deletion by retention policy
diff --git a/docs/vmbackupmanager_latest_folder.png b/docs/vmbackupmanager_latest_folder.png
deleted file mode 100644
index 23162c905..000000000
Binary files a/docs/vmbackupmanager_latest_folder.png and /dev/null differ
diff --git a/docs/vmbackupmanager_latest_folder.webp b/docs/vmbackupmanager_latest_folder.webp
new file mode 100644
index 000000000..a435ad965
Binary files /dev/null and b/docs/vmbackupmanager_latest_folder.webp differ
diff --git a/docs/vmbackupmanager_root_folder.png b/docs/vmbackupmanager_root_folder.png
deleted file mode 100644
index 5d64e5525..000000000
Binary files a/docs/vmbackupmanager_root_folder.png and /dev/null differ
diff --git a/docs/vmbackupmanager_root_folder.webp b/docs/vmbackupmanager_root_folder.webp
new file mode 100644
index 000000000..342b725c0
Binary files /dev/null and b/docs/vmbackupmanager_root_folder.webp differ
diff --git a/docs/vmbackupmanager_rp_daily_1.png b/docs/vmbackupmanager_rp_daily_1.png
deleted file mode 100644
index f2f050aa7..000000000
Binary files a/docs/vmbackupmanager_rp_daily_1.png and /dev/null differ
diff --git a/docs/vmbackupmanager_rp_daily_1.webp b/docs/vmbackupmanager_rp_daily_1.webp
new file mode 100644
index 000000000..9485eaad6
Binary files /dev/null and b/docs/vmbackupmanager_rp_daily_1.webp differ
diff --git a/docs/vmbackupmanager_rp_daily_2.png b/docs/vmbackupmanager_rp_daily_2.png
deleted file mode 100644
index e716c5d6a..000000000
Binary files a/docs/vmbackupmanager_rp_daily_2.png and /dev/null differ
diff --git a/docs/vmbackupmanager_rp_daily_2.webp b/docs/vmbackupmanager_rp_daily_2.webp
new file mode 100644
index 000000000..83df2cdc4
Binary files /dev/null and b/docs/vmbackupmanager_rp_daily_2.webp differ
diff --git a/docs/vmgateway-access-control.jpg b/docs/vmgateway-access-control.jpg
deleted file mode 100644
index 91988329a..000000000
Binary files a/docs/vmgateway-access-control.jpg and /dev/null differ
diff --git a/docs/vmgateway-access-control.webp b/docs/vmgateway-access-control.webp
new file mode 100644
index 000000000..6b4133f95
Binary files /dev/null and b/docs/vmgateway-access-control.webp differ
diff --git a/docs/vmgateway-overview.jpeg b/docs/vmgateway-overview.jpeg
deleted file mode 100644
index adb30aa59..000000000
Binary files a/docs/vmgateway-overview.jpeg and /dev/null differ
diff --git a/docs/vmgateway-overview.webp b/docs/vmgateway-overview.webp
new file mode 100644
index 000000000..dca7e3178
Binary files /dev/null and b/docs/vmgateway-overview.webp differ
diff --git a/docs/vmgateway-rate-limiting.jpg b/docs/vmgateway-rate-limiting.jpg
deleted file mode 100644
index 2849a0094..000000000
Binary files a/docs/vmgateway-rate-limiting.jpg and /dev/null differ
diff --git a/docs/vmgateway-rate-limiting.webp b/docs/vmgateway-rate-limiting.webp
new file mode 100644
index 000000000..5d68e2b6b
Binary files /dev/null and b/docs/vmgateway-rate-limiting.webp differ
diff --git a/docs/vmgateway.md b/docs/vmgateway.md
index 84f74b78a..ace78a710 100644
--- a/docs/vmgateway.md
+++ b/docs/vmgateway.md
@@ -15,7 +15,7 @@ aliases:
It is available for download and evaluation at [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest).
See how to request a free trial license [here](https://victoriametrics.com/products/enterprise/trial/).***
-
+
`vmgateway` is a proxy for the VictoriaMetrics Time Series Database (TSDB). It provides the following features:
@@ -30,7 +30,7 @@ See how to request a free trial license [here](https://victoriametrics.com/produ
## Access Control
-
+
`vmgateway` supports jwt based authentication. With jwt payload can be configured to give access to specific tenants and labels as well as to read/write.
@@ -95,7 +95,7 @@ curl 'http://localhost:8431/api/v1/series/count' -H 'Authorization: Bearer incor
## Rate Limiter
-
+
Limits incoming requests by given, pre-configured limits. It supports read and write limiting by tenant.
diff --git a/ports/OpenBSD/VictoriaMetrics/pkg/PLIST b/ports/OpenBSD/VictoriaMetrics/pkg/PLIST
index 314704d3c..f4319b166 100644
--- a/ports/OpenBSD/VictoriaMetrics/pkg/PLIST
+++ b/ports/OpenBSD/VictoriaMetrics/pkg/PLIST
@@ -19,27 +19,25 @@ share/doc/vmetrics/Home.md
share/doc/vmetrics/LICENSE
share/doc/vmetrics/MetricsQL.md
share/doc/vmetrics/PerTenantStatistic.md
+share/doc/vmetrics/PerTenantStatistic-stats.webp
share/doc/vmetrics/Quick-Start.md
share/doc/vmetrics/README.md
share/doc/vmetrics/Release-Guide.md
share/doc/vmetrics/Single-server-VictoriaMetrics.md
-share/doc/vmetrics/latest.png
-share/doc/vmetrics/logo.png
-share/doc/vmetrics/per-tenant-stats.jpg
+share/doc/vmetrics/logo.webp
share/doc/vmetrics/robots.txt
-share/doc/vmetrics/root.png
-share/doc/vmetrics/rp_daily_1.png
-share/doc/vmetrics/rp_daily_2.png
share/doc/vmetrics/vmagent.md
-share/doc/vmetrics/vmagent.png
+share/doc/vmetrics/vmagent.webp
share/doc/vmetrics/vmalert.md
share/doc/vmetrics/vmauth.md
share/doc/vmetrics/vmbackup.md
share/doc/vmetrics/vmbackupmanager.md
+share/doc/vmetrics/vmbackupmanager_rp_daily_1.webp
+share/doc/vmetrics/vmbackupmanager_rp_daily_2.webp
share/doc/vmetrics/vmctl.md
-share/doc/vmetrics/vmgateway-access-control.jpg
-share/doc/vmetrics/vmgateway-overview.jpeg
-share/doc/vmetrics/vmgateway-rate-limiting.jpg
+share/doc/vmetrics/vmgateway-access-control.webp
+share/doc/vmetrics/vmgateway-overview.webp
+share/doc/vmetrics/vmgateway-rate-limiting.webp
share/doc/vmetrics/vmgateway.md
share/doc/vmetrics/vmrestore.md
@mode 0755
+ 
VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database.
@@ -518,7 +518,7 @@ DataDog agent allows configuring destinations for metrics sending via ENV variab
or via [configuration file](https://docs.datadoghq.com/agent/guide/agent-configuration-files/) in section `dd_url`.
+ 