diff --git a/docs/operator/CHANGELOG.md b/docs/operator/CHANGELOG.md
index c9d2022ca..ad05d2f91 100644
--- a/docs/operator/CHANGELOG.md
+++ b/docs/operator/CHANGELOG.md
@@ -1,16 +1,22 @@
+---
+sort: 10
+weight: 10
+title: CHANGELOG
+---
+
# CHANGELOG
## Next release
### Features
-- [vmoperator](https://docs.victoriametrics.com/operator/): upgrade vmagent/vmauth's default config-reloader image.
+- [vmoperator](./README.md): upgrade vmagent/vmauth's default config-reloader image.
### Fixes
-- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): remove redundant annotation `operator.victoriametrics/last-applied-spec` from created workloads like vmstorage statefulset.
-- [vmoperator](https://docs.victoriametrics.com/operator/): properly resize statefulset's multiple pvc when needed and allowable, before they could be updated with wrong size.
-- [vmoperator](https://docs.victoriametrics.com/operator/): fix wrong api group of endpointsices, before vmagent won't able to access endpointsices resources with default rbac rule.
+- [vmcluster](./api.html#vmcluster): remove redundant annotation `operator.victoriametrics/last-applied-spec` from created workloads like vmstorage statefulset.
+- [vmoperator](./README.md): properly resize statefulset's multiple pvc when needed and allowable, before they could be updated with wrong size.
+- [vmoperator](./README.md): fix wrong api group of endpointsices, before vmagent won't able to access endpointsices resources with default rbac rule.
## [v0.38.0](https://github.com/VictoriaMetrics/operator/releases/tag/v0.38.0) - 11 Sep 2023
@@ -19,13 +25,13 @@
### Fixes
-- [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): [Enterprise] fixes ip_filters indent for url_prefix. Previously it wasn't possible to use ip_filters with multiple target refs
-- [vmoperator](https://docs.victoriametrics.com/operator/): turn off `EnableStrictSecurity` by default. Before, upgrade operator to v0.36.0+ could fail components with volume attached, see [this issue](https://github.com/VictoriaMetrics/operator/issues/749) for details.
-- [vmoperator](https://docs.victoriametrics.com/operator/): bump default version of VictoriaMetrics components to [1.93.4](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.4).
+- [vmuser](./api.md#vmuser): [Enterprise] fixes ip_filters indent for url_prefix. Previously it wasn't possible to use ip_filters with multiple target refs
+- [vmoperator](./README.md): turn off `EnableStrictSecurity` by default. Before, upgrade operator to v0.36.0+ could fail components with volume attached, see [this issue](https://github.com/VictoriaMetrics/operator/issues/749) for details.
+- [vmoperator](./README.md): bump default version of VictoriaMetrics components to [1.93.4](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.4).
### Features
-- [vmoperator](https://docs.victoriametrics.com/operator/) add ability to print default values for all [operator variables](https://docs.victoriametrics.com/operator/vars.html). See [this issue](https://github.com/VictoriaMetrics/operator/issues/675) for details.
+- [vmoperator](./README.md) add ability to print default values for all [operator variables](./vars.md). See [this issue](https://github.com/VictoriaMetrics/operator/issues/675) for details.
## [v0.37.1](https://github.com/VictoriaMetrics/operator/releases/tag/v0.37.1) - 02 Sep 2023
@@ -41,18 +47,18 @@
### Fixes
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fix unmarshalling for streaming aggregation `match` field.
+- [vmagent](./api.md#vmagent): fix unmarshalling for streaming aggregation `match` field.
### Features
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): support [multiple if conditions](https://docs.victoriametrics.com/vmagent.html#relabeling:~:text=the%20if%20option%20may%20contain%20more%20than%20one%20filter) for relabeling. See [this issue](https://github.com/VictoriaMetrics/operator/issues/730) for details.
+- [vmagent](./api.md#vmagent): support [multiple if conditions](https://docs.victoriametrics.com/vmagent.html#relabeling:~:text=the%20if%20option%20may%20contain%20more%20than%20one%20filter) for relabeling. See [this issue](https://github.com/VictoriaMetrics/operator/issues/730) for details.
## [v0.36.1](https://github.com/VictoriaMetrics/operator/releases/tag/v0.36.0) - 25 Aug 2023
### Fixes
-- [vmselect](https://docs.victoriametrics.com/operator/api.html#vmcluster): fix cache directory when `cacheDataPath` not specified, before it will use `/tmp` which is protect by default strict securityContext.
+- [vmselect](./api.md#vmcluster): fix cache directory when `cacheDataPath` not specified, before it will use `/tmp` which is protect by default strict securityContext.
### Features
@@ -61,7 +67,7 @@
### Breaking changes
-- **[vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): Field `OAuth2` was renamed to `oauth2` due to compatibility issue. If you defined `OAuth2` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `oauth2` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/689) for details.**
+- **[vmalert](./api.md#vmalert): Field `OAuth2` was renamed to `oauth2` due to compatibility issue. If you defined `OAuth2` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `oauth2` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/689) for details.**
- **Affected fields:**
- **`VMAlert.spec.datasource.OAuth2` -> `VMAlert.spec.datasource.oauth2`,**
- **`VMAlert.spec.notifier.OAuth2` -> `VMAlert.spec.notifier.oauth2`,**
@@ -69,7 +75,7 @@
- **`VMAlert.spec.remoteRead.OAuth2` -> `VMAlert.spec.remoteRead.oauth2`,**
- **`VMAlert.spec.remoteWrite.OAuth2` -> `VMAlert.spec.remoteWrite.oauth2`,**
-- **[vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): Field `bearerTokenFilePath` was renamed to `bearerTokenFile` due to compatibility issue. If you defined `bearerTokenFilePath` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `bearerTokenFile` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/688/) for details.**
+- **[vmalert](./api.md#vmalert): Field `bearerTokenFilePath` was renamed to `bearerTokenFile` due to compatibility issue. If you defined `bearerTokenFilePath` with below fields in vmalert objects using operator before v0.36.0, these fields must be reapplied with new tag `bearerTokenFile` after upgrading. See [this issue](https://github.com/VictoriaMetrics/operator/issues/522) and [this PR](https://github.com/VictoriaMetrics/operator/pull/688/) for details.**
- **Affected fields:**
- **`VMAlert.spec.datasource.bearerTokenFilePath` --> `VMAlert.spec.datasource.bearerTokenFile`,**
- **`VMAlert.spec.notifier.bearerTokenFilePath` --> `VMAlert.spec.notifier.bearerTokenFile`,**
@@ -82,22 +88,22 @@
- operator set resource requests for config-reloader container by default. See [this PR](https://github.com/VictoriaMetrics/operator/pull/695/) for details.
- fix `attachMetadata` value miscovert for scrape objects. See [this issue](https://github.com/VictoriaMetrics/operator/issues/697) and [this PR](https://github.com/VictoriaMetrics/operator/pull/698) for details.
- fix volumeClaimTemplates change check for objects that generate statefulset, like vmstorage, vmselect. Before, the statefulset won't be recreated if additional `claimTemplates` object changed. See [this issue](https://github.com/VictoriaMetrics/operator/issues/507) and [this PR](https://github.com/VictoriaMetrics/operator/pull/719) for details.
-- [vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): fix `tlsCAFile` argument value generation when using secret or configMap. See [this issue](https://github.com/VictoriaMetrics/operator/issues/699) and [this PR](https://github.com/VictoriaMetrics/operator/issues/699) for details.
-- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): fix default request memory and apply default resources if not set. See [this issue](https://github.com/VictoriaMetrics/operator/issues/706) and [this PR](https://github.com/VictoriaMetrics/operator/pull/710) for details.
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fix missing additional VolumeClaimTemplates when using `ClaimTemplates` under StatefulMode.
+- [vmalert](./api.md#vmalert): fix `tlsCAFile` argument value generation when using secret or configMap. See [this issue](https://github.com/VictoriaMetrics/operator/issues/699) and [this PR](https://github.com/VictoriaMetrics/operator/issues/699) for details.
+- [vmalertmanager](./api.md#vmalertmanager): fix default request memory and apply default resources if not set. See [this issue](https://github.com/VictoriaMetrics/operator/issues/706) and [this PR](https://github.com/VictoriaMetrics/operator/pull/710) for details.
+- [vmagent](./api.md#vmagent): fix missing additional VolumeClaimTemplates when using `ClaimTemplates` under StatefulMode.
### Features
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmagent_stateful_with_sharding.yaml) for vmagent statefulmode.
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent)/[vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): adapt new features in streaming aggregation:
+- [vmagent](./api.md#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmagent_stateful_with_sharding.yaml) for vmagent statefulmode.
+- [vmagent](./api.md#vmagent)/[vmsingle](./api.md#vmsingle): adapt new features in streaming aggregation:
- support `streamAggr.dropInput`, see [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4243) for details;
- support list for `match` parameter, see [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4635) for details;
- support `staleness_interval`, see [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4667) for details.
-- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmcluster_with_additional_claim.yaml) for cluster with custom storage claims.
-- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): support `update_entries_limit` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/691) for details.
-- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): support `keep_firing_for` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/711) for details.
-- [vmoperator parameters](https://docs.victoriametrics.com/operator/vars.html): Add option `VM_ENABLESTRICTSECURITY` and enable strict security context by default. See [this issue](https://github.com/VictoriaMetrics/operator/issues/637), [this](https://github.com/VictoriaMetrics/operator/pull/692/) and [this](https://github.com/VictoriaMetrics/operator/pull/712) PR for details.
-- [vmoperator parameters](https://docs.victoriametrics.com/operator/vars.html): change option `VM_PSPAUTOCREATEENABLED` default value from `true` to `false` cause PodSecurityPolicy already got deprecated since [kubernetes v1.25](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#psp-v125). See [this pr](https://github.com/VictoriaMetrics/operator/pull/726) for details.
+- [vmcluster](./api.md#vmagent): add [example config](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmcluster_with_additional_claim.yaml) for cluster with custom storage claims.
+- [vmrule](./api.md#vmrule): support `update_entries_limit` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/691) for details.
+- [vmrule](./api.md#vmrule): support `keep_firing_for` field in rules, refer to [alerting rules](https://docs.victoriametrics.com/vmalert.html#alerting-rules). See [this PR](https://github.com/VictoriaMetrics/operator/pull/711) for details.
+- [vmoperator parameters](./vars.md): Add option `VM_ENABLESTRICTSECURITY` and enable strict security context by default. See [this issue](https://github.com/VictoriaMetrics/operator/issues/637), [this](https://github.com/VictoriaMetrics/operator/pull/692/) and [this](https://github.com/VictoriaMetrics/operator/pull/712) PR for details.
+- [vmoperator parameters](./vars.md): change option `VM_PSPAUTOCREATEENABLED` default value from `true` to `false` cause PodSecurityPolicy already got deprecated since [kubernetes v1.25](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#psp-v125). See [this pr](https://github.com/VictoriaMetrics/operator/pull/726) for details.
[Changes][v0.36.0]
@@ -106,7 +112,7 @@
### Fixes
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fixes regression with remoteWrite authorization (basicAuth/token). When `UseCustomConfigReloader` option was set, operator incorrectly rendered mounts for `vmagent` container. https://github.com/VictoriaMetrics/operator/commit/f2b8cf701a33f91cef19848c857fd6efb7db59dd
+- [vmagent](./api.md#vmagent): fixes regression with remoteWrite authorization (basicAuth/token). When `UseCustomConfigReloader` option was set, operator incorrectly rendered mounts for `vmagent` container. https://github.com/VictoriaMetrics/operator/commit/f2b8cf701a33f91cef19848c857fd6efb7db59dd
[Changes][v0.35.1]
@@ -116,19 +122,19 @@
### Fixes
-* [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): fix vmselect url_map in vmuser. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/655). Thanks [@Haleygo](https://github.com/Haleygo)
-* [vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): correctly set default port for vmauth components discovery. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/658). Thanks [@Haleygo](https://github.com/Haleygo)
-* [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): remove rate limit on delete. In https://github.com/VictoriaMetrics/operator/pull/672. Thanks [@Haleygo](https://github.com/Haleygo)
-* [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): fix spec change check. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
+* [vmuser](./api.md#vmuser): fix vmselect url_map in vmuser. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/655). Thanks [@Haleygo](https://github.com/Haleygo)
+* [vmalert](./api.md#vmalert): correctly set default port for vmauth components discovery. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/658). Thanks [@Haleygo](https://github.com/Haleygo)
+* [vmuser](./api.md#vmuser): remove rate limit on delete. In https://github.com/VictoriaMetrics/operator/pull/672. Thanks [@Haleygo](https://github.com/Haleygo)
+* [vmcluster](./api.md#vmcluster): fix spec change check. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
* Correctly publish multi-arch release at https://github.com/VictoriaMetrics/operator/pull/681. Thanks [@Haleygo](https://github.com/Haleygo)
### Features
-* [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): add validation when generate static scrape config. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
-* [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): add validation for slack receiver url. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/661). Thanks [@Haleygo](https://github.com/Haleygo)
-* [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth)/[vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): implement configuration initiation for custom config reloader. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/619). Thanks [@Haleygo](https://github.com/Haleygo)
+* [vmagent](./api.md#vmagent): add validation when generate static scrape config. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/677). Thanks [@Haleygo](https://github.com/Haleygo)
+* [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): add validation for slack receiver url. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/661). Thanks [@Haleygo](https://github.com/Haleygo)
+* [vmauth](./api.md#vmauth)/[vmagent](./api.md#vmagent): implement configuration initiation for custom config reloader. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/619). Thanks [@Haleygo](https://github.com/Haleygo)
* add more generators Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/668
-* [vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): add status field. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/670). Thanks [@Haleygo](https://github.com/Haleygo)
+* [vmsingle](./api.md#vmsingle): add status field. See [this issue for details](https://github.com/VictoriaMetrics/operator/issues/670). Thanks [@Haleygo](https://github.com/Haleygo)
[Changes][v0.35.0]
@@ -138,9 +144,9 @@
### Fixes
-- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): fail fast on misconfigured or missing kubernetes pods. It should prevent rare bug with cascade pod deletion. See this [issue](https://github.com/VictoriaMetrics/operator/issues/643) for details
-- [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth)/[vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): correctly renders initConfig image with global container registry domain. See this [issue](https://github.com/VictoriaMetrics/operator/issues/654) for details.
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): correctly set RBAC permissions for single namespace mode and custom config reloader image. See this [issue](https://github.com/VictoriaMetrics/operator/issues/653) for details.
+- [vmcluster](./api.md#vmcluster): fail fast on misconfigured or missing kubernetes pods. It should prevent rare bug with cascade pod deletion. See this [issue](https://github.com/VictoriaMetrics/operator/issues/643) for details
+- [vmauth](./api.md#vmauth)/[vmagent](./api.md#vmagent): correctly renders initConfig image with global container registry domain. See this [issue](https://github.com/VictoriaMetrics/operator/issues/654) for details.
+- [vmagent](./api.md#vmagent): correctly set RBAC permissions for single namespace mode and custom config reloader image. See this [issue](https://github.com/VictoriaMetrics/operator/issues/653) for details.
[Changes][v0.34.1]
@@ -154,18 +160,18 @@
### Fixes
-- [vmnodescrape](https://docs.victoriametrics.com/operator/api.html#vmnodescrape): fixed selectors for Exists and NotExists operators with empty label Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/646
-- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): Add config for vmrule in validating webhook Thanks in https://github.com/VictoriaMetrics/operator/pull/650
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): skips misconfigured objects with missed secret references: https://github.com/VictoriaMetrics/operator/issues/648
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): correctly renders initContainer for configuration download: https://github.com/VictoriaMetrics/operator/issues/649
+- [vmnodescrape](./api.md#vmnodescrape): fixed selectors for Exists and NotExists operators with empty label Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/646
+- [vmrule](./api.md#vmrule): Add config for vmrule in validating webhook Thanks in https://github.com/VictoriaMetrics/operator/pull/650
+- [vmagent](./api.md#vmagent): skips misconfigured objects with missed secret references: https://github.com/VictoriaMetrics/operator/issues/648
+- [vmagent](./api.md#vmagent): correctly renders initContainer for configuration download: https://github.com/VictoriaMetrics/operator/issues/649
### Features
-- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): Bump alertmanager to v0.25.0 Thanks [@tamcore](https://github.com/tamcore) in https://github.com/VictoriaMetrics/operator/pull/636
-- [vmcluster](https://docs.victoriametrics.com/operator/api.html#vmcluster): added `clusterNativePort` field to VMSelect/VMInsert for multi-level cluster setup ([#634](https://github.com/VictoriaMetrics/operator/issues/634)) Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/639
-- [vmrule](https://docs.victoriametrics.com/operator/api.html#vmrule): add notifierHeader field in vmrule spec Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/622
-- [vmpodscrape](https://docs.victoriametrics.com/operator/api.html#vmpodscrape): adds FilterRunning option as prometheus does in https://github.com/VictoriaMetrics/operator/pull/640
-- [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth): adds latest features in https://github.com/VictoriaMetrics/operator/pull/642
+- [vmalertmanager](./api.md#vmalertmanager): Bump alertmanager to v0.25.0 Thanks [@tamcore](https://github.com/tamcore) in https://github.com/VictoriaMetrics/operator/pull/636
+- [vmcluster](./api.md#vmcluster): added `clusterNativePort` field to VMSelect/VMInsert for multi-level cluster setup ([#634](https://github.com/VictoriaMetrics/operator/issues/634)) Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/639
+- [vmrule](./api.md#vmrule): add notifierHeader field in vmrule spec Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/622
+- [vmpodscrape](./api.md#vmpodscrape): adds FilterRunning option as prometheus does in https://github.com/VictoriaMetrics/operator/pull/640
+- [vmauth](./api.md#vmauth): adds latest features in https://github.com/VictoriaMetrics/operator/pull/642
[Changes][v0.34.0]
@@ -175,22 +181,22 @@
### Fixes
-- [vmalert](https://docs.victoriametrics.com/operator/api.html#vmalert): skip bad rules and improve logging for rules exceed max configmap size https://github.com/VictoriaMetrics/operator/commit/bb754d5c20bb371a197cd6ff5afac1ba86a4d92b
-- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): fixed error with headers in VMAlertmanagerConfig.Receivers.EmailConfigs.Headers unmarshalling. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/610
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): fixed keepInput setting for streaming aggregation. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/618
-- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): fix webhook config maxAlerts not work. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/625
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): Remove single quotes from remote write headers. Thanks [@axelsccp](https://github.com/axelsccp) in https://github.com/VictoriaMetrics/operator/pull/613
-- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): fix parse route error and some comments. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/630
-- [vmuser](https://docs.victoriametrics.com/operator/api.html#vmuser): properly removes finalizers for objects https://github.com/VictoriaMetrics/operator/commit/8f10113920a353f21fbcc8637076905f2e57bb34
+- [vmalert](./api.md#vmalert): skip bad rules and improve logging for rules exceed max configmap size https://github.com/VictoriaMetrics/operator/commit/bb754d5c20bb371a197cd6ff5afac1ba86a4d92b
+- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): fixed error with headers in VMAlertmanagerConfig.Receivers.EmailConfigs.Headers unmarshalling. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/610
+- [vmagent](./api.md#vmagent): fixed keepInput setting for streaming aggregation. Thanks [@Amper](https://github.com/Amper) in https://github.com/VictoriaMetrics/operator/pull/618
+- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): fix webhook config maxAlerts not work. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/625
+- [vmagent](./api.md#vmagent): Remove single quotes from remote write headers. Thanks [@axelsccp](https://github.com/axelsccp) in https://github.com/VictoriaMetrics/operator/pull/613
+- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): fix parse route error and some comments. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/630
+- [vmuser](./api.md#vmuser): properly removes finalizers for objects https://github.com/VictoriaMetrics/operator/commit/8f10113920a353f21fbcc8637076905f2e57bb34
### Features
-- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): add option to disable route continue enforce. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/621
-- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): support set require_tls to false. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/624
-- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): add sanity check. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/627
+- [vmalertmanager](./api.md#vmalertmanager): add option to disable route continue enforce. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/621
+- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): support set require_tls to false. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/624
+- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): add sanity check. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/627
- Makefile: bump Alpine base image to latest v3.17.3. Thanks [@denisgolius](https://github.com/denisgolius) in https://github.com/VictoriaMetrics/operator/pull/628
-- [vmalertmanagerconfig](https://docs.victoriametrics.com/operator/api.html#vmalertmanagerconfig): support sound field in pushover config. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/631
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent)/[vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth): download initial config with initContainer https://github.com/VictoriaMetrics/operator/commit/612e7c8f40659731e7938ef9556eb088c67eb4b7
+- [vmalertmanagerconfig](./api.md#vmalertmanagerconfig): support sound field in pushover config. Thanks [@Haleygo](https://github.com/Haleygo) in https://github.com/VictoriaMetrics/operator/pull/631
+- [vmagent](./api.md#vmagent)/[vmauth](./api.md#vmauth): download initial config with initContainer https://github.com/VictoriaMetrics/operator/commit/612e7c8f40659731e7938ef9556eb088c67eb4b7
[Changes][v0.33.0]
@@ -201,7 +207,7 @@
### Fixes
- config: fixes typo at default vm apps version https://github.com/VictoriaMetrics/operator/issues/608
-- [vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): conditionally adds stream aggregation config https://github.com/VictoriaMetrics/operator/commit/4a0ca54113afcde439ca4c77e22d3ef1c0d36241
+- [vmsingle](./api.md#vmsingle): conditionally adds stream aggregation config https://github.com/VictoriaMetrics/operator/commit/4a0ca54113afcde439ca4c77e22d3ef1c0d36241
[Changes][v0.32.1]
@@ -215,10 +221,10 @@
### Features
-- [vmauth](https://docs.victoriametrics.com/operator/api.html#vmauth): automatically configures `proxy-protocol` client and `reloadAuthKey` for `config-reloader` container. https://github.com/VictoriaMetrics/operator/commit/611819233bf595a4dbd04b07d7be24b7e994379c
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): adds `scrapeTimeout` global configuration for `VMAgent` https://github.com/VictoriaMetrics/operator/commit/d1d5024c6befa0961f8d56c82a0554935a4b1878
-- [vmagent](https://docs.victoriametrics.com/operator/api.html#vmagent): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) for `remoteWrite` targets https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
-- [vmsingle](https://docs.victoriametrics.com/operator/api.html#vmsingle): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) as global configuration for database https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
+- [vmauth](./api.md#vmauth): automatically configures `proxy-protocol` client and `reloadAuthKey` for `config-reloader` container. https://github.com/VictoriaMetrics/operator/commit/611819233bf595a4dbd04b07d7be24b7e994379c
+- [vmagent](./api.md#vmagent): adds `scrapeTimeout` global configuration for `VMAgent` https://github.com/VictoriaMetrics/operator/commit/d1d5024c6befa0961f8d56c82a0554935a4b1878
+- [vmagent](./api.md#vmagent): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) for `remoteWrite` targets https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
+- [vmsingle](./api.md#vmsingle): adds [streaming aggregation](https://docs.victoriametrics.com/stream-aggregation.html) as global configuration for database https://github.com/VictoriaMetrics/operator/commit/b8baa6c2b72bdda64ebfcc9c3d86d846cd9b3c98 Thanks [@Amper](https://github.com/Amper)
[Changes][v0.32.0]
@@ -233,8 +239,8 @@
### Features
-- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): Add support of vmalertmanager.spec.templates and autoreload dirs for templates and configmaps thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/590 https://github.com/VictoriaMetrics/operator/issues/592
-- [vmalertmanager](https://docs.victoriametrics.com/operator/api.html#vmalertmanager): Add support "%SHARD_NUM%" placeholder for vmagent sts/deployment Thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/508
+- [vmalertmanager](./api.md#vmalertmanager): Add support of vmalertmanager.spec.templates and autoreload dirs for templates and configmaps thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/590 https://github.com/VictoriaMetrics/operator/issues/592
+- [vmalertmanager](./api.md#vmalertmanager): Add support "%SHARD_NUM%" placeholder for vmagent sts/deployment Thanks [@Amper](https://github.com/Amper) https://github.com/VictoriaMetrics/operator/issues/508
[Changes][v0.31.0]
diff --git a/docs/operator/FAQ.md b/docs/operator/FAQ.md
index 79ca2662d..04d9a44c1 100644
--- a/docs/operator/FAQ.md
+++ b/docs/operator/FAQ.md
@@ -1,17 +1,14 @@
---
-sort: 15
-weight: 15
+sort: 9
+weight: 9
title: FAQ
-menu:
- docs:
- parent: "operator"
- weight: 15
- identifier: "faq-operator"
-aliases:
-- /operator/FAQ.html
---
-# FAQ
+# FAQ (Frequency Asked Questions)
+
+## How do you monitor the operator itself?
+
+You can read about vmoperator monitoring in [this document](./monitoring.md).
## How to change VMStorage PVC storage class
@@ -29,3 +26,53 @@ With Operator deployment:
1. Run `kubectl delete statefulset --cascade=orphan {vmstorage-sts}`
1. Update VMCluster spec to use new storage class
1. Apply cluster configuration
+
+## How to override image registry
+
+You can use `VM_CONTAINERREGISTRY` parameter for operator:
+
+- See details about tuning [operator settings here](./setup.md#settings).
+- See [available operator settings](./vars.md) here.
+
+## How to set up automatic backups?
+
+You can read about backups:
+
+- for `VMSingle`: [Backup automation](./resources/vmsingle.md#backup-automation)
+- for `VMCluster`: [Backup automation](./resources/vmcluster.md#backup-automation)
+
+## How to migrate from Prometheus-operator to VictoriaMetrics operator?
+
+You can read about migration from prometheus operator on [this page](./migration.md).
+
+## How to turn off conversion for prometheus resources
+
+You can read about it on [this page](./migration.md#objects-convesion).
+
+## My VM objects are not deleted/changed when I delete/change Prometheus objects
+
+You can read about it in following sections of "Migration from prometheus-operator" docs:
+
+- [Deletion synchronization](./migration.md#deletion-synchronization)
+- [Update synchronization](./migration.md#update-synchronization)
+- [Labels synchronization](./migration.md#labels-synchronization)
+
+## What permissions does an operator need to run in a cluster?
+
+You can read about needed permissions for operator in [this document](./security.md#roles).
+
+## How to know the version of VM components in the operator?
+
+See [printDefaults mode](./configuration.md).
+
+In addition, you can use [Release notes](https://github.com/VictoriaMetrics/operator/releases)
+or [CHANGELOG](https://github.com/VictoriaMetrics/operator/blob/master/docs/CHANGELOG.md).
+- that's where we describe default version of VictoriaMetrics components.
+
+## How to run VictoriaMetrics operator with permissions for one namespace only?
+
+See this document for details: [Configuration -> Namespaced mode](./configuration.md#namespaced-mode).
+
+## What versions of Kubernetes is the operator compatible with?
+
+Operator tested at kubernetes versions from 1.16 to 1.27.
diff --git a/docs/operator/README.md b/docs/operator/README.md
index 765872593..9185fdf7d 100644
--- a/docs/operator/README.md
+++ b/docs/operator/README.md
@@ -1,21 +1,91 @@
---
-sort: 27
+sort: 0
+weight: 0
title: VictoriaMetrics Operator
-disableToc: true
---
# VictoriaMetrics Operator
-1. [VictoriaMetrics Operator](VictoriaMetrics-Operator.html)
-1. [Additional Scrape Configuration](additional-scrape.html)
-1. [API Docs](api.html)
-1. [Authorization and exposing components](auth.html)
-1. [vmbackupmanager](backups.html)
-1. [Design](design.html)
-1. [High Availability](high-availability.html)
-1. [VMAlert, VMAgent, VMAlertmanager, VMSingle version](managing-versions.html)
-1. [Victoria Metrics Operator Quick Start](quick-start.html)
-1. [VMAgent relabel](relabeling.html)
-1. [CRD Validation](resources-validation.html)
-1. [Security](security.html)
-1. [Auto Generated vars for package config](vars.html)
+Operator serves to make running VictoriaMetrics applications on top of Kubernetes as easy as possible while preserving Kubernetes-native configuration options.
+
+VictoriaMetrics Operator (`vmoperator`) is the classic kubernetes-operator for VictoriaMetrics with many [great features](#features).
+It allows you to manage Victoria Metrics components in Kubernetes or OpenShift clusters
+in a declarative style according to [GitOps](https://www.redhat.com/en/topics/devops/what-is-gitops)
+and [IaC](https://en.wikipedia.org/wiki/Infrastructure_as_code) concepts.
+
+VictoriaMetrics also provides [helm charts](https://github.com/VictoriaMetrics/helm-charts) without operator.
+Operator makes the same, simplifies it and provides [advanced features](#features).
+
+Learn more about [key concepts](#key-concepts) of `vmoperator` and follow the **[quick start guide](./quick-start.md)** for a better experience.
+
+## Features of vmoperator
+
+- Deployment and management in a kubernetes clusters of any number of VictoriaMetrics applications (like vmsingle/vmcluster instances and another components like vmauth, vmagent, vmalert, etc...)
+- Seamless [migration from prometheus-operator](./migration.md) with auto-conversion of prometheus [custom resources](#custom-resources)
+- Simple VictoriaMetrics cluster installation, configuring, upgrading and managing with [crd-objects](./resources/README.md).
+- Ability to delegate the configuration (parts of configuration) of applications monitoring to the end-users and managing access to different configurations or configuration sections.
+- Integration with VictoriaMetrics [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager.html) - advanced tools for making backups. Check [Backup automation for VMSingle](./resources/vmsingle.md#backup-automation) or [Backup automation for VMCluster](./resources/vmcluster.md#backup-automation).
+- Everything you need for monitoring out of the box in [k8s-stack helm chart](https://victoriametrics.github.io/helm-charts/charts/victoria-metrics-k8s-stack/) with ready-made usecases and solutions.
+- Ability to template your own deployment scenarios.
+
+## Key Concepts
+
+### Kubernetes-operators
+
+[Kubernetes-operators](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) are software extensions
+for Kubernetes that make use of [custom resources](#custom-resources) to manage applications and their components.
+Operators follow Kubernetes principles, notably the control loop.
+It can be said that operators are custom controllers for Kubernetes that allow you to create business logic for custom resources.
+
+Design and implementation of `vmoperator` inspired by [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator).
+
+Useful links:
+- [Custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
+- [Custom resource definitions](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)
+- [Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/)
+- [Operator best practices](https://sdk.operatorframework.io/docs/best-practices/)
+
+### Custom resources
+
+Kubernetes-Operators use [custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
+for interaction. Custom resources are a mechanism built into Kubernetes that allows you to create your own extensions for Kubernetes,
+working on the same principles as those built into Kubernetes APIs. Custom resources make Kubernetes so modular and extensible.
+
+In addition, thanks to CRD ([Custom Resource Definitions](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)),
+the mechanism of custom resources allows you to declare an API in the format of the OpenAPI specification and verify that the resources correspond to this API.
+
+### Reconciliation cycle
+
+The main task of the operator is to bring the state of the cluster in line with what is declared by the user in the custom resources.
+This process of constant monitoring and adjustment is called the "Reconciliation cycle" - it is the operator's workflow.
+
+The basic workflow of working with the operator can be simplified as the following diagram:
+
+
+
+- Operator declares and owns [resources of Victoria Metrics](./resources/README.md).
+- Kubernetes validates of the resource according to the specification from CRD (see more in [custom resources](#custom-resources)).
+- Operator subscribed to change events (`create`, `update`, `delete`) for related resources.
+- When an event occurs, the operator reacts and updates the state of the objects in the cluster.
+- For some objects in the cluster the reconciliation cycle is performed at a given interval, even without the occurrence of change events (see `VM_FORCERESYNCINTERVAL`).
+
+### Next steps
+
+- [Quick Start Guide](./quick-start.md)
+- [Setup](./setup.md)
+- [Security](./security.md)
+- [Configuration](./configuration.md)
+- [Migration from Prometheus](./migration.md)
+- [Monitoring](./monitoring.md)
+- [Authorization and exposing components](./auth.md)
+- [High Availability](./high-availability.md)
+- [Enterprise](./enterprise.md)
+- [Custom resources](./resources/README.md)
+
+If you have any questions, check out our [FAQ](./FAQ.md)
+and feel free to can ask them:
+- [VictoriaMetrics Slack](https://victoriametrics.slack.com/)
+- [VictoriaMetrics Telegram](https://t.me/VictoriaMetrics_en)
+
+If you have any suggestions or find a bug, please create an issue
+on [GitHub](https://github.com/VictoriaMetrics/operator/issues/new).
diff --git a/docs/operator/api.md b/docs/operator/api.md
index ffb60c700..dd14239c8 100644
--- a/docs/operator/api.md
+++ b/docs/operator/api.md
@@ -1,15 +1,11 @@
---
-sort: 16
+sort: 12
+weight: 12
title: API Docs
-weight: 16
-menu:
- docs:
- parent: "operator"
- weight: 16
-aliases:
-- /operator/api.html
---
+
+
# API Docs
This Document documents the types introduced by the VictoriaMetrics to be consumed by users.
diff --git a/docs/operator/auth.md b/docs/operator/auth.md
index 2dd2f0df8..0773f42b9 100644
--- a/docs/operator/auth.md
+++ b/docs/operator/auth.md
@@ -1,27 +1,23 @@
---
-sort: 4
-weight: 4
+sort: 7
+weight: 7
title: Authorization and exposing components
-menu:
- docs:
- parent: "operator"
- weight: 4
-aliases:
-- /operator/auth.html
---
# Authorization and exposing components
## Exposing components
+CRD objects doesn't have `ingress` configuration.
+Instead, you can use [VMAuth](./resources/vmauth.md) as proxy between ingress-controller and VictoriaMetrics components.
- CRD objects doesn't have `ingress` configuration. Instead, you can use `VMAuth` as proxy between ingress-controller and VM app components.
- It adds missing authorization and access control features and enforces it.
+It adds missing authorization and access control features and enforces it.
- Access can be given with `VMUser` definition. It supports basic auth and bearer token authentication.
+Access can be given with [VMUser](./resources/vmuser.md) definition.
+
+It supports basic auth and bearer token authentication:
```yaml
-cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAuth
metadata:
@@ -30,45 +26,43 @@ spec:
userNamespaceSelector: {}
userSelector: {}
ingress: {}
-EOF
+ unauthorizedAccessConfig: []
```
- Advanced configuration with cert-manager annotations:
+Advanced configuration with cert-manager annotations:
+
```yaml
-cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAuth
metadata:
- name: router-main
+ name: router-main
spec:
- podMetadata:
- labels:
- component: vmauth
- userSelector: {}
- userNamespaceSelector: {}
- replicaCount: 2
- resources:
- requests:
- cpu: "250m"
- memory: "350Mi"
- limits:
- cpu: "500m"
- memory: "850Mi"
- ingress:
- tlsSecretName: vmauth-tls
- annotations:
- cert-manager.io/cluster-issuer: base
- class_name: nginx
- tlsHosts:
- - vm-access.example.com
-EOF
+ podMetadata:
+ labels:
+ component: vmauth
+ userSelector: {}
+ userNamespaceSelector: {}
+ replicaCount: 2
+ resources:
+ requests:
+ cpu: "250m"
+ memory: "350Mi"
+ limits:
+ cpu: "500m"
+ memory: "850Mi"
+ ingress:
+ tlsSecretName: vmauth-tls
+ annotations:
+ cert-manager.io/cluster-issuer: base
+ class_name: nginx
+ tlsHosts:
+ - vm-access.example.com
```
-
-simple static routing with read-only access to vmagent for username - `user-1` with password `Asafs124142`
+Simple static routing with read-only access to vmagent for username - `user-1` with password `Asafs124142`:
+
```yaml
# curl vmauth:8427/metrics -u 'user-1:Asafs124142'
-cat << EOF | kubectl apply -f
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
@@ -79,14 +73,12 @@ spec:
- static:
url: http://vmagent-base.default.svc:8429
paths: ["/targets/api/v1","/targets","/metrics"]
-EOF
```
- With bearer token access:
+With bearer token access:
```yaml
# curl vmauth:8427/metrics -H 'Authorization: Bearer Asafs124142'
-cat << EOF | kubectl apply -f
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
@@ -97,13 +89,12 @@ spec:
- static:
url: http://vmagent-base.default.svc:8429
paths: ["/targets/api/v1","/targets","/metrics"]
-EOF
```
- It's also possible to use service discovery for objects:
+It's also possible to use service discovery for objects:
+
```yaml
# curl vmauth:8427/metrics -H 'Authorization: Bearer Asafs124142'
-cat << EOF | kubectl apply -f
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
@@ -116,12 +107,11 @@ spec:
name: base
namespace: default
paths: ["/targets/api/v1","/targets","/metrics"]
-EOF
```
- Cluster components supports auto path generation for single tenant view:
+Cluster components supports auto path generation for single tenant view:
+
```yaml
-cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
@@ -143,17 +133,15 @@ spec:
url: http://vmselect-test-persistent.default.svc:8481/
paths:
- /internal/resetRollupResultCache
-EOF
```
- For each `VMUser` operator generates corresponding secret with username/password or bearer token at the same namespace as `VMUser`.
+For each `VMUser` operator generates corresponding secret with username/password or bearer token at the same namespace as `VMUser`.
## Basic auth for targets
-To authenticate a `VMServiceScrape`s over a metrics endpoint use [`basicAuth`](https://docs.victoriametrics.com/operator/api.html#basicauth)
+To authenticate a `VMServiceScrape`s over a metrics endpoint use [`basicAuth`](./api.md#basicauth):
```yaml
-cat < Unauthorized access](./resources/vmauth.md#unauthorized-access).
+
+More details about features of `VMAuth` and `VMUser` you can read in:
+- [VMAuth docs](./resources/vmauth.md),
+- [VMUser docs](./resources/vmuser.md).
diff --git a/docs/operator/configuration.md b/docs/operator/configuration.md
new file mode 100644
index 000000000..ec3921c9d
--- /dev/null
+++ b/docs/operator/configuration.md
@@ -0,0 +1,260 @@
+---
+sort: 4
+weight: 4
+title: Configuration
+---
+
+# Configuration
+
+Operator configured by env variables, list of it can be found
+on [Variables](./vars.md) page.
+
+It defines default configuration options, like images for components, timeouts, features.
+
+In addition, the operator has a special startup mode for outputting all variables, their types and default values.
+For instance, with this mode you can know versions of VM components, which are used by default:
+
+```console
+./operator --printDefaults
+
+# This application is configured via the environment. The following environment variables can be used:
+#
+# KEY TYPE DEFAULT REQUIRED DESCRIPTION
+# VM_USECUSTOMCONFIGRELOADER True or False false
+# VM_CUSTOMCONFIGRELOADERIMAGE String victoriametrics/operator:config-reloader-v0.32.0
+# VM_VMALERTDEFAULT_IMAGE String victoriametrics/vmalert
+# VM_VMALERTDEFAULT_VERSION String v1.93.3
+# VM_VMALERTDEFAULT_USEDEFAULTRESOURCES True or False true
+# VM_VMALERTDEFAULT_RESOURCE_LIMIT_MEM String 500Mi
+# VM_VMALERTDEFAULT_RESOURCE_LIMIT_CPU String 200m
+# ...
+```
+
+You can choose output format for variables with `--printFormat` flag, possible values: `json`, `yaml`, `list` and `table` (default):
+
+```console
+.operator --printDefaults --printFormat=json
+
+# {
+# 'VM_USECUSTOMCONFIGRELOADER': 'false',
+# 'VM_CUSTOMCONFIGRELOADERIMAGE': 'victoriametrics/operator:config-reloader-v0.32.0',
+# 'VM_VMALERTDEFAULT_IMAGE': 'victoriametrics/vmalert',
+# 'VM_VMALERTDEFAULT_VERSION': 'v1.93.3',
+# ...
+# 'VM_FORCERESYNCINTERVAL': '60s',
+# 'VM_ENABLESTRICTSECURITY': 'true'
+# }
+```
+
+## Conversion of prometheus-operator objects
+
+You can read detailed instructions about configuring prometheus-objects conversion in [this document](./migration.md).
+
+## Helm-charts
+
+In [helm-charts](https://github.com/VictoriaMetrics/helm-charts) some important configuration parameters are implemented as separate flags in `values.yaml`:
+
+### victoria-metrics-k8s-stack
+
+For possible values refer to [parameters](https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-k8s-stack#parameters).
+
+Also, checkout [here possible ENV variables](./vars.md) to configure operator behaviour.
+ENV variables can be set in the `victoria-metrics-operator.env` section.
+
+```yaml
+# values.yaml
+
+victoria-metrics-operator:
+ image:
+ # -- Image repository
+ repository: victoriametrics/operator
+ # -- Image tag
+ tag: v0.35.0
+ # -- Image pull policy
+ pullPolicy: IfNotPresent
+
+ # -- Tells helm to remove CRD after chart remove
+ cleanupCRD: true
+ cleanupImage:
+ repository: gcr.io/google_containers/hyperkube
+ tag: v1.18.0
+ pullPolicy: IfNotPresent
+
+ operator:
+ # -- By default, operator converts prometheus-operator objects.
+ disable_prometheus_converter: false
+ # -- Compare-options and sync-options for prometheus objects converted by operator for properly use with ArgoCD
+ prometheus_converter_add_argocd_ignore_annotations: false
+ # -- Enables ownership reference for converted prometheus-operator objects,
+ # it will remove corresponding victoria-metrics objects in case of deletion prometheus one.
+ enable_converter_ownership: false
+ # -- By default, operator creates psp for its objects.
+ psp_auto_creation_enabled: true
+ # -- Enables custom config-reloader, bundled with operator.
+ # It should reduce vmagent and vmauth config sync-time and make it predictable.
+ useCustomConfigReloader: false
+
+ # -- extra settings for the operator deployment. full list Ref: [https://github.com/VictoriaMetrics/operator/blob/master/vars.md](https://github.com/VictoriaMetrics/operator/blob/master/vars.md)
+ env:
+ # -- default version for vmsingle
+ - name: VM_VMSINGLEDEFAULT_VERSION
+ value: v1.43.0
+ # -- container registry name prefix, e.g. docker.io
+ - name: VM_CONTAINERREGISTRY
+ value: ""
+ # -- image for custom reloader (see the useCustomConfigReloader parameter)
+ - name: VM_CUSTOMCONFIGRELOADERIMAGE
+ value: victoriametrics/operator:config-reloader-v0.32.0
+
+ # By default, the operator will watch all the namespaces
+ # If you want to override this behavior, specify the namespace it needs to watch separated by a comma.
+ # Ex: my_namespace1,my_namespace2
+ watchNamespace: ""
+
+ # Count of operator instances (can be increased for HA mode)
+ replicaCount: 1
+
+ # -- VM operator log level
+ # -- possible values: info and error.
+ logLevel: "info"
+
+ # -- Resource object
+ resources:
+ {}
+ # limits:
+ # cpu: 120m
+ # memory: 320Mi
+ # requests:
+ # cpu: 80m
+ # memory: 120Mi
+```
+
+### victoria-metrics-operator
+
+For possible values refer to [parameters](https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-operator#parameters).
+
+Also, checkout [here possible ENV variables](./vars.md) to configure operator behaviour.
+ENV variables can be set in the `env` section.
+
+```yaml
+# values.yaml
+
+image:
+ # -- Image repository
+ repository: victoriametrics/operator
+ # -- Image tag
+ tag: v0.35.0
+ # -- Image pull policy
+ pullPolicy: IfNotPresent
+
+operator:
+ # -- By default, operator converts prometheus-operator objects.
+ disable_prometheus_converter: false
+ # -- Compare-options and sync-options for prometheus objects converted by operator for properly use with ArgoCD
+ prometheus_converter_add_argocd_ignore_annotations: false
+ # -- Enables ownership reference for converted prometheus-operator objects,
+ # it will remove corresponding victoria-metrics objects in case of deletion prometheus one.
+ enable_converter_ownership: false
+ # -- By default, operator creates psp for its objects.
+ psp_auto_creation_enabled: true
+ # -- Enables custom config-reloader, bundled with operator.
+ # It should reduce vmagent and vmauth config sync-time and make it predictable.
+ useCustomConfigReloader: false
+
+# -- extra settings for the operator deployment. full list Ref: [https://github.com/VictoriaMetrics/operator/blob/master/vars.md](https://github.com/VictoriaMetrics/operator/blob/master/vars.md)
+env:
+ # -- default version for vmsingle
+ - name: VM_VMSINGLEDEFAULT_VERSION
+ value: v1.43.0
+ # -- container registry name prefix, e.g. docker.io
+ - name: VM_CONTAINERREGISTRY
+ value: ""
+ # -- image for custom reloader (see the useCustomConfigReloader parameter)
+ - name: VM_CUSTOMCONFIGRELOADERIMAGE
+ value: victoriametrics/operator:config-reloader-v0.32.0
+
+# By default, the operator will watch all the namespaces
+# If you want to override this behavior, specify the namespace it needs to watch separated by a comma.
+# Ex: my_namespace1,my_namespace2
+watchNamespace: ""
+
+# Count of operator instances (can be increased for HA mode)
+replicaCount: 1
+
+# -- VM operator log level
+# -- possible values: info and error.
+logLevel: "info"
+
+# -- Resource object
+resources:
+ {}
+ # limits:
+ # cpu: 120m
+ # memory: 320Mi
+ # requests:
+ # cpu: 80m
+ # memory: 120Mi
+```
+
+## Namespaced mode
+
+By default, the operator will watch all namespaces, but it can be configured to watch only specific namespace.
+
+If you want to override this behavior, specify the namespace:
+
+- in the `WATCH_NAMESPACE` environment variable.
+- in the `watchNamespace` field in the `values.yaml` file of helm-charts.
+
+The operator supports only single namespace for watching.
+
+You can find example of RBAC manifests for single-namespace mode in
+[this file](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/operator_rbac_for_single_namespace.yaml).
+
+## Monitoring of cluster components
+
+By default, operator creates [VMServiceScrape](./resources/vmservicescrape.md)
+object for each component that it manages.
+
+You can disable this behaviour with `VM_DASABLESELFSERVICASCRAPECREATION` environment variable:
+
+```shell
+VM_DASABLESELFSERVICASCRAPECREATION=false
+```
+
+Also, you can override default configuration for self-scraping with `ServiceScrapeSpec` field in each deployable resource
+(`vmcluster/select`, `vmcluster/insert`, `vmcluster/storage`, `vmagent`, `vmalert`, `vmalertmanager`, `vmauth`, `vmsingle`):
+
+## CRD Validation
+
+Operator supports validation admission webhook [docs](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/)
+
+It checks resources configuration and returns errors to caller before resource will be created at kubernetes api.
+This should reduce errors and simplify debugging.
+
+Validation hooks at operator side must be enabled with flags:
+
+```console
+./operator
+ --webhook.enable
+ # optional configuration for certDir and tls names.
+ --webhook.certDir=/tmp/k8s-webhook-server/serving-certs/
+ --webhook.keyName=tls.key
+ --webhook.certName=tls.crt
+```
+
+You have to mount correct certificates at give directory.
+It can be simplified with cert-manager and kustomize command:
+
+```console
+kustomize build config/deployments/webhook/
+```
+
+### Requirements
+
+- Valid certificate with key must be provided to operator
+- Valid CABundle must be added to the `ValidatingWebhookConfiguration`
+
+### Useful links
+
+- [k8s admission webhooks](https://banzaicloud.com/blog/k8s-admission-webhooks/)
+- [olm webhooks](https://docs.openshift.com/container-platform/4.5/operators/user/olm-webhooks.html)
diff --git a/docs/operator/enterprise.md b/docs/operator/enterprise.md
new file mode 100644
index 000000000..49e889f07
--- /dev/null
+++ b/docs/operator/enterprise.md
@@ -0,0 +1,36 @@
+---
+sort: 13
+weight: 13
+title: Enterprise features
+---
+
+# Using operator with enterprise features
+
+Operator doesn't have enterprise version for itself, but it supports
+[enterprise features for VictoriaMetrics components](https://docs.victoriametrics.com/enterprise.html):
+
+- [VMAgent Enterprise features](./resources/vmagent.md#enterprise-features):
+ - [Reading metrics from kafka](./resources/vmagent.md#reading-metrics-from-kafka)
+ - [Writing metrics to kafka](./resources/vmagent.md#writing-metrics-to-kafka)
+- [VMAlert Enterprise features](./resources/vmalert.md#enterprise-features):
+ - [Reading rules from object storage](./resources/vmalert.md#reading-rules-from-object-storage)
+ - [Multitenancy](./resources/vmalert.md#multitenancy)
+- [VMAuth Enterprise features](./resources/vmauth.md#enterprise-features)
+ - [IP Filters](./resources/vmauth.md#ip-filters)
+- [VMCluster Enterprise features](./resources/vmcluster.md#enterprise-features)
+ - [Downsampling](./resources/vmcluster.md#downsampling)
+ - [Multiple retentions / Retention filters](./resources/vmcluster.md#retention-filters)
+ - [Advanced per-tenant statistic](./resources/vmcluster.md#advanced-per-tenant-statistic)
+ - [mTLS protection](./resources/vmcluster.md#mtls-protection)
+ - [Backup atomation](./resources/vmcluster.md#backup-atomation)
+- [VMRule Enterprise features](./resources/vmrule.md#enterprise-features)
+ - [Multitenancy](./resources/vmrule.md#multitenancy)
+- [VMSingle Enterprise features](./resources/vmsingle.md#enterprise-features)
+ - [Downsampling](./resources/vmsingle.md#downsampling)
+ - [Retention filters](./resources/vmsingle.md#retention-filters)
+ - [Backup atomation](./resources/vmsingle.md#backup-atomation)
+- [VMUser Enterprise features](./resources/vmuser.md#enterprise-features)
+ - [IP Filters](./resources/vmuser.md#ip-filters)
+
+More information about enterprise features you can read
+on [VictoriaMetrics Enterprise page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
diff --git a/docs/operator/high-availability.md b/docs/operator/high-availability.md
index 38ec21574..6f648ef34 100644
--- a/docs/operator/high-availability.md
+++ b/docs/operator/high-availability.md
@@ -1,381 +1,43 @@
---
-sort: 7
-weight: 7
+sort: 8
+weight: 8
title: High Availability
-menu:
- docs:
- parent: "operator"
- weight: 7
-aliases:
-- /operator/high-availability.html
---
# High Availability
-High availability is not only important for customer-facing software but if the monitoring infrastructure is not highly available, then there is a risk that operations people are not notified of alerts. Therefore, high availability must be just as thought through for the monitoring stack, as for anything else.
+High availability is not only important for customer-facing software but if the monitoring infrastructure is not highly available, then there is a risk that operations people are not notified of alerts.
+Therefore, high availability must be just as thought through for the monitoring stack, as for anything else.
-## VMAgent
+## Components
-To run VMAgent in a highly available manner you have to configure deduplication at Victoria Metrics first [doc](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/Single-server-VictoriaMetrics.md#deduplication)
+VictoriaMetrics operator support high availability for each component of the monitoring stack:
-Then increase replicas for VMAgent.
+- [VMAgent](./resources/vmagent.md#high-availability)
+- [VMAlert](./resources/vmalert.md#high-availability)
+- [VMAlertmanager](./resources/vmalertmanager.md#high-availability)
+- [VMAuth](./resources/vmauth.md#high-availability)
+- [VMCluster](./resources/vmcluster.md#high-availability)
-create `VMSingle` with dedup flag:
+More details you can find in the section **[High Availability for resources](./resources/README.md#high-availability)**.
-```yaml
-cat < 8480/TCP 69s
-vmselect-example-vmcluster-persistent ClusterIP None 8481/TCP 79s
-vmstorage-example-vmcluster-persistent ClusterIP None 8482/TCP,8400/TCP,8401/TCP 85s
-```
-
-Now you can connect vmagent to vminsert and vmalert to vmselect
-
->NOTE do not forget to create rbac for vmagent
-
-```yaml
-cat << EOF | kubectl apply -f -
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMAgent
-metadata:
- name: example-vmagent
-spec:
- serviceScrapeNamespaceSelector: {}
- serviceScrapeSelector: {}
- podScrapeNamespaceSelector: {}
- podScrapeSelector: {}
- # Add fields here
- replicaCount: 1
- remoteWrite:
- - url: "http://vminsert-example-vmcluster-persistent.default.svc.cluster.local:8480/insert/0/prometheus/api/v1/write"
-EOF
-```
-
-Config for vmalert
-
-```yaml
-cat << EOF | kubectl apply -f -
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMAlert
-metadata:
- name: example-vmalert
-spec:
- # Add fields here
- replicas: 1
- datasource:
- url: "http://vmselect-example-vmcluster-persistent.default.svc.cluster.local:8481/select/0/prometheus"
- notifier:
- url: "http://alertmanager-operated.default.svc:9093"
- evaluationInterval: "10s"
- ruleSelector: {}
-EOF
-```
-
-
-## Alertmanager
-
-The final step of the high availability scheme is Alertmanager, when an alert triggers, actually fire alerts against *all* instances of an Alertmanager cluster.
-
-The Alertmanager, starting with the `v0.5.0` release, ships with a high availability mode. It implements a gossip protocol to synchronize instances of an Alertmanager cluster regarding notifications that have been sent out, to prevent duplicate notifications. It is an AP (available and partition tolerant) system. Being an AP system means that notifications are guaranteed to be sent at least once.
-
-The Victoria Metrics Operator ensures that Alertmanager clusters are properly configured to run highly available on Kubernetes.
+In addition, don't forget about [monitoring for the operator](./monitoring.md).
diff --git a/docs/operator/migration.md b/docs/operator/migration.md
new file mode 100644
index 000000000..2609c3733
--- /dev/null
+++ b/docs/operator/migration.md
@@ -0,0 +1,203 @@
+---
+sort: 5
+weight: 5
+title: Migration from Prometheus
+---
+
+# Migration from prometheus-operator
+
+Design and implementation inspired by [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator).
+It's great a tool for managing monitoring configuration of your applications. VictoriaMetrics operator has api capability with it.
+
+So you can use familiar CRD objects: `ServiceMonitor`, `PodMonitor`, `PrometheusRule`, `Probe` and `AlertmanagerConfig`.
+
+Or you can use VictoriaMetrics CRDs:
+
+- `VMServiceScrape` (instead of `ServiceMonitor`) - defines scraping metrics configuration from pods backed by services. [See details](./resources/vmservicescrape.md).
+- `VMPodScrape` (instead of `PodMonitor`) - defines scraping metrics configuration from pods. [See details](./resources/vmpodscrape.md).
+- `VMRule` (instead of `PrometheusRule`) - defines alerting or recording rules. [See details](./resources/vmrule.md).
+- `VMProbe` (instead of `Probe`) - defines a probing configuration for targets with blackbox exporter. [See details](./resources/vmprobe.md).
+- `VMAlertmanagerConfig` (instead of `AlertmanagerConfig`) - defines a configuration for AlertManager. [See details](./resources/vmalertmanagerconfig.md).
+
+Note that Prometheus CRDs are not supplied with the VictoriaMetrics operator,
+so you need to [install them separately](https://github.com/prometheus-operator/prometheus-operator/releases).
+VictoriaMetrics operator supports conversion from Prometheus CRD of
+version `monitoring.coreos.com/v1` for kinds `ServiceMonitor`, `PodMonitor`, `PrometheusRule`, `Probe`
+and version `monitoring.coreos.com/v1alpha1` for kind `AlertmanagerConfig`.
+
+The default behavior of the operator is as follows:
+
+- It **converts** all existing Prometheus `ServiceMonitor`, `PodMonitor`, `PrometheusRule` and `Probe` objects into corresponding VictoriaMetrics Operator objects.
+- It **syncs** updates (including labels) from Prometheus `ServiceMonitor`, `PodMonitor`, `PrometheusRule` and `Probe` objects to corresponding VictoriaMetrics Operator objects.
+- It **DOES NOT delete** converted objects after original ones are deleted.
+
+With this configuration removing prometheus-operator API objects wouldn't delete any converted objects. So you can safely migrate or run two operators at the same time.
+
+You can change default behavior with operator configuration - [see details below](#objects-conversion).
+
+## Objects conversion
+
+By default, the vmoperator converts all existing [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator)
+API objects into corresponding VictoriaMetrics Operator objects ([see above](#migration-from-prometheus-operator)),
+i.e. creates resources of VictoriaMetrics similar to Prometheus resources in the same namespace.
+
+You can control this behaviour by setting env variable for operator:
+
+```console
+# disable convertion for each object
+VM_ENABLEDPROMETHEUSCONVERTER_PODMONITOR=false
+VM_ENABLEDPROMETHEUSCONVERTER_SERVICESCRAPE=false
+VM_ENABLEDPROMETHEUSCONVERTER_PROMETHEUSRULE=false
+VM_ENABLEDPROMETHEUSCONVERTER_PROBE=false
+```
+
+For [victoria-metrics-operator helm-chart](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-operator/README.md) you can use following way:
+
+```yaml
+# values.yaml
+
+# ...
+operator:
+ # -- By default, operator converts prometheus-operator objects.
+ disable_prometheus_converter: true
+# ...
+```
+
+Otherwise, VictoriaMetrics Operator would try to discover prometheus-operator API and convert it.
+
+
+
+For more information about the operator's workflow, see [this doc](./README.md).
+
+## Deletion synchronization
+
+By default, the operator doesn't make converted objects disappear after original ones are deleted. To change this behaviour
+configure adding `OwnerReferences` to converted objects with following [operator parameter](./setup.md#settings):
+
+```console
+VM_ENABLEDPROMETHEUSCONVERTEROWNERREFERENCES=true
+```
+
+For [victoria-metrics-operator helm-chart](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-operator/README.md) you can use following way:
+
+```yaml
+# values.yaml
+
+# ...
+operator:
+ # -- Enables ownership reference for converted prometheus-operator objects,
+ # it will remove corresponding victoria-metrics objects in case of deletion prometheus one.
+ enable_converter_ownership: true
+# ...
+```
+
+Converted objects will be linked to the original ones and will be deleted by kubernetes after the original ones are deleted.
+
+## Update synchronization
+
+Conversion of api objects can be controlled by annotations, added to `VMObject`s.
+
+Annotation `operator.victoriametrics.com/ignore-prometheus-updates` controls updates from Prometheus api objects.
+
+By default, it set to `disabled`. You define it to `enabled` state and all updates from Prometheus api objects will be ignored.
+
+Example:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMServiceScrape
+metadata:
+ annotations:
+ meta.helm.sh/release-name: prometheus
+ operator.victoriametrics.com/ignore-prometheus-updates: enabled
+ labels:
+ release: prometheus
+ name: prometheus-monitor
+spec:
+ endpoints: []
+```
+
+Annotation `operator.victoriametrics.com/ignore-prometheus-updates` can be set on one of the resources:
+
+- [VMServiceScrape](./resources/vmservicescrape.md)
+- [VMPodScrape](./resources/vmpodscrape.md)
+- [VMRule](./resources/vmrule.md)
+- [VMProbe](./resources/vmprobe.md)
+- [VMAlertmanagerConfig](./resources/vmalertmanagerconfig.md)
+
+And annotation doesn't make sense for [VMStaticScrape](./resources/vmstaticscrape.md)
+and [VMNodeScrape](./resources/vmnodescrape.md) because these objects are not created as a result of conversion.
+
+## Labels and annotations synchronization
+
+Conversion of api objects can be controlled by annotations, added to `VMObject`s.
+
+Annotation `operator.victoriametrics.com/merge-meta-strategy` controls syncing of metadata labels and annotations
+between `VMObject`s and `Prometheus` api objects during updates to `Prometheus` objects.
+
+By default, it has `prefer-prometheus`. And annotations and labels will be used from `Prometheus` objects, manually set values will be dropped.
+
+You can set it to `prefer-victoriametrics`. In this case all labels and annotations applied to `Prometheus` object will be ignored and `VMObject` will use own values.
+
+Two additional strategies annotations -`merge-victoriametrics-priority` and `merge-prometheus-priority` merges labelSets into one combined labelSet, with priority.
+
+Example:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMServiceScrape
+metadata:
+ annotations:
+ meta.helm.sh/release-name: prometheus
+ operator.victoriametrics.com/merge-meta-strategy: prefer-victoriametrics
+ labels:
+ release: prometheus
+ name: prometheus-monitor
+spec:
+ endpoints: []
+```
+
+Annotation `operator.victoriametrics.com/merge-meta-strategy` can be set on one of the resources:
+
+- [VMServiceScrape](./resources/vmservicescrape.md)
+- [VMPodScrape](./resources/vmpodscrape.md)
+- [VMRule](./resources/vmrule.md)
+- [VMProbe](./resources/vmprobe.md)
+- [VMAlertmanagerConfig](./resources/vmalertmanagerconfig.md)
+
+And annotation doesn't make sense for [VMStaticScrape](./resources/vmstaticscrape.md)
+and [VMNodeScrape](./resources/vmnodescrape.md) because these objects are not created as a result of conversion.
+
+You can filter labels for syncing
+with [operator parameter](./setup.md#settings) `VM_FILTERPROMETHEUSCONVERTERLABELPREFIXES`:
+
+```console
+# it excludes all labels that start with "helm.sh" or "argoproj.io" from synchronization
+VM_FILTERPROMETHEUSCONVERTERLABELPREFIXES=helm.sh,argoproj.io
+```
+
+In the same way, annotations with specified prefixes can be excluded from synchronization
+with [operator parameter](./setup.md#settings) `VM_FILTERPROMETHEUSCONVERTERANNOTATIONPREFIXES`:
+
+```console
+# it excludes all annotations that start with "helm.sh" or "argoproj.io" from synchronization
+VM_FILTERPROMETHEUSCONVERTERANNOTATIONPREFIXES=helm.sh,argoproj.io
+```
+
+## Using converter with ArgoCD
+
+If you use ArgoCD, you can allow ignoring objects at ArgoCD converted from Prometheus CRD
+with [operator parameter](./setup.md#settings) `VM_PROMETHEUSCONVERTERADDARGOCDIGNOREANNOTATIONS`.
+
+It helps to properly use converter with ArgoCD and should help prevent out-of-sync issues with argo-cd based deployments:
+
+```console
+# adds compare-options and sync-options for prometheus objects converted by operator
+VM_PROMETHEUSCONVERTERADDARGOCDIGNOREANNOTATIONS=true
+```
+
+## Data migration
+
+You can use [vmctl](https://docs.victoriametrics.com/vmctl.html) for migrating your data from Prometheus to VictoriaMetrics.
+
+See [this doc](https://docs.victoriametrics.com/vmctl.html#migrating-data-from-prometheus) for more details.
diff --git a/docs/operator/monitoring.md b/docs/operator/monitoring.md
new file mode 100644
index 000000000..2fdc295d8
--- /dev/null
+++ b/docs/operator/monitoring.md
@@ -0,0 +1,72 @@
+---
+sort: 6
+weight: 6
+title: Monitoring
+---
+
+# Monitoring of VictoriaMetrics Operator
+
+VictoriaMetrics operator exports internal metrics in Prometheus exposition format at `/metrics` page.
+
+These metrics can be scraped via [vmagent](./resources/vmagent.md) or Prometheus.
+
+## Dashboard
+
+Official Grafana dashboard available for [vmoperator](https://grafana.com/grafana/dashboards/17869-victoriametrics-operator/).
+
+
+
+Graphs on the dashboards contain useful hints - hover the `i` icon in the top left corner of each graph to read it.
+
+
+
+## Configuration
+
+### Helm-chart victoria-metrics-k8s-stack
+
+In [victoria-metrics-k8s-stack](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-k8s-stack/README.md) helm-chart operator self-scrapes metrics by default.
+
+This helm-chart also includes [official grafana dashboard for operator](#dashboard).
+
+### Helm-chart victoria-metrics-operator
+
+With [victoria-metrics-operator](https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-operator/README.md) you can use following parameter in `values.yaml`:
+
+```yaml
+# values.yaml
+#...
+# -- configures monitoring with serviceScrape. VMServiceScrape must be pre-installed
+serviceMonitor:
+ enabled: true
+```
+
+This parameter makes helm-chart to create a scrape-object for installed operator instance.
+
+You will also need to deploy a (vmsingle)[./resources/vmsingle.md] where the metrics will be collected.
+
+### Pure operator installation
+
+With pure operator installation you can use config with separate vmsingle and scrape object for operator like that:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMServiceScrape
+metadata:
+ name: vmoperator
+ namespace: monitoring
+spec:
+ selector:
+ matchLabels:
+ app.kubernetes.io/instance: vm-operator
+ app.kubernetes.io/name: victoria-metrics-operator
+ endpoints:
+ - port: http
+ namespaceSelector:
+ matchNames:
+ - monitoring
+```
+
+See more info about object [VMServiceScrape](./resources/vmservicescrape.md).
+
+You will also need a [vmsingle](https://docs.victoriametrics.com/vmoperatos/resources/vmsingle.html) where the metrics will be collected.
+
diff --git a/docs/operator/quick-start.md b/docs/operator/quick-start.md
index 402c384d1..d1511ae1b 100644
--- a/docs/operator/quick-start.md
+++ b/docs/operator/quick-start.md
@@ -1,221 +1,187 @@
---
-sort: 9
-weight: 9
-title: Quick start
-menu:
- docs:
- parent: "operator"
- weight: 9
- identifier: "quickstartoperator"
-aliases:
-- /operator/quick-start.html
+sort: 1
+weight: 1
+title: QuickStart
---
-# Quick start
+# VictoriaMetrics Operator QuickStart
-Operator serves to make running VictoriaMetrics applications on top of Kubernetes as easy as possible while preserving Kubernetes-native configuration options.
+VictoriaMetrics Operator serves to make running VictoriaMetrics applications on top of Kubernetes as easy as possible
+while preserving Kubernetes-native configuration options.
-## Installing by Manifest
+The shortest way to deploy full-stack monitoring cluster with VictoriaMetrics Operator is
+to use Helm-chart [victoria-metrics-k8s-stack](https://victoriametrics.github.io/helm-charts/charts/victoria-metrics-k8s-stack/).
-Obtain release from releases page:
-[https://github.com/VictoriaMetrics/operator/releases](https://github.com/VictoriaMetrics/operator/releases)
+Also you can follow the other steps in documentation to use VictoriaMetrics Operator:
- We suggest use the latest release.
+- [Setup](./setup.md)
+- [Security](./security.md)
+- [Configuration](./configuration.md)
+- [Migration from Prometheus](./migration.md)
+- [Monitoring](./monitoring.md)
+- [Authorization and exposing components](./auth.md)
+- [High Availability](./high-availability.md)
+- [Enterprise](./enterprise.md)
+- [Custom resources](./resources/README.md)
+- [FAQ (Frequency Asked Questions)](./FAQ.md)
-```console
-# Get latest release version from https://github.com/VictoriaMetrics/operator/releases/latest
-export VM_VERSION=`basename $(curl -fs -o/dev/null -w %{redirect_url} https://github.com/VictoriaMetrics/operator/releases/latest)`
-wget https://github.com/VictoriaMetrics/operator/releases/download/$VM_VERSION/bundle_crd.zip
-unzip bundle_crd.zip
+But if you want to deploy VictoriaMetrics Operator quickly from scratch (without using templating for custom resources),
+you can follow this guide:
+
+- [Setup operator](#setup-operator)
+- [Deploy components](#deploy-components)
+ - [VMCluster](#vmcluster-vmselect-vminsert-vmstorage)
+ - [Scraping](#scraping)
+ - [VMAgent](#vmagent)
+ - [VMServiceScrape](#vmservicescrape)
+ - [Access](#access)
+ - [VMAuth](#vmauth)
+ - [VMUser](#vmuser)
+ - [Alerting](#alerting)
+ - [VMAlertmanager](#vmalertmanager)
+ - [VMAlert](#vmalert)
+ - [VMRule](#vmrule)
+ - [VMUser](#vmuser-update)
+- [Anythings else?](#anythings-else)
+
+Let's start!
+
+## Setup operator
+
+You can find out how to and instructions for installing the VictoriaMetrics operator into your kubernetes cluster
+on the [Setup page](./setup.md).
+
+Here we will elaborate on just one of the ways - for instance, we will install operator via Helm-chart
+[victoria-metrics-operator](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-operator/README.md):
+
+Add repo with helm-chart:
+
+```shell
+helm repo add vm https://victoriametrics.github.io/helm-charts/
+helm repo update
```
-> TIP, operator use monitoring-system namespace, but you can install it to specific namespace with command
-> sed -i "s/namespace: monitoring-system/namespace: YOUR_NAMESPACE/g" release/operator/*
+Render `values.yaml` with default operator configuration:
-First of all, you have to create [custom resource definitions](https://github.com/VictoriaMetrics/operator)
-```console
-kubectl apply -f release/crds
-```
-
-Then you need RBAC for operator, relevant configuration for the release can be found at release/operator/rbac.yaml
-
-Change configuration for operator at `release/operator/manager.yaml`, possible settings: [operator-settings](/operator/vars.html)
-and apply it:
-```console
-kubectl apply -f release/operator/
+```shell
+helm show values vm/victoria-metrics-operator > values.yaml
```
-Check the status of operator
-
-```console
-kubectl get pods -n monitoring-system
-
-#NAME READY STATUS RESTARTS AGE
-#vm-operator-667dfbff55-cbvkf 1/1 Running 0 101s
+Now you can configure operator - open rendered `values.yaml` file in your text editor. For example:
+```shell
+code values.yaml
```
+
-## Installing by Kustomize
+Now you can change configuration in `values.yaml`. For more details about configuration options and methods,
+see [configuration -> victoria-metrics-operator](./configuration.md#victoria-metrics-operator).
-You can install operator using [Kustomize](https://kustomize.io/) by pointing to the remote kustomization file.
+If you migrated from prometheus-operator, you can read about prometheus-operator objects conversion on
+the [migration from prometheus-operator](./migration.md).
+
+Since we're looking at installing from scratch, let's disable prometheus-operator objects conversion,
+and also let's set some resources for operator in `values.yaml`:
```yaml
-# Get latest release version from https://github.com/VictoriaMetrics/operator/releases/latest
-export VM_VERSION=`basename $(curl -fs -o/dev/null -w %{redirect_url} https://github.com/VictoriaMetrics/operator/releases/latest)`
+# ...
-cat << EOF > kustomization.yaml
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
+operator:
+ # -- By default, operator converts prometheus-operator objects.
+ disable_prometheus_converter: true
+
+# -- Resources for operator
resources:
-- github.com/VictoriaMetrics/operator/config/default?ref=${VM_VERSION}
+ limits:
+ cpu: 500m
+ memory: 500Mi
+ requests:
+ cpu: 100m
+ memory: 150Mi
-images:
-- name: victoriametrics/operator
- newTag: ${VM_VERSION}
-EOF
+# ...
```
+You will need a kubernetes namespace to deploy the operator and VM components. Let's create it:
-You can change [operator-settings](/vars.MD), or use your custom namespace see [kustomize-example](https://github.com/YuriKravetc/yurikravetc.github.io/tree/main/Operator/kustomize-example).
-
-
-
-Build template
-
-```console
-kustomize build . -o monitoring.yaml
+```shell
+kubectl create namespace vm
```
-Apply manifests
+After finishing with `values.yaml` and creating namespace, you can test the installation with command:
-```console
-kubectl apply -f monitoring.yaml
+```shell
+helm install vmoperator vm/victoria-metrics-operator -f values.yaml -n vm --debug --dry-run
```
-Check the status of operator
+Where `vm` is the namespace where you want to install operator.
-```console
-kubectl get pods -n monitoring-system
+If everything is ok, you can install operator with command:
-#NAME READY STATUS RESTARTS AGE
-#vm-operator-667dfbff55-cbvkf 1/1 Running 0 101s
+```shell
+helm install vmoperator vm/victoria-metrics-operator -f values.yaml -n vm
+# NAME: vmoperator
+# LAST DEPLOYED: Thu Sep 14 15:13:04 2023
+# NAMESPACE: vm
+# STATUS: deployed
+# REVISION: 1
+# TEST SUITE: None
+# NOTES:
+# victoria-metrics-operator has been installed. Check its status by running:
+# kubectl --namespace vm get pods -l "app.kubernetes.io/instance=vmoperator"
+#
+# Get more information on https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-operator.
+# See "Getting started guide for VM Operator" on https://docs.victoriametrics.com/guides/getting-started-with-vm-operator.html .
```
-## Installing to ARM
+And check that operator is running:
- There is no need in an additional configuration for ARM. Operator and VictoriaMetrics have full support for it.
+```shell
+kubectl get pods -n vm -l "app.kubernetes.io/instance=vmoperator"
-## Create related resources
+# NAME READY STATUS RESTARTS AGE
+# vmoperator-victoria-metrics-operator-7b88bd6df9-q9qwz 1/1 Running 0 98s
+```
-The VictoriaMetrics Operator introduces additional resources in Kubernetes to declare the desired state of a Victoria Metrics applications and Alertmanager cluster as well as the Prometheus resources configuration. The resources it introduces are:
+## Deploy components
-* [VMSingle](#vmsingle)
-* [VMCluster](#vmcluster)
-* [VMAgent](#vmagent)
-* [VMAlert](#vmalert)
-* [VMAlertmanager](#vmalertmanager)
-* [VMServiceScrape](#vmservicescrape)
-* [VMRule](#vmrule)
-* [VMPodScrape](#vmpodscrape)
-* [VMProbe](#vmprobe)
-* [VMStaticScrape](#vmstaticscrape)
-* [VMAuth](#vmauth)
-* [VMUser](#vmuser)
-* [Selectors](#object-selectors)
+Now you can create instances of VictoriaMetrics applications.
+Let's create fullstack monitoring cluster with
+[`vmagent`](./resources/vmagent.md),
+[`vmauth`](./resources/vmauth.md),
+[`vmalert`](./resources/vmalert.md),
+[`vmalertmanager`](./resources/vmalertmanager.md),
+[`vmcluster`](./resources/vmcluster.md)
+(a component for deploying a cluster version of
+[VictoriaMetrics](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#architecture-overview)
+consisting of `vmstorage`, `vmselect` and `vminsert`):
-## VMSingle
+
-[VMSingle](https://github.com/VictoriaMetrics/VictoriaMetrics/) represents database for storing metrics, for all possible config options check api [doc](https://docs.victoriametrics.com/operator/api.html#vmsingle):
-
-```yaml
-cat < 8482/TCP,8400/TCP,8401/TCP 8m3s
+# vmselect-demo ClusterIP None 8481/TCP 8m3s
+# vminsert-demo ClusterIP 192.168.194.183 8480/TCP 8m3s
+```
+
+We'll need them in the next steps.
+
+More information about `vmcluster` resource you can find on
+the [vmcluster page](./resources/vmcluster.md).
+
+### Scraping
+
+#### VMAgent
+
+Now let's deploy [`vmagent`](./resources/vmagent.md) resource.
+
+Create file `vmagent.yaml`
+
+```shell
+code vmagent.yaml
+```
+
+with the following content:
```yaml
-cat << EOF | kubectl apply -f -
----
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMCluster
-metadata:
- name: example-vmcluster
-spec:
- # Add fields here
- retentionPeriod: "1"
- vmselect:
- replicaCount: 2
- extraArgs:
- storageNode: "node-1:8401,node-2:8401"
- vminsert:
- replicaCount: 2
- extraArgs:
- storageNode: "node-1:8401,node-2:8401"
-EOF
-```
-
-## VMAgent
-
-[VMAgent](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmagent) - is a tiny but brave agent, which helps you collect metrics from various sources and stores them in [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics).
-It requires access to Kubernetes API and you can create RBAC for it first, it can be found at `release/examples/VMAgent_rbac.yaml`
-Or you can use default rbac account, that will be created for `VMAgent` by operator automatically.
-
-```console
- kubectl apply -f release/examples/vmagent_rbac.yaml
-```
-
-Modify `VMAgent` config parameters at `release/examples/vmagent.yaml` and apply it, config options [doc](https://docs.victoriametrics.com/operator/api.html#vmagent)
-
-Example:
-
-```yaml
-cat <
+
+
+
+### Alerting
+
+The remaining components will be needed for alerting.
+
+#### VMAlertmanager
+
+Let's start with [`vmalertmanager`](./resources/vmalertmanager.md).
+
+Create file `vmuser.yaml`
+
+```shell
+code vmuser.yaml
+```
+
+with the following content:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlertmanager
+metadata:
+ name: demo
+spec:
+ configRawYaml: |
global:
resolve_timeout: 5m
route:
- group_by: ['job']
group_wait: 30s
group_interval: 5m
repeat_interval: 12h
@@ -359,417 +488,99 @@ stringData:
receivers:
- name: 'webhook'
webhook_configs:
- - url: 'http://alertmanagerwh:30500/'
-EOF
+ - url: 'http://your-webhook-url'
```
-Then add `Alertmanager` object, other config options at [doc](https://docs.victoriametrics.com/operator/api.html#alertmanager)
-you have to set configSecret with name of secret, that we created before - `alertmanager-config`.
-```yaml
-cat << EOF | kubectl apply -f -
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMAlertmanager
-metadata:
- name: example-alertmanager
-spec:
- # Add fields here
- replicaCount: 1
- configSecret: alertmanager-config
- selectAllByDefault: true
-EOF
+where webhook-url is the address of the webhook to receive notifications
+(configuration of AlertManager notifications will remain out of scope).
+You can find more details about `alertmanager` configuration in
+the [Alertmanager documentation](https://prometheus.io/docs/alerting/latest/configuration/).
+
+After that you can deploy `vmalertmanager` resource to the kubernetes cluster:
+
+```shell
+kubectl apply -f vmalertmanager.yaml -n vm
+
+# vmalertmanager.operator.victoriametrics.com/demo created
```
-Alertmanager config with raw yaml configuration, use it with care about secret information:
-```yaml
-cat << EOF | kubectl apply -f -
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMAlertmanager
-metadata:
- name: example-alertmanager-raw-config
-spec:
- # Add fields here
- replicaCount: 1
- configSecret: alertmanager-config
- configRawYaml: |
- global:
- resolve_timeout: 5m
- route:
- group_wait: 30s
- group_interval: 5m
- repeat_interval: 12h
- receiver: 'webhook'
- receivers:
- - name: 'webhook'
- webhook_configs:
- - url: 'http://localhost:30502/'
-EOF
+Check that `vmalertmanager` is running:
+
+```shell
+kubectl get pods -n vm -l "app.kubernetes.io/instance=demo" -l "app.kubernetes.io/name=vmalertmanager"
+
+# NAME READY STATUS RESTARTS AGE
+# vmalertmanager-demo-0 2/2 Running 0 107s
```
-
-## VMAlertmanagerConfig
+#### VMAlert
- `VMAlertmanagerConfig` allows managing `VMAlertmanager` configuration.
+And now you can create [`vmalert`](./resources/vmalert.md) resource.
-```yaml
+Create file `vmalert.yaml`
-cat << EOF | kubectl apply -f
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMAlertmanagerConfig
-metadata:
- name: example
- namespace: default
-spec:
- inhibit_rules:
- - equals: []
- target_matchers: []
- source_matchers: []
- route:
- routes:
- - receiver: webhook
- continue: true
- receiver: email
- group_by: []
- continue: false
- matchers:
- - job = "alertmanager"
- group_wait: 30s
- group_interval: 45s
- repeat_interval: 1h
- mute_time_intervals:
- - name: base
- time_intervals:
- - times:
- - start_time: ""
- end_time: ""
- weekdays: []
- days_of_month: []
- months: []
- years: []
- receivers:
- email_configs: []
- webhook_configs:
- - url: http://some-other-wh
- pagerduty_configs: []
- pushover_configs: []
- slack_configs: []
- opsgenie_configs: []
- victorops_configs: []
- wechat_configs: []
-EOF
+```shell
+code vmalert.yaml
```
-## VMAlert
-
-[VMAlert](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmalert) - executes a list of given [alerting](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) or [recording](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) rules against configured address. It
-has few required config options - `datasource` and `notifier` are required, for other config parameters check [doc](https://docs.victoriametrics.com/operator/api.html#vmalert).
+with the following content:
```yaml
-cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAlert
metadata:
- name: example-vmalert
+ name: demo
spec:
- replicaCount: 1
datasource:
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429"
- notifier:
- url: "http://vmalertmanager-example-alertmanager.default.svc:9093"
- evaluationInterval: "30s"
- selectAllByDefault: true
-
-EOF
-```
-
-## VMServiceScrape
-
- It generates part of `VMAgent` configuration with `Endpoint` kubernetes_sd role for service discovery targets
- by corresponding `Service` and it's `Endpoint`s.
- It has various options for scraping configuration of target (with basic auth,tls access, by specific port name etc.).
-
-Let's make some demo, you have to deploy [VMAgent](#vmagent) and [VMSingle](#vmsingle) from previous step with match any selectors:
-
-```yaml
-cat < 1`
+Check that `vmalert` is running:
+
+```shell
+kubectl get pods -n vm -l "app.kubernetes.io/instance=demo" -l "app.kubernetes.io/name=vmalert"
+
+# NAME READY STATUS RESTARTS AGE
+# vmalert-demo-bf75c67cb-hh4qd 2/2 Running 0 5s
+```
+
+#### VMRule
+
+Now you can create [vmrule](./resources/vmrule.md) resource
+for [vmalert](./resources/vmalert.md).
+
+Create file `vmrule.yaml`
+
+```shell
+code vmrule.yaml
+```
+
+with the following content:
-{% raw %}
```yaml
-cat << 'EOF' | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMRule
metadata:
- name: example-vmrule-reload-config
- labels:
- project: devops
+ name: demo
spec:
groups:
- name: vmalert
@@ -783,753 +594,101 @@ spec:
annotations:
value: "{{ $value }}"
description: 'error reloading vmalert config, reload count for 5 min {{ $value }}'
-EOF
-```
-{% endraw %}
-
- Ensure, that new alert was started:
- ```console
-kubectl logs vmalert-example-vmalert-6f8748c6f9-hcfrr vmalert
-2020-08-03T09:07:49.772Z info VictoriaMetrics/app/vmalert/web.go:45 api config reload was called, sending sighup
-2020-08-03T09:07:49.772Z info VictoriaMetrics/app/vmalert/main.go:115 SIGHUP received. Going to reload rules ["/etc/vmalert/config/vm-example-vmalert-rulefiles-0/*.yaml"] ...
-2020-08-03T09:07:49.772Z info VictoriaMetrics/app/vmalert/manager.go:83 reading rules configuration file from "/etc/vmalert/config/vm-example-vmalert-rulefiles-0/*.yaml"
-2020-08-03T09:07:49.773Z info VictoriaMetrics/app/vmalert/group.go:169 group "vmAlertGroup": received stop signal
-2020-08-03T09:07:49.773Z info VictoriaMetrics/app/vmalert/main.go:124 Rules reloaded successfully from ["/etc/vmalert/config/vm-example-vmalert-rulefiles-0/*.yaml"]
-2020-08-03T09:07:49.773Z info VictoriaMetrics/app/vmalert/group.go:153 group "vmalert" started with interval 30s
-
```
- Let's trigger it by adding some incorrect rule
-
-{% raw %}
-```yaml
-cat << 'EOF' | kubectl apply -f -
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMRule
-metadata:
- name: example-vmrule-incorrect-rule
- labels:
- project: devops
-spec:
- groups:
- - name: incorrect rule
- rules:
- - alert: vmalert bad config
- expr: bad expression
- for: 10s
- labels:
- severity: major
- annotations:
- value: "{{ $badValue | bad function }}"
-EOF
-```
-{% endraw %}
+After that you can deploy `vmrule` resource to the kubernetes cluster:
-`VMAlert` will report incorrect rule config and fire alert:
-```console
-2020-08-03T09:11:40.672Z info VictoriaMetrics/app/vmalert/main.go:115 SIGHUP received. Going to reload rules ["/etc/vmalert/config/vm-example-vmalert-rulefiles-0/*.yaml"] ...
-2020-08-03T09:11:40.672Z info VictoriaMetrics/app/vmalert/manager.go:83 reading rules configuration file from "/etc/vmalert/config/vm-example-vmalert-rulefiles-0/*.yaml"
-2020-08-03T09:11:40.673Z error VictoriaMetrics/app/vmalert/main.go:119 error while reloading rules: cannot parse configuration file: invalid group "incorrect rule" in file "/etc/vmalert/config/vm-example-vmalert-rulefiles-0/default-example-vmrule-incorrect-rule.yaml": invalid rule "incorrect rule"."vmalert bad config": invalid expression: unparsed data left: "expression"
+```shell
+kubectl apply -f vmrule.yaml -n vm
+
+# vmrule.operator.victoriametrics.com/demo created
```
-Clean up incorrect rule:
-```console
-kubectl delete vmrule example-vmrule-incorrect-rule
+#### VMUser update
+
+Let's update our user with access to `vmalert` and `vmalertmanager`:
+
+```shell
+code vmuser.yaml
```
-## VMNodeScrape
-
- `VMNodeScrape` is useful for node exporters monitoring, lets create scraper for cadvisor metrics:
-
```yaml
-cat << EOF | kubectl apply -f -
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMNodeScrape
-metadata:
- name: cadvisor-metrics
-spec:
- scheme: "https"
- tlsConfig:
- insecureSkipVerify: true
- caFile: "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
- bearerTokenFile: "/var/run/secrets/kubernetes.io/serviceaccount/token"
- relabelConfigs:
- - action: labelmap
- regex: __meta_kubernetes_node_label_(.+)
- - targetLabel: __address__
- replacement: kubernetes.default.svc:443
- - sourceLabels: [__meta_kubernetes_node_name]
- regex: (.+)
- targetLabel: __metrics_path__
- replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
-EOF
-```
-
-
-
-
-
-## VMProbe
-
- `VMProbe` required `VMAgent` and some external prober, blackbox exporter in our case. Ensure that you have `VMAgent` and `VMSingle`:
- ```yaml
-cat < 443/TCP 4h21m
-prometheus-blackbox-exporter ClusterIP 10.105.251.80 9115/TCP 4m36s
-vmagent-example-vmagent ClusterIP 10.102.31.47 8429/TCP 12m
-vmsingle-example-vmsingle-persisted ClusterIP 10.107.69.7 8429/TCP 12m
-```
-
-So, we will probe `VMAgent` with url - `vmagent-example-vmagent.default.svc:9115/heath` with blackbox url:
-`prometheus-blackbox-exporter.default.svc:9115` and module: `http_2xx` it was specified at blackbox configmap.
-
-```yaml
-cat << EOF | kubectl apply -f -
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMProbe
-metadata:
- name: probe-agent
-spec:
- jobName: static-probe
- vmProberSpec:
- # by default scheme http, and path is /probe
- url: prometheus-blackbox-exporter.default.svc:9115
- module: http_2xx
- targets:
- staticConfig:
- targets:
- - vmagent-example-vmagent.default.svc:8429/health
- interval: 2s
-EOF
-```
-
-Now new target must be added to `VMAgent` configuration, and it starts probing itself throw blackbox exporter.
-
-Let's try another target probe type - `Ingress`. Create ingress rule for `VMSingle` and create `VMProbe` for it:
-
-```yaml
-
-cat << EOF | kubectl apply -f -
-apiVersion: networking.k8s.io/v1beta1
-kind: Ingress
-metadata:
- labels:
- app: victoria-metrics-single
- name: victoria-metrics-single
-spec:
- rules:
- - host: vmsingle.example.com
- http:
- paths:
- - backend:
- serviceName: vmsingle-example-vmsingle-persisted
- servicePort: 8428
- path: /
- - host: vmsingle2.example.com
- http:
- paths:
- - backend:
- serviceName: vmsingle-example-vmsingle-persisted
- servicePort: 8428
- path: /
-
----
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMProbe
-metadata:
- name: probe-single-ingress
-spec:
- vmProberSpec:
- # by default scheme http, and path is /probe
- url: prometheus-blackbox-exporter.default.svc:9115
- module: http_2xx
- targets:
- ingress:
- selector:
- matchLabels:
- app: victoria-metrics-single
- interval: 10s
-EOF
-```
-
-This configuration will add 2 additional targets for probing: `vmsingle2.example.com` and `vmsingle.example.com`.
-
-But probes will be unsuccessful, coz there is no such hosts.
-
-## VMStaticScrape
-
-It generates config part of `VMAgent` with static_configs, targets for targetEndpoint is a required parameter.
-It has various options for scraping configuration of target (with basic auth,tls access, by specific port name etc).
-
-Add `VMAgent` and Example app from step above and continue this step.
-
-With simple configuration:
-```yaml
-cat << EOF | kubectl apply -f -
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMStaticScrape
-metadata:
- name: vmstaticscrape-sample
-spec:
- jobName: static
- targetEndpoints:
- - targets: ["192.168.0.1:9100","196.168.0.50:9100"]
- labels:
- env: dev
- project: operator
-EOF
-```
- 2 targets must be added to `VMAgent` scrape config:
-```console
-static_configs: added targets: 2, removed targets: 0; total targets: 2
-```
-
-
-## VMAuth
-
-[VMAuth](https://docs.victoriametrics.com/vmauth.html) allows protecting application with authentication and route traffic by rules.
-
-api docs [link](https://docs.victoriametrics.com/operator/api.html#vmauthspec)
-
- First create `VMAuth` configuration:
-```yaml
-cat << EOF | kubectl apply -f -
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMAuth
-metadata:
- name: example
- namespace: default
-spec:
- ingress: {}
- selectAllByDefault: true
-EOF
-```
- It will catch all `VMUser` at any kubernetes namespace and create `Ingress` record for it.
-```text
-kubectl get pods
-NAME READY STATUS RESTARTS AGE
-vmauth-example-ffcc78fcc-xddk7 2/2 Running 0 84s
-kubectl get ingress
-NAME CLASS HOSTS ADDRESS PORTS AGE
-vmauth-example * 80 106s
-kubectl get secret -l app.kubernetes.io/name=vmauth
-NAME TYPE DATA AGE
-vmauth-config-example Opaque 1 2m32s
-```
-
- Generated configuration can be retrieved with command:
-{% raw %}
-```text
-kubectl get secrets/vmauth-config-example -o=go-template='{{index .data "config.yaml.gz"}}' | base64 -d | gunzip
-
-users:
-- url_prefix: http://localhost:8428
- bearer_token: some-default-token
-```
-{% endraw %}
-
- Operator generates default config, if `VMUser`s for given `VMAuth` wasn't found.
-
-## VMUser
-
- `VMUser` configures `VMAuth`. api doc [link](https://docs.victoriametrics.com/operator/api.html#vmuserspec)
-
- There are two authentication mechanisms: `bearerToken` and `basicAuth` with `username` and `password`. Only one of them can be used with `VMUser` at one time.
-If you need to provide access with different mechanisms for single endpoint, create multiple `VMUsers`.
- If `username` is empty, metadata.name from `VMUser` used as `username`.
- If `password` is empty, operator generates random password for `VMUser`. This password added to the `Secret` for this `VMUser` at `data.password` field.
- Operator creates `Secret` for every `VMUser` with name - `vmuser-{VMUser.metadata.name}`. It places `username` + `password` or `bearerToken` into `data` section.
-
-`TargetRefs` is required field for `VMUser`, it allows to configure routing with:
-- `static` ref:
-```yaml
-- static:
- url: http://vmalertmanager.service.svc:9093
- ```
-- `crd` ref, allows to target CRD kind of operator, this `CRDObject` must exist.
-```yaml
-- crd:
- kind: VMAgent
- name: example
- namespace: default
-```
- Supported kinds are: `VMAgent, VMSingle, VMAlert, VMAlertmanager, VMCluster/vminsert, VMCluster/vmselect, VMCluster/vmstorage`
-
-`paths` - configures allowed routing paths for given `targetRef`.
-
- Let's create example, with access to `VMSingle` and `VMAlert` as static target:
-
-```yaml
-cat << EOF | kubectl apply -f -
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMSingle
-metadata:
- name: example
- namespace: default
-spec:
- retentionPeriod: "2d"
----
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMAlert
-metadata:
- name: example
-spec:
- replicaCount: 1
- datasource:
- url: "http://vmsingle-example.default.svc:8429"
- notifier:
- url: "http://vmalertmanager-example.default.svc:9093"
- evaluationInterval: "20s"
- ruleSelector: {}
-EOF
-```
-
- Check its status
-```console
-
-kubectl get pods
-NAME READY STATUS RESTARTS AGE
-vmalert-example-775b8dfbc9-vzlnv 1/2 Running 0 3s
-vmauth-example-ffcc78fcc-xddk7 2/2 Running 0 29m
-vmsingle-example-6496b5c95d-k6hhp 1/1 Running 0 3s
-```
-
- Then create `VMUser`
-```yaml
-cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
- name: example
+ name: demo
spec:
- username: simple-user
- password: simple-password
+ name: demo
+ username: demo
+ generatePassword: true
targetRefs:
+ # vmui + vmselect
- crd:
- kind: VMSingle
- name: example
- namespace: default
- paths: ["/.*"]
- - static:
- url: http://vmalert-example.default.svc:8080
- paths: ["/api/v1/groups","/api/v1/alerts"]
-EOF
-```
-
- Configuration changes for `VMAuth` takes some time, coz of mounted secret, its eventually updated by kubelet. Check vmauth log for changes:
-
-```console
-kubectl logs vmauth-example-ffcc78fcc-xddk7 vmauth -f --tail 10
-2021-05-31T10:46:40.171Z info VictoriaMetrics/app/vmauth/auth_config.go:168 Loaded information about 1 users from "/opt/vmauth/config.yaml"
-2021-05-31T10:46:40.171Z info VictoriaMetrics/app/vmauth/main.go:37 started vmauth in 0.000 seconds
-2021-05-31T10:46:40.171Z info VictoriaMetrics/lib/httpserver/httpserver.go:82 starting http server at http://:8427/
-2021-05-31T10:46:40.171Z info VictoriaMetrics/lib/httpserver/httpserver.go:83 pprof handlers are exposed at http://:8427/debug/pprof/
-2021-05-31T10:46:45.077Z info VictoriaMetrics/app/vmauth/auth_config.go:143 SIGHUP received; loading -auth.config="/opt/vmauth/config.yaml"
-2021-05-31T10:46:45.077Z info VictoriaMetrics/app/vmauth/auth_config.go:168 Loaded information about 1 users from "/opt/vmauth/config.yaml"
-2021-05-31T10:46:45.077Z info VictoriaMetrics/app/vmauth/auth_config.go:150 Successfully reloaded -auth.config="/opt/vmauth/config.yaml"
-2021-05-31T11:18:21.313Z info VictoriaMetrics/app/vmauth/auth_config.go:143 SIGHUP received; loading -auth.config="/opt/vmauth/config.yaml"
-2021-05-31T11:18:21.313Z info VictoriaMetrics/app/vmauth/auth_config.go:168 Loaded information about 1 users from "/opt/vmauth/config.yaml"
-2021-05-31T11:18:21.313Z info VictoriaMetrics/app/vmauth/auth_config.go:150 Successfully reloaded -auth.config="/opt/vmauth/config.yaml"
-```
-
- Now lets try to access protected endpoints, i will use port-forward for that:
-
-```console
-kubectl port-forward vmauth-example-ffcc78fcc-xddk7 8427
-
-# at separate terminal execute:
-
-# vmsingle response
-curl http://localhost:8427 -u 'simple-user:simple-password'
-
-# vmalert response
-curl localhost:8427/api/v1/groups -u 'simple-user:simple-password'
-```
-
- Check create secret for application access:
-
-```console
-kubectl get secrets vmuser-example
-NAME TYPE DATA AGE
-vmuser-example Opaque 2 6m33s
-```
-
-## Migration from prometheus-operator objects
-
-By default, the operator converts all existing prometheus-operator API objects into corresponding VictoriaMetrics Operator objects
-
-You can control this behaviour by setting env variable for operator:
-
-```console
-#disable convertion for each object
-VM_ENABLEDPROMETHEUSCONVERTER_PODMONITOR=false
-VM_ENABLEDPROMETHEUSCONVERTER_SERVICESCRAPE=false
-VM_ENABLEDPROMETHEUSCONVERTER_PROMETHEUSRULE=false
-VM_ENABLEDPROMETHEUSCONVERTER_PROBE=false
-```
-Otherwise, VictoriaMetrics Operator would try to discover prometheus-operator API and convert it.
-
-
- Conversion of api objects can be controlled by annotations, added to `VMObject`s, there are following annotations:
- - `operator.victoriametrics.com/merge-meta-strategy` - it controls syncing of metadata labels and annotations between
- `VMObject`s and `Prometheus` api objects during updates to `Prometheus` objects. By default, it has `prefer-prometheus`.
- And annotations and labels will be used from `Prometheus` objects, manually set values will be dropped.
- You can set it to `prefer-victoriametrics`. In this case all labels and annotations applied to `Prometheus` object
- will be ignored and `VMObject` will use own values.
- Two additional strategies annotations -`merge-victoriametrics-priority` and `merge-prometheus-priority` merges labelSets into one combined labelSet, with priority.
- Example:
-```yaml
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMServiceScrape
-metadata:
- annotations:
- meta.helm.sh/release-name: prometheus
- operator.victoriametrics.com/merge-meta-strategy: prefer-victoriametrics
- labels:
- release: prometheus
- name: prometheus-monitor
-spec:
- endpoints: []
-```
-
-- `operator.victoriametrics.com/ignore-prometheus-updates` - it controls updates from Prometheus api objects.
- By default, it set to `disabled`. You define it to `enabled` state and all updates from Prometheus api objects will be
- ignored.
-```yaml
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMServiceScrape
-metadata:
- annotations:
- meta.helm.sh/release-name: prometheus
- operator.victoriametrics.com/ignore-prometheus-updates: enabled
- labels:
- release: prometheus
- name: prometheus-monitor
-spec:
- endpoints: []
-```
-
-By default the operator doesn't make converted objects disappear after original ones are deleted. To change this behaviour
-configure adding `OwnerReferences` to converted objects:
-```console
-VM_ENABLEDPROMETHEUSCONVERTEROWNERREFERENCES=true
-```
-Converted objects will be linked to the original ones and will be deleted by kubernetes after the original ones are deleted.
-
-### prometheus Rule duplication
- `Prometheus` allows to specify rules with the same content with-in one group at Rule spec, but its forbidden by vmalert.
- You can tell operator to deduplicate this rules by adding annotation to the `VMAlert` crd definition. In this case operator
- skips rule with the same values, see example below.
- ```yaml
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMAlert
-metadata:
- name: example-vmalert-with-dedup
- annotations:
- operator.victoriametrics.com/vmalert-deduplicate-rules: "true"
-spec:
- replicaCount: 1
- datasource:
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429"
- notifier:
- url: "http://vmalertmanager-example-alertmanager.default.svc:9093"
- evaluationInterval: "30s"
- ruleNamespaceSelector: {}
- ruleSelector: {}
-```
- Now operator will transform this `VMRule`:
- ```yaml
-apiVersion: operator.victoriametrics.com/v1beta1
-kind: VMRule
-metadata:
- name: example-vmrule-reload-config
- labels:
- project: devops
-spec:
- groups:
- - name: vmalert
- rules:
- - alert: vmalert config reload error
- expr: delta(vmalert_config_last_reload_errors_total[5m]) > 0
- for: 10s
- labels:
- severity: major
- - alert: vmalert config reload error
- expr: delta(vmalert_config_last_reload_errors_total[5m]) > 0
- for: 10s
- labels:
- severity: major
- - alert: vmalert config reload error
- expr: delta(vmalert_config_last_reload_errors_total[5m]) > 0
- for: 2m
- labels:
- severity: critical
-```
-to the rule config:
-
-```yaml
- groups:
- - name: vmalert
- rules:
- - alert: vmalert config reload error
- expr: delta(vmalert_config_last_reload_errors_total[5m]) > 0
- for: 10s
- labels:
- severity: major
- - alert: vmalert config reload error
- expr: delta(vmalert_config_last_reload_errors_total[5m]) > 0
- for: 2m
- labels:
- severity: critical
-```
-## Expose the VMSingle API
-
-
-> WARNING: Please protect delete endpoint before exposing it [doc](https://github.com/VictoriaMetrics/VictoriaMetrics#how-to-delete-time-series)
-
-Example for Kubernetes Nginx ingress [doc](https://kubernetes.github.io/ingress-nginx/examples/auth/basic/)
-
-```console
-#generate creds
-htpasswd -c auth foo
-
-#create basic auth secret
-cat <
+
+
+
+## Anything else
+
+That's it. We obtained a monitoring cluster corresponding to the target topology:
+
+
+
+You have a full-stack monitoring cluster with VictoriaMetrics Operator.
+
+You can find information about these and other resources of operator on the [Custom resources page](./resources/README.md).
+
+In addition, check out other sections of the documentation for VictoriaMetrics Operator:
+
+- [Setup](./setup.md)
+- [Security](./security.md)
+- [Configuration](./configuration.md)
+- [Migration from Prometheus](./migration.md)
+- [Monitoring](./monitoring.md)
+- [Authorization and exposing components](./auth.md)
+- [High Availability](./high-availability.md)
+- [Enterprise](./enterprise.md)
+
+If you have any questions, check out our [FAQ](./FAQ.md)
+and feel free to can ask them:
+- [VictoriaMetrics Slack](https://victoriametrics.slack.com/)
+- [VictoriaMetrics Telegram](https://t.me/VictoriaMetrics_en)
+
+If you have any suggestions or find a bug, please create an issue
+on [GitHub](https://github.com/VictoriaMetrics/operator/issues/new).
diff --git a/docs/operator/resources/README.md b/docs/operator/resources/README.md
new file mode 100644
index 000000000..52691151b
--- /dev/null
+++ b/docs/operator/resources/README.md
@@ -0,0 +1,220 @@
+---
+sort: 14
+weight: 14
+title: Custom resources
+---
+
+# Custom resource kinds
+
+This documentation section describes the design and interaction between the custom resource definitions (CRD) that the Victoria
+Metrics Operator introduces.
+
+[Operator](../README.md) introduces the following custom resources:
+
+- [VMAgent](./vmagent.md)
+- [VMAlert](./vmalert.md)
+- [VMAlertManager](./vmalertmanager.md)
+- [VMAlertManagerConfig](./vmalertmanagerconfig.md)
+- [VMAuth](./vmauth.md)
+- [VMCluster](./vmcluster.md)
+- [VMNodeScrape](./vmnodescrape.md)
+- [VMPodScrape](./vmpodscrape.md)
+- [VMProbe](./vmprobe.md)
+- [VMRule](./vmrule.md)
+- [VMServiceScrape](./vmservicescrape.md)
+- [VMStaticScrape](./vmstaticscrape.md)
+- [VMSingle](./vmsingle.md)
+- [VMUser](./vmuser.md)
+
+Here is the scheme of relations between the custom resources:
+
+
+
+## Specification
+
+You can find the specification for the custom resources on **[API Docs](../api.md)**.
+
+### Extra arguments
+
+If you can't find necessary field in the specification of custom resource,
+you can use `extraArgs` field for passing additional arguments to the application.
+
+Field `extraArgs` is supported for the following custom resources:
+
+- [VMAgent spec](../api.md#vmagentspec)
+- [VMAlert spec](../api.md#vmalertspec)
+- [VMAlertManager spec](../api.md#vmalertmanagerspec)
+- [VMAuth spec](../api.md#vmauthspec)
+- [VMCluster/vmselect spec](../api.md#vmselect)
+- [VMCluster/vminsert spec](../api.md#vminsert)
+- [VMCluster/vmstorage spec](../api.md#vmstorage)
+- [VMSingle spec](../api.md#vmsinglespec)
+
+Supported flags for each application can be found the in the corresponding documentation:
+
+- [VMAgent flags](https://docs.victoriametrics.com/vmagent.html#advanced-usage)
+- [VMAlert](https://docs.victoriametrics.com/vmalert.html#configuration)
+- [VMAuth](https://docs.victoriametrics.com/vmauth.html#advanced-usage)
+- [VMCluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#list-of-command-line-flags)
+- [VMSingle](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#list-of-command-line-flags)
+
+Usage example:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMSingle
+metadata:
+ name: vmsingle-example-exrtaargs
+spec:
+ retentionPeriod: "1"
+ extraArgs:
+ dedup.minScrapeInterval: 60s
+ # ...
+```
+
+### Extra environment variables
+
+Flag can be replaced with environment variable, it's useful for retrieving value from secret.
+You can use `extraEnvs` field for passing additional arguments to the application.
+
+Usage example:
+
+```yaml
+kind: VMSingle
+metadata:
+ name: vmsingle-example--exrtaenvs
+spec:
+ retentionPeriod: "1"
+ extraEnvs:
+ - name: DEDUP_MINSCRAPEINTERVAL
+ valueFrom:
+ secretKeyRef:
+ name: vm-secret
+ key: dedup
+```
+
+This feature really useful for using with
+[`-envflag.enable` command-line argument](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#environment-variables).
+
+## Examples
+
+Page for every custom resource contains examples section:
+
+- [VMAgent examples](./vmagent.md#examples)
+- [VMAlert examples](./vmalert.md#examples)
+- [VMAlertmanager examples](./vmalertmanager.md#examples)
+- [VMAlertmanagerConfig examples](./vmalertmanagerconfig.md#examples)
+- [VMAuth examples](./vmauth.md#examples)
+- [VMCluster examples](./vmcluster.md#examples)
+- [VMNodeScrape examples](./vmnodescrape.md#examples)
+- [VMPodScrape examples](./vmpodscrape.md#examples)
+- [VMProbe examples](./vmprobe.md#examples)
+- [VMRule examples](./vmrule.md#examples)
+- [VMServiceScrape examples](./vmservicescrape.md#examples)
+- [VMStaticScrape examples](./vmstaticscrape.md#examples)
+- [VMSingle examples](./vmsingle.md#examples)
+- [VMUser examples](./vmuser.md#examples)
+
+In addition, you can find examples of the custom resources for VIctoriMetrics operator in
+the **[examples directory](https://github.com/VictoriaMetrics/operator/tree/master/config/examples) of operator repository**.
+
+## Managing versions of VM
+
+Every custom resource with deployable application has a fields for specifying version (docker image) of component:
+
+- [Managing versions for VMAgent](./vmagent.md#version-management)
+- [Managing versions for VMAlert](./vmalert.md#version-management)
+- [Managing versions for VMAlertmanager](./vmalertmanager.md#version-management)
+- [Managing versions for VMAuth](./vmauth.md#version-management)
+- [Managing versions for VMCluster](./vmcluster.md#version-management)
+- [Managing versions for VMSingle](./vmsingle.md#version-management)
+
+## High availability
+
+VictoriaMetrics operator support high availability for each component of the monitoring stack:
+
+- [VMAgent](./vmagent.md#high-availability)
+- [VMAlert](./vmalert.md#high-availability)
+- [VMAlertmanager](./vmalertmanager.md#high-availability)
+- [VMAuth](./vmauth.md#high-availability)
+- [VMCluster](./vmcluster.md#high-availability)
+
+In addition, these CRD support common features, that can be used to increase high availability - resources above have the following fields:
+
+- `affinity` - to schedule pods on different nodes ([affinity and anti-affinity in kubernetes docs](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)),
+- `tolerations` - to schedule pods on nodes with taints ([taints and tolerations in kubernetes docs](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/)),
+- `nodeSelector` - to schedule pods on nodes with specific labels ([node selector in kubernetes docs](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector)),
+- `topologySpreadConstraints` - to schedule pods on different nodes in the same topology ([topology spread constraints in kubernetes docs](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#pod-topology-spread-constraints)).
+
+See details about these fields in the [Specification](#specification).
+
+## Enterprise features
+
+Operator supports following [Enterprise features for VictoriaMetrics components](https://docs.victoriametrics.com/enterprise.html):
+
+- [VMAgent Enterprise features](./vmagent.md#enterprise-features):
+ - [Reading metrics from kafka](./vmagent.md#reading-metrics-from-kafka)
+ - [Writing metrics to kafka](./vmagent.md#writing-metrics-to-kafka)
+- [VMAlert Enterprise features](./vmalert.md#enterprise-features):
+ - [Reading rules from object storage](./vmalert.md#reading-rules-from-object-storage)
+ - [Multitenancy](./vmalert.md#multitenancy)
+- [VMAuth Enterprise features](./vmauth.md#enterprise-features)
+ - [IP Filters](./vmauth.md#ip-filters)
+- [VMCluster Enterprise features](./vmcluster.md#enterprise-features)
+ - [Downsampling](./vmcluster.md#downsampling)
+ - [Multiple retentions / Retention filters](./vmcluster.md#retention-filters)
+ - [Advanced per-tenant statistic](./vmcluster.md#advanced-per-tenant-statistic)
+ - [mTLS protection](./vmcluster.md#mtls-protection)
+ - [Backup atomation](./vmcluster.md#backup-atomation)
+- [VMRule Enterprise features](./vmrule.md#enterprise-features)
+ - [Multitenancy](./vmrule.md#multitenancy)
+- [VMSingle Enterprise features](./vmsingle.md#enterprise-features)
+ - [Downsampling](./vmsingle.md#downsampling)
+ - [Retention filters](./vmsingle.md#retention-filters)
+ - [Backup atomation](./vmsingle.md#backup-atomation)
+- [VMUser Enterprise features](./vmuser.md#enterprise-features)
+ - [IP Filters](./vmuser.md#ip-filters)
+
+More information about enterprise features you can read
+on [VictoriaMetrics Enterprise page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
+
+## Configuration synchronization
+
+### Basic concepts
+
+VictoriaMetrics applications, like many other applications with configuration file deployed at Kubernetes, uses `ConfigMaps` and `Secrets` for configuration files.
+Usually, it's out of application scope to watch for configuration on-disk changes.
+Applications reload their configuration by a signal from a user or some other tool, that knows how to watch for updates.
+At Kubernetes, the most popular design for this case is a sidecar container, that watches for configuration file changes and sends an HTTP request to the application.
+
+`Configmap` or `Secret` that mounted at `Pod` holds a copy of its content.
+Kubernetes component `kubelet` is responsible for content synchronization between an object at Kubernetes API and a file served on disk.
+It's not efficient to sync its content immediately, and `kubelet` eventually synchronizes it. There is a configuration option, that controls this period.
+
+That's why, applications managed by operator don't receive changes immediately. It usually takes 1-2 min, before content will be updated.
+
+It may trigger errors when an application was deleted, but [`VMAgent`](./README.mdvmagent) still tries to scrape it.
+
+### Possible mitigations
+
+The naive solution for this case decrease the synchronization period. But it configures globally and may be hard for operator users.
+
+That's why operator uses a few hacks.
+
+For `ConfigMap` updates, operator changes annotation with a time of `Configmap` content update. It triggers `ConfigMap`'s content synchronization by kubelet immediately.
+It's the case for `VMAlert`, it uses `ConfigMap` as a configuration source.
+
+For `Secret` it doesn't work. And operator offers its implementation for side-car container. It can be configured with env variable for operator:
+
+```
+- name: VM_USECUSTOMCONFIGRELOADER
+ value: "true"
+```
+
+If it's defined, operator uses own [config-reloader](https://github.com/VictoriaMetrics/operator/tree/master/internal/config-reloader)
+instead of [prometheus-config-reload](https://github.com/prometheus-operator/prometheus-operator/tree/main/cmd/prometheus-config-reloader).
+
+It watches corresponding `Secret` for changes with Kubernetes API watch call and writes content into emptyDir.
+This emptyDir shared with the application.
+In case of content changes, `config-reloader` sends HTTP requests to the application.
+It greatly reduces the time for configuration synchronization.
diff --git a/docs/operator/resources/vmagent.md b/docs/operator/resources/vmagent.md
new file mode 100644
index 000000000..73126c82d
--- /dev/null
+++ b/docs/operator/resources/vmagent.md
@@ -0,0 +1,720 @@
+# VMAgent
+
+`VMAgent` represents agent, which helps you collect metrics from various sources and stores them in VictoriaMetrics.
+The `VMAgent` CRD declaratively defines a desired [VMAgent](https://docs.victoriametrics.com/vmagent)
+setup to run in a Kubernetes cluster.
+
+It requires access to Kubernetes API and you can create RBAC for it first, it can be found
+at [`examples/vmagent_rbac.yaml`](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmagent_rbac.yaml)
+Or you can use default rbac account, that will be created for `VMAgent` by operator automatically.
+
+For each `VMAgent` resource Operator deploys a properly configured `Deployment` in the same namespace.
+The VMAgent `Pod`s are configured to mount a `Secret` prefixed with `` containing the configuration
+for VMAgent.
+
+For each `VMAgent` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
+name ``.
+
+The CRD specifies which `VMServiceScrape` should be covered by the deployed VMAgent instances based on label selection.
+The Operator then generates a configuration based on the included `VMServiceScrape`s and updates the `Secret` which
+contains the configuration. It continuously does so for all changes that are made to the `VMServiceScrape`s or the
+`VMAgent` resource itself.
+
+If no selection of `VMServiceScrape`s is provided - Operator leaves management of the `Secret` to the user,
+so user can set custom configuration while still benefiting from the Operator's capabilities of managing VMAgent setups.
+
+## Specification
+
+You can see the full actual specification of the `VMAgent` resource in the **[API docs -> VMAgent](../api.md#vmagent)**.
+
+If you can't find necessary field in the specification of the custom resource,
+see [Extra arguments section](./README.md#extra-arguments).
+
+Also, you can check out the [examples](#examples) section.
+
+## Scraping
+
+`VMAgent` supports scraping targets with:
+
+- [VMServiceScrape](./vmservicescrape.md),
+- [VMPodScrape](./vmpodscrape.md),
+- [VMNodeScrape](./vmnodescrape.md),
+- [VMStaticScrape](./vmstaticscrape.md),
+- [VMProbe](./vmprobe.md).
+
+These objects tell VMAgent from which targets and how to collect metrics and
+generate part of [VMAgent](./vmagent.md) scrape configuration.
+
+For filtering scrape objects `VMAgent` uses selectors.
+Selectors are defined with suffixes - `NamespaceSelector` and `Selector` for each type of scrape objects in spec of `VMAgent`:
+
+- `serviceScrapeNamespaceSelector` and `serviceScrapeSelector` for selecting [VMServiceScrape](./vmservicescrape.md) objects,
+- `podScrapeNamespaceSelector` and `podScrapeSelector` for selecting [VMPodScrape](./vmpodscrape.md) objects,
+- `probeNamespaceSelector` and `probeSelector` for selecting [VMProbe](./vmprobe.md) objects,
+- `staticScrapeNamespaceSelector` and `staticScrapeSelector` for selecting [VMStaticScrape](./vmstaticscrape.md) objects,
+- `nodeScrapeNamespaceSelector` and `nodeScrapeSelector` for selecting [VMNodeScrape](./vmnodescrape.md) objects.
+
+It allows configuring objects access control across namespaces and different environments.
+Specification of selectors you can see in [this doc](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#labelselector-v1-meta).
+
+In addition to the above selectors, the filtering of objects in a cluster is affected by the field `selectAllByDefault` of `VMAgent` spec and environment variable `WATCH_NAMESPACE` for operator.
+
+Following rules are applied:
+
+- If `*NamespaceSelector` and `*Selector` both undefined, then by default select nothing. With option set - `spec.selectAllByDefault: true`, select all objects of given type.
+- If `*NamespaceSelector` defined, `*Selector` undefined, then all objects are matching at namespaces for given `*NamespaceSelector`.
+- If `*NamespaceSelector` undefined, `*Selector` defined, then all objects at `VMAgent`'s namespaces are matching for given `*Selector`.
+- If `*NamespaceSelector` and `*Selector` both defined, then only objects at namespaces matched `*NamespaceSelector` for given `*Selector` are matching.
+
+Here's a more visual and more detailed view:
+
+| `*NamespaceSelector` | `*Selector` | `selectAllByDefault` | `WATCH_NAMESPACE` | Selected objects |
+|----------------------|-------------|----------------------|-------------------|-------------------------------------------------------------------------------------------------------|
+| undefined | undefined | false | undefined | nothing |
+| undefined | undefined | **true** | undefined | all objects of given type (`*`) in the cluster |
+| **defined** | undefined | any | undefined | all objects of given type (`*`) at namespaces for given `*NamespaceSelector` |
+| undefined | **defined** | any | undefined | all objects of given type (`*`) only at `VMAgent`'s namespace are matching for given `Selector |
+| **defined** | **defined** | any | undefined | all objects of given type (`*`) only at namespaces matched `*NamespaceSelector` for given `*Selector` |
+| any | undefined | any | **defined** | all objects of given type (`*`) only at `VMAgent`'s namespace |
+| any | **defined** | any | **defined** | all objects of given type (`*`) only at `VMAgent`'s namespace for given `*Selector` |
+
+More details about `WATCH_NAMESPACE` variable you can read in [this doc](../configuration.md#namespaced-mode).
+
+Here are some examples of `VMAgent` configuration with selectors:
+
+```yaml
+# select all scrape objects in the cluster
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-select-all
+spec:
+ # ...
+ selectAllByDefault: true
+
+---
+
+# select all scrape objects in specific namespace (my-namespace)
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-select-ns
+spec:
+ # ...
+ serviceScrapeNamespaceSelector:
+ matchLabels:
+ kubernetes.io/metadata.name: my-namespace
+ podScrapeNamespaceSelector:
+ matchLabels:
+ kubernetes.io/metadata.name: my-namespace
+ nodeScrapeNamespaceSelector:
+ matchLabels:
+ kubernetes.io/metadata.name: my-namespace
+ staticScrapeNamespaceSelector:
+ matchLabels:
+ kubernetes.io/metadata.name: my-namespace
+ probeNamespaceSelector:
+ matchLabels:
+ kubernetes.io/metadata.name: my-namespace
+```
+
+## High availability
+
+
+
+### Replication and deduplication
+
+To run VMAgent in a highly available manner at first you have to configure deduplication in Victoria Metrics
+according [this doc for VMSingle](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#deduplication)
+or [this doc for VMCluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#deduplication).
+
+You can do it with `extraArgs` on [`VMSingle`](./vmsingle.md):
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMSingle
+metadata:
+ name: vmsingle-example
+spec:
+ # ...
+ extraArgs:
+ dedup.minScrapeInterval: 30s
+ # ...
+```
+
+For [`VMCluster`](./vmcluster.md) you can do it with `vmstorage.extraArgs` and `vmselect.extraArgs`:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMCluster
+metadata:
+ name: vmcluster-example
+spec:
+ # ...
+ vmselect:
+ extraArgs:
+ dedup.minScrapeInterval: 30s
+ # ...
+ vmstorage:
+ extraArgs:
+ dedup.minScrapeInterval: 30s
+ # ...
+```
+
+Deduplication is automatically enabled with `replicationFactor > 1` on `VMCLuster`.
+
+After enabling deduplication you can increase replicas for VMAgent.
+
+For instance, let's create `VMAgent` with 2 replicas:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-ha-example
+spec:
+ # ...
+ selectAllByDefault: true
+ vmAgentExternalLabelName: vmagent_ha
+ remoteWrite:
+ - url: "http://vmsingle-example.default.svc:8429/api/v1/write"
+ # Replication:
+ scrapeInterval: 30s
+ replicaCount: 2
+ # ...
+```
+
+Now, even if something happens to one of the vmagent, you'll still have the data.
+
+### StatefulMode
+
+VMAgent supports [persistent buffering](https://docs.victoriametrics.com/vmagent.html#replication-and-high-availability)
+for sending data to remote storage. By default, operator set `-remoteWrite.tmpDataPath` for `VMAgent` to `/tmp` (that use k8s ephemeral storage)
+and `VMAgent` loses state of the PersistentQueue on pod restarts.
+
+In `StatefulMode` `VMAgent` doesn't lose state of the PersistentQueue (file-based buffer size for unsent data) on pod restarts.
+Operator creates `StatefulSet` and, with provided `PersistentVolumeClaimTemplate` at `StatefulStorage` configuration param, metrics queue is stored on disk.
+
+Example of configuration for `StatefulMode`:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-ha-example
+spec:
+ # ...
+ selectAllByDefault: true
+ vmAgentExternalLabelName: vmagent_ha
+ remoteWrite:
+ - url: "http://vmsingle-example.default.svc:8429/api/v1/write"
+ # Replication:
+ scrapeInterval: 30s
+ replicaCount: 2
+ # StatefulMode:
+ statefulMode: true
+ statefulStorage:
+ volumeClaimTemplate:
+ spec:
+ resources:
+ requests:
+ storage: 20Gi
+ # ...
+```
+
+### Sharding
+
+Operator supports sharding with [cluster mode of vmagent](https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets)
+for **scraping big number of targets**.
+
+Sharding for `VMAgent` distributes scraping between multiple deployments of `VMAgent`.
+
+Example usage (it is a complete example of `VMAgent` with high availability features):
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-ha-example
+spec:
+ # ...
+ selectAllByDefault: true
+ vmAgentExternalLabelName: vmagent_ha
+ remoteWrite:
+ - url: "http://vmsingle-example.default.svc:8429/api/v1/write"
+ # Replication:
+ scrapeInterval: 30s
+ replicaCount: 2
+ # StatefulMode:
+ statefulMode: true
+ statefulStorage:
+ volumeClaimTemplate:
+ spec:
+ resources:
+ requests:
+ storage: 20Gi
+ # Sharding
+ shardCount: 5
+ affinity:
+ podAntiAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - podAffinityTerm:
+ labelSelector:
+ matchLabels:
+ shard-num: '%SHARD_NUM%'
+ topologyKey: kubernetes.io/hostname
+ # ...
+```
+
+This configuration produces `5` deployments with `2` replicas at each.
+Each deployment has its own shard num and scrapes only `1/5` of all targets.
+
+Also, you can use special placeholder `%SHARD_NUM%` in fields of `VMAgent` specification
+and operator will replace it with current shard num of vmagent when creating deployment or statefullset for vmagent.
+
+In the example above, the `%SHARD_NUM%` placeholder is used in the `podAntiAffinity` section,
+which recommend to scheduler that pods with the same shard num (label `shard-num` in the pod template)
+are not deployed on the same node. You can use another `topologyKey` for availability zone or region instead of nodes.
+
+**Note** that at the moment operator doesn't use `-promscrape.cluster.replicationFactor` parameter of `VMAgent` and
+creates `replicaCount` of replicas for each shard (which leads greater resource consumption).
+This will be fixed in the future, more details can be seen in [this issue](https://github.com/VictoriaMetrics/operator/issues/604).
+
+Also see [this example](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmagent_stateful_with_sharding.yaml).
+
+## Additional scrape configuration
+
+AdditionalScrapeConfigs is an additional way to add scrape targets in `VMAgent` CRD.
+
+There are two options for adding targets into `VMAgent`:
+
+- [inline configuration into CRD](#inline-additional-scrape-configuration-in-vmagent-crd),
+- [defining it as a Kubernetes Secret](#define-additional-scrape-configuration-as-a-kubernetes-secret).
+
+No validation happens during the creation of configuration. However, you must validate job specs, and it must follow job spec configuration.
+Please check [scrape_configs documentation](https://docs.victoriametrics.com/sd_configs.html#scrape_configs) as references.
+
+### Inline Additional Scrape Configuration in VMAgent CRD
+
+You need to add scrape configuration directly to the `vmagent spec.inlineScrapeConfig`. It is raw text in YAML format.
+See example below
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-example
+spec:
+ # ...
+ selectAllByDefault: true
+ inlineScrapeConfig: |
+ - job_name: "prometheus"
+ static_configs:
+ - targets: ["localhost:9090"]
+ remoteWrite:
+ - url: "http://vmsingle-example.default.svc:8429/api/v1/write"
+ # ...
+```
+
+**Note**: Do not use passwords and tokens with inlineScrapeConfig use Secret instead.
+
+## Define Additional Scrape Configuration as a Kubernetes Secret
+
+You need to define Kubernetes Secret with a key.
+
+The key is `prometheus-additional.yaml` in the example below:
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+ name: additional-scrape-configs
+stringData:
+ prometheus-additional.yaml: |
+ - job_name: "prometheus"
+ static_configs:
+ - targets: ["localhost:9090"]
+```
+
+After that, you need to specify the secret's name and key in VMAgent CRD in `additionalScrapeConfigs` section:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-example
+spec:
+ # ...
+ selectAllByDefault: true
+ additionalScrapeConfigs:
+ name: additional-scrape-configs
+ key: prometheus-additional.yaml
+ remoteWrite:
+ - url: "http://vmsingle-example.default.svc:8429/api/v1/write"
+ # ...
+```
+
+**Note**: You can specify only one Secret in the VMAgent CRD configuration so use it for all additional scrape configurations.
+
+## Relabeling
+
+`VMAgent` supports global relabeling for all metrics and per remoteWrite target relabel config.
+
+Note in some cases, you don't need relabeling, `key=value` label pairs can be added to the all scrapped metrics with `spec.externalLabels` for `VMAgent`:
+
+```yaml
+# simple label add config
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-example
+spec:
+ externalLabels:
+ clusterid: some_cluster
+```
+
+`VMAgent` CR supports relabeling with [custom configMap](#relabeling-config-in-configmap)
+or [inline defined at CRD](#inline-relabeling-config).
+
+### Relabeling config in Configmap
+
+Quick tour how to create `ConfigMap` with relabeling configuration:
+
+ ```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: vmagent-relabel
+data:
+ global-relabel.yaml: |
+ - target_label: bar
+ - source_labels: [aa]
+ separator: "foobar"
+ regex: "foo.+bar"
+ target_label: aaa
+ replacement: "xxx"
+ - action: keep
+ source_labels: [aaa]
+ - action: drop
+ source_labels: [aaa]
+ target-1-relabel.yaml: |
+ - action: keep_if_equal
+ source_labels: [foo, bar]
+ - action: drop_if_equal
+ source_labels: [foo, bar]
+```
+
+Second, add `relabelConfig` to `VMagent` spec for global relabeling with name of `Configmap` - `vmagent-relabel` and key `global-relabel.yaml`.
+
+For relabeling per remoteWrite target, add `urlRelabelConfig` name of `Configmap` - `vmagent-relabel`
+and key `target-1-relabel.yaml` to one of remoteWrite target for relabeling only for those target:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-example
+spec:
+ # ...
+ selectAllByDefault: true
+ relabelConfig:
+ name: "vmagent-relabel"
+ key: "global-relabel.yaml"
+ remoteWrite:
+ - url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
+ - url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
+ urlRelabelConfig:
+ name: "vmagent-relabel"
+ key: "target-1-relabel.yaml"
+```
+
+### Inline relabeling config
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-example
+spec:
+ # ...
+ selectAllByDefault: true
+ inlineRelabelConfig:
+ - target_label: bar
+ - source_labels: [aa]
+ separator: "foobar"
+ regex: "foo.+bar"
+ target_label: aaa
+ replacement: "xxx"
+ - action: keep
+ source_labels: [aaa]
+ - action: drop
+ source_labels: [aaa]
+ remoteWrite:
+ - url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
+ - url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
+ inlineUrlRelabelConfig:
+ - action: keep_if_equal
+ source_labels: [foo, bar]
+ - action: drop_if_equal
+ source_labels: [foo, bar]
+```
+
+### Combined example
+
+It's also possible to use both features in combination.
+
+First will be added relabeling configs from `inlineRelabelConfig`, then `relabelConfig` from configmap.
+
+ ```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: vmagent-relabel
+data:
+ global-relabel.yaml: |
+ - target_label: bar
+ - source_labels: [aa]
+ separator: "foobar"
+ regex: "foo.+bar"
+ target_label: aaa
+ replacement: "xxx"
+ - action: keep
+ source_labels: [aaa]
+ - action: drop
+ source_labels: [aaa]
+ target-1-relabel.yaml: |
+ - action: keep_if_equal
+ source_labels: [foo, bar]
+ - action: drop_if_equal
+ source_labels: [foo, bar]
+```
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: example-vmagent
+spec:
+ # ...
+ selectAllByDefault: true
+ inlineRelabelConfig:
+ - target_label: bar1
+ - source_labels: [aa]
+ relabelConfig:
+ name: "vmagent-relabel"
+ key: "global-relabel.yaml"
+ remoteWrite:
+ - url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
+ - url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
+ urlRelabelConfig:
+ name: "vmagent-relabel"
+ key: "target-1-relabel.yaml"
+ inlineUrlRelabelConfig:
+ - action: keep_if_equal
+ source_labels: [foo1, bar2]
+```
+
+Resulted configmap, mounted to `VMAgent` pod:
+
+```yaml
+apiVersion: v1
+data:
+ global_relabeling.yaml: |
+ - target_label: bar1
+ - source_labels:
+ - aa
+ - target_label: bar
+ - source_labels: [aa]
+ separator: "foobar"
+ regex: "foo.+bar"
+ target_label: aaa
+ replacement: "xxx"
+ - action: keep
+ source_labels: [aaa]
+ - action: drop
+ source_labels: [aaa]
+ url_rebaling-1.yaml: |
+ - source_labels:
+ - foo1
+ - bar2
+ action: keep_if_equal
+ - action: keep_if_equal
+ source_labels: [foo, bar]
+ - action: drop_if_equal
+ source_labels: [foo, bar]
+kind: ConfigMap
+metadata:
+ finalizers:
+ - apps.victoriametrics.com/finalizer
+ labels:
+ app.kubernetes.io/component: monitoring
+ app.kubernetes.io/instance: example-vmagent
+ app.kubernetes.io/name: vmagent
+ managed-by: vm-operator
+ name: relabelings-assets-vmagent-example-vmagent
+ namespace: default
+ ownerReferences:
+ - apiVersion: operator.victoriametrics.com/v1beta1
+ blockOwnerDeletion: true
+ controller: true
+ kind: VMAgent
+ name: example-vmagent
+ uid: 7e9fb838-65da-4443-a43b-c00cd6c4db5b
+```
+
+### Additional information
+
+`VMAgent` also has some extra options for relabeling actions, you can check it [docs](https://docs.victoriametrics.com/vmagent#relabeling).
+
+## Version management
+
+To set `VMAgent` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: example-vmagent
+spec:
+ image:
+ repository: victoriametrics/victoria-metrics
+ tag: v1.93.4
+ pullPolicy: Always
+ # ...
+```
+
+Also, you can specify `imagePullSecrets` if you are pulling images from private repo:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: example-vmagent
+spec:
+ image:
+ repository: victoriametrics/victoria-metrics
+ tag: v1.93.4
+ pullPolicy: Always
+ imagePullSecrets:
+ - name: my-repo-secret
+# ...
+```
+
+## Enterprise features
+
+VMAgent supports feature [Kafka integration](https://docs.victoriametrics.com/vmagent.html#kafka-integration)
+from [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
+
+For using Enterprise version of [vmagent](https://docs.victoriametrics.com/vmagent.html)
+you need to change version of `vmagent` to version with `-enterprise` suffix using [Version management](#version-management).
+
+All the enterprise apps require `-eula` command-line flag to be passed to them.
+This flag acknowledges that your usage fits one of the cases listed on [this page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
+So you can use [extraArgs](./README.md#extra-arguments) for passing this flag to `VMAgent`:
+
+After that you can pass [Kafka integration](https://docs.victoriametrics.com/vmagent.html#kafka-integration)
+flags to `VMAgent` with [extraArgs](./README.md#extra-arguments).
+
+### Reading metrics from Kafka
+
+Here are complete example for [Reading metrics from Kafka](https://docs.victoriametrics.com/vmagent.html#reading-metrics-from-kafka):
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-ent-example
+spec:
+ # enabling enterprise features
+ image:
+ # enterprise version of vmagent
+ tag: v1.93.5-enterprise
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmagent enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: reading metrics from kafka
+ # more details about kafka integration you can read on https://docs.victoriametrics.com/vmagent.html#kafka-integration
+ # more details about these and other flags you can read on https://docs.victoriametrics.com/vmagent.html#command-line-flags-for-kafka-consumer
+ kafka.consumer.topic.brokers: localhost:9092
+ kafka.consumer.topic.format: influx
+ kafka.consumer.topic: metrics-by-telegraf
+ kafka.consumer.topic.groupID: some-id
+
+ # ...other fields...
+```
+
+### Writing metrics to Kafka
+
+Here are complete example for [Writing metrics to Kafka](https://docs.victoriametrics.com/vmagent.html#writing-metrics-to-kafka):
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: vmagent-ent-example
+spec:
+ # enabling enterprise features
+ image:
+ # enterprise version of vmagent
+ tag: v1.93.5-enterprise
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmagent enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: writing metrics to Kafka
+ # more details about kafka integration you can read on https://docs.victoriametrics.com/vmagent.html#kafka-integration
+ remoteWrite:
+ # sasl with username and password
+ - url: kafka://broker-1:9092/?topic=prom-rw-1&security.protocol=SASL_SSL&sasl.mechanisms=PLAIN
+ # it requires to create kubernetes secret `kafka-basic-auth` with keys `username` and `password` in the same namespace
+ basicAuth:
+ username:
+ name: kafka-basic-auth
+ key: username
+ password:
+ name: kafka-basic-auth
+ key: password
+ # sasl with username and password from secret and tls
+ - url: kafka://localhost:9092/?topic=prom-rw-2&security.protocol=SSL
+ # it requires to create kubernetes secret `kafka-tls` with keys `ca.pem`, `cert.pem` and `key.pem` in the same namespace
+ tlsConfig:
+ ca:
+ secret:
+ name: kafka-tls
+ key: ca.pem
+ cert:
+ secret:
+ name: kafka-tls
+ key: cert.pem
+ keySecret:
+ name: kafka-tls
+ key: key.pem
+
+ # ...other fields...
+```
+
+## Examples
+
+```yaml
+kind: VMAgent
+metadata:
+ name: vmagent-example
+spec:
+ selectAllByDefault: true
+ replicaCount: 1
+ scrapeInterval: 30s
+ scrapeTimeout: 10s
+ vmAgentExternalLabelName: example
+ externalLabels:
+ cluster: my-cluster
+ remoteWrite:
+ - url: "http://vmsingle-example.default.svc:8428/api/v1/write"
+ inlineRelabelConfig:
+ - action: labeldrop
+ regex: "temp.*"
+```
diff --git a/docs/operator/resources/vmalert.md b/docs/operator/resources/vmalert.md
new file mode 100644
index 000000000..9e5b9c272
--- /dev/null
+++ b/docs/operator/resources/vmalert.md
@@ -0,0 +1,362 @@
+# VMAlert
+
+`VMAlert` - executes a list of given [alerting](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/)
+or [recording](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) rules against configured address.
+
+The `VMAlert` CRD declaratively defines a desired [VMAlert](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmalert)
+setup to run in a Kubernetes cluster.
+
+It has few required config options - `datasource` and `notifier` are required, for other config parameters
+check [doc](../api.md#vmalert).
+
+For each `VMAlert` resource, the Operator deploys a properly configured `Deployment` in the same namespace.
+The VMAlert `Pod`s are configured to mount a list of `Configmaps` prefixed with `-number` containing
+the configuration for alerting rules.
+
+For each `VMAlert` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
+name ``.
+
+## Specification
+
+You can see the full actual specification of the `VMAlert` resource in the **[API docs -> VMAlert](../api.md#vmalert)**.
+
+If you can't find necessary field in the specification of the custom resource,
+see [Extra arguments section](./README.md#extra-arguments).
+
+Also, you can check out the [examples](#examples) section.
+
+## Rules
+
+The CRD specifies which `VMRule`s should be covered by the deployed `VMAlert` instances based on label selection.
+The Operator then generates a configuration based on the included `VMRule`s and updates the `Configmaps` containing
+the configuration. It continuously does so for all changes that are made to `VMRule`s or to the `VMAlert` resource itself.
+
+Alerting rules are filtered by selectors `ruleNamespaceSelector` and `ruleSelector` in `VMAlert` CRD definition.
+For selecting rules from all namespaces you must specify it to empty value:
+
+```yaml
+spec:
+ ruleNamespaceSelector: {}
+```
+
+[VMRUle](./vmrule.md) objects generate part of `VMAlert` configuration.
+
+For filtering rules `VMAlert` uses selectors `ruleNamespaceSelector` and `ruleSelector`.
+It allows configuring rules access control across namespaces and different environments.
+Specification of selectors you can see in [this doc](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#labelselector-v1-meta).
+
+In addition to the above selectors, the filtering of objects in a cluster is affected by the field `selectAllByDefault` of `VMAlert` spec and environment variable `WATCH_NAMESPACE` for operator.
+
+Following rules are applied:
+
+- If `ruleNamespaceSelector` and `ruleSelector` both undefined, then by default select nothing. With option set - `spec.selectAllByDefault: true`, select all vmrules.
+- If `ruleNamespaceSelector` defined, `ruleSelector` undefined, then all vmrules are matching at namespaces for given `ruleNamespaceSelector`.
+- If `ruleNamespaceSelector` undefined, `ruleSelector` defined, then all vmrules at `VMAgent`'s namespaces are matching for given `ruleSelector`.
+- If `ruleNamespaceSelector` and `ruleSelector` both defined, then only vmrules at namespaces matched `ruleNamespaceSelector` for given `ruleSelector` are matching.
+
+Here's a more visual and more detailed view:
+
+| `ruleNamespaceSelector` | `ruleSelector` | `selectAllByDefault` | `WATCH_NAMESPACE` | Selected rules |
+|-------------------------|----------------|----------------------|-------------------|------------------------------------------------------------------------------------------------------|
+| undefined | undefined | false | undefined | nothing |
+| undefined | undefined | **true** | undefined | all vmrules in the cluster |
+| **defined** | undefined | any | undefined | all vmrules are matching at namespaces for given `ruleNamespaceSelector` |
+| undefined | **defined** | any | undefined | all vmrules only at `VMAlert`'s namespace are matching for given `ruleSelector` |
+| **defined** | **defined** | any | undefined | all vmrules only at namespaces matched `ruleNamespaceSelector` for given `ruleSelector` are matching |
+| any | undefined | any | **defined** | all vmrules only at `VMAlert`'s namespace |
+| any | **defined** | any | **defined** | all vmrules only at `VMAlert`'s namespace for given `ruleSelector` are matching |
+
+More details about `WATCH_NAMESPACE` variable you can read in [this doc](../configuration.md#namespaced-mode).
+
+Here are some examples of `VMAlert` configuration with selectors:
+
+```yaml
+# select all rule objects in the cluster
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlert
+metadata:
+ name: vmalert-select-all
+spec:
+ # ...
+ selectAllByDefault: true
+
+---
+
+# select all rule objects in specific namespace (my-namespace)
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlert
+metadata:
+ name: vmalert-select-ns
+spec:
+ # ...
+ ruleNamespaceSelector:
+ matchLabels:
+ kubernetes.io/metadata.name: my-namespace
+```
+
+## High availability
+
+`VMAlert` can be launched with multiple replicas without an additional configuration as far [alertmanager](./vmalertmanager.md) is responsible for alert deduplication.
+
+Note, if you want to use `VMAlert` with high-available [`VMAlertmanager`](./vmalertmanager.md), which has more than 1 replica.
+You have to specify all pod fqdns at `VMAlert.spec.notifiers.[url]`. Or you can use service discovery for notifier, examples:
+
+- alertmanager:
+ ```yaml
+ apiVersion: v1
+ kind: Secret
+ metadata:
+ name: vmalertmanager-example-alertmanager
+ labels:
+ app: vm-operator
+ type: Opaque
+ stringData:
+ alertmanager.yaml: |
+ global:
+ resolve_timeout: 5m
+ route:
+ group_by: ['job']
+ group_wait: 30s
+ group_interval: 5m
+ repeat_interval: 12h
+ receiver: 'webhook'
+ receivers:
+ - name: 'webhook'
+ webhook_configs:
+ - url: 'http://alertmanagerwh:30500/'
+ # ...
+
+ ---
+
+ apiVersion: operator.victoriametrics.com/v1beta1
+ kind: VMAlertmanager
+ metadata:
+ name: example
+ namespace: default
+ labels:
+ usage: dedicated
+ spec:
+ replicaCount: 2
+ configSecret: vmalertmanager-example-alertmanager
+ configSelector: {}
+ configNamespaceSelector: {}
+ # ...
+ ```
+- vmalert with fqdns:
+ ```yaml
+ apiVersion: operator.victoriametrics.com/v1beta1
+ kind: VMAlert
+ metadata:
+ name: example-ha
+ namespace: default
+ spec:
+ replicaCount: 2
+ datasource:
+ url: http://vmsingle-example.default.svc:8429
+ notifiers:
+ - url: http://vmalertmanager-example-0.vmalertmanager-example.default.svc:9093
+ - url: http://vmalertmanager-example-1.vmalertmanager-example.default.svc:9093
+ evaluationInterval: "10s"
+ ruleSelector: {}
+ # ...
+ ```
+- vmalert with service discovery:
+ ```yaml
+ apiVersion: operator.victoriametrics.com/v1beta1
+ kind: VMAlert
+ metadata:
+ name: example-ha
+ namespace: default
+ spec:
+ replicaCount: 2
+ datasource:
+ url: http://vmsingle-example.default.svc:8429
+ notifiers:
+ - selector:
+ namespaceSelector:
+ matchNames:
+ - default
+ labelSelector:
+ matchLabels:
+ usage: dedicated
+ evaluationInterval: "10s"
+ ruleSelector: {}
+ # ...
+ ```
+
+In addition, you need to specify `remoteWrite` and `remoteRead` urls for restoring alert states after restarts:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlert
+metadata:
+ name: example-ha
+ namespace: default
+spec:
+ replicaCount: 2
+ evaluationInterval: "10s"
+ selectAllByDefault: true
+ datasource:
+ url: http://vmselect-demo.vm.svc:8481/select/0/prometheus
+ notifiers:
+ - url: http://vmalertmanager-example-0.vmalertmanager-example.default.svc:9093
+ - url: http://vmalertmanager-example-1.vmalertmanager-example.default.svc:9093
+ remoteWrite:
+ url: http://vminsert-demo.vm.svc:8480/insert/0/prometheus
+ remoteRead:
+ url: http://vmselect-demo.vm.svc:8481/select/0/prometheus
+```
+
+More details about `remoteWrite` and `remoteRead` you can read in [vmalert docs](https://docs.victoriametrics.com/vmalert.html#alerts-state-on-restarts).
+
+## Version management
+
+To set `VMAlert` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlert
+metadata:
+ name: example-vmalert
+spec:
+ image:
+ repository: victoriametrics/victoria-metrics
+ tag: v1.93.4
+ pullPolicy: Always
+ # ...
+```
+
+Also, you can specify `imagePullSecrets` if you are pulling images from private repo:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlert
+metadata:
+ name: example-vmalert
+spec:
+ image:
+ repository: victoriametrics/victoria-metrics
+ tag: v1.93.4
+ pullPolicy: Always
+ imagePullSecrets:
+ - name: my-repo-secret
+# ...
+```
+
+## Enterprise features
+
+VMAlert supports features [Reading rules from object storage](https://docs.victoriametrics.com/vmalert.html#reading-rules-from-object-storage)
+and [Multitenancy](https://docs.victoriametrics.com/vmalert.html#multitenancy)
+from [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
+
+For using Enterprise version of [vmalert](https://docs.victoriametrics.com/vmalert.html)
+you need to change version of `VMAlert` to version with `-enterprise` suffix using [Version management](#version-management).
+
+All the enterprise apps require `-eula` command-line flag to be passed to them.
+This flag acknowledges that your usage fits one of the cases listed on [this page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
+So you can use [extraArgs](./README.md#extra-arguments) for passing this flag to `VMAlert`:
+
+### Reading rules from object storage
+
+After that you can pass `-rule` command-line argument with `s3://` or `gs://`
+to `VMAlert` with [extraArgs](./README.md#extra-arguments).
+
+More details about reading rules from object storage you can read in [vmalert docs](https://docs.victoriametrics.com/vmalert.html#reading-rules-from-object-storage).
+
+Here are complete example for [Reading rules from object storage](https://docs.victoriametrics.com/vmalert.html#reading-rules-from-object-storage):
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlert
+metadata:
+ name: vmalert-ent-example
+spec:
+ # enabling enterprise features
+ image:
+ # enterprise version of vmalert
+ tag: v1.93.5-enterprise
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmalert enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: Reading rules from object storage
+ # more details about reading rules from object storage you can read on https://docs.victoriametrics.com/vmalert.html#reading-rules-from-object-storage
+ rule: s3://bucket/dir/alert.rules
+
+ # ...other fields...
+```
+
+### Multitenancy
+
+After enabling enterprise version you can use [Multitenancy](https://docs.victoriametrics.com/vmalert.html#multitenancy)
+feature in `VMAlert`.
+
+For that you need to set `clusterMode` commad-line flag
+with [extraArgs](./README.md#extra-arguments)
+and specify `tenant` field for groups
+in [VMRule](./vmrule.md#enterprise-features):
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlert
+metadata:
+ name: vmalert-ent-example
+spec:
+ # enabling enterprise features
+ image:
+ # enterprise version of vmalert
+ tag: v1.93.5-enterprise
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmalert enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: Multitenancy
+ # more details about multitenancy you can read on https://docs.victoriametrics.com/vmalert.html#multitenancy
+ clusterMode: true
+
+ # ...other fields...
+
+---
+
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMRule
+metadata:
+ name: vmrule-ent-example
+spec:
+ groups:
+ - name: vmalert-1
+ rules:
+ # using enterprise features: Multitenancy
+ # more details about multitenancy you can read on https://docs.victoriametrics.com/vmalert.html#multitenancy
+ - tenant: 1
+ alert: vmalert config reload error
+ expr: delta(vmalert_config_last_reload_errors_total[5m]) > 0
+ for: 10s
+ labels:
+ severity: major
+ job: "{{ $labels.job }}"
+ annotations:
+ value: "{{ $value }}"
+ description: 'error reloading vmalert config, reload count for 5 min {{ $value }}'
+```
+
+## Examples
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlert
+metadata:
+ name: example-vmalert
+spec:
+ replicaCount: 1
+ datasource:
+ url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429"
+ notifier:
+ url: "http://vmalertmanager-example-alertmanager.default.svc:9093"
+ evaluationInterval: "30s"
+ selectAllByDefault: true
+```
diff --git a/docs/operator/resources/vmalertmanager.md b/docs/operator/resources/vmalertmanager.md
new file mode 100644
index 000000000..559c8b5d8
--- /dev/null
+++ b/docs/operator/resources/vmalertmanager.md
@@ -0,0 +1,270 @@
+# VMAlertmanager
+
+`VMAlertmanager` - represents [alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/) configuration.
+
+The `VMAlertmanager` CRD declaratively defines a desired Alertmanager setup to run in a Kubernetes cluster.
+It provides options to configure replication and persistent storage.
+
+For each `Alertmanager` resource, the Operator deploys a properly configured `StatefulSet` in the same namespace.
+The Alertmanager pods are configured to include a `Secret` called `` which holds the used
+configuration file in the key `alertmanager.yaml`.
+
+When there are two or more configured replicas the Operator runs the Alertmanager instances in high availability mode.
+
+## Specification
+
+You can see the full actual specification of the `VMAlertmanager` resource in the **[API docs -> VMAlert](../api.md#vmalertmanager)**.
+
+If you can't find necessary field in the specification of the custom resource,
+see [Extra arguments section](./README.md#extra-arguments).
+
+Also, you can check out the [examples](#examples) section.
+
+## Configuration
+
+The operator generates a configuration file for `VMAlertmanager` based on user input at the definition of `CRD`.
+
+Generated config stored at `Secret` created by the operator, it has the following name template `vmalertmanager-CRD_NAME-config`.
+
+This configuration file is mounted at `VMAlertmanager` `Pod`. A special side-car container tracks its changes and sends config-reload signals to `alertmanager` container.
+
+### Using secret
+
+Basically, you can use the global configuration defined at manually created `Secret`. This `Secret` must be created before `VMAlertmanager`.
+
+Name of the `Secret` must be defined at `VMAlertmanager` `spec.configSecret` option:
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+ name: vmalertmanager-example-alertmanager
+ labels:
+ app: vm-operator
+type: Opaque
+stringData:
+ alertmanager.yaml: |
+ global:
+ resolve_timeout: 5m
+ route:
+ receiver: 'webhook'
+ receivers:
+ - name: 'webhook'
+ webhook_configs:
+ - url: 'http://alertmanagerwh:30500/'
+
+---
+
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlertmanager
+metadata:
+ name: example-alertmanager
+spec:
+ replicaCount: 2
+ configSecret: vmalertmanager-example-alertmanager
+```
+
+### Using inline raw config
+
+Also, if there is no secret data at configuration, or you just want to redefine some global variables for `alertmanager`.
+You can define configuration at `spec.configRawYaml` section of `VMAlertmanager` configuration:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlertmanager
+metadata:
+ name: example-alertmanager
+spec:
+ replicaCount: 2
+ configRawYaml: |
+ global:
+ resolve_timeout: 5m
+ route:
+ receiver: 'default'
+ group_interval: 5m
+ repeat_interval: 12h
+ receivers:
+ - name: 'default'
+```
+
+If both `configSecret` and `configRawYaml` are defined, only configuration from `configRawYaml` will be used. Values from `configRawYaml` will be ignored.
+
+### Using VMAlertmanagerConfig
+
+See details at [VMAlertmanagerConfig](./vmalertmanagerconfig.md).
+
+The CRD specifies which `VMAlertmanagerConfig`s should be covered by the deployed `VMAlertmanager` instances based on label selection.
+The Operator then generates a configuration based on the included `VMAlertmanagerConfig`s and updates the `Configmaps` containing
+the configuration. It continuously does so for all changes that are made to `VMAlertmanagerConfig`s or to the `VMAlertmanager` resource itself.
+
+Configs are filtered by selectors `configNamespaceSelector` and `configSelector` in `VMAlertmanager` CRD definition.
+For selecting rules from all namespaces you must specify it to empty value:
+
+```yaml
+spec:
+ configNamespaceSelector: {}
+```
+
+[VMAlertmanagerConfig](./vmalertmanagerconfig.md) objects are
+generates part of [VMAlertmanager](./vmalertmanager.md) configuration.
+
+For filtering rules `VMAlertmanager` uses selectors `configNamespaceSelector` and `configSelector`.
+It allows configuring rules access control across namespaces and different environments.
+Specification of selectors you can see in [this doc](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#labelselector-v1-meta).
+
+In addition to the above selectors, the filtering of objects in a cluster is affected by the field `selectAllByDefault`
+of `VMAlertmanager` spec and environment variable `WATCH_NAMESPACE` for operator.
+
+Following rules are applied:
+
+- If `configNamespaceSelector` and `configSelector` both undefined, then by default select nothing. With option set - `spec.selectAllByDefault: true`, select all vmalertmanagerconfigs.
+- If `configNamespaceSelector` defined, `configSelector` undefined, then all vmalertmaangerconfigs are matching at namespaces for given `configNamespaceSelector`.
+- If `configNamespaceSelector` undefined, `configSelector` defined, then all vmalertmaangerconfigs at `VMAgent`'s namespaces are matching for given `configSelector`.
+- If `configNamespaceSelector` and `configSelector` both defined, then only vmalertmaangerconfigs at namespaces matched `configNamespaceSelector` for given `configSelector` are matching.
+
+Here's a more visual and more detailed view:
+
+| `configNamespaceSelector` | `configSelector` | `selectAllByDefault` | `WATCH_NAMESPACE` | Selected rules |
+|---------------------------|------------------|----------------------|-------------------|------------------------------------------------------------------------------------------------------------------------|
+| undefined | undefined | false | undefined | nothing |
+| undefined | undefined | **true** | undefined | all vmalertmaangerconfigs in the cluster |
+| **defined** | undefined | any | undefined | all vmalertmaangerconfigs are matching at namespaces for given `configNamespaceSelector` |
+| undefined | **defined** | any | undefined | all vmalertmaangerconfigs only at `VMAlertmanager`'s namespace are matching for given `ruleSelector` |
+| **defined** | **defined** | any | undefined | all vmalertmaangerconfigs only at namespaces matched `configNamespaceSelector` for given `configSelector` are matching |
+| any | undefined | any | **defined** | all vmalertmaangerconfigs only at `VMAlertmanager`'s namespace |
+| any | **defined** | any | **defined** | all vmalertmaangerconfigs only at `VMAlertmanager`'s namespace for given `configSelector` are matching |
+
+More details about `WATCH_NAMESPACE` variable you can read in [this doc](../configuration.md#namespaced-mode).
+
+Here are some examples of `VMAlertmanager` configuration with selectors:
+
+```yaml
+# select all config objects in the cluster
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlertmanager
+metadata:
+ name: vmalertmanager-select-all
+spec:
+ # ...
+ selectAllByDefault: true
+
+---
+
+# select all config objects in specific namespace (my-namespace)
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlertmanager
+metadata:
+ name: vmalertmanager-select-ns
+spec:
+ # ...
+ configNamespaceSelector:
+ matchLabels:
+ kubernetes.io/metadata.name: my-namespace
+```
+
+### Extra configuration files
+
+`VMAlertmanager` specification has the following fields, that can be used to configure without editing raw configuration file:
+
+- `spec.templates` - list of keys in `ConfigMaps`, that contains template files for `alertmanager`, e.g.:
+
+ ```yaml
+ apiVersion: operator.victoriametrics.com/v1beta1
+ kind: VMAlertmanager
+ metadata:
+ name: example-alertmanager
+ spec:
+ replicaCount: 2
+ templates:
+ - Name: alertmanager-templates
+ Key: my-template-1.tmpl
+ - Name: alertmanager-templates
+ Key: my-template-2.tmpl
+ ---
+ apiVersion: v1
+ kind: ConfigMap
+ metadata:
+ name: alertmanager-templates
+ data:
+ my-template-1.tmpl: |
+ {{ define "hello" -}}
+ hello, Victoria!
+ {{- end }}
+ my-template-2.tmpl: """
+ ```
+
+These templates will be automatically added to `VMAlertmanager` configuration and will be automatically reloaded on changes in source `ConfigMap`.
+- `spec.configMaps` - list of `ConfigMap` names (in the same namespace) that will be mounted at `VMAlertmanager`
+ workload and will be automatically reloaded on changes in source `ConfigMap`. Mount path is `/etc/vm/configs/`.
+
+### Behavior without provided config
+
+If no configuration is provided, operator configures stub configuration with blackhole route.
+
+## High Availability
+
+The final step of the high availability scheme is Alertmanager, when an alert triggers, actually fire alerts against *all* instances of an Alertmanager cluster.
+
+The Alertmanager, starting with the `v0.5.0` release, ships with a high availability mode.
+It implements a gossip protocol to synchronize instances of an Alertmanager cluster
+regarding notifications that have been sent out, to prevent duplicate notifications.
+It is an AP (available and partition tolerant) system. Being an AP system means that notifications are guaranteed to be sent at least once.
+
+The Victoria Metrics Operator ensures that Alertmanager clusters are properly configured to run highly available on Kubernetes.
+
+## Version management
+
+To set `VMAlertmanager` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlertmanager
+metadata:
+ name: example-vmalertmanager
+spec:
+ image:
+ repository: prom/alertmanager
+ tag: v0.25.0
+ pullPolicy: Always
+ # ...
+```
+
+Also, you can specify `imagePullSecrets` if you are pulling images from private repo:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlertmanager
+metadata:
+ name: example-vmalertmanager
+spec:
+ image:
+ repository: prom/alertmanager
+ tag: v0.25.0
+ pullPolicy: Always
+ imagePullSecrets:
+ - name: my-repo-secret
+# ...
+```
+
+## Examples
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlertmanager
+metadata:
+ name: vmalertmanager-example
+spec:
+ replicaCount: 1
+ configRawYaml: |
+ global:
+ resolve_timeout: 5m
+ route:
+ group_wait: 30s
+ group_interval: 5m
+ repeat_interval: 12h
+ receiver: 'webhook'
+ receivers:
+ - name: 'webhook'
+ webhook_configs:
+ - url: 'http://localhost:30502/'
+```
diff --git a/docs/operator/resources/vmalertmanagerconfig.md b/docs/operator/resources/vmalertmanagerconfig.md
new file mode 100644
index 000000000..a615b72df
--- /dev/null
+++ b/docs/operator/resources/vmalertmanagerconfig.md
@@ -0,0 +1,101 @@
+# VMAlertmanagerConfig
+
+The `VMAlertmanagerConfig` provides way to configure [VMAlertmanager](./vmalertmanager.md)
+configuration with CRD. It allows to define different configuration parts, which will be merged by operator into config.
+
+It behaves like other config parts - `VMServiceScrape` and etc.
+
+Read [Usage](#usage) and [Special case](#special-case) before using.
+
+## Specification
+
+You can see the full actual specification of the `VMAlertmanagerConfig` resource in
+the **[API docs -> VMAlertmanagerConfig](../api.md#vmalertmanagerconfig)**.
+
+Also, you can check out the [examples](#examples) section.
+
+## Usage
+
+`VMAlertmanagerConfig` allows delegating notification configuration to the kubernetes cluster users.
+The application owner may configure notifications by defining it at `VMAlertmanagerConfig`.
+
+With the combination of `VMRule` and `VMServiceScrape` it allows delegating configuration observability to application owners, and uses popular `GitOps` practice.
+
+Operator combines `VMAlertmanagerConfig`s into a single configuration file for `VMAlertmanager`.
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlertmanagerConfig
+metadata:
+ name: example-email-web
+ namespace: production
+spec:
+ route:
+ receiver: email
+ group_interval: 1m
+ routes:
+ - receiver: email
+ matchers:
+ - {severity =~ "warning|critical", app_name = "blog"}
+ receivers:
+ - name: email
+ email_configs:
+ - to: some-email@example.com
+ from: alerting@example.com
+ smarthost: example.com:25
+ text: ALARM
+```
+
+#### Special Case
+
+VMAlertmanagerConfig has enforced namespace matcher.
+Alerts must have a proper namespace label, with the same value as name of namespace for VMAlertmanagerConfig.
+
+It can be disabled, by setting the following value to the VMAlertmanager: `spec.disableNamespaceMatcher: true`.
+
+## Examples
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAlertmanagerConfig
+metadata:
+ name: example
+ namespace: default
+spec:
+ inhibit_rules:
+ - equals: []
+ target_matchers: []
+ source_matchers: []
+ route:
+ routes:
+ - receiver: webhook
+ continue: true
+ receiver: email
+ group_by: []
+ continue: false
+ matchers:
+ - job = "alertmanager"
+ group_wait: 30s
+ group_interval: 45s
+ repeat_interval: 1h
+ mute_time_intervals:
+ - name: base
+ time_intervals:
+ - times:
+ - start_time: ""
+ end_time: ""
+ weekdays: []
+ days_of_month: []
+ months: []
+ years: []
+ receivers:
+ email_configs: []
+ webhook_configs:
+ - url: http://some-other-wh
+ pagerduty_configs: []
+ pushover_configs: []
+ slack_configs: []
+ opsgenie_configs: []
+ victorops_configs: []
+ wechat_configs: []
+```
diff --git a/docs/operator/resources/vmauth.md b/docs/operator/resources/vmauth.md
new file mode 100644
index 000000000..d3aa3cc72
--- /dev/null
+++ b/docs/operator/resources/vmauth.md
@@ -0,0 +1,237 @@
+# VMAuth
+
+The `VMAuth` CRD provides mechanism for exposing application with authorization to outside world or to other applications inside kubernetes cluster.
+
+For first case, user can configure `ingress` setting at `VMAuth` CRD. For second one, operator will create secret with `username` and `password` at `VMUser` CRD name.
+So it will be possible to access these credentials from any application by targeting corresponding kubernetes secret.
+
+## Specification
+
+You can see the full actual specification of the `VMAuth` resource in
+the **[API docs -> VMAuth](../api.md#vmauth)**.
+
+If you can't find necessary field in the specification of the custom resource,
+see [Extra arguments section](./README.md#extra-arguments).
+
+Also, you can check out the [examples](#examples) section.
+
+## Users
+
+The CRD specifies which `VMUser`s should be covered by the deployed `VMAuth` instances based on label selection.
+The Operator then generates a configuration based on the included `VMUser`s and updates the `Configmaps` containing
+the configuration. It continuously does so for all changes that are made to `VMUser`s or to the `VMAuth` resource itself.
+
+[VMUser](./vmrule.md) objects generate part of `VMAuth` configuration.
+
+For filtering users `VMAuth` uses selectors `userNamespaceSelector` and `userSelector`.
+It allows configuring rules access control across namespaces and different environments.
+Specification of selectors you can see in [this doc](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#labelselector-v1-meta).
+
+In addition to the above selectors, the filtering of objects in a cluster is affected by the field `selectAllByDefault` of `VMAuth` spec and environment variable `WATCH_NAMESPACE` for operator.
+
+Following rules are applied:
+
+- If `userNamespaceSelector` and `userSelector` both undefined, then by default select nothing. With option set - `spec.selectAllByDefault: true`, select all vmusers.
+- If `userNamespaceSelector` defined, `userSelector` undefined, then all vmusers are matching at namespaces for given `userNamespaceSelector`.
+- If `userNamespaceSelector` undefined, `userSelector` defined, then all vmusers at `VMAgent`'s namespaces are matching for given `userSelector`.
+- If `userNamespaceSelector` and `userSelector` both defined, then only vmusers at namespaces matched `userNamespaceSelector` for given `userSelector` are matching.
+
+Here's a more visual and more detailed view:
+
+| `userNamespaceSelector` | `userSelector` | `selectAllByDefault` | `WATCH_NAMESPACE` | Selected rules |
+|-------------------------|----------------|----------------------|-------------------|------------------------------------------------------------------------------------------------------|
+| undefined | undefined | false | undefined | nothing |
+| undefined | undefined | **true** | undefined | all vmusers in the cluster |
+| **defined** | undefined | any | undefined | all vmusers are matching at namespaces for given `userNamespaceSelector` |
+| undefined | **defined** | any | undefined | all vmusers only at `VMAuth`'s namespace are matching for given `userSelector` |
+| **defined** | **defined** | any | undefined | all vmusers only at namespaces matched `userNamespaceSelector` for given `userSelector` are matching |
+| any | undefined | any | **defined** | all vmusers only at `VMAuth`'s namespace |
+| any | **defined** | any | **defined** | all vmusers only at `VMAuth`'s namespace for given `userSelector` are matching |
+
+More details about `WATCH_NAMESPACE` variable you can read in [this doc](../configuration.md#namespaced-mode).
+
+Here are some examples of `VMAuth` configuration with selectors:
+
+```yaml
+# select all user objects in the cluster
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAuth
+metadata:
+ name: vmauth-select-all
+spec:
+ # ...
+ selectAllByDefault: true
+
+---
+
+# select all user objects in specific namespace (my-namespace)
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAuth
+metadata:
+ name: vmauth-select-ns
+spec:
+ # ...
+ userNamespaceSelector:
+ matchLabels:
+ kubernetes.io/metadata.name: my-namespace
+```
+
+## Unauthorized access
+
+You can configure `VMAuth` to allow unauthorized access for specified routes with `unauthorizedAccessConfig` field.
+
+For instance:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAuth
+metadata:
+ name: vmauth-unauthorized-example
+spec:
+ unauthorizedAccessConfig:
+ - paths: ["/metrics"]
+ urls:
+ - http://vmsingle-example.default.svc:8428
+```
+
+In this example every user can access `/metrics` route and get vmsingle metrics without authorization.
+
+In addition, `unauthorizedAccessConfig` in [Enterprise version](#enterprise-features) supports [IP Filters](#ip-filters)
+with `ip_filters` field.
+
+## High availability
+
+The `VMAuth` resource is stateless, so it can be scaled horizontally by increasing the number of replicas:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAuth
+metadata:
+ name: vmauth-example
+spec:
+ replicas: 3
+ # ...
+```
+
+## Version management
+
+To set `VMAuth` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAuth
+metadata:
+ name: example-vmauth
+spec:
+ image:
+ repository: victoriametrics/victoria-metrics
+ tag: v1.93.4
+ pullPolicy: Always
+ # ...
+```
+
+Also, you can specify `imagePullSecrets` if you are pulling images from private repo:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAuth
+metadata:
+ name: example-vmauth
+spec:
+ image:
+ repository: victoriametrics/victoria-metrics
+ tag: v1.93.4
+ pullPolicy: Always
+ imagePullSecrets:
+ - name: my-repo-secret
+# ...
+```
+
+## Enterprise features
+
+Custom resource `VMAuth` supports feature [IP filters](https://docs.victoriametrics.com/vmauth.html#ip-filters)
+from [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
+
+For using Enterprise version of [vmauth](https://docs.victoriametrics.com/vmauth.html)
+you need to change version of `vmauth` to version with `-enterprise` suffix using [Version management](#version-management).
+
+All the enterprise apps require `-eula` command-line flag to be passed to them.
+This flag acknowledges that your usage fits one of the cases listed on [this page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
+So you can use [extraArgs](./README.md#extra-arguments) for passing this flag to `VMAuth`:
+
+### IP Filters
+
+After that you can use [IP filters for `VMUser`](./vmuser.md#enterprise-features)
+and field `ip_filters` for `VMAuth`.
+
+Here are complete example with described above:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAuth
+metadata:
+ name: vmauth-ent-example
+spec:
+ # enabling enterprise features
+ image:
+ # enterprise version of vmauth
+ tag: v1.93.5-enterprise
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmauth enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: ip filters for vmauth
+ # more details about ip filters you can read in https://docs.victoriametrics.com/vmauth.html#ip-filters
+ ip_filters:
+ allow_list:
+ - 10.0.0.0/24
+ - 1.2.3.4
+ deny_list:
+ - 5.6.7.8
+ # allow read vmsingle metrics without authorization for users from internal network
+ unauthorizedAccessConfig:
+ - paths: ["/metrics"]
+ urls: ["http://vmsingle-example.default.svc:8428"]
+ ip_filters:
+ allow_list:
+ - 192.168.0.0/16
+ - 10.0.0.0/8
+
+ # ...other fields...
+
+---
+
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMUser
+metadata:
+ name: vmuser-ent-example
+spec:
+ username: simple-user
+ password: simple-password
+
+ # using enterprise features: ip filters for vmuser
+ # more details about ip filters you can read in https://docs.victoriametrics.com/vmuser.html#enterprise-features
+ ip_filters:
+ allow_list:
+ - 10.0.0.0/24
+ - 1.2.3.4
+ deny_list:
+ - 5.6.7.8
+```
+
+## Examples
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAuth
+metadata:
+ name: example
+ namespace: default
+spec:
+ selectAllByDefault: true
+ ingress:
+ class_name: nginx # <-- change this to your ingress-controller
+ host: vm-demo.k8s.orb.local # <-- change this to your domain
+```
diff --git a/docs/operator/resources/vmcluster.md b/docs/operator/resources/vmcluster.md
new file mode 100644
index 000000000..722271277
--- /dev/null
+++ b/docs/operator/resources/vmcluster.md
@@ -0,0 +1,612 @@
+# VMCluster
+
+`VMCluster` represents a high-available and fault-tolerant version of VictoriaMetrics database.
+The `VMCluster` CRD defines a [cluster version VM](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html).
+
+For each `VMCluster` resource, the Operator creates:
+
+- `VMStorage` as `StatefulSet`,
+- `VMSelect` as `StatefulSet`
+- and `VMInsert` as deployment.
+
+For `VMStorage` and `VMSelect` headless services are created. `VMInsert` is created as service with clusterIP.
+
+There is a strict order for these objects creation and reconciliation:
+
+1. `VMStorage` is synced - the Operator waits until all its pods are ready;
+1. Then it syncs `VMSelect` with the same manner;
+1. `VMInsert` is the last object to sync.
+
+All [statefulsets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) are created
+with [OnDelete](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#on-delete) update type.
+It allows to manually manage the rolling update process for Operator by deleting pods one by one and waiting for the ready status.
+
+Rolling update process may be configured by the operator env variables.
+The most important is `VM_PODWAITREADYTIMEOUT=80s` - it controls how long to wait for pod's ready status.
+
+## Specification
+
+You can see the full actual specification of the `VMCluster` resource in the **[API docs -> VMCluster](../api.md#vmcluster)**.
+
+If you can't find necessary field in the specification of the custom resource,
+see [Extra arguments section](./README.md#extra-arguments).
+
+Also, you can check out the [examples](#examples) section.
+
+## High availability
+
+The cluster version provides a full set of high availability features - metrics replication, node failover, horizontal scaling.
+
+First, we recommend familiarizing yourself with the high availability tools provided by "VictoriaMetrics Cluster" itself:
+
+- [High availability](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#high-availability),
+- [Cluster availability](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#cluster-availability),
+- [Replication and data safety](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#replication-and-data-safety).
+
+`VMCluster` supports all listed in the above-mentioned articles parameters and features:
+
+- `replicationFactor` - the number of replicas for each metric.
+- for every component of cluster (`vmstorage` / `vmselect` / `vminsert`):
+ - `replicaCount` - the number of replicas for components of cluster.
+ - `affinity` - the affinity (the pod's scheduling constraints) for components pods. See more details in [kubernetes docs](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity).
+ - `topologySpreadConstraints` - controls how pods are spread across your cluster among failure-domains such as regions, zones, nodes, and other user-defined topology domains. See more details in [kubernetes docs](https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/).
+
+In addition, operator:
+
+- uses k8s services or vmauth for load balancing between `vminsert` and `vmselect` components,
+- uses health checks for to determine the readiness of components for work after restart,
+- allows to horizontally scale all cluster components just by changing `replicaCount` field.
+
+Here is an example of a `VMCluster` resource with HA features:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMCluster
+metadata:
+ name: example-vmcluster-persistent
+spec:
+ replicationFactor: 2
+ vmstorage:
+ replicaCount: 10
+ storageDataPath: "/vm-data"
+ affinity:
+ podAntiAffinity:
+ requiredDuringSchedulingIgnoredDuringExecution:
+ - labelSelector:
+ matchExpressions:
+ - key: "app.kubernetes.io/name"
+ operator: In
+ values:
+ - "vmstorage"
+ topologyKey: "kubernetes.io/hostname"
+ storage:
+ volumeClaimTemplate:
+ spec:
+ resources:
+ requests:
+ storage: 10Gi
+ resources:
+ limits:
+ cpu: "2"
+ memory: 2048Mi
+ vmselect:
+ replicaCount: 3
+ cacheMountPath: "/select-cache"
+ affinity:
+ podAntiAffinity:
+ requiredDuringSchedulingIgnoredDuringExecution:
+ - labelSelector:
+ matchExpressions:
+ - key: "app.kubernetes.io/name"
+ operator: In
+ values:
+ - "vmselect"
+ topologyKey: "kubernetes.io/hostname"
+ storage:
+ volumeClaimTemplate:
+ spec:
+ resources:
+ requests:
+ storage: 2Gi
+ resources:
+ limits:
+ cpu: "1"
+ memory: "500Mi"
+ vminsert:
+ replicaCount: 4
+ affinity:
+ podAntiAffinity:
+ requiredDuringSchedulingIgnoredDuringExecution:
+ - labelSelector:
+ matchExpressions:
+ - key: "app.kubernetes.io/name"
+ operator: In
+ values:
+ - "vminsert"
+ topologyKey: "kubernetes.io/hostname"
+ resources:
+ limits:
+ cpu: "1"
+ memory: "500Mi"
+```
+
+## Version management
+
+For `VMCluster` you can specify tag name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) and repository setting per cluster object:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMCluster
+metadata:
+ name: example-vmcluster
+spec:
+ vmstorage:
+ replicaCount: 2
+ image:
+ repository: victoriametrics/vmstorage
+ tag: v1.93.4-cluster
+ pullPolicy: Always
+ vmselect:
+ replicaCount: 2
+ image:
+ repository: victoriametrics/vmselect
+ tag: v1.93.4-cluster
+ pullPolicy: Always
+ vminsert:
+ replicaCount: 2
+ image:
+ repository: victoriametrics/vminsert
+ tag: v1.93.4-cluster
+ pullPolicy: Always
+```
+
+Also, you can specify `imagePullSecrets` if you are pulling images from private repo,
+but `imagePullSecrets` is global setting for all `VMCluster` specification:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMCluster
+metadata:
+ name: example-vmcluster
+spec:
+ vmstorage:
+ replicaCount: 2
+ image:
+ repository: victoriametrics/vmstorage
+ tag: v1.93.4-cluster
+ pullPolicy: Always
+ vmselect:
+ replicaCount: 2
+ image:
+ repository: victoriametrics/vmselect
+ tag: v1.93.4-cluster
+ pullPolicy: Always
+ vminsert:
+ replicaCount: 2
+ image:
+ repository: victoriametrics/vminsert
+ tag: v1.93.4-cluster
+ pullPolicy: Always
+ imagePullSecrets:
+ - name: my-repo-secret
+ # ...
+```
+
+## Enterprise features
+
+VMCluster supports following features
+from [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise):
+
+- [Downsampling](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#downsampling)
+- [Multiple retentions / Retention filters](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#retention-filters)
+- [Advanced per-tenant statistic](https://docs.victoriametrics.com/PerTenantStatistic.html)
+- [mTLS for cluster components](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection)
+- [Backup automation](https://docs.victoriametrics.com/vmbackupmanager.html)
+
+VMCluster doesn't support yet feature
+[Automatic discovery for vmstorage nodes](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#automatic-vmstorage-discovery).
+
+For using Enterprise version of [vmcluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html)
+you need to change version of `VMCluster` to version with `-enterprise` suffix using [Version management](#version-management).
+
+All the enterprise apps require `-eula` command-line flag to be passed to them.
+This flag acknowledges that your usage fits one of the cases listed on [this page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
+So you can use [extraArgs](./README.md#extra-arguments) for passing this flag to `VMCluster`.
+
+### Downsampling
+
+After that you can pass [Downsampling](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#downsampling)
+flag to `VMCluster/vmselect` and `VMCluster/vmstorage` with [extraArgs](./README.md#extra-arguments) too.
+
+Here are complete example for [Downsampling](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#downsampling):
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMCluster
+metadata:
+ name: vmcluster-ent-example
+spec:
+
+ vmselect:
+ # enabling enterprise features for vmselect
+ image:
+ # enterprise version of vmselect
+ tag: v1.93.5-enterprise-cluster
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmselect enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: Downsampling
+ # more details about downsampling you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#downsampling
+ downsampling.period: 30d:5m,180d:1h,1y:6h,2y:1d
+
+ vmstorage:
+ # enabling enterprise features for vmstorage
+ image:
+ # enterprise version of vmstorage
+ tag: v1.93.5-enterprise-cluster
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmstorage enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: Downsampling
+ # more details about downsampling you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#downsampling
+ downsampling.period: 30d:5m,180d:1h,1y:6h,2y:1d
+
+ # ...other fields...
+```
+
+### Retention filters
+
+You can pass [Retention filters](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#retention-filters)
+flag to `VMCluster/vmstorage` with [extraArgs](./README.md#extra-arguments).
+
+Here are complete example for [Retention filters](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#retention-filters):
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMCluster
+metadata:
+ name: vmcluster-ent-example
+spec:
+
+ vmstorage:
+ # enabling enterprise features for vmstorage
+ image:
+ # enterprise version of vmstorage
+ tag: v1.93.5-enterprise-cluster
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmstorage enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: Retention filters
+ # more details about retention filters you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#retention-filters
+ retentionFilter: '{vm_account_id="5",env="dev"}:5d,{vm_account_id="5",env="prod"}:5y'
+
+ # ...other fields...
+```
+
+### Advanced per-tenant statistic
+
+For using [Advanced per-tenant statistic](https://docs.victoriametrics.com/PerTenantStatistic.html)
+you only need to [enable Enterprise version of vmcluster components](#enterprise-features)
+and operator will automatically create
+[Scrape objects](./vmagent.md#scraping) for cluster components.
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMCluster
+metadata:
+ name: vmcluster-ent-example
+spec:
+
+ vmselect:
+ # enabling enterprise features for vmselect
+ image:
+ # enterprise version of vmselect
+ tag: v1.93.5-enterprise-cluster
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmselect enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ vminsert:
+ # enabling enterprise features for vminsert
+ image:
+ # enterprise version of vminsert
+ tag: v1.93.5-enterprise-cluster
+ extraArgs:
+ # should be true and means that you have the legal right to run a vminsert enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ vmstorage:
+ # enabling enterprise features for vmstorage
+ image:
+ # enterprise version of vmstorage
+ tag: v1.93.5-enterprise-cluster
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmstorage enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # ...other fields...
+```
+
+After that [VMAgent](./vmagent.md) will automatically
+scrape [Advanced per-tenant statistic](https://docs.victoriametrics.com/PerTenantStatistic.html) for cluster components.
+
+### mTLS protection
+
+You can pass [mTLS protection](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection)
+flags to `VMCluster/vmstorage`, `VMCluster/vmselect` and `VMCluster/vminsert` with [extraArgs](./README.md#extra-arguments) and mount secret files
+with `extraVolumes` and `extraVolumeMounts` fields.
+
+Here are complete example for [mTLS protection](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection)
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMCluster
+metadata:
+ name: vmcluster-ent-example
+spec:
+
+ vmselect:
+ # enabling enterprise features for vmselect
+ image:
+ # enterprise version of vmselect
+ tag: v1.93.5-enterprise-cluster
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmselect enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: mTLS protection
+ # more details about mTLS protection you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
+ cluster.tls: true
+ cluster.tlsCAFile: /etc/mtls/ca.crt
+ cluster.tlsCertFile: /etc/mtls/vmselect.crt
+ cluster.tlsKeyFile: /etc/mtls/vmselect.key
+ extraVolumes:
+ - name: mtls
+ secret:
+ secretName: mtls
+ extraVolumeMounts:
+ - name: mtls
+ mountPath: /etc/mtls
+
+ vminsert:
+ # enabling enterprise features for vminsert
+ image:
+ # enterprise version of vminsert
+ tag: v1.93.5-enterprise-cluster
+ extraArgs:
+ # should be true and means that you have the legal right to run a vminsert enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: mTLS protection
+ # more details about mTLS protection you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
+ cluster.tls: true
+ cluster.tlsCAFile: /etc/mtls/ca.crt
+ cluster.tlsCertFile: /etc/mtls/vminsert.crt
+ cluster.tlsKeyFile: /etc/mtls/vminsert.key
+ extraVolumes:
+ - name: mtls
+ secret:
+ secretName: mtls
+ extraVolumeMounts:
+ - name: mtls
+ mountPath: /etc/mtls
+
+ vmstorage:
+ # enabling enterprise features for vmstorage
+ image:
+ # enterprise version of vmstorage
+ tag: v1.93.5-enterprise-cluster
+ env:
+ - name: POD
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.name
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmstorage enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: mTLS protection
+ # more details about mTLS protection you can read on https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#mtls-protection
+ cluster.tls: true
+ cluster.tlsCAFile: /etc/mtls/ca.crt
+ cluster.tlsCertFile: /etc/mtls/$(POD).crt
+ cluster.tlsKeyFile: /etc/mtls/$(POD).key
+ extraVolumes:
+ - name: mtls
+ secret:
+ secretName: mtls
+ extraVolumeMounts:
+ - name: mtls
+ mountPath: /etc/mtls
+
+ # ...other fields...
+
+---
+
+apiVersion: v1
+kind: Secret
+metadata:
+ name: mtls
+ namespace: default
+stringData:
+ ca.crt: |
+ -----BEGIN CERTIFICATE-----
+ ...
+ -----END CERTIFICATE-----
+ mtls-vmstorage-0.crt: |
+ -----BEGIN CERTIFICATE-----
+ ...
+ -----END CERTIFICATE-----
+ mtls-vmstorage-0.key: |
+ -----BEGIN PRIVATE KEY-----
+ ...
+ -----END PRIVATE KEY-----
+ mtls-vmstorage-1.crt: |
+ -----BEGIN CERTIFICATE-----
+ ...
+ -----END CERTIFICATE-----
+ mtls-vmstorage-1.key: |
+ -----BEGIN PRIVATE KEY-----
+ ...
+ -----END PRIVATE KEY-----
+ vminsert.crt: |
+ -----BEGIN CERTIFICATE-----
+ ...
+ -----END CERTIFICATE-----
+ vminsert.key: |
+ -----BEGIN PRIVATE KEY-----
+ ...
+ -----END PRIVATE KEY-----
+ vmselect.crt: |
+ -----BEGIN CERTIFICATE-----
+ ...
+ -----END CERTIFICATE-----
+ vmselect.key: |
+ -----BEGIN PRIVATE KEY-----
+ ...
+ -----END PRIVATE KEY-----
+
+```
+
+Example commands for generating certificates you can read
+on [this page](https://gist.github.com/f41gh7/76ed8e5fb1ebb9737fe746bae9175ee6#generate-self-signed-ca-with-key).
+
+### Backup automation
+
+You can check [vmbackupmanager documentation](https://docs.victoriametrics.com/vmbackupmanager.html) for backup automation.
+It contains a description of the service and its features. This section covers vmbackumanager integration in vmoperator.
+
+`VMCluster` has built-in backup configuration, it uses `vmbackupmanager` - proprietary tool for backups.
+It supports incremental backups (hourly, daily, weekly, monthly) with popular object storages (aws s3, google cloud storage).
+
+Here is a complete example for backup configuration:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMCluster
+metadata:
+ name: vmcluster-ent-example
+spec:
+
+ vmstorage:
+ vmBackup:
+ # should be true and means that you have the legal right to run a vmstorage enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ acceptEULA: true
+
+ # using enterprise features: Backup automation
+ # more details about backup automation you can read on https://docs.victoriametrics.com/vmbackupmanager.html
+ destination: "s3://your_bucket/folder"
+ credentialsSecret:
+ name: remote-storage-keys
+ key: credentials
+
+ # ...other fields...
+
+---
+
+apiVersion: v1
+kind: Secret
+metadata:
+ name: remote-storage-keys
+type: Opaque
+stringData:
+ credentials: |-
+ [default]
+ aws_access_key_id = your_access_key_id
+ aws_secret_access_key = your_secret_access_key
+```
+
+**NOTE**: for cluster version operator adds suffix for destination: `"s3://your_bucket/folder"`, it becomes `"s3://your_bucket/folder/$(POD_NAME)"`.
+It's needed to make consistent backups for each storage node.
+
+You can read more about backup configuration options and mechanics [here](https://docs.victoriametrics.com/vmbackupmanager.html)
+
+Possible configuration options for backup crd can be found at [link](../api.md#vmbackup)
+
+**Using VMBackupmanager for restoring backups** in Kubernetes environment is described [here](https://docs.victoriametrics.com/vmbackupmanager.html#how-to-restore-in-kubernetes).
+
+Also see VMCLuster example spec [here](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmcluster_with_backuper.yaml).
+
+## Examples
+
+### Minimal example without persistence
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMCluster
+metadata:
+ name: vmcluster-example-minimal
+spec:
+ # ...
+ retentionPeriod: "1"
+ vmstorage:
+ replicaCount: 2
+ vmselect:
+ replicaCount: 2
+ vminsert:
+ replicaCount: 2
+```
+
+### With persistence
+
+```yaml
+kind: VMCluster
+metadata:
+ name: vmcluster-example-persistent
+spec:
+ # ...
+ retentionPeriod: "4"
+ replicationFactor: 2
+ vmstorage:
+ replicaCount: 2
+ storageDataPath: "/vm-data"
+ storage:
+ volumeClaimTemplate:
+ spec:
+ storageClassName: standard
+ resources:
+ requests:
+ storage: 10Gi
+ resources:
+ limits:
+ cpu: "0.5"
+ memory: 500Mi
+ vmselect:
+ replicaCount: 2
+ cacheMountPath: "/select-cache"
+ storage:
+ volumeClaimTemplate:
+ spec:
+ resources:
+ requests:
+ storage: 2Gi
+ resources:
+ limits:
+ cpu: "0.3"
+ memory: "300Mi"
+ vminsert:
+ replicaCount: 2
+```
diff --git a/docs/operator/resources/vmnodescrape.md b/docs/operator/resources/vmnodescrape.md
new file mode 100644
index 000000000..5ace0b19f
--- /dev/null
+++ b/docs/operator/resources/vmnodescrape.md
@@ -0,0 +1,46 @@
+# VMNodeScrape
+
+The `VMNodeScrape` CRD provides discovery mechanism for scraping metrics kubernetes nodes,
+it is useful for node exporters monitoring.
+
+`VMNodeScrape` object generates part of [VMAgent](./vmagent.md) configuration.
+It has various options for scraping configuration of target (with basic auth,tls access, by specific port name etc.).
+
+By specifying configuration at CRD, operator generates config
+for [VMAgent](./vmagent.md) and syncs it. It's useful for cadvisor scraping,
+node-exporter or other node-based exporters. `VMAgent` `nodeScrapeSelector` must match `VMNodeScrape` labels.
+
+More information about selectors you can find in [this doc](./vmagent.md#scraping).
+
+## Specification
+
+You can see the full actual specification of the `VMNodeScrape` resource in
+the **[API docs -> VMNodeScrape](../api.md#vmnodescrape)**.
+
+Also, you can check out the [examples](#examples) section.
+
+## Examples
+
+### Cadvisor scraping
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMNodeScrape
+metadata:
+ name: cadvisor-metrics
+spec:
+ scheme: "https"
+ tlsConfig:
+ insecureSkipVerify: true
+ caFile: "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
+ bearerTokenFile: "/var/run/secrets/kubernetes.io/serviceaccount/token"
+ relabelConfigs:
+ - action: labelmap
+ regex: __meta_kubernetes_node_label_(.+)
+ - targetLabel: __address__
+ replacement: kubernetes.default.svc:443
+ - sourceLabels: [__meta_kubernetes_node_name]
+ regex: (.+)
+ targetLabel: __metrics_path__
+ replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
+```
diff --git a/docs/operator/resources/vmpodscrape.md b/docs/operator/resources/vmpodscrape.md
new file mode 100644
index 000000000..cf829af21
--- /dev/null
+++ b/docs/operator/resources/vmpodscrape.md
@@ -0,0 +1,64 @@
+# VMPodScrape
+
+The `VMPodScrape` CRD allows to declaratively define how a dynamic set of pods should be monitored.
+Use label selections to match pods for scraping. This allows an organization to introduce conventions
+for how metrics should be exposed. Following these conventions new services will be discovered automatically without
+need to reconfigure.
+
+`VMPodScrape` object generates part of [VMAgent](./vmagent.md) configuration with
+[kubernetes service discovery](https://docs.victoriametrics.com/sd_configs.html#kubernetes_sd_configs) role `pod` having specific labels and ports.
+It has various options for scraping configuration of target (with basic auth,tls access, by specific port name etc.).
+
+A `Pod` is a collection of one or more containers which can expose Prometheus metrics on a number of ports.
+
+The `VMPodScrape` object discovers pods and generates the relevant scraping configuration.
+
+The `PodMetricsEndpoints` section of the `VMPodScrapeSpec` is used to configure which ports of a pod are going to be
+scraped for metrics and with which parameters.
+
+Both `VMPodScrapes` and discovered targets may belong to any namespace. It is important for cross-namespace monitoring
+use cases, e.g. for meta-monitoring. Using the `namespaceSelector` of the `VMPodScrapeSpec` one can restrict the
+namespaces from which `Pods` are discovered from. To discover targets in all namespaces the `namespaceSelector` has to
+be empty:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMPodScrape
+metadata:
+ name: example-pod-scrape
+spec:
+ namespaceSelector:
+ any: true
+```
+
+More information about selectors you can find in [this doc](./vmagent.md#scraping).
+
+## Specification
+
+You can see the full actual specification of the `VMPodScrape` resource in
+the **[API docs -> VMPodScrape](../api.md#vmpodscrape)**.
+
+Also, you can check out the [examples](#examples) section.
+
+## Migration from Prometheus
+
+The `VMPodScrape` CRD from VictoriaMetrics Operator is a drop-in replacement
+for the Prometheus `PodMonitor` from prometheus-operator.
+
+More details about migration from prometheus-operator you can read in [this doc](../migration.md).
+
+## Examples
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMPodScrape
+metadata:
+ name: example-pod-scrape
+spec:
+ podMetricsEndpoints:
+ - port: web
+ scheme: http
+ selector:
+ matchLabels:
+ owner: dev
+```
diff --git a/docs/operator/resources/vmprobe.md b/docs/operator/resources/vmprobe.md
new file mode 100644
index 000000000..c2ed443af
--- /dev/null
+++ b/docs/operator/resources/vmprobe.md
@@ -0,0 +1,226 @@
+# VMProbe
+
+The `VMProbe` CRD provides probing target ability with some external prober.
+The most common prober is [blackbox exporter](https://github.com/prometheus/blackbox_exporter).
+By specifying configuration at CRD, operator generates config for [VMAgent](./vmagent.md)
+and syncs it. It's possible to use static targets or use standard k8s discovery mechanism with `Ingress`.
+
+`VMProbe` object generates part of [VMAgent](./vmagent.md) configuration;
+It has various options for scraping configuration of target (with basic auth, tls access, by specific port name etc.).
+
+You have to configure blackbox exporter before you can use this feature.
+The second requirement is [VMAgent](./vmagent.md) selectors,
+it must match your `VMProbe` by label or namespace selector. `VMAgent` `probeSelector` must match `VMProbe` labels.
+
+See more details about selectors [here](./vmagent.md#scraping).
+
+## Specification
+
+You can see the full actual specification of the `VMProbe` resource in
+the **[API docs -> VMProbe](../api.md#vmprobe)**.
+
+Also, you can check out the [examples](#examples) section.
+
+## Migration from Prometheus
+
+The `VMProbe` CRD from VictoriaMetrics Operator is a drop-in replacement
+for the Prometheus `Probe` from prometheus-operator.
+
+More details about migration from prometheus-operator you can read in [this doc](../migration.md).
+
+## Examples
+
+### Static targets
+
+It will probe `VMAgent` with url - `vmagent-example-vmagent.default.svc:9115/heath` with blackbox url:
+`prometheus-blackbox-exporter.default.svc:9115` and module `http_2xx`
+(it was specified at [blackbox configmap](#blackbox-exporter)).
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMProbe
+metadata:
+ name: vmprobe-static-example
+spec:
+ jobName: static-probe
+ vmProberSpec:
+ # by default scheme http, and path is /probe
+ url: prometheus-blackbox-exporter.default.svc:9115
+ module: http_2xx
+ targets:
+ staticConfig:
+ targets:
+ - vmagent-example-vmagent.default.svc:8429/health
+ interval: 2s
+```
+
+After adding target to `VMAgent` configuration it starts probing itself throw blackbox exporter.
+
+### Ingress targets
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMProbe
+metadata:
+ name: vmprobe-ingress-example
+spec:
+ vmProberSpec:
+ # by default scheme http, and path is /probe
+ url: prometheus-blackbox-exporter.default.svc:9115
+ module: http_2xx
+ targets:
+ ingress:
+ selector:
+ matchLabels:
+ app: victoria-metrics-single
+ interval: 10s
+```
+
+This configuration will add 2 additional targets for probing: `vmsingle2.example.com` and `vmsingle.example.com`.
+
+But probes will be unsuccessful, because there is no such hosts.
+
+### Related resources
+
+Following resources will be used for the examples below:
+
+#### Blackbox exporter
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: prometheus-blackbox-exporter
+ labels:
+ app: prometheus-blackbox-exporter
+data:
+ blackbox.yaml: |
+ modules:
+ http_2xx:
+ http:
+ preferred_ip_protocol: ip4
+ valid_http_versions:
+ - HTTP/1.1
+ - HTTP/2.0
+ valid_status_codes: []
+ prober: http
+ timeout: 5s
+
+---
+
+kind: Service
+apiVersion: v1
+metadata:
+ name: prometheus-blackbox-exporter
+ labels:
+ app: prometheus-blackbox-exporter
+spec:
+ type: ClusterIP
+ ports:
+ - name: http
+ port: 9115
+ protocol: TCP
+ selector:
+ app: prometheus-blackbox-exporter
+
+---
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: prometheus-blackbox-exporter
+ labels:
+ app: prometheus-blackbox-exporter
+spec:
+ replicas: 1
+ selector:
+ matchLabels:
+ app: prometheus-blackbox-exporter
+ template:
+ metadata:
+ labels:
+ app: prometheus-blackbox-exporter
+ spec:
+ containers:
+ - name: blackbox-exporter
+ image: "prom/blackbox-exporter:v0.17.0"
+ args:
+ - "--config.file=/config/blackbox.yaml"
+ resources:
+ {}
+ ports:
+ - containerPort: 9115
+ name: http
+ livenessProbe:
+ httpGet:
+ path: /health
+ port: http
+ readinessProbe:
+ httpGet:
+ path: /health
+ port: http
+ volumeMounts:
+ - mountPath: /config
+ name: config
+ volumes:
+ - name: config
+ configMap:
+ name: prometheus-blackbox-exporter
+```
+
+### VMSingle
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMSingle
+metadata:
+ name: example-vmsingle-persisted
+spec:
+ retentionPeriod: "1"
+ removePvcAfterDelete: true
+ storage:
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 1Gi
+
+---
+
+apiVersion: networking.k8s.io/v1beta1
+kind: Ingress
+metadata:
+ labels:
+ app: victoria-metrics-single
+ name: victoria-metrics-single
+spec:
+ rules:
+ - host: vmsingle.example.com
+ http:
+ paths:
+ - backend:
+ serviceName: vmsingle-example-vmsingle-persisted
+ servicePort: 8428
+ path: /
+ - host: vmsingle2.example.com
+ http:
+ paths:
+ - backend:
+ serviceName: vmsingle-example-vmsingle-persisted
+ servicePort: 8428
+ path: /
+```
+
+### VMAgent
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMAgent
+metadata:
+ name: example-vmagent
+spec:
+ selectAllByDefault: true
+ replicaCount: 1
+ remoteWrite:
+ - url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
+```
diff --git a/docs/operator/resources/vmrule.md b/docs/operator/resources/vmrule.md
new file mode 100644
index 000000000..8f7416160
--- /dev/null
+++ b/docs/operator/resources/vmrule.md
@@ -0,0 +1,99 @@
+# VMRule
+
+`VMRule` represents [alerting](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/)
+or [recording](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) rules
+for [VMAlert](./vmalert.md) instances.
+
+The `VMRule` CRD declaratively defines a desired Prometheus rule to be consumed by one or more VMAlert instances.
+
+`VMRule` object generates [VMAlert](./vmalert.md)
+with ruleset defined at `VMRule` spec.
+
+Alerts and recording rules can be saved and applied as YAML files, and dynamically loaded without requiring any restart.
+
+See more details about rule configuration in [VMAlert docs](https://docs.victoriametrics.com/vmalert.html#quickstart).
+
+## Specification
+
+You can see the full actual specification of the `VMRule` resource in
+the **[API docs -> VMRule](../api.md#vmrule)**.
+
+Also, you can check out the [examples](#examples) section.
+
+## Enterprise features
+
+Custom resource `VMRule` supports feature [Multitenancy](https://docs.victoriametrics.com/vmalert.html#multitenancy)
+from [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
+
+### Multitenancy
+
+For using [Multitenancy](https://docs.victoriametrics.com/vmalert.html#multitenancy) in `VMRule`
+you need to **[enable VMAlert Enterprise](./vmalert.md#enterprise-features)**.
+
+After that you can add `tenant` field for groups in `VMRule`:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMRule
+metadata:
+ name: vmrule-ent-example
+spec:
+ groups:
+ - name: vmalert-1
+ rules:
+ # using enterprise features: Multitenancy
+ # more details about multitenancy you can read on https://docs.victoriametrics.com/vmalert.html#multitenancy
+ - tenant: 1
+ alert: vmalert config reload error
+ expr: delta(vmalert_config_last_reload_errors_total[5m]) > 0
+ for: 10s
+ labels:
+ severity: major
+ job: "{{ $labels.job }}"
+ annotations:
+ value: "{{ $value }}"
+ description: 'error reloading vmalert config, reload count for 5 min {{ $value }}'
+```
+
+## Examples
+
+### Alerting rule
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMRule
+metadata:
+ name: vmrule-alerting-example
+spec:
+ groups:
+ - name: vmalert
+ rules:
+ - alert: vmalert config reload error
+ expr: delta(vmalert_config_last_reload_errors_total[5m]) > 0
+ for: 10s
+ labels:
+ severity: major
+ job: "{{ $labels.job }}"
+ annotations:
+ value: "{{ $value }}"
+ description: 'error reloading vmalert config, reload count for 5 min {{ $value }}'
+```
+
+### Recording rule
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMRule
+metadata:
+ name: vmrule-recording-example
+spec:
+ groups:
+ - name: vmalert
+ interval: 1m
+ rules:
+ - alert: vmalert config reload error
+ expr: |-
+ sum by (cluster, namespace, job) (
+ rate(vm_http_request_errors_total[5m])
+ )
+```
diff --git a/docs/operator/resources/vmservicescrape.md b/docs/operator/resources/vmservicescrape.md
new file mode 100644
index 000000000..3f3ac770d
--- /dev/null
+++ b/docs/operator/resources/vmservicescrape.md
@@ -0,0 +1,77 @@
+# VMServiceScrape
+
+The `VMServiceScrape` CRD allows to define a dynamic set of services for monitoring. Services
+and scraping configurations can be matched via label selections. This allows an organization to introduce conventions
+for how metrics should be exposed. Following these conventions new services will be discovered automatically without
+need to reconfigure.
+
+`VMServiceScrape` object generates part of [VMAgent](./vmagent.md) configuration with
+[kubernetes service discovery](https://docs.victoriametrics.com/sd_configs.html#kubernetes_sd_configs) targets by corresponding `Service`.
+It has various options for scraping configuration of target (with basic auth,tls access, by specific port name etc.).
+
+Monitoring configuration based on `discoveryRole` setting. By default, `endpoints` is used to get objects from kubernetes api.
+It's also possible to use `discoveryRole: service` or `discoveryRole: endpointslices`.
+
+`Endpoints` objects are essentially lists of IP addresses.
+Typically, `Endpoints` objects are populated by `Service` object. `Service` object discovers `Pod`s by a label
+selector and adds those to the `Endpoints` object.
+
+A `Service` may expose one or more service ports backed by a list of one or multiple endpoints pointing to
+specific `Pod`s. The same reflected in the respective `Endpoints` object as well.
+
+The `VMServiceScrape` object discovers `Endpoints` objects and configures [VMAgent](./vmagent.md) to monitor `Pod`s.
+
+The `Endpoints` section of the `VMServiceScrapeSpec` is used to configure which `Endpoints` ports should be scraped.
+For advanced use cases, one may want to monitor ports of backing `Pod`s, which are not a part of the service endpoints.
+Therefore, when specifying an endpoint in the `endpoints` section, they are strictly used.
+
+**Note:** `endpoints` (lowercase) is the field in the `VMServiceScrape` CRD, while `Endpoints` (capitalized) is the Kubernetes object kind.
+
+Both `VMServiceScrape` and discovered targets may belong to any namespace. It is important for cross-namespace monitoring
+use cases, e.g. for meta-monitoring. Using the `serviceScrapeSelector` of the `VMAgentSpec`
+one can restrict the namespaces from which `VMServiceScrape`s are selected from by the respective [VMAgent](./vmagent.md) server.
+Using the `namespaceSelector` of the `VMServiceScrape` one can restrict the namespaces from which `Endpoints` can be
+discovered from. To discover targets in all namespaces the `namespaceSelector` has to be empty:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMServiceScrape
+metadata:
+ name: example-service-scrape
+spec:
+ namespaceSelector: {}
+ # ...
+```
+
+More information about selectors you can find in [this doc](./vmagent.md#scraping).
+
+## Specification
+
+You can see the full actual specification of the `VMServiceScrape` resource in
+the **[API docs -> VMServiceScrape](../api.md#vmservicescrape)**.
+
+Also, you can check out the [examples](#examples) section.
+
+## Migration from Prometheus
+
+The `VMServiceScrape` CRD from VictoriaMetrics Operator is a drop-in replacement
+for the Prometheus `ServiceMonitor` from prometheus-operator.
+
+More details about migration from prometheus-operator you can read in [this doc](../migration.md).
+
+## Examples
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMServiceScrape
+metadata:
+ name: example-app
+ labels:
+ team: frontend
+spec:
+ selector:
+ matchLabels:
+ app: example-app
+ endpoints:
+ - port: web
+```
diff --git a/docs/operator/resources/vmsingle.md b/docs/operator/resources/vmsingle.md
new file mode 100644
index 000000000..62835c229
--- /dev/null
+++ b/docs/operator/resources/vmsingle.md
@@ -0,0 +1,282 @@
+# VMSingle
+
+`VMSingle` represents database for storing metrics.
+The `VMSingle` CRD declaratively defines a [single-node VM](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html)
+installation to run in a Kubernetes cluster.
+
+For each `VMSingle` resource, the Operator deploys a properly configured `Deployment` in the same namespace.
+The VMSingle `Pod`s are configured to mount an empty dir or `PersistentVolumeClaimSpec` for storing data.
+Deployment update strategy set to [recreate](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#recreate-deployment).
+No more than one replica allowed.
+
+For each `VMSingle` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with name from `VMSingle.metadata.name`.
+
+## Specification
+
+You can see the full actual specification of the `VMSingle` resource in the **[API docs -> VMSingle](../api.md#vmsingle)**.
+
+If you can't find necessary field in the specification of the custom resource,
+see [Extra arguments section](./README.md#extra-arguments).
+
+Also, you can check out the [examples](#examples) section.
+
+## High availability
+
+`VMSingle` doesn't support high availability by default, for such purpose
+use [`VMCluster`](./vmcluster.md) instead or duplicate the setup.
+
+## Version management
+
+To set `VMSingle` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMSingle
+metadata:
+ name: example-vmsingle
+spec:
+ image:
+ repository: victoriametrics/victoria-metrics
+ tag: v1.93.4
+ pullPolicy: Always
+ # ...
+```
+
+Also, you can specify `imagePullSecrets` if you are pulling images from private repo:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMSingle
+metadata:
+ name: example-vmsingle
+spec:
+ image:
+ repository: victoriametrics/victoria-metrics
+ tag: v1.93.4
+ pullPolicy: Always
+ imagePullSecrets:
+ - name: my-repo-secret
+# ...
+```
+
+## Enterprise features
+
+VMSingle supports features from [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise):
+
+- [Downsampling](https://docs.victoriametrics.com/#downsampling)
+- [Multiple retentions / Retention filters](https://docs.victoriametrics.com/#retention-filters)
+- [Backup automation](https://docs.victoriametrics.com/vmbackupmanager.html)
+
+For using Enterprise version of [vmsingle](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html)
+you need to change version of `VMSingle` to version with `-enterprise` suffix using [Version management](#version-management).
+
+All the enterprise apps require `-eula` command-line flag to be passed to them.
+This flag acknowledges that your usage fits one of the cases listed on [this page](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
+So you can use [extraArgs](./README.md#extra-arguments) for passing this flag to `VMSingle`.
+
+### Downsampling
+
+After that you can pass [Downsampling](https://docs.victoriametrics.com/#downsampling)
+flag to `VMSingle` with [extraArgs](./README.md#extra-arguments) too.
+
+Here are complete example for [Downsampling](https://docs.victoriametrics.com/#downsampling):
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMSingle
+metadata:
+ name: vmsingle-ent-example
+spec:
+ # enabling enterprise features
+ image:
+ # enterprise version of vmsingle
+ tag: v1.93.5-enterprise
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmsingle enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: Downsampling
+ # more details about downsampling you can read on https://docs.victoriametrics.com/#downsampling
+ downsampling.period: 30d:5m,180d:1h,1y:6h,2y:1d
+
+ # ...other fields...
+```
+
+### Retention filters
+
+The same method is used to enable retention filters - here are complete example for [Retention filters](https://docs.victoriametrics.com/#retention-filters).
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMSingle
+metadata:
+ name: vmsingle-ent-example
+spec:
+ # enabling enterprise features
+ image:
+ # enterprise version of vmsingle
+ tag: v1.93.5-enterprise
+ extraArgs:
+ # should be true and means that you have the legal right to run a vmsingle enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ eula: true
+
+ # using enterprise features: Retention filters
+ # more details about retention filters you can read on https://docs.victoriametrics.com/#retention-filters
+ retentionFilter: '{team="juniors"}:3d,{env=~"dev|staging"}:30d'
+
+ # ...other fields...
+```
+
+### Backup automation
+
+You can check [vmbackupmanager documentation](https://docs.victoriametrics.com/vmbackupmanager.html) for backup automation.
+It contains a description of the service and its features. This section covers vmbackumanager integration in vmoperator.
+
+`VMSingle` has built-in backup configuration, it uses `vmbackupmanager` - proprietary tool for backups.
+It supports incremental backups (hourly, daily, weekly, monthly) with popular object storages (aws s3, google cloud storage).
+
+Here is a complete example for backup configuration:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMSingle
+metadata:
+ name: example-vmsingle
+spec:
+
+ vmBackup:
+ # should be true and means that you have the legal right to run a vmsingle enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ acceptEULA: true
+
+ # using enterprise features: Backup automation
+ # more details about backup automation you can read on https://docs.victoriametrics.com/vmbackupmanager.html
+ destination: "s3://your_bucket/folder"
+ credentialsSecret:
+ name: remote-storage-keys
+ key: credentials
+
+ # ...other fields...
+
+---
+
+apiVersion: v1
+kind: Secret
+metadata:
+ name: remote-storage-keys
+type: Opaque
+stringData:
+ credentials: |-
+ [default]
+ aws_access_key_id = your_access_key_id
+ aws_secret_access_key = your_secret_access_key
+```
+
+You can read more about backup configuration options and mechanics [here](https://docs.victoriametrics.com/vmbackupmanager.html)
+
+Possible configuration options for backup crd can be found at [link](../api.md#vmbackup)
+
+#### Restoring backups
+
+There are several ways to restore with [vmrestore](https://docs.victoriametrics.com/vmrestore.html) or [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager.html).
+
+##### Manually mounting disk
+
+You have to stop `VMSingle` by scaling it replicas to zero and manually restore data to the database directory.
+
+Steps:
+
+1. Edit `VMSingle` CRD, set `replicaCount: 0`
+1. Wait until database stops
+1. SSH to some server, where you can mount `VMSingle` disk and mount it manually
+1. Restore files with `vmrestore`
+1. Umount disk
+1. Edit `VMSingle` CRD, set `replicaCount: 1`
+1. Wait database start
+
+##### Using VMRestore init container
+
+1. Add init container with `vmrestore` command to `VMSingle` CRD, example:
+ ```yaml
+ apiVersion: operator.victoriametrics.com/v1beta1
+ kind: VMSingle
+ metadata:
+ name: example-vmsingle
+ spec:
+
+ vmBackup:
+ # should be true and means that you have the legal right to run a vmsingle enterprise
+ # that can either be a signed contract or an email with confirmation to run the service in a trial period
+ # https://victoriametrics.com/legal/esa/
+ acceptEULA: true
+
+ # using enterprise features: Backup automation
+ # more details about backup automation you can read on https://docs.victoriametrics.com/vmbackupmanager.html
+ destination: "s3://your_bucket/folder"
+ credentialsSecret:
+ name: remote-storage-keys
+ key: credentials
+
+ extraArgs:
+ runOnStart: "true"
+
+ initContainers:
+ - name: vmrestore
+ image: victoriametrics/vmrestore:latest
+ volumeMounts:
+ - mountPath: /victoria-metrics-data
+ name: data
+ - mountPath: /etc/vm/creds
+ name: secret-remote-storage-keys
+ readOnly: true
+ args:
+ - -storageDataPath=/victoria-metrics-data
+ - -src=s3://your_bucket/folder/latest
+ - -credsFilePath=/etc/vm/creds/credentials
+
+ # ...other fields...
+ ```
+1. Apply it, and db will be restored from S3
+1. Remove `initContainers` and apply CRD.
+
+Note that using `VMRestore` will require adjusting `src` for each pod because restore will be handled per-pod.
+
+##### Using VMBackupmanager init container
+
+Using VMBackupmanager restore in Kubernetes environment is described [here](https://docs.victoriametrics.com/vmbackupmanager.html#how-to-restore-in-kubernetes).
+
+Advantages of using `VMBackupmanager` include:
+
+- Automatic adjustment of `src` for each pod when backup is requested
+- Graceful handling of case when no restore is required - `VMBackupmanager` will exit with successful status code and won't prevent pod from starting
+
+## Examples
+
+```yaml
+kind: VMSingle
+metadata:
+ name: vmsingle-example
+spec:
+ retentionPeriod: "12"
+ removePvcAfterDelete: true
+ storage:
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 50Gi
+ extraArgs:
+ dedup.minScrapeInterval: 60s
+ resources:
+ requests:
+ memory: 500Mi
+ cpu: 500m
+ limits:
+ memory: 10Gi
+ cpu: 5
+```
diff --git a/docs/operator/resources/vmstaticscrape.md b/docs/operator/resources/vmstaticscrape.md
new file mode 100644
index 000000000..13cf6793c
--- /dev/null
+++ b/docs/operator/resources/vmstaticscrape.md
@@ -0,0 +1,37 @@
+# VMStaticScrape
+
+The `VMStaticScrape` CRD provides mechanism for scraping metrics from static targets, configured by CRD targets.
+
+`VMStaticScrape` object generates part of [VMAgent](./vmagent.md)
+configuration with [static "service discovery"](https://docs.victoriametrics.com/sd_configs.html#static_configs).
+It has various options for scraping configuration of target (with basic auth,tls access, by specific port name etc.).
+
+By specifying configuration at CRD, operator generates config
+for [VMAgent](./vmagent.md) and syncs it.
+It's useful for external targets management, when service-discovery is not available.
+`VMAgent` `staticScrapeSelector` must match `VMStaticScrape` labels.
+
+More information about selectors you can find in [this doc](./vmagent.md#scraping).
+
+## Specification
+
+You can see the full actual specification of the `VMStaticScrape` resource in
+the **[API docs -> VMStaticScrape](../api.md#vmstaticscrape)**.
+
+Also, you can check out the [examples](#examples) section.
+
+## Examples
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMStaticScrape
+metadata:
+ name: vmstaticscrape-sample
+spec:
+ jobName: static
+ targetEndpoints:
+ - targets: ["192.168.0.1:9100", "196.168.0.50:9100"]
+ labels:
+ env: dev
+ project: operator
+```
diff --git a/docs/operator/resources/vmuser.md b/docs/operator/resources/vmuser.md
new file mode 100644
index 000000000..6c1fdf04f
--- /dev/null
+++ b/docs/operator/resources/vmuser.md
@@ -0,0 +1,135 @@
+# VMUser
+
+The `VMUser` CRD describes user configuration, its authentication methods `basic auth` or `Authorization` header.
+User access permissions, with possible routing information.
+
+User can define routing target with `static` config, by entering target `url`, or with `CRDRef`, in this case,
+operator queries kubernetes API, retrieves information about CRD and builds proper url.
+
+## Specification
+
+You can see the full actual specification of the `VMUser` resource in
+the **[API docs -> VMUser](../api.md#vmuser)**.
+
+Also, you can check out the [examples](#examples) section.
+
+## Authentication methods
+
+There are two authentication mechanisms: ["Bearer token"](#bearer-token) and ["Basic auth"](#basic-auth) with `username` and `password`.
+Only one of them can be used with `VMUser` at one time.
+
+Operator creates `Secret` for every `VMUser` with name - `vmuser-{VMUser.metadata.name}`.
+It places `username` + `password` or `bearerToken` into `data` section.
+
+### Bearer token
+
+Bearer token is a way to authenticate user with `Authorization` header.
+User defines `token` field in `auth` section.
+
+Also, you can check out the [examples](#examples) section.
+
+### Basic auth
+
+Basic auth is the simplest way to authenticate user. User defines `username` and `password` fields in `auth` section.
+
+If `username` is empty, `metadata.name` from `VMUser` used as `username`.
+
+You can automatically generate `password` if:
+- Set `generatePassword: true` field
+- Don't fill `password` field
+
+Operator generates random password for this `VMUser`,
+this password will be added to the `Secret` for this `VMUser` at `data.password` field.
+
+Also, you can check out the [examples](#examples) section.
+
+## Routing
+
+You can define routes for user in `targetRefs` section.
+
+For every entry in `targetRefs` you can define routing target with `static` config, by entering target `url`,
+or with `crd`, in this case, operator queries kubernetes API, retrieves information about CRD and builds proper url.
+
+Here are details about other fields in `targetRefs`:
+
+- `paths` is the same as `src_paths` from [auth config](https://docs.victoriametrics.com/vmauth.html#auth-config)
+- `headers` is the same as `headers` from [auth config](https://docs.victoriametrics.com/vmauth.html#auth-config)
+- `targetPathSuffix` is the suffix for `url_prefix` (target URL) from [auth config](https://docs.victoriametrics.com/vmauth.html#auth-config)
+
+### Static
+
+The `static` field is the same as `url_prefix` (target URL) from [auth config](https://docs.victoriametrics.com/vmauth.html#auth-config),
+it allows you to set a specific static URL.
+
+### CRDRef
+
+The `crd` field is a more convenient form for specifying the components handled by the operator as auth targets.
+
+User can define routing target with `crd` config, by entering `kind`, `name` and `namespace` of CRD.
+
+Operator supports following kinds in `kind` field:
+
+- `VMAgent` for [VMAgent](./vmagent.md)
+- `VMAlert` for [VMAlert](./vmalert.md)
+- `VMAlertmanager` for [VMAlertmanager](./vmalertmanager.md)
+- `VMSingle` for [VMSingle](./vmsingle.md)
+- `VMCluster/vmselect`, `VMCluster/vminsert` and `VMCluster/vmstorage` for [VMCluster](./vmcluster.md)
+
+Also, you can check out the [examples](#examples) section.
+
+Additional fields like `path` and `scheme` can be added to `CRDRef` config.
+
+## Enterprise features
+
+Custom resource `VMUser` supports feature [IP filters](https://docs.victoriametrics.com/vmauth.html#ip-filters)
+from [VictoriaMetrics Enterprise](https://docs.victoriametrics.com/enterprise.html#victoriametrics-enterprise).
+
+### IP Filters
+
+For using [IP filters](https://docs.victoriametrics.com/vmauth.html#ip-filters)
+you need to **[enable VMAuth Enterprise](./vmauth.md#enterprise-features)**.
+
+After that you can add `ip_filters` field to `VMUser`:
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMUser
+metadata:
+ name: vmuser-ent-example
+spec:
+ username: simple-user
+ password: simple-password
+
+ # using enterprise features: ip filters for vmuser
+ # more details about ip filters you can read in https://docs.victoriametrics.com/vmuser.html#enterprise-features
+ ip_filters:
+ allow_list:
+ - 10.0.0.0/24
+ - 1.2.3.4
+ deny_list:
+ - 5.6.7.8
+```
+
+## Examples
+
+```yaml
+apiVersion: operator.victoriametrics.com/v1beta1
+kind: VMUser
+metadata:
+ name: example
+spec:
+ username: simple-user
+ password: simple-password
+ targetRefs:
+ - crd:
+ kind: VMSingle
+ name: example
+ namespace: default
+ paths: ["/.*"]
+ - static:
+ url: http://vmalert-example.default.svc:8080
+ paths: ["/api/v1/groups","/api/v1/alerts"]
+```
+
+More examples see on [Authorization and exposing components](../auth.md) page
+and in [Quickstart guide](../quick-start.md#vmuser).
diff --git a/docs/operator/security.md b/docs/operator/security.md
index 64a0e9849..774640ab1 100644
--- a/docs/operator/security.md
+++ b/docs/operator/security.md
@@ -1,25 +1,45 @@
---
-sort: 12
-weight: 12
+sort: 3
+weight: 3
title: Security
-menu:
- docs:
- parent: "operator"
- weight: 12
-aliases:
-- /operator/security.html
---
# Security
-VictoriaMetrics operator provides several security features, such as [PodSecurityPolicies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/), [PodSecurityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/).
+## Access control
+### Roles
-## PodSecurityPolicy.
+To run in a cluster the operator needs certain permissions, you can see them in [this directory](https://github.com/VictoriaMetrics/operator/tree/master/config/rbac):
- By default, operator creates serviceAccount for each cluster resource and binds default `PodSecurityPolicy` to it.
+- [`role.yaml` file](https://github.com/VictoriaMetrics/operator/blob/master/config/rbac/role.yaml) - basic set of cluster roles for launching an operator.
+- [`leader_election_role.yaml` file](https://github.com/VictoriaMetrics/operator/blob/master/config/rbac/leader_election_role.yaml) - set of roles with permissions to do leader election (is necessary to run the operator in several replicas for high availability).
- Default psp:
+Also, you can use single-namespace mode with minimal permissions, see [this section](./configuration.md#namespaced-mode) for details.
+
+Also in [the same directory](https://github.com/VictoriaMetrics/operator/tree/master/config/rbac) are files with a set of separate permissions to view or edit [operator resources](./resources/README.md) to organize fine-grained access:
+
+- file `_viewer_role.yaml` - permissions for viewing (`get`, `list` and `watch`) some resource of vmoperator.
+- file `_editor_role.yaml` - permissions for editing (`create`, `delete`, `patch`, `update` and `deletecollection`) some resource of vmoperator (also includes viewing permissions).
+
+For instance, [`vmalert_editor_role.yaml` file](https://github.com/VictoriaMetrics/operator/blob/master/config/rbac/vmalert_editor_role.yaml) contain permission
+for editing [`vmagent` custom resources](./resources/vmagent.md).
+
+
+
+
+## Security policies
+
+VictoriaMetrics operator provides several security features, such as [PodSecurityPolicies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/),
+[PodSecurityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/).
+
+### PodSecurityPolicy
+
+> PodSecurityPolicy was [deprecated](https://kubernetes.io/docs/concepts/security/pod-security-policy/) in Kubernetes v1.21, and removed from Kubernetes in v1.25.
+
+If your Kubernetes version is under v1.25 and want to use PodSecurityPolicy, you can set env `VM_PSPAUTOCREATEENABLED: "true"` in operator, it will create serviceAccount for each cluster resource and binds default `PodSecurityPolicy` to it.
+
+Default psp:
```yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
@@ -48,19 +68,39 @@ spec:
- nfs
```
- This behaviour may be disabled with env variable passed to operator:
- ```yaml
- - name: VM_PSPAUTOCREATEENABLED
- value: "false"
-```
-
- User may also override default pod security policy with setting: `spec.podSecurityPolicyName: "psp-name"`.
-
+User may also override default pod security policy with setting: `spec.podSecurityPolicyName: "psp-name"`.
## PodSecurityContext
- `PodSecurityContext` can be configured with spec setting. It may be useful for mounted volumes, with `VMSingle` for example:
-
+VictoriaMetrics operator will add default Security Context to managed pods and containers if env `EnableStrictSecurity: "true"` is set.
+The following SecurityContext will be applied:
+
+### Pod SecurityContext
+
+1. **RunAsNonRoot: true**
+1. **RunAsUser/RunAsGroup/FSGroup: 65534**
+
+ '65534' refers to 'nobody' in all the used default images like alpine, busybox.
+
+ If you're using customize image, please make sure '65534' is a valid uid in there or specify SecurityContext.
+1. **FSGroupChangePolicy: &onRootMismatch**
+
+ If KubeVersion>=1.20, use `FSGroupChangePolicy="onRootMismatch"` to skip the recursive permission change
+ when the root of the volume already has the correct permissions
+1. **SeccompProfile: {type: RuntimeDefault}**
+
+ Use `RuntimeDefault` seccomp profile by default, which is defined by the container runtime,
+ instead of using the Unconfined (seccomp disabled) mode.
+
+### Container SecurityContext
+
+1. **AllowPrivilegeEscalation: false**
+1. **ReadOnlyRootFilesystem: true**
+1. **Capabilities: {drop: [all]}**
+
+
+Also `SecurityContext` can be configured with spec setting. It may be useful for mounted volumes, with `VMSingle` for example:
+
```yaml
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMSingle
@@ -89,5 +129,4 @@ spec:
limits:
cpu: "1"
memory: "1512Mi"
-
```
diff --git a/docs/operator/setup.md b/docs/operator/setup.md
new file mode 100644
index 000000000..3c0e6c378
--- /dev/null
+++ b/docs/operator/setup.md
@@ -0,0 +1,117 @@
+---
+sort: 2
+weight: 2
+title: Setup
+---
+
+# VictoriaMetrics Operator Setup
+
+## Installing by helm-charts
+
+You can use one of the following official helm-charts with `vmoperator`:
+
+- [victoria-metrics-operator helm-chart](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-operator/README.md)
+- [victoria-metrics-k8s-stack helm chart](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-k8s-stack/README.md)
+ (includes the `victoria-metrics-operator` helm-chart and other components for full-fledged k8s monitoring, is an alternative for [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack)).
+
+For installing VictoriaMetrics operator with helm-chart follow the instructions from README of the corresponding helm-chart
+([this](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-operator/README.md)
+or [this](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-k8s-stack/README.md)).
+
+in addition, you can use [quickstart guide](./quick-start.md) for
+installing VictoriaMetrics operator with helm-chart.
+
+## Installing by Manifest
+
+Obtain release from releases page:
+[https://github.com/VictoriaMetrics/operator/releases](https://github.com/VictoriaMetrics/operator/releases)
+
+We suggest use the latest release.
+
+```console
+# Get latest release version from https://github.com/VictoriaMetrics/operator/releases/latest
+export VM_VERSION=`basename $(curl -fs -o/dev/null -w %{redirect_url} https://github.com/VictoriaMetrics/operator/releases/latest)`
+wget https://github.com/VictoriaMetrics/operator/releases/download/$VM_VERSION/bundle_crd.zip
+unzip bundle_crd.zip
+```
+
+Operator use `monitoring-system` namespace, but you can install it to specific namespace with command:
+
+```console
+sed -i "s/namespace: monitoring-system/namespace: YOUR_NAMESPACE/g" release/operator/*
+```
+
+First of all, you have to create [custom resource definitions](https://github.com/VictoriaMetrics/operator):
+
+```console
+kubectl apply -f release/crds
+```
+
+Then you need RBAC for operator, relevant configuration for the release can be found at `release/operator/rbac.yaml`.
+
+Change configuration for operator at `release/operator/manager.yaml`, possible settings: [operator-settings](/operator/vars.html)
+and apply it:
+
+```console
+kubectl apply -f release/operator/
+```
+
+Check the status of operator
+
+```console
+kubectl get pods -n monitoring-system
+
+#NAME READY STATUS RESTARTS AGE
+#vm-operator-667dfbff55-cbvkf 1/1 Running 0 101s
+```
+
+## Installing by Kustomize
+
+You can install operator using [Kustomize](https://kustomize.io/) by pointing to the remote kustomization file.
+
+```console
+# Get latest release version from https://github.com/VictoriaMetrics/operator/releases/latest
+export VM_VERSION=`basename $(curl -fs -o/dev/null -w %{redirect_url} https://github.com/VictoriaMetrics/operator/releases/latest)`
+
+cat << EOF > kustomization.yaml
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+- github.com/VictoriaMetrics/operator/config/default?ref=${VM_VERSION}
+
+images:
+- name: victoriametrics/operator
+ newTag: ${VM_VERSION}
+EOF
+```
+
+You can change [operator configuration](#configuring), or use your custom namespace see [kustomize-example](https://github.com/YuriKravetc/yurikravetc.github.io/tree/main/Operator/kustomize-example).
+
+Build template
+
+```console
+kustomize build . -o monitoring.yaml
+```
+
+Apply manifests
+
+```console
+kubectl apply -f monitoring.yaml
+```
+
+Check the status of operator
+
+```console
+kubectl get pods -n monitoring-system
+
+#NAME READY STATUS RESTARTS AGE
+#vm-operator-667dfbff55-cbvkf 1/1 Running 0 101s
+```
+
+## Installing to ARM
+
+There is no need in an additional configuration for ARM. Operator and VictoriaMetrics have full support for it.
+
+## Configuring
+
+You can read detailed instructions about operator configuring in [this document](./configuration.md).
diff --git a/docs/operator/vars.md b/docs/operator/vars.md
index 5142a05d1..f127dcfdb 100644
--- a/docs/operator/vars.md
+++ b/docs/operator/vars.md
@@ -1,16 +1,12 @@
---
-sort: 14
-weight: 14
+sort: 11
+weight: 11
title: Variables
-menu:
- docs:
- parent: "operator"
- weight: 14
-aliases:
-- /operator/vars.html
---
+
+
# Auto Generated vars for package config
- updated at Wed Sep 27 00:09:29 UTC 2023
+ updated at Mon Oct 2 12:46:32 UTC 2023
| varible name | variable default value | variable required | variable description |