Docker add vmauth (#5057)

* docker-compose: add vmauth to cluster env vmauth acts as a balancer and used as an example of how to interconnect VM components via vmauth. Signed-off-by: hagen1778 <roman@victoriametrics.com> * docker-compose: add vmauth to cluster env vmauth acts as a balancer and used as an example of how to interconnect VM components via vmauth. Signed-off-by: hagen1778 <roman@victoriametrics.com> --------- Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-11-21 14:44:00 +00:00 · 2023-09-26 10:50:10 +02:00 · 2023-09-26 10:50:10 +02:00 · 4d1b572f46
commit 4d1b572f46
parent f897d5241d
8 changed files with 79 additions and 24 deletions
--- a/app/vmauth/README.md
+++ b/app/vmauth/README.md
@ -25,6 +25,7 @@ The auth config can be reloaded via the following ways:
  and apply new changes every 5 seconds.

 Docker images for `vmauth` are available [here](https://hub.docker.com/r/victoriametrics/vmauth/tags).
+See how `vmauth` used in [docker-compose env](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/README.md#victoriametrics-cluster).

 Pass `-help` to `vmauth` in order to see all the supported command-line flags with their descriptions.

--- a/deployment/docker/README.md
+++ b/deployment/docker/README.md
@ -42,30 +42,36 @@ The communication scheme between components is the following:
  and recording rules back to it;
 * [alertmanager](#alertmanager) is configured to receive notifications from `vmalert`.

-To access `vmalert` use link [http://localhost:8428/vmalert](http://localhost:8428/vmalert/).
+To access Grafana use link [http://localhost:3000](http://localhost:3000).

 To access [vmui](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui)
 use link [http://localhost:8428/vmui](http://localhost:8428/vmui).

+To access `vmalert` use link [http://localhost:8428/vmalert](http://localhost:8428/vmalert/).
+
+
 ## VictoriaMetrics cluster

 VictoriaMetrics cluster environment consists of `vminsert`, `vmstorage` and `vmselect` components.
-`vmselect` has exposed port `:8481`, `vminsert` has exposed port `:8480` and the rest of components 
-are available only inside the environment.
+`vminsert` has exposed port `:8480`, access to `vmselect` components goes through `vmauth` on port `:8427`,
+and the rest of components are available only inside the environment.

 The communication scheme between components is the following:
 * [vmagent](#vmagent) sends scraped metrics to `vminsert`;
 * `vminsert` forwards data to `vmstorage`;
-* `vmselect` is connected to `vmstorage` for querying data;
-* [grafana](#grafana) is configured with datasource pointing to `vmselect`;
-* [vmalert](#vmalert) is configured to query `vmselect` and send alerts state
+* `vmselect`s are connected to `vmstorage` for querying data;
+* [vmauth](#vmauth) balances incoming read requests among `vmselect`s;
+* [grafana](#grafana) is configured with datasource pointing to `vmauth`;
+* [vmalert](#vmalert) is configured to query `vmselect`s via `vmauth` and send alerts state
  and recording rules to `vminsert`;
 * [alertmanager](#alertmanager) is configured to receive notifications from `vmalert`.

-To access `vmalert` use link [http://localhost:8481/select/0/prometheus/vmalert](http://localhost:8481/select/0/prometheus/vmalert/).
+To access Grafana use link [http://localhost:3000](http://localhost:3000).

-To access [vmui](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui) 
-use link [http://localhost:8481/select/0/prometheus/vmui](http://localhost:8481/select/0/prometheus/vmui).
+To access [vmui](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#vmui)
+use link [http://localhost:8427/select/0/prometheus/vmui/](http://localhost:8427/select/0/prometheus/vmui/).
+
+To access `vmalert` use link [http://localhost:8427/select/0/prometheus/vmalert/](http://localhost:8427/select/0/prometheus/vmalert/).

 ## vmagent

@ -75,6 +81,13 @@ with listed targets for scraping.

 [Web interface link](http://localhost:8429/).

+## vmauth
+
+[vmauth](https://docs.victoriametrics.com/vmauth.html) acts as a [balancer](https://docs.victoriametrics.com/vmauth.html#load-balancing)
+to spread the load across `vmselect`'s. [Grafana](#grafana) and [vmalert](#vmalert) use vmauth for read queries.
+vmauth config is available [here](ttps://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/auth-cluster.yml)
+
+
 ## vmalert

 vmalert evaluates alerting rules [alerts.yml](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/alerts.yml)
--- a/deployment/docker/auth-cluster.yml
+++ b/deployment/docker/auth-cluster.yml
@ -0,0 +1,6 @@
+# balance load among vmselects
+# see https://docs.victoriametrics.com/vmauth.html#load-balancing
+unauthorized_user:
+  url_prefix:
+    - http://vmselect-1:8481
+    - http://vmselect-2:8481
--- a/deployment/docker/docker-compose-cluster.yml
+++ b/deployment/docker/docker-compose-cluster.yml
@ -2,7 +2,7 @@ version: '3.5'
 services:
  vmagent:
    container_name: vmagent
-    image: victoriametrics/vmagent:v1.93.4
+    image: victoriametrics/vmagent:v1.93.5
    depends_on:
      - "vminsert"
    ports:
@ -19,7 +19,8 @@ services:
    container_name: grafana
    image: grafana/grafana:9.2.7
    depends_on:
-      - "vmselect"
+      - "vmselect-1"
+      - "vmselect-2"
    ports:
      - 3000:3000
    restart: always
@ -32,7 +33,7 @@ services:

  vmstorage-1:
    container_name: vmstorage-1
-    image: victoriametrics/vmstorage:v1.93.4-cluster
+    image: victoriametrics/vmstorage:v1.93.5-cluster
    ports:
      - 8482
      - 8400
@ -44,7 +45,7 @@ services:
    restart: always
  vmstorage-2:
    container_name: vmstorage-2
-    image: victoriametrics/vmstorage:v1.93.4-cluster
+    image: victoriametrics/vmstorage:v1.93.5-cluster
    ports:
      - 8482
      - 8400
@ -54,9 +55,10 @@ services:
    command:
      - '--storageDataPath=/storage'
    restart: always
+
  vminsert:
    container_name: vminsert
-    image: victoriametrics/vminsert:v1.93.4-cluster
+    image: victoriametrics/vminsert:v1.93.5-cluster
    depends_on:
      - "vmstorage-1"
      - "vmstorage-2"
@ -66,9 +68,10 @@ services:
    ports:
      - 8480:8480
    restart: always
-  vmselect:
-    container_name: vmselect
-    image: victoriametrics/vmselect:v1.93.4-cluster
+
+  vmselect-1:
+    container_name: vmselect-1
+    image: victoriametrics/vmselect:v1.93.5-cluster
    depends_on:
      - "vmstorage-1"
      - "vmstorage-2"
@ -77,14 +80,44 @@ services:
      - '--storageNode=vmstorage-2:8401'
      - '--vmalert.proxyURL=http://vmalert:8880'
    ports:
-      - 8481:8481
+      - 8481
+    restart: always
+
+  vmselect-2:
+    container_name: vmselect-2
+    image: victoriametrics/vmselect:v1.93.5-cluster
+    depends_on:
+      - "vmstorage-1"
+      - "vmstorage-2"
+    command:
+      - '--storageNode=vmstorage-1:8401'
+      - '--storageNode=vmstorage-2:8401'
+      - '--vmalert.proxyURL=http://vmalert:8880'
+    ports:
+      - 8481
+    restart: always
+
+  vmauth:
+    container_name: vmauth
+    image: victoriametrics/vmauth:v1.93.5
+    depends_on:
+      - "vmselect-1"
+      - "vmselect-2"
+    volumes:
+      - ./auth-cluster.yml:/etc/auth.yml
+#      - /var/run/docker.sock:/var/run/docker.sock
+    command:
+      - '--auth.config=/etc/auth.yml'
+    ports:
+      - 8427:8427
    restart: always

  vmalert:
    container_name: vmalert
-    image: victoriametrics/vmalert:v1.93.4
+    image: victoriametrics/vmalert:v1.93.5
    depends_on:
-      - "vmselect"
+      - "vmselect-1"
+      - "vmselect-2"
    ports:
      - 8880:8880
    volumes:
@ -93,8 +126,8 @@ services:
      - ./alerts-vmagent.yml:/etc/alerts/alerts-vmagent.yml
      - ./alerts-vmalert.yml:/etc/alerts/alerts-vmalert.yml
    command:
-      - '--datasource.url=http://vmselect:8481/select/0/prometheus'
-      - '--remoteRead.url=http://vmselect:8481/select/0/prometheus'
+      - '--datasource.url=http://vmauth:8427/select/0/prometheus'
+      - '--remoteRead.url=http://vmauth:8427/select/0/prometheus'
      - '--remoteWrite.url=http://vminsert:8480/insert/0/prometheus'
      - '--notifier.url=http://alertmanager:9093/'
      - '--rule=/etc/alerts/*.yml'
--- a/deployment/docker/prometheus-cluster.yml
+++ b/deployment/docker/prometheus-cluster.yml
@ -13,7 +13,7 @@ scrape_configs:
      - targets: ['vminsert:8480']
  - job_name: 'vmselect'
    static_configs:
-      - targets: ['vmselect:8481']
+      - targets: ['vmselect-1:8481', 'vmselect-2:8481']
  - job_name: 'vmstorage'
    static_configs:
      - targets: ['vmstorage-1:8482', 'vmstorage-2:8482']
--- a/deployment/docker/provisioning/datasources/datasource.yml
+++ b/deployment/docker/provisioning/datasources/datasource.yml
@ -10,5 +10,5 @@ datasources:
    - name: VictoriaMetrics - cluster
      type: prometheus
      access: proxy
-      url: http://vmselect:8481/select/0/prometheus
+      url: http://vmauth:8427/select/0/prometheus
      isDefault: false
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@ -49,6 +49,7 @@ The sandbox cluster installation is running under the constant load generated by
 * FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): add `eval_offset` attribute for [Groups](https://docs.victoriametrics.com/vmalert.html#groups). If specified, Group will be evaluated at the exact time offset on the range of [0...evaluationInterval]. The setting might be useful for cron-like rules which must be evaluated at specific moments of time. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3409) for details.
 * FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): validate [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html) function names in alerting and recording rules when `vmalert` runs with `-dryRun` command-line flag. Previously it was allowed to use unknown (aka invalid) MetricsQL function names there. For example, `foo()` was counted as a valid query. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4933).
 * FEATURE: limit the length of string params in log messages to 500 chars. Longer string params are replaced with the `first_250_chars..last_250_chars`. This prevents from too long log lines, which can be emitted by VictoriaMetrics components.
+* FEATURE: [docker compose environment](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker): add `vmauth` component to cluster's docker-compose example for balancing load among multiple `vmselect` components.
 * FEATURE: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): make sure that `q2` series are returned after `q1` series in the results of `q1 or q2` query, in the same way as Prometheus does. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763).
 * FEATURE: stop exposing `vm_merge_need_free_disk_space` metric, since it has been appeared that it confuses users while doesn't bring any useful information. See [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686#issuecomment-1733844128).

--- a/docs/vmauth.md
+++ b/docs/vmauth.md
@ -36,6 +36,7 @@ The auth config can be reloaded via the following ways:
  and apply new changes every 5 seconds.

 Docker images for `vmauth` are available [here](https://hub.docker.com/r/victoriametrics/vmauth/tags).
+See how `vmauth` used in [docker-compose env](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/README.md#victoriametrics-cluster).

 Pass `-help` to `vmauth` in order to see all the supported command-line flags with their descriptions.