3407006cdb
The purpose of the update is to make the dash more usable for large installations with many instances. Panels which showed metrics per-instance (Mem, CPU) now are showing metrics per-job or min/max/avg aggregations in % instead. This supposed to help immediately to identify resource shortage and remain usable for small and big installations. For cases when detailed info is needed, to the bottom of the dashboard a new row `Drilldown` was added. Panels like Mem or CPU now contain a `data-link` named `Drilldown` (cis shown on line click) which takes user to more detailed panel. The change list is the following: * bump Grafana version to 9.1.0; * replace old "Graph" panel with "TimeSeries" panel; * improve Uptime panel to show number of instances per job; * show % usage of Mem and CPU instead of absolute values; * `Caches` row was removed. All needed info for caches is now part of `Troubleshooting`; * add `Drilldown` section for detailed resource usage; * add Annotations for Alert triggers. Not all alerts are supposed to be displayed on the dashboard, but only those with label `show_at: dashboard`. See `alerts-cluster.yml` change. Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com> |
||
---|---|---|
.. | ||
base | ||
builder | ||
provisioning | ||
alertmanager.yml | ||
alerts-cluster.yml | ||
alerts-health.yml | ||
alerts-vmagent.yml | ||
alerts-vmalert.yml | ||
alerts.yml | ||
docker-compose-cluster.yml | ||
docker-compose.yml | ||
Makefile | ||
prometheus-cluster.yml | ||
prometheus.yml | ||
README.md |
Docker compose environment for VictoriaMetrics
Docker compose environment for VictoriaMetrics includes VictoriaMetrics components, Alertmanager and Grafana.
For starting the docker-compose environment ensure you have docker installed and running and access to the Internet. All commands should be executed from the root directory of this repo.
To spin-up environment for single server VictoriaMetrics run the following command :
make docker-single-up
To shutdown the docker compose environment for single server run the following command:
make docker-single-down
For cluster version the command will be the following:
make docker-cluster-up
To shutdown the docker compose environment for cluster version run the following command:
make docker-cluster-down
VictoriaMetrics single server
VictoriaMetrics will be accessible on the following ports:
--graphiteListenAddr=:2003
--opentsdbListenAddr=:4242
--httpListenAddr=:8428
The communication scheme between components is the following:
- vmagent sends scraped metrics to VictoriaMetrics;
- grafana is configured with datasource pointing to VictoriaMetrics;
- vmalert is configured to query VictoriaMetrics and send alerts state and recording rules back to it;
- alertmanager is configured to receive notifications from vmalert.
To access vmalert
via vmselect
use link http://localhost:8428/vmalert.
To access vmui use link http://localhost:8428/vmui.
VictoriaMetrics cluster
VictoriaMetrics cluster environemnt consists of vminsert, vmstorage and vmselect components. vmselect
has exposed port :8481
, vminsert has exposed port :8480
and the rest of components are available
only inside of environment.
The communication scheme between components is the following:
- vmagent sends scraped metrics to vminsert;
- vminsert forwards data to vmstorage;
- vmselect is connected to vmstorage for querying data;
- grafana is configured with datasource pointing to vmselect;
- vmalert is configured to query vmselect and send alerts state and recording rules to vminsert;
- alertmanager is configured to receive notifications from vmalert.
To access vmalert
via vmselect
use link http://localhost:8481/select/0/prometheus/vmalert.
To access vmui use link http://localhost:8481/select/0/prometheus/vmui.
vmagent
vmagent is used for scraping and pushing timeseries to
VictoriaMetrics instance. It accepts Prometheus-compatible
configuration prometheus.yml
with listed targets for scraping.
vmalert
vmalert evaluates alerting rules (alerts.yml
) to track VictoriaMetrics
health state. It is connected with AlertManager for firing alerts,
and with VictoriaMetrics for executing queries and storing alert's state.
alertmanager
AlertManager accepts notifications from vmalert
and fires alerts.
All notifications are blackholed according to alertmanager.yml
config.
Grafana
To access service open following link.
Default credential:
- login -
admin
- password -
admin
Grafana is provisioned by default with following entities:
VictoriaMetrics
datasourceVictoriaMetrics - cluster
datasourceVictoriaMetrics overview
dashboardVictoriaMetrics - cluster
dashboardVictoriaMetrics - vmagent
dashboardVictoriaMetrics - vmalert
dashboard
Remember to pick VictoriaMetrics - cluster
datasource when viewing VictoriaMetrics - cluster
dashboard.