github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
VictoriaMetrics/docs/guides/guide-vmcluster-multiple-retention-setup.md
Yurii Kravets 0afec0259b
Create guide-vmcluster-multiple-retention-setup.md (#1888) 
* Create guide-vmcluster-multiple-retention-setup.md

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-01 12:42:16 +02:00

2.1 KiB
Raw Blame History

Multi Retention Setup within VictoriaMetrics Cluster

Objective

Setup Victoria Metrics TSDB with support of multiple retention periods within one installation.

Challenge

VictoriaMetrics instance (single node or vmstorage node) supports only one retention period.

Solution

A multi-retention setup can be implemented by dividing a victoriametrics cluster into logical groups with different retentions.

Example: Setup should handle 3 different retention groups 3months, 1year and 3 years. Solution contains 3 groups of vmstorages + vminserst and one group of vmselects. Routing is done by vmagent and relabeling configuration

The diagram below shows a proposed solution

Implementation Details

  1. Groups of vminserts A know about only vmstorages A and this is explicitly specified in -storageNode configuration.
  2. Groups of vminserts B know about only vmstorages B and this is explicitly specified in -storageNode configuration.
  3. Groups of vminserts C know about only vmstorages A and this is explicitly specified in -storageNode configuration.
  4. Vmselect reads data from all vmstorage nodes.
  5. Vmagent routes incoming metrics to the given set of vminsert nodes using relabeling rules specified at -remoteWrite.urlRelabelConfig. See these docs.

Multi-Tenant Setup

Every group of vmstorages can handle one tenant or multiple one. Different groups can have overlapping tenants. As vmselect reads from all vmstorage nodes, the data is aggregated on its level.

Additional Enhancements

You can set up vmauth for routing data to the given vminsert group depending on the needed retention.