github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity

docs: Add backup and restore doc for VictoriaLogs

Browse source 
This commit adds doc for VictoriaLogs backup and restore based on `rsync`.
This commit is contained in:
Zhu Jiekun 2024-08-28 03:41:42 +08:00 committed by GitHub
parent 9fcfba3927
commit 52043796c9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 49 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

49
docs/VictoriaLogs/README.md
View file

@ -129,6 +129,55 @@ For example, the following command starts VictoriaLogs, which stores the data at
VictoriaLogs automatically creates the `-storageDataPath` directory on the first run if it is missing.
## Backup and restore
VictoriaLogs currently does not have a snapshot feature and a tool like vmbackup as VictoriaMetrics does.
So backing up VictoriaLogs requires manually executing the `rsync` command.
The files in VictoriaLogs have the following properties:
- All the data files are immutable. Small metadata files can be modified.
- Old data files are periodically merged into new data files.
Therefore, for a complete data **backup**, you need to run the `rsync` command **twice**.
```sh
# example of rsync to remote host
rsync -avh --progress <path-to-victorialogs-data> <username>@<host>:<path-to-victorialogs-backup>
```
The first `rsync` will sync the majority of the data, which can be time-consuming.
As VictoriaLogs continues to run, new data is ingested, potentially creating new data files and modifying metadata files.
```sh
# example output
sending incremental file list
victoria-logs-data/
victoria-logs-data/flock.lock
0 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=78/80)
...
victoria-logs-data/partitions/20240809/indexdb/17E9ED7EF89BF422/metaindex.bin
51 100% 5.53kB/s 0:00:00 (xfr#64, to-chk=0/80)
sent 12.19K bytes received 1.30K bytes 3.86K bytes/sec
total size is 7.31K speedup is 0.54
```
The second `rsync` **requires a brief shutdown of VictoriaLogs** to ensure all data and metadata files are consistent and no longer changing.
This `rsync` will cover any changes that have occurred since the last `rsync` and should not take a significant amount of time.
To **restore** from a backup, simply `rsync` the backup files from a remote location to the original directory during downtime.
VictoriaLogs will automatically load this data upon startup.
```sh
# example of rsync from remote backup to local
rsync -avh --progress <username>@<host>:<path-to-victorialogs-backup> <path-to-victorialogs-data>
```
It is also possible to use **the disk snapshot** in order to perform a backup. This feature could be provided by your operating system,
cloud provider, or third-party tools. Note that the snapshot must be **consistent** to ensure reliable backup.
## Multitenancy
VictoriaLogs supports multitenancy. A tenant is identified by `(AccountID, ProjectID)` pair, where `AccountID` and `ProjectID` are arbitrary 32-bit unsigned integers.