diff --git a/app/vmbackup/README.md b/app/vmbackup/README.md index ffe6ee64d..d8e1a8916 100644 --- a/app/vmbackup/README.md +++ b/app/vmbackup/README.md @@ -180,7 +180,7 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time- -customS3Endpoint string Custom S3 endpoint for use with S3-compatible storages (e.g. MinIO). S3 is used if not set -dst string - Where to put the backup on the remote storage. Example: gs://bucket/path/to/backup/dir, s3://bucket/path/to/backup/dir or fs:///path/to/local/backup/dir + Where to put the backup on the remote storage. Example: gs://bucket/path/to/backup, s3://bucket/path/to/backup, azblob://bucket/path/to/backup or fs:///path/to/local/backup/dir -dst can point to the previous backup. In this case incremental backup is performed, i.e. only changed data is uploaded -enableTCP6 Whether to enable IPv6 for listening and dialing. By default only IPv4 TCP and UDP is used diff --git a/app/vmbackup/main.go b/app/vmbackup/main.go index bda03ee64..001f1d6d8 100644 --- a/app/vmbackup/main.go +++ b/app/vmbackup/main.go @@ -30,7 +30,7 @@ var ( snapshotDeleteURL = flag.String("snapshot.deleteURL", "", "VictoriaMetrics delete snapshot url. Optional. Will be generated from -snapshot.createURL if not provided. "+ "All created snapshots will be automatically deleted. Example: http://victoriametrics:8428/snapshot/delete") dst = flag.String("dst", "", "Where to put the backup on the remote storage. "+ - "Example: gs://bucket/path/to/backup/dir, s3://bucket/path/to/backup/dir or fs:///path/to/local/backup/dir\n"+ + "Example: gs://bucket/path/to/backup, s3://bucket/path/to/backup, azblob://bucket/path/to/backup or fs:///path/to/local/backup/dir\n"+ "-dst can point to the previous backup. In this case incremental backup is performed, i.e. only changed data is uploaded") origin = flag.String("origin", "", "Optional origin directory on the remote storage with old backup for server-side copying when performing full backup. This speeds up full backups") concurrency = flag.Int("concurrency", 10, "The number of concurrent workers. Higher concurrency may reduce backup duration") diff --git a/app/vmrestore/README.md b/app/vmrestore/README.md index 67793a6b9..b0e2aec41 100644 --- a/app/vmrestore/README.md +++ b/app/vmrestore/README.md @@ -1,7 +1,6 @@ # vmrestore `vmrestore` restores data from backups created by [vmbackup](https://docs.victoriametrics.com/vmbackup.html). -VictoriaMetrics `v1.29.0` and newer versions must be used for working with the restored data. Restore process can be interrupted at any time. It is automatically resumed from the interruption point when restarting `vmrestore` with the same args. @@ -10,19 +9,28 @@ when restarting `vmrestore` with the same args. VictoriaMetrics must be stopped during the restore process. -```console -vmrestore -src=gs:/// -storageDataPath= +Run the following command to restore backup from the given `-src` into the given `-storageDataPath`: +```console +vmrestore -src=:// -storageDataPath= ``` -* `` is [GCS bucket](https://cloud.google.com/storage/docs/creating-buckets) name. -* `` is the path to backup made with [vmbackup](https://docs.victoriametrics.com/vmbackup.html) on GCS bucket. +* `://` is the path to backup made with [vmbackup](https://docs.victoriametrics.com/vmbackup.html). + `vmrestore` can restore backups from the following storage types: + * [GCS](https://cloud.google.com/storage/). Example: `-src=gs:///` + * [S3](https://aws.amazon.com/s3/). Example: `-src=s3:///` + * [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs/). Example: `-src=azblob:///` + * Any S3-compatible storage such as [MinIO](https://github.com/minio/minio), [Ceph](https://docs.ceph.com/en/pacific/radosgw/s3/) + or [Swift](https://platform.swiftstack.com/docs/admin/middleware/s3_middleware.html). See [these docs](#advanced-usage) for details. + * Local filesystem. Example: `-src=fs://`. Note that `vmbackup` prevents from storing the backup + into the directory pointed by `-storageDataPath` command-line flag, since this directory should be managed solely by VictoriaMetrics or `vmstorage`. * `` is the path to folder where data will be restored. This folder must be passed to VictoriaMetrics in `-storageDataPath` command-line flag after the restore process is complete. The original `-storageDataPath` directory may contain old files. They will be substituted by the files from backup, i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/questions/476041/how-do-i-make-rsync-delete-files-that-have-been-deleted-from-the-source-folder). + ## Troubleshooting * If `vmrestore` eats all the network bandwidth, then set `-maxBytesPerSecond` to the desired value. @@ -154,7 +162,7 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q -skipBackupCompleteCheck Whether to skip checking for 'backup complete' file in -src. This may be useful for restoring from old backups, which were created without 'backup complete' file -src string - Source path with backup on the remote storage. Example: gs://bucket/path/to/backup/dir, s3://bucket/path/to/backup/dir or fs:///path/to/local/backup/dir + Source path with backup on the remote storage. Example: gs://bucket/path/to/backup, s3://bucket/path/to/backup, azblob://bucket/path/to/backup or fs:///path/to/local/backup -storageDataPath string Destination path where backup must be restored. VictoriaMetrics must be stopped when restoring from backup. -storageDataPath dir can be non-empty. In this case the contents of -storageDataPath dir is synchronized with -src contents, i.e. it works like 'rsync --delete' (default "victoria-metrics-data") -tls diff --git a/app/vmrestore/main.go b/app/vmrestore/main.go index a45989c9b..08d07827a 100644 --- a/app/vmrestore/main.go +++ b/app/vmrestore/main.go @@ -20,7 +20,7 @@ import ( var ( httpListenAddr = flag.String("httpListenAddr", ":8421", "TCP address for exporting metrics at /metrics page") src = flag.String("src", "", "Source path with backup on the remote storage. "+ - "Example: gs://bucket/path/to/backup/dir, s3://bucket/path/to/backup/dir or fs:///path/to/local/backup/dir") + "Example: gs://bucket/path/to/backup, s3://bucket/path/to/backup, azblob://bucket/path/to/backup or fs:///path/to/local/backup") storageDataPath = flag.String("storageDataPath", "victoria-metrics-data", "Destination path where backup must be restored. "+ "VictoriaMetrics must be stopped when restoring from backup. -storageDataPath dir can be non-empty. In this case the contents of -storageDataPath dir "+ "is synchronized with -src contents, i.e. it works like 'rsync --delete'") diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index feffc89d6..3d7bb60f4 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -46,12 +46,13 @@ See [these docs](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#m * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): minimize the time needed for reading large responses from scrape targets in [stream parsing mode](https://docs.victoriametrics.com/vmagent.html#stream-parsing-mode). This should reduce scrape durations for such targets as [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics) running in a big Kubernetes cluster. * FEATURE: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): add [sort_by_label_numeric](https://docs.victoriametrics.com/MetricsQL.html#sort_by_label_numeric) and [sort_by_label_numeric_desc](https://docs.victoriametrics.com/MetricsQL.html#sort_by_label_numeric_desc) functions for [numeric sort](https://www.gnu.org/software/coreutils/manual/html_node/Version-sort-is-not-the-same-as-numeric-sort.html) of input time series by the specified labels. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2938). * FEATURE: [vmbackup](https://docs.victoriametrics.com/vmbackup.html) and [vmrestore](https://docs.victoriametrics.com/vmrestore.html): retry GCS operations for up to 3 minutes on temporary failures. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3147). +* FEATURE: [vmbackup](https://docs.victoriametrics.com/vmbackup.html): add support for saving / restoring backups to / from [Azure blob storage](https://azure.microsoft.com/en-us/products/storage/blobs/). See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1029). +* FEATURE: [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager.html): expose `vm_backup_in_flight` metric, which can be used for determining which backup types - latest, hourly, daily, weekly or monthly - are currently executed. * FEATURE: [vmgateway](https://docs.victoriametrics.com/vmgateway.html): add ability to extract JWT authorization token from non-standard HTTP header by passing it via `-auth.httpHeader` command-line flag. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3054). * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): expose `__meta_ec2_region` label for [ec2_sd_config](https://docs.victoriametrics.com/sd_configs.html#ec2_sd_configs) in the same way as [Prometheus 2.39 does](https://github.com/prometheus/prometheus/pull/11326). * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): accept data ingestion requests via paths starting from `/prometheus` prefix in the same way as [VictoriaMetrics does](https://docs.victoriametrics.com/#how-to-import-time-series-data). For example, `vmagent` now accepts Prometheus `remote_write` data via both `/api/v1/write` and `/prometheus/api/v1/write`. This simplifies switching between single-node VictoriaMetrics and `vmagent`. * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add `external_labels` from `global` section at `-promscrape.config` after the [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling) is applied to scraped metrics. This aligns with Prometheus behaviour. Previously the `external_labels` were added to scrape targets, so they could be modified during relabeling. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3137). * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): allow specifying per-`-remoteWrite.url` limits for on-disk size for pending data via `-remoteWrite.maxDiskUsagePerURL` command-line flag. Thanks to @rbizos for [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3071). -* FEATURE: [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager.html): expose `vm_backup_in_flight` metric, which can be used for determining which backup types - latest, hourly, daily, weekly or monthly - are currently executed. * BUGFIX: do not export stale metrics via [/federate api](https://docs.victoriametrics.com/#federation) after the staleness markers. Previously such metrics were exported with `NaN` values. this could break some setups. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3185). * BUGFIX: [vmauth](https://docs.victoriametrics.com/vmauth.html): properly handle request paths ending with `/` such as `/vmui/`. Previously `vmui` was dropping the traling `/`, which could prevent from using `vmui` via `vmauth`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1752). diff --git a/docs/vmbackup.md b/docs/vmbackup.md index 2b37be491..804ee2060 100644 --- a/docs/vmbackup.md +++ b/docs/vmbackup.md @@ -10,6 +10,7 @@ Supported storage systems for backups: * [GCS](https://cloud.google.com/storage/). Example: `gs:///` * [S3](https://aws.amazon.com/s3/). Example: `s3:///` +* [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs/). Example: `azblob:///` * Any S3-compatible storage such as [MinIO](https://github.com/minio/minio), [Ceph](https://docs.ceph.com/en/pacific/radosgw/s3/) or [Swift](https://platform.swiftstack.com/docs/admin/middleware/s3_middleware.html). See [these docs](#advanced-usage) for details. * Local filesystem. Example: `fs://`. Note that `vmbackup` prevents from storing the backup into the directory pointed by `-storageDataPath` command-line flag, since this directory should be managed solely by VictoriaMetrics or `vmstorage`. @@ -183,7 +184,7 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time- -customS3Endpoint string Custom S3 endpoint for use with S3-compatible storages (e.g. MinIO). S3 is used if not set -dst string - Where to put the backup on the remote storage. Example: gs://bucket/path/to/backup/dir, s3://bucket/path/to/backup/dir or fs:///path/to/local/backup/dir + Where to put the backup on the remote storage. Example: gs://bucket/path/to/backup, s3://bucket/path/to/backup, azblob://bucket/path/to/backup or fs:///path/to/local/backup/dir -dst can point to the previous backup. In this case incremental backup is performed, i.e. only changed data is uploaded -enableTCP6 Whether to enable IPv6 for listening and dialing. By default only IPv4 TCP and UDP is used diff --git a/docs/vmrestore.md b/docs/vmrestore.md index 13e04b7b0..4a9955f16 100644 --- a/docs/vmrestore.md +++ b/docs/vmrestore.md @@ -5,7 +5,6 @@ sort: 7 # vmrestore `vmrestore` restores data from backups created by [vmbackup](https://docs.victoriametrics.com/vmbackup.html). -VictoriaMetrics `v1.29.0` and newer versions must be used for working with the restored data. Restore process can be interrupted at any time. It is automatically resumed from the interruption point when restarting `vmrestore` with the same args. @@ -14,19 +13,28 @@ when restarting `vmrestore` with the same args. VictoriaMetrics must be stopped during the restore process. -```console -vmrestore -src=gs:/// -storageDataPath= +Run the following command to restore backup from the given `-src` into the given `-storageDataPath`: +```console +vmrestore -src=:// -storageDataPath= ``` -* `` is [GCS bucket](https://cloud.google.com/storage/docs/creating-buckets) name. -* `` is the path to backup made with [vmbackup](https://docs.victoriametrics.com/vmbackup.html) on GCS bucket. +* `://` is the path to backup made with [vmbackup](https://docs.victoriametrics.com/vmbackup.html). + `vmrestore` can restore backups from the following storage types: + * [GCS](https://cloud.google.com/storage/). Example: `-src=gs:///` + * [S3](https://aws.amazon.com/s3/). Example: `-src=s3:///` + * [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs/). Example: `-src=azblob:///` + * Any S3-compatible storage such as [MinIO](https://github.com/minio/minio), [Ceph](https://docs.ceph.com/en/pacific/radosgw/s3/) + or [Swift](https://platform.swiftstack.com/docs/admin/middleware/s3_middleware.html). See [these docs](#advanced-usage) for details. + * Local filesystem. Example: `-src=fs://`. Note that `vmbackup` prevents from storing the backup + into the directory pointed by `-storageDataPath` command-line flag, since this directory should be managed solely by VictoriaMetrics or `vmstorage`. * `` is the path to folder where data will be restored. This folder must be passed to VictoriaMetrics in `-storageDataPath` command-line flag after the restore process is complete. The original `-storageDataPath` directory may contain old files. They will be substituted by the files from backup, i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/questions/476041/how-do-i-make-rsync-delete-files-that-have-been-deleted-from-the-source-folder). + ## Troubleshooting * If `vmrestore` eats all the network bandwidth, then set `-maxBytesPerSecond` to the desired value. @@ -158,7 +166,7 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q -skipBackupCompleteCheck Whether to skip checking for 'backup complete' file in -src. This may be useful for restoring from old backups, which were created without 'backup complete' file -src string - Source path with backup on the remote storage. Example: gs://bucket/path/to/backup/dir, s3://bucket/path/to/backup/dir or fs:///path/to/local/backup/dir + Source path with backup on the remote storage. Example: gs://bucket/path/to/backup, s3://bucket/path/to/backup, azblob://bucket/path/to/backup or fs:///path/to/local/backup -storageDataPath string Destination path where backup must be restored. VictoriaMetrics must be stopped when restoring from backup. -storageDataPath dir can be non-empty. In this case the contents of -storageDataPath dir is synchronized with -src contents, i.e. it works like 'rsync --delete' (default "victoria-metrics-data") -tls