11 commits

Author	SHA1	Message	Date
Zakhar Bessarab	455415e4ff	lib/backup: fix issue with inconsistent copying of appliedRetention.txt (#5027) ... * lib/backup: fix issue with inconsistent copying of appliedRetention.txt appliedRetention.txt can be modified in place, so it should be always copied just the same as parts.json Updates: https://github.com/victoriaMetrics/victoriaMetrics/issues/5005 Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> * docs: add changelog entry for appliedRetention.txt copying fix Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> --------- Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>	2023-09-21 11:27:56 +02:00
Aliaksandr Valialkin	bbd46e9b2c	lib/backup: properly copy parts.json files inside indexdb directory additional to data directory ... This is a follow-up for 264ffe3fa1 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5006	2023-09-19 00:40:16 +02:00
Aliaksandr Valialkin	5612052a27	lib/backup/common: consistently use canonical path with / directory separators at Part.Path ... Previously Part.Path could contain `\` directory separators on Windows OS, which could result in incorrect filepaths generation when making backups at object storage. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4704 This is a follow-up for f2df8ad480	2023-09-19 00:37:36 +02:00
Zakhar Bessarab	3d2c078cee	lib/backup: force copying of parts.json (#5006) ... * lib/backup: force copying of parts.json Copying of parts.json is required because `part.key()` comparison can create same key value for files with different contents. This will result in inconsistent backup being created or restored. See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005 Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> * lib/backup: ensure parts.json is only copied once Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> --------- Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> Co-authored-by: Nikolay <nik@victoriametrics.com>	2023-09-18 16:19:28 +02:00
Nikolay	7bfa1d7d9e	lib/backup: fixes path generation for windows (#4133) ... replaces custom fsync function with standard Fsync methods for files. fixes pattern matching for parts and properly generate backup path for local fs. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70	2023-05-08 23:16:26 -07:00
Nikolay	cf1d2f289b	removes FileSize from backup part key (#1872) ... * removes FileSize from backup part key it should fix download restoration for backups * Update lib/backup/common/part.go Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-01 12:30:03 +02:00
Aliaksandr Valialkin	81789da731	lib/backup: increase backup chunk size from 128MB to 1GB ... This should reduce costs for object storage API calls by 8x. See https://cloud.google.com/storage/pricing#operations-pricing	2021-01-13 12:16:39 +02:00
Aliaksandr Valialkin	de1c07b937	lib/backup: add MustStop() method for all remote filesystems	2020-10-09 15:32:13 +03:00
Aliaksandr Valialkin	705af61587	app/{vmbackup,vmrestore}: add backup complete file to backup when it is complete and check for this file before restoring from backup ... This should prevent from restoring from incomplete backups. Add `-skipBackupCompleteCheck` command-line flag to `vmrestore` in order to be able restoring from old backups without `backup complete` file.	2020-01-09 15:35:45 +02:00
Oleg Kovalov	74ba42d111	fix misspelled words (#229)	2019-11-12 00:18:24 +02:00
Aliaksandr Valialkin	7d7fbf890e	app/{vmbackup,vmrestore}: add vmbackup and vmrestore tools for creating backups on s3 or gcs from instant snapshots ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/203 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/38	2019-11-07 21:26:43 +02:00