VictoriaMetrics/lib
Aliaksandr Valialkin 12cd32fd75
lib/protoparser/promremotewrite: fall back to Snappy decoding if zstd decoding fails
This case is possible after the following steps:

1. vmagent tries to perform handshake with the -remoteWrite.url in order to determine whether
   the remote storage supports zstd-compressed data.
2. The remote storage is unavailable during the handshake. In this case vmagent falls back to Snappy compression
   for the data sent to the remote storage.
3. vmagent compresses the collected data into blocks with Snappy and puts these blocks to persistent queue on disk.
4. The remote storage becomes available.
5. vmagent restarts, performs the handshake with the remote storage and detects that it supports zstd-compressed data.
6. vmagent starts sending Snappy-compressed data from persistent queue to the remote storage,
   while falsely advertizing it sends zstd-compressed data.
7. The remote storage receives Snappy-compressed data and fails unpacking it with zstd.

The solution is to just fall back to Snappy decompression if zstd decompression fails.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5301
2023-11-13 21:25:39 +01:00
..
appmetrics all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:48:26 -07:00
auth lib/auth: add NewTokenPossibleMultitenant() for parsing auth token, which can be multitenant 2023-08-30 14:13:51 +02:00
awsapi .golangci.yml: properly enable revive linter and fix all the warnings it detects 2023-02-26 12:19:58 -08:00
backup fixed error when creating a full backup using the -origin flag (#5180) 2023-10-16 14:01:16 +02:00
blockcache lib/blockcache: do not cache entries, which were attempted to be accessed 1 or 2 times 2023-11-13 09:38:57 +01:00
bloomfilter all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:48:26 -07:00
bufferedwriter app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils 2023-07-06 17:22:23 -07:00
buildinfo all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
bytesutil lib/logger: add -loggerMaxArgLen command-line flag for fine-tuning the maximum length of logged args 2023-11-13 09:43:49 +01:00
cgroup lib/cgroup: add SetGOGC() function 2023-07-06 17:24:31 -07:00
consts app/vminsert: reduce the max packet size, which vminsert can send to vmstorage 2022-04-05 15:39:58 +03:00
decimal lib/decimal: use consistent randomizer in tests 2023-01-23 19:24:05 -08:00
encoding all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:15:42 +01:00
envflag lib/envflag: do not allow unsupported form for boolean command-line flags in the form -boolFlag value 2023-08-17 13:37:05 +02:00
envtemplate allowed using dashes and dots in environment variables names (#4009) 2023-03-24 17:57:19 -07:00
fastnum Makefile: add build and test rules with enabled race detector. These rules have -race suffix 2020-03-05 12:05:16 +02:00
fasttime lib: extract common code for returning fast unix timestamp into lib/fasttime 2020-05-14 23:06:50 +03:00
filestream lib/filestream: add -filestream.disableFadvise syscall for unconditional disabling of fadvise syscall 2023-10-04 16:30:19 +02:00
flagutil docs: update -help output after recent changes to VictoriaMetrics components 2023-11-02 20:27:16 +01:00
formatutil app/vmbackupmanager: add metrics for better observability (#488) 2022-12-20 14:18:43 -08:00
fs all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:15:42 +01:00
handshake lib/handshake: do not pollute logs with cannot read hello messages on TCP health checks 2023-05-18 10:37:59 -07:00
htmlcomponents lib/htmlcomponents: use relative links for the top page and for favicon.ico 2023-11-13 20:28:17 +01:00
httpserver lib/htmlcomponents: use relative links for the top page and for favicon.ico 2023-11-13 20:28:17 +01:00
httputils lib/httputils: fix test after b49d04b3dc 2023-07-06 22:21:43 -07:00
influxutils lib/flagutil: rename Array to ArrayString 2022-10-01 18:28:19 +03:00
ingestserver lib/netutil: init implimentation of proxy protocol (#3687) 2023-01-26 23:25:22 -08:00
leveledbytebufferpool all: make fmt via the upcoming Go1.19 2022-07-11 19:23:25 +03:00
logger all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:15:42 +01:00
logjson app/vlinsert/jsonline: code prettifying 2023-07-06 21:35:55 -07:00
logstorage all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-26 09:44:40 +02:00
lrucache Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:25:49 +02:00
memory all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:15:42 +01:00
mergeset lib/{mergeset,storage}: consistently reset isInMerge field in parts passed to mergeParts() before returning from the function 2023-10-02 20:34:52 +02:00
metricsql all: make fmt via the upcoming Go1.19 2022-07-11 19:23:25 +03:00
netutil all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:15:42 +01:00
persistentqueue lib/persistentqueue: properly re-create flock.lock file inside directory if persistent queue is broken. 2023-10-31 19:10:26 +01:00
procutil all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:15:42 +01:00
promauth lib/promauth: typo fix in the error message after d5a599badc: obtaine -> obtain 2023-10-26 09:56:20 +02:00
prompb app/vminsert: moved -maxInsertRequestSize command-line flag out of lib/prompb in order to prevent its inclusion in vmselect and vmstorage apps 2020-01-28 22:53:50 +02:00
prompbmarshal all: use %w instead of %s for wrapping errors in fmt.Errorf 2020-06-30 23:33:46 +03:00
promrelabel all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-26 09:44:40 +02:00
promscrape lib/promscrape/discovery/kubernetes: propagate possible errors at newAPIWatcher() to the caller 2023-10-27 20:27:58 +02:00
promutils lib/promutils, app/vmalert-tool/unittest: move promutils.Duration.ParseTime() to app/vmalert-tool/unittest.durationToTime() 2023-10-16 14:23:41 +02:00
protoparser lib/protoparser/promremotewrite: fall back to Snappy decoding if zstd decoding fails 2023-11-13 21:25:39 +01:00
proxy lib/promauth: follow-up for e16d3f5639 2023-10-26 09:55:47 +02:00
pushmetrics fixed typos in documentation and commandline flags descriptions (#4275) 2023-05-10 02:22:06 -07:00
querytracer all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-26 09:44:40 +02:00
regexutil lib/regexutil: properly handle alternate regexps surrounded by .+ or .* 2023-11-13 18:25:57 +01:00
snapshot Makefile: update golangci-lint version from v1.54.2 to v1.55.1 2023-11-02 21:42:35 +01:00
storage lib/logger: add -loggerMaxArgLen command-line flag for fine-tuning the maximum length of logged args 2023-11-13 09:43:49 +01:00
streamaggr lib/streamaggr: respect streamAgg.dropInput with empty stream aggr config (#5213) 2023-10-26 09:30:12 +02:00
stringsutil lib/stringsutil: add tests for LimitStringLen() function 2023-11-13 10:33:07 +01:00
syncwg all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
tenantmetrics lib/encoding/zstd: switch back from atomic.Pointer to atomic.Value for map[...]... 2023-07-20 21:54:51 -07:00
timerpool lib/timerpool: use timer pool in concurrency limiters 2019-05-28 17:30:10 +03:00
uint64set lib/uint64set: use repeatable randomizer in tests 2023-01-23 19:24:05 -08:00
vmselectapi all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-26 09:44:40 +02:00
workingsetcache lib/workingsetcache: fix outdated comments for Load() and New() functions 2023-10-26 09:43:04 +02:00
writeconcurrencylimiter lib/writeconcurrencylimiter: initialize concurrencyLimitCh before exporting vm_concurrent_insert_capacity and vm_concurrent_insert_current metrics 2023-02-07 11:08:39 -08:00