VictoriaMetrics/lib
Hui Wang cd7e84b995
lib/protoparser/promremotewrite: fall back to zstd decoding if Snappy-decoding fails (#5344)
This case is possible after the following steps:
1. vmagent successfully performed handshake with the -remoteWrite.url and the remote storage supports zstd-compressed data.
2. remote storage became unavailable or slow to ingest data, vmagent compressed the collected data into blocks with zstd and puts these blocks to persistent queue on disk.
3. vmagent restarts and the remote storage is unavailable during the handshake, then vmagent falls back to Snappy compression.
4. vmagent starts sending zstd-compressed data from persistent queue to the remote storage, while falsely advertizing it sends Snappy-compressed data.
5. The remote storage receives zstd-compressed data and fails unpacking it with Snappy.

The solution is the same as 12cd32fd75, just fall back to zstd decompression if Snappy decompression fails.
2023-11-23 20:47:10 +02:00
..
appmetrics all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:48:26 -07:00
auth lib/auth: add NewTokenPossibleMultitenant() for parsing auth token, which can be multitenant 2023-08-30 14:21:30 +02:00
awsapi .golangci.yml: properly enable revive linter and fix all the warnings it detects 2023-02-26 12:19:58 -08:00
backup lib/backup: fix issue with inconsistent copying of appliedRetention.txt (#5027) 2023-09-21 11:27:56 +02:00
blockcache Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 11:15:51 +02:00
bloomfilter all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:48:26 -07:00
bufferedwriter app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils 2023-07-06 17:22:23 -07:00
buildinfo all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
bytesutil Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 11:15:51 +02:00
cgroup lib/cgroup: add SetGOGC() function 2023-07-06 17:24:31 -07:00
consts app/vminsert: reduce the max packet size, which vminsert can send to vmstorage 2022-04-05 15:39:58 +03:00
decimal lib/decimal: use consistent randomizer in tests 2023-01-23 19:24:05 -08:00
encoding lib/encoding/zstd: switch back from atomic.Pointer to atomic.Value for map[...]... 2023-07-20 21:54:51 -07:00
envflag lib/envflag: do not allow unsupported form for boolean command-line flags in the form -boolFlag value 2023-08-17 14:15:48 +02:00
envtemplate allowed using dashes and dots in environment variables names (#4009) 2023-03-24 17:57:19 -07:00
fastnum Makefile: add build and test rules with enabled race detector. These rules have -race suffix 2020-03-05 12:05:16 +02:00
fasttime lib: extract common code for returning fast unix timestamp into lib/fasttime 2020-05-14 23:06:50 +03:00
filestream lib/filestream: change Create() to MustCreate() 2023-04-14 15:14:24 -07:00
flagutil lib/flagutil: add defaultValue arg to NewArray{Int,Bytes,Duration} functions 2023-08-12 04:19:34 -07:00
formatutil app/vmbackupmanager: add metrics for better observability (#488) 2022-12-20 14:18:43 -08:00
fs lib/fs: add ReaderAt.Path() function 2023-07-06 17:25:19 -07:00
handshake lib/handshake: do not pollute logs with cannot read hello messages on TCP health checks 2023-05-18 10:37:59 -07:00
htmlcomponents app/vmselect: remove dependency on lib/promscrape from app/vmselect 2023-01-03 23:27:36 -08:00
httpserver lib/httpserver: properly quote the returned address from GetQuotedRemoteAddr() for requests with X-Forwarded-For header 2023-08-11 05:47:28 -07:00
httputils lib/httputils: fix test after b49d04b3dc 2023-07-06 22:21:43 -07:00
influxutils lib/flagutil: rename Array to ArrayString 2022-10-01 18:28:19 +03:00
ingestserver lib/netutil: init implimentation of proxy protocol (#3687) 2023-01-26 23:25:22 -08:00
leveledbytebufferpool all: make fmt via the upcoming Go1.19 2022-07-11 19:23:25 +03:00
logger add error handler for parsing prometheus text format to vmagent and v… (#3693) 2023-01-23 22:36:23 -08:00
logjson app/vlinsert/jsonline: code prettifying 2023-07-06 21:35:55 -07:00
logstorage Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 11:15:51 +02:00
lrucache Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 11:15:51 +02:00
memory docs: follow-up after aec4b5db81 (#4638) 2023-07-19 14:48:17 -07:00
mergeset Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 11:15:51 +02:00
metricsql all: make fmt via the upcoming Go1.19 2022-07-11 19:23:25 +03:00
netutil Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 11:15:51 +02:00
persistentqueue lib/persistentqueue: properly re-create flock.lock file inside directory if persistent queue is broken. 2023-10-31 18:37:00 +01:00
procutil lib/procutil: stop immediately after receiving the second SIGINT or SIGTERM signal 2022-10-20 21:58:49 +03:00
promauth lib/promauth: properly parse string contents for ca, cert and key fields at tls_config 2023-10-27 14:23:15 +02:00
prompb app/vminsert: moved -maxInsertRequestSize command-line flag out of lib/prompb in order to prevent its inclusion in vmselect and vmstorage apps 2020-01-28 22:53:50 +02:00
prompbmarshal all: use %w instead of %s for wrapping errors in fmt.Errorf 2020-06-30 23:33:46 +03:00
promrelabel lib/promrelabel: stop emitting DEBUG log lines when parsing if expressions 2023-08-17 13:57:41 +02:00
promscrape lib/promscrape/discovery/kubernetes: stop all the url watchers, which belong to a particular groupWatcher, at once 2023-10-27 14:37:45 +02:00
promutils vmalert: revert unittest feature (#4734) 2023-07-28 11:09:52 +02:00
protoparser lib/protoparser/promremotewrite: fall back to zstd decoding if Snappy-decoding fails (#5344) 2023-11-23 20:47:10 +02:00
proxy lib/proxy: consistently use gopkg.in/yaml.v2 across all the code 2023-09-01 11:16:04 +02:00
pushmetrics fixed typos in documentation and commandline flags descriptions (#4275) 2023-05-10 02:22:06 -07:00
querytracer lib/querytracer: add missing blank comment line after 3121d76bee 2023-11-15 16:06:25 +01:00
regexutil lib/regexutil: properly handle alternate regexps surrounded by .+ or .* 2023-11-13 18:27:25 +01:00
snapshot lib/snapshot: make golagci-lint happy 2023-11-15 16:06:25 +01:00
storage lib/storage: follow-up for 29cebd82fb 2023-10-31 16:28:09 +01:00
streamaggr lib/streamaggr: respect streamAgg.dropInput with empty stream aggr config (#5213) 2023-10-26 09:31:59 +02:00
syncwg all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
tenantmetrics lib/encoding/zstd: switch back from atomic.Pointer to atomic.Value for map[...]... 2023-07-20 21:54:51 -07:00
timerpool lib/timerpool: use timer pool in concurrency limiters 2019-05-28 17:30:10 +03:00
uint64set lib/uint64set: use repeatable randomizer in tests 2023-01-23 19:24:05 -08:00
vmselectapi lib/vmselect: bump maxSearchQuerySize to 5MB (#5158) 2023-10-16 13:49:06 +02:00
workingsetcache all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:48:26 -07:00
writeconcurrencylimiter lib/writeconcurrencylimiter: initialize concurrencyLimitCh before exporting vm_concurrent_insert_capacity and vm_concurrent_insert_current metrics 2023-02-07 11:08:39 -08:00