gozstd - go wrapper for zstd

Features

Vendors upstream zstd without any modifications.
Simple API.
Optimized for speed. The API may be easily used in zero allocations mode.
Compress* and Decompress* functions are optimized for high concurrency.
Proper Writer.Flush for network apps.
Supports the following features from upstream zstd:
- Block / stream compression / decompression with all the supported compression levels and with dictionary support.
- Dictionary building from a sample set. The created dictionary may be saved to persistent storage / transfered over the network.
- Dictionary loading for compression / decompression.
Pull requests for missing upstream zstd features are welcome.

Quick start

How to install `gozstd`?

go get -u github.com/valyala/gozstd

How to compress data?

The easiest way is just to use Compress:

	compressedData := Compress(nil, data)

There is also StreamCompress and Writer for stream compression.

How to decompress data?

The easiest way is just to use Decompress:

	data, err := Decompress(nil, compressedData)

There is also StreamDecompress and Reader for stream decompression.

How to cross-compile gozstd?

If you're cross-compiling some code that uses gozstd and you stumble upon the following error:

# github.com/valyala/gozstd
/go/pkg/mod/github.com/valyala/gozstd@v1.6.2/stream.go:31:59: undefined: CDict
/go/pkg/mod/github.com/valyala/gozstd@v1.6.2/stream.go:35:64: undefined: CDict
/go/pkg/mod/github.com/valyala/gozstd@v1.6.2/stream.go:47:20: undefined: Writer

You can easily fix it by enabling CGO and using a cross-compiler (e.g. arm-linux-gnueabi-gcc):

env CC=arm-linux-gnueabi-gcc GOOS=linux GOARCH=arm CGO_ENABLED=1 go build ./main.go

NOTE: Check #21 for more info.

Who uses gozstd?

VictoriaMetrics

FAQ

Q: Which go version is supported?
A: go1.10 and newer. Pull requests for older go versions are accepted.
Q: Which platforms/architectures are supported?
A: linux/amd64, linux/arm, linux/arm64, freebsd/amd64, darwin/amd64, windows/amd64. Pull requests for other platforms/architectures are accepted.
Q: I don't trust libzstd*.a binary files from the repo or these files dont't work on my OS/ARCH. How to rebuild them? A: Just run make clean libzstd.a if your OS/ARCH is supported.
Q: How do I specify custom build flags when recompiling libzstd*.a? A: You can specify MOREFLAGS=... variable when running make like this: MOREFLAGS=-fPIC make clean libzstd.a.
Q: Why the repo contains libzstd*.a binary files?
A: This simplifies package installation to usual go get without additional steps for building the libzstd*.a

4 KiB Raw Blame History