mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2025-03-21 15:45:01 +00:00
c05ffa906d
Previously, performance of stream.Parse could be limited by mutex.Lock on callback function. It used shared writeContext. With complicated relabeling rules and any slowness at pushData function, it could significantly decrease parsed rows processing performance. This commit removes locks and makes parsed rows processing lock-free in the same manner as `stream.Parse` processing implemented at push ingestion processing. Implementation details: - Removing global lock around stream.Parse callback. - Using atomic operations for counters - Creating write contexts per callback instead of sharing - Improving series limit checking with sync.Once - Optimizing labels hash calculation with buffer pooling - Adding comprehensive tests for concurrency correctness Benchmark performance: ``` # before BenchmarkScrapeWorkScrapeInternalStreamBigData-10 13 81973945 ns/op 37.68 MB/s 18947868 B/op 197 allocs/op # after goos: darwin goarch: arm64 pkg: github.com/VictoriaMetrics/VictoriaMetrics/lib/promscrape cpu: Apple M1 Pro BenchmarkScrapeWorkScrapeInternalStreamBigData-10 74 15761331 ns/op 195.98 MB/s 15487399 B/op 148 allocs/op PASS ok github.com/VictoriaMetrics/VictoriaMetrics/lib/promscrape 1.806s ``` Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8159 --------- Signed-off-by: Maksim Kotlyar <kotlyar.maksim@gmail.com> Co-authored-by: Roman Khavronenko <hagen1778@gmail.com> |
||
|---|---|---|
| .. | ||
| csvimport | ||
| datadogsketches | ||
| datadogutils | ||
| datadogv1 | ||
| datadogv2 | ||
| graphite | ||
| influx | ||
| native/stream | ||
| newrelic | ||
| opentelemetry | ||
| opentsdb | ||
| opentsdbhttp | ||
| prometheus | ||
| promremotewrite/stream | ||
| protoparserutil | ||
| vmimport | ||