Roman Khavronenko c6e0780f4b app/vmalert: update parsing for instant responses (#6859) ... This change is made in attempt to reduce memory usage by vmalert when parsing big instant responses from VM/Prometheus. In a5c427bac4 vmalert switched from std json lib to fastjson lib in order to reduce amount of allocations, as according to highloaded profiles of vmalert the CPU is mostly spent on GC. But switching to fastjson resulted into excessive memory usage for cases when vmalert has to parse long json lines, which usually happens when instant response contains many `metric` objects. In this change we do a mixed parsing: 1. Slice of `metric` objects is parsed with std lib to keep mem low 2. Each `metric` object is parsed with fastjson to reduce allocs The benchmark results are the following: ``` pkg: github.com/VictoriaMetrics/VictoriaMetrics/app/vmalert/datasource BenchmarkParsePrometheusResponse/Instant_std+fastjson-10 1760 668959 ns/op 280147 B/op 5781 allocs/op MBs allocated at heap: 493.078392 mallocs: 18655472 BenchmarkParsePrometheusResponse/Instant_fastjson-10 6109 198258 ns/op 172839 B/op 5548 allocs/op MBs allocated at heap: 1056.384464 mallocs: 34457184 BenchmarkParsePrometheusResponse/Instant_std-10 1287 950987 ns/op 451677 B/op 9619 allocs/op MBs allocated at heap: 580.802976 mallocs: 13351636 ``` The benchmark function code with mem measurement is available here https://gist.github.com/hagen1778/b9c3ca7f8ca7d6b21aec9777112c5810 The benchmark contains 3 results: 1. Instant_std+fastjson is the implementation in this change 2. Instant_fastjson-10 is the implementation from a5c427bac4 3. BenchmarkParsePrometheusResponse/Instant_std-10 is implementation before a5c427bac4 According to these results, this new implementation is slower than previous, but faster than before switching to fastjson. It also has lower number of allocations and roughly the same memory allocation on heap with GC turned off. --------- Other changes: 1. rm BenchmarkMetrics as it doesn't measure anything 2. simplify BenchmarkParsePrometheusResponse into BenchmarkPromInstantUnmarshal ### Describe Your Changes Please provide a brief description of the changes you made. Be as specific as possible to help others understand the purpose and impact of your modifications. ### Checklist The following checks are **mandatory**: - [ ] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). Signed-off-by: hagen1778 <roman@victoriametrics.com>		2024-08-22 23:56:11 +02:00
..
testdata	app/vmalert: update parsing for instant responses (#6859)	2024-08-22 23:56:11 +02:00
datasource.go	vmalert: add evalAlignment for rule group and fix evalutaion timstamp (#5066)	2023-10-10 12:45:37 +02:00
datasource_test.go	vmalert: support rules backfilling (aka replay) (#1358)	2021-06-09 12:30:54 +03:00
faker.go	vmalert-tool: implement unittest (#4789)	2023-10-16 14:12:06 +02:00
init.go	lib/httputils: parse URL before creating HTTP transport (#6820)	2024-08-16 11:34:49 +02:00
vm.go	app/vmalert: adds idleConnTimeout flags and retry trivial network errors (#6382)	2024-06-03 11:52:58 +02:00
vm_graphite_api.go	vmalert: deprecate cmd-line flag -datasource.lookback (#5877)	2024-03-12 16:17:40 +01:00
vm_prom_api.go	app/vmalert: update parsing for instant responses (#6859)	2024-08-22 23:56:11 +02:00
vm_prom_api_test.go	app/vmalert/datasource: reduce number of allocations when parsing instant responses (#6272)	2024-05-16 09:35:58 +02:00
vm_prom_api_timing_test.go	app/vmalert: update parsing for instant responses (#6859)	2024-08-22 23:56:11 +02:00
vm_test.go	app/vmalert: switch from table-driven tests to f-tests	2024-07-12 22:45:50 +02:00