Aliaksandr Valialkin
22c1e29284
lib/protoparser: propagate callback error to the caller of ParseStream for every supported data ingestion protocols
...
The caller of ParseStream then can generate HTTP 503 responses for non-nil errors occured in callbacks when processing incoming requests.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/896
2020-11-13 13:05:34 +02:00
Aliaksandr Valialkin
ed724d25ba
lib/promscrape: add stream parse mode for efficient scraping of targets that expose millions of metrics
2020-11-01 23:12:26 +02:00
Aliaksandr Valialkin
1481d6d8ff
lib/protoparser: avoid copying of buffer read from the network to unmarshal buffer
2020-09-28 17:19:04 +03:00
Aliaksandr Valialkin
8df33bd5c1
app/{vminsert,vmagent}: improve data ingestion speed over a single connection
...
Process data obtianed from a single connection on all the available CPU cores.
2020-09-28 04:14:51 +03:00
Aliaksandr Valialkin
7072db75cb
lib/protoparser: use 64KB read buffer instead of default 4KB buffer provided by net/http.Server
...
This should reduce syscall overhead when reading big amounts of data
2020-09-28 02:07:19 +03:00
Aliaksandr Valialkin
cd87ca303f
lib/protoparser: report more errors for incorrect timestamps and/or values
...
Previously certain errors in timestamps and/or values could be silently skipped,
which could lead to samples with zero values stored in the database.
Updates https://github.com/VictoriaMetrics/vmctl/issues/25
2020-09-16 02:16:15 +03:00
Aliaksandr Valialkin
d9f7ea1c6e
lib/protoparser: removed unnecessary call to SetReadDeadline when reading a stream of data
...
The OS should return any buffered data in the stream without the need to set the read timeout.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/696
2020-08-15 15:38:35 +03:00
Aliaksandr Valialkin
3efa4e4e1c
lib/protoparser: move common code for detecting timeouts to ReadLinesBlockExt
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/696
2020-08-14 20:39:51 +03:00
Aliaksandr Valialkin
59d95961b8
lib/protoparser: clarify that the string passed to Unmarshal() function must remain available when the parsed rows are in use
2020-08-11 17:05:21 +03:00
Aliaksandr Valialkin
fe77d661b3
all: use errors.As instead of type assertion for detecting net.Error
2020-07-01 00:16:13 +03:00
Aliaksandr Valialkin
d962568e93
all: use %w instead of %s for wrapping errors in fmt.Errorf
...
This will simplify examining the returned errors such as httpserver.ErrorWithStatusCode .
See https://blog.golang.org/go1.13-errors for details.
2020-06-30 23:33:46 +03:00
Aliaksandr Valialkin
3845420a8f
lib: extract common code for returning fast unix timestamp into lib/fasttime
2020-05-14 23:06:50 +03:00
Aliaksandr Valialkin
661cfb03e2
lib/protoparser: add -*TrimTimstamp command-line flags for Influx, Graphite, OpenTSDB and CSV data
...
These flags can be used for reducing disk space usage for timestamps data ingested over the given protocols
2020-04-10 12:44:46 +03:00
Aliaksandr Valialkin
6a1aab88fd
lib/protoparser: metrics renaming: vm_protoparser_<type>_* -> vm_protoparser_*{type="<type>"}
...
This should improve composability of these metrics in PromQL queries
2020-02-28 20:19:59 +02:00
Aliaksandr Valialkin
7ee7614e90
app/vmagent: initial implementation for vmagent
2020-02-23 17:31:54 +02:00
Aliaksandr Valialkin
0cda6afa8e
app/vminsert: move ingestion protocol parsers to lib/protoparser, so they could be re-used in the upcoming vmagent
2020-01-24 16:55:18 +02:00
