Commit graph

4760 commits

Author SHA1 Message Date
Aliaksandr Valialkin
3802ae9269 README.md: recommend checking which metrics will be deleted before deleting them 2019-08-27 15:01:16 +03:00
Artem Navoiev
b0090dbd86 add github actions (#160) 2019-08-27 14:42:46 +03:00
Aliaksandr Valialkin
603a79b357 app/vmstorage: increase default values for search.maxTagKeys, search.maxTagValues and search.maxUniqueTimeseries 2019-08-27 14:29:53 +03:00
Aliaksandr Valialkin
2655220c58 lib/storage: go fmt 2019-08-27 14:29:51 +03:00
Aliaksandr Valialkin
bf915fc0db lib/storage: report proper maxMetrics limit when more than -search.maxUniqueTimeseries series match the given filters 2019-08-27 14:21:42 +03:00
Aliaksandr Valialkin
2fc157ff7a lib/storage: properly handle (?i) in the tag filter regexp
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/161
2019-08-26 00:44:45 +03:00
Aliaksandr Valialkin
0dc0006f34 lib/storage: calculate the maximum number of rows per small part from -memory.allowedPercent
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/159

This simplifies error detection additionally to the `vm_rows_ignored_total` counters.
2019-08-25 15:31:47 +03:00
Aliaksandr Valialkin
4b688fffee lib/storage: calculate the maximum number of rows per small part from -memory.allowedPercent
This should improve query speed over recent data on machines with big amounts of RAM
2019-08-25 14:41:12 +03:00
Aliaksandr Valialkin
1402a6b981 lib/storage: properly limit the number of output rows in small and big parts storage
Previously small parts storage didn't take into account the available disk space for big parts.
2019-08-25 14:41:12 +03:00
Aliaksandr Valialkin
3308279c4e lib/storage: remove outdated comment on maxRowsPerSmallPart
The commend became outdated after the commit ed6ac1a5df027f0dfc22448e3b27c26b6f77c67a,
which stops merging of small parts on graceful shutdown instead of waiting
for their completion.
2019-08-25 13:47:32 +03:00
Aliaksandr Valialkin
fb909cf710 app/vminsert/influx: set db label only if Influx line doesnt have db tag 2019-08-24 13:52:48 +03:00
Aliaksandr Valialkin
c4e75f09dc README.md: mention that -retentionPeriod must cover the backfilled data 2019-08-24 13:52:48 +03:00
Aliaksandr Valialkin
fb8840ac38 vendor: update github.com/valyala/quicktemplate from v1.1.1 to v1.2.0 2019-08-24 13:41:15 +03:00
Aliaksandr Valialkin
9c9221d1b2 app/vminsert: skip empty tags 2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
70ca018a57 app/vminsert/opentsdbhttp: skip invalid rows and continue parsing the remaining rows
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb-http"}` metric
2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
4266091e4f app/vminsert/opentsdb: skip invalid rows and continue parsing the remaining rows
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb"}` metric
2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
8001d29b6e app/vminsert/graphite: skip invalid rows and continue parsing the remaining rows
Invalid rows are logged and counted in `vm_rows_invalid_total{type="graphite"}` metric
2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
9d3f1fcbb9 app/vminsert/influx: skip invalid rows and continue parsing the remaining rows
Invalid influx lines are logged and counted in `vm_rows_invalid_total{type="influx"}` metric.
2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
ba7b3806be app/vminsert/influx: do not allow escaping newline char, since they dont occur in real life
The prefious report with escaped newline chars in influx line protocol was false alarm.
2019-08-23 18:42:05 +03:00
Aliaksandr Valialkin
7fa88c6efc app/vminsert/opentsdbhttp: allow timestamp as float64 and as string, since it occurs in real life 2019-08-23 18:35:41 +03:00
Aliaksandr Valialkin
4da34b11f8 app/vminsert/influx: handle \r\n aka crlf influx line endings from windows world
Such lines exist in real life.
2019-08-23 18:28:49 +03:00
Aliaksandr Valialkin
a18317adbc app/vminsert/influx: allow escaping newline char
Though newline char isn't mentioned in escape rules at https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_tutorial/ ,
there are reports that such chars occur in real life
2019-08-23 15:14:46 +03:00
Aliaksandr Valialkin
44d7fc599d app/vminsert/influx: skip comments starting with # in influx line protocol 2019-08-23 14:43:09 +03:00
Aliaksandr Valialkin
dce6079379 README.md: add a section about Go profiling 2019-08-23 13:37:09 +03:00
Aliaksandr Valialkin
98419c00ef vendor: make vendor-update 2019-08-23 10:02:10 +03:00
Aliaksandr Valialkin
ac004665b5 all: return 503 http error if service is temporarily unavailable
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/156
2019-08-23 09:55:07 +03:00
Aliaksandr Valialkin
8c03a8c4b4 app/vminsert: allow setting the maximum number of labels per time series via -maxLabelsPerTimeseries 2019-08-23 08:45:26 +03:00
Aliaksandr Valialkin
8a126c2865 README.md: mention that VictoriaMetrics supports enterprise workloads 2019-08-22 18:00:47 +03:00
Aliaksandr Valialkin
380cae23a0 lib/storage: add benchmarks for regexp filter match / mismatch
These benchmarks allow estimate the performance of regexp filters in promql
2019-08-22 16:36:42 +03:00
Aliaksandr Valialkin
1272e407b2 app/vmselect/promql: attempt to repair invalid bucket counts passed to histogram_quantile
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/136
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/154
2019-08-22 14:39:46 +03:00
Aliaksandr Valialkin
5f33fc8e46 app/vminsert: add ability to ingest data via HTTP OpenTSDB /api/put requests
This is manual merge of the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/152
Thanks to nustinov@gmail.com for the initial pull request.
2019-08-22 12:28:32 +03:00
Aliaksandr Valialkin
ec8125606d app/vminsert/opentsdb: fix BenchmarkRowsUnmarshal by adding missing put prefixes to each line 2019-08-21 19:14:47 +03:00
Aliaksandr Valialkin
f4a38f7fb1 app/vmselect/promql: fix panic on -search.disableCache
Reset the cache if it is disabled instead of stopping, since it is stopped on graceful shutdown.
2019-08-21 17:11:52 +03:00
Aliaksandr Valialkin
ab740afd0d app/vmselect/promql: explain why empty timeseries arent removed in transformLabelValue 2019-08-21 11:29:24 +03:00
Aliaksandr Valialkin
7b5168adfb app/vmselect/promql: remove NaNs from /api/v1/query_range output like Prometheus does
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/153
2019-08-20 23:01:41 +03:00
Aliaksandr Valialkin
a0d480fbf3 app/vmselect/promql: pre-allocate memory for map for checking for duplicate timeseries
This should reduce memory allocations for big number of timeseries
2019-08-20 23:01:39 +03:00
Aliaksandr Valialkin
0dfc1ace53 README.md: add a section about backfilling 2019-08-20 00:34:51 +03:00
Aliaksandr Valialkin
d3fd113a80 app/vmselect/promql: add label_value(q, label_name) func, which returns numeric value labels with name label_name in q 2019-08-20 00:28:34 +03:00
Aliaksandr Valialkin
4f738c8a15 lib/storage: try slower path for searching the tag filter with the minimum number of matching time series before giving up with increase -search.maxUniqueTimeseries error 2019-08-19 16:04:21 +03:00
Aliaksandr Valialkin
dd86e6130c app/vmselect/promql: independently track offset hints for tStart and tEnd
This should improve performance if timeseries starts or ends on the selected time range
2019-08-19 13:40:14 +03:00
Aliaksandr Valialkin
6a27657d73 app/vmselect/promql: optimize search for timestamp boundaries in rollupConfig.Do
This should improve the performance of queries over big number of time series
with big number of output points.
2019-08-19 13:03:29 +03:00
Aliaksandr Valialkin
c23b66a1ad lib/storage: pre-allocate memory for blockHeader slice in unmarshalBlockHeaders
This reduces memory usage and memory fragmentation when working with big number of time series
2019-08-19 12:46:33 +03:00
Aliaksandr Valialkin
be39414f9c deployment/docker: switch Go builder from go1.12.8 to go1.12.9 2019-08-18 22:07:58 +03:00
Aliaksandr Valialkin
e74fb23189 app/vmselect/promql: add scrape_interval(q[d]) function, which would return scrape interval for q over d 2019-08-18 21:08:26 +03:00
Aliaksandr Valialkin
582fdc059a app/vmselect/promql: hande comparisons with NaN similar to Prometheus
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150
2019-08-18 00:25:50 +03:00
Aliaksandr Valialkin
1c108fc494 app/vmselect/promql: add lifetime(q[d]) function, which returns the lifetime of q over d in seconds.
This function is useful for determining time series lifetime.
`d` must exceed the expected lifetime of the time series, otherwise
the function would return values close to `d`.
2019-08-16 11:59:32 +03:00
Aliaksandr Valialkin
d6b5ed6d39 app/vmselect/promql: fix corner-case calculation for ideriv 2019-08-16 11:59:28 +03:00
Aliaksandr Valialkin
639b14e8ab app/vmselect/promql: properly handle corner cases for rollup functions 2019-08-15 23:29:59 +03:00
Aliaksandr Valialkin
483de1cc06 lib/workingsetcache: automatically detect when it is better to double cache capacity 2019-08-15 22:57:55 +03:00
Aliaksandr Valialkin
9e0896055d deployment/docker: switch Go builder from go1.12.7 to go1.12.8 2019-08-15 20:43:36 +03:00