Commit graph

82 commits

Author SHA1 Message Date
Aliaksandr Valialkin
62a7353479 app/vmselect/prometheus: set start arg in /api/v1/series to the minimum allowed time by default as Prometheus does
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/91
2019-07-11 17:10:14 +03:00
Aliaksandr Valialkin
54bd21eb4a app/vmselect/prometheus: convert negative times to 0, since they arent supported by the storage 2019-07-11 17:07:20 +03:00
Artem Navoiev
cd4833d3d0 integration tests 2019-07-11 15:48:08 +03:00
Aliaksandr Valialkin
101fa258e5 app/vmstorage: prepare for integration tests with multiple Init / Stop cycles 2019-07-11 15:34:50 +03:00
Aliaksandr Valialkin
a336bb4e22 app/vmselect/promql: reduce RAM usage for aggregates over big number of time series
Calculate incremental aggregates for `aggr(metric_selector)` function instead of
keeping all the time series matching the given `metric_selector` in memory.
2019-07-10 13:04:39 +03:00
Aliaksandr Valialkin
1fe6d784d8 all: consistency renaming: bytesSize -> sizeBytes 2019-07-10 00:47:36 +03:00
Aliaksandr Valialkin
55fe36149c app/vmselect/promql: mention -search.logSlowQueryDuration flag value in the slow query log message 2019-07-10 00:41:24 +03:00
Aliaksandr Valialkin
9203170eb2 app/vmselect/promql: extract rmoeveGroupTags function for removing unneeded tags from MetricName according to the given modifierExpr 2019-07-09 23:20:48 +03:00
Aliaksandr Valialkin
2db685c19c app/vmselect/promql: properly preserve metric name after applying functions in any case from transformFuncsKeepMetricGroup 2019-07-09 23:10:35 +03:00
Aliaksandr Valialkin
40a6c0d672 app/vmselect/prometheus: typo fix 2019-07-07 23:34:23 +03:00
Aliaksandr Valialkin
1371024747 app/vmselect/prometheus: handle minTime and maxTime values that may be set by Promxy or Prometheus client
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/88
2019-07-07 21:53:48 +03:00
Aliaksandr Valialkin
4dbd642c86 app/vmselect/promql: remove empty timeseries left after topk call 2019-07-04 19:42:39 +03:00
Aliaksandr Valialkin
56c154f45b all: add vm_data_size_bytes metrics for easy monitoring of on-disk data size and on-disk inverted index size 2019-07-04 19:42:30 +03:00
Aliaksandr Valialkin
9a4b2b8315 app/vmselect/prometheus: update adjustLastPoints function
- Do not overwrite last points by the previous NaNs, since this may result in empty time series.
- Overwrite the last 2 points instead of 3. This should be enough in most cases.
2019-07-04 09:14:18 +03:00
Aliaksandr Valialkin
e06866005d app/vmselect/promql: gracefully handle duplicate timestamps in irate and rollup_rate funcs
Previously such timestamps result in `+Inf` results. Now the previous timestamp is used
for the calculations.
2019-07-03 12:39:55 +03:00
Aliaksandr Valialkin
b9166a60ff app/vmselect: do not return empty time series in /api/v1/query result 2019-07-01 17:16:34 +03:00
Aliaksandr Valialkin
74c0fb04f3 app/vmselect/promql: consistency renaming: candlestick -> rollup_candlestick 2019-06-29 03:13:02 +03:00
Aliaksandr Valialkin
55bcf60ea6 app/vmselect: fix 32bit arm build
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/83
2019-06-27 19:36:17 +03:00
Aliaksandr Valialkin
796b010139 app/vmselect: add candlestick(m[d]) func for returning open, close, low and high rollups on the given time range d
This function is frequently used in financial apps. See https://en.wikipedia.org/wiki/Candlestick_chart
2019-06-27 18:46:13 +03:00
Jiri Tyr
e0fccc6c60 Change the default influxMeasurementFieldSeparator 2019-06-26 13:22:03 +03:00
Aliaksandr Valialkin
1f7d9a213a app/vminsert: fix inifinite loop when reading two lines without newline in the end
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/82
2019-06-26 02:51:56 +03:00
Aliaksandr Valialkin
844ce4731e app/vmselect/promql: suppress error when template func is used inside modifier list. Just leave it as is
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/78
2019-06-25 20:43:22 +03:00
Aliaksandr Valialkin
feacfffe89 app/vmselect/promql: increase default value for -search.maxPointsPerTimeSeries from 10k to 30k
This may be required for subqueries with small steps. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/77
2019-06-24 22:53:18 +03:00
Aliaksandr Valialkin
4bb738ddd9 app/vmselect/promql: adjust value returned by linearRegression to the end of time range like Prometheus does
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/71
2019-06-24 22:45:58 +03:00
Aliaksandr Valialkin
90e72c2a42 app/vmselect/promql: add sum2 and sum2_over_time, geomean and geomean_over_time funcs.
These functions may be useful for statistic calculations.
2019-06-24 16:44:44 +03:00
Aliaksandr Valialkin
61926bae01 app/vmselect/promql: adjust the provided window only for range functions with dt in denominator
This should fix range function calculations such as `changes(m[d])` where `d` is smaller
than the scrape interval.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/72
2019-06-23 19:27:31 +03:00
Aliaksandr Valialkin
ee13256f74 app/vmselect/promql: use deriv_fast instead of deriv in ttf, since deriv calculations have been changed recently 2019-06-23 15:54:18 +03:00
Aliaksandr Valialkin
3b3b2f1e6e app/vmselect/promql: adjust ttf calculation, so deriv(freev) for freev=m[d] could be properly calculated 2019-06-23 14:31:19 +03:00
Aliaksandr Valialkin
146c6e1f72 app/vmselect/promql: typo fixes in comments 2019-06-21 23:22:59 +03:00
Aliaksandr Valialkin
d261fa2885 app/vmselect/promql: add deriv_fast function for calculating fast derivative
`deriv_fast` calculates derivative based on the first and the last point on the interval
instead of calculating linear regression based on all the data points on the interval.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/73
2019-06-21 23:05:39 +03:00
Aliaksandr Valialkin
5b47c00910 app/vmselect/promql: use linear regression in deriv func like Prometheus does
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/73
2019-06-21 22:59:46 +03:00
Aliaksandr Valialkin
9e1119dab8 app/vmselect/promql: ajdust data model to the model used in Prometheus
Do not take into account data points on the range `[timestamp .. timestamp+step)`
when calculating value on the given `timestamp`.
Use only data points from the past when performing these calculations like Prometheus does.

This should reduce discrepancies between results returned by VictoriaMetrics
and results returned by Prometheus.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/72
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/71
2019-06-21 21:54:48 +03:00
Aliaksandr Valialkin
47a3228108 app/vmselect/promql: do not strip __name__ form time series after binary comparison operation
Example:

  foo > 10

Would leave `foo` name for all the matching time series on the left.
2019-06-21 13:09:38 +03:00
Aliaksandr Valialkin
973e4b5b76 app/vmselect/promql: remove unused func keepLastValue; updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69 2019-06-20 14:35:11 +03:00
Aliaksandr Valialkin
7aadec8e3c app/vmselect/promql: typo fix; updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69 2019-06-20 14:33:47 +03:00
Aliaksandr Valialkin
4b2523fb40 app/vminsert/opentsdb: remove unused const maxReadPacketSize; update https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69 2019-06-20 14:30:06 +03:00
Aliaksandr Valialkin
70ba36fa37 app/vmselect/prometheus: return better error messages on missing args to /api/v1/* 2019-06-20 14:07:55 +03:00
Aliaksandr Valialkin
a78b3dba7f app/vmstorage: add vm_cache_entries{type="storage/hour_metric_ids"} metric for tracking active time series count 2019-06-19 18:36:47 +03:00
Aliaksandr Valialkin
a8d4224828 app/vminsert/graphite: allow skipping timestamps in Graphite plaintext protocol
In this case VictoriaMetrics uses the ingestion time as a timestamp.
2019-06-18 19:04:04 +03:00
Aliaksandr Valialkin
86d3d907a5 app/vminsert/influx: add -influxSkipSingleField flag for using {measurement} instead of {measurement}{separator}{field_name} for Influx lines with a single field
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/66
2019-06-17 19:05:57 +03:00
Aliaksandr Valialkin
269285848f app/vminsert/influx: add -influxMeasurementFieldSeparator flag for the ability to change separator for {measurement}{separator}{field_name} metric name 2019-06-14 10:00:12 +03:00
Aliaksandr Valialkin
beb479b8f1 app/vmselect/promql: use dynamic limit on memory for concurrent queries 2019-06-12 23:18:44 +03:00
Aliaksandr Valialkin
a8db528930 app/vmselect/promql: merge non-overlapping duplicate time series in group_left and group_right joins 2019-06-12 20:32:32 +03:00
Aliaksandr Valialkin
15613e5338 app/vmselect/promql: swap binary operation with modifier in the error message for improved readability 2019-06-12 17:14:39 +03:00
Aliaksandr Valialkin
3237d0309c app/vmselect/promql: list a sample of duplicate time series in the error message for group_left or group_right
This should improve troubleshooting for complex queries involving `group_left` and `group_right` modifiers.
2019-06-12 16:57:37 +03:00
Aliaksandr Valialkin
419197ba08 lib/fs: consolidate *RemoveAll* funcs into a single MustRemoveAll func
The func syncs parent dir in order to persist directory removal
in the event of power loss
2019-06-12 01:53:46 +03:00
Aliaksandr Valialkin
ac7b186f13 all: try hard removing directory with contents
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
2019-06-11 01:57:59 +03:00
Aliaksandr Valialkin
1c33b5937e app/vmselect/promql: prevent from count_values explosion of timeseries, which could result in OOM 2019-06-11 01:03:13 +03:00
Aliaksandr Valialkin
8bb6bc986d app/vmselect/promql: skip superflouos timestamps copying in count_values 2019-06-11 00:44:01 +03:00
Aliaksandr Valialkin
d2be567482 app/vmselect/promql: remove superflouos timeseries copy in histogram_quantile func 2019-06-11 00:39:41 +03:00