github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
305 commits 150 branches 748 tags 347 MiB
Commit graph

111 commits

Author SHA1 Message Date
Aliaksandr Valialkin
09fc6e22e5 all: use workingsetcache instead of fastcache 
This should reduce the amount of RAM required for processing time series
with non-zero churn rate.

The previous cache behavior can be restored with `-cache.oldBehavior` command-line flag.
 2019-08-13 21:39:34 +03:00
Aliaksandr Valialkin
b8bb74ffc6 app/vmstorage: add vm_concurrent_addrows_* metrics for tracking concurrency for Storage.AddRows calls 
Track also the number of dropped rows due to the exceeded timeout
on concurrency limit for Storage.AddRows. This number is tracked in `vm_concurrent_addrows_dropped_rows_total`
 2019-08-06 15:08:33 +03:00
Aliaksandr Valialkin
5c83f8e203 app: add vm_concurrent_ metrics for visibility in concurrency limiters for vminsert and vmselect 2019-08-05 18:30:57 +03:00
Aliaksandr Valialkin
99e048c9df app/vmselect: allow passing match[], start and time to /api/v1/label/<label_name>/values 
`/api/v1/label/<label_name>/values?match[]=q` emulates emulates `label_values(q, <label_name>)`
call in Grafana templating.
 2019-08-04 23:09:21 +03:00
Aliaksandr Valialkin
47e4b50112 app/vmselect: optimize /api/v1/series by skipping storage data 
Fetch and process only time series metainfo.
 2019-08-04 23:01:28 +03:00
Aliaksandr Valialkin
241170dc05 app/vmselect/prometheus: prevent from fetching and scanning all the data on /api/v1/searies call by default 2019-08-04 19:42:36 +03:00
Aliaksandr Valialkin
1c69f4eadc app/vmselect/promql: tune automatic window adjustement 
Increase the windows adjustement for small scrape intervals,
since they usually have higher jitter.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/139
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/134
 2019-08-04 19:34:05 +03:00
Aliaksandr Valialkin
8d93b15b86 app/vmselect/promql: further increase the allowed jitter for scrape interval 
Real-world production data shows higher jitter than 1/8 of scrape interval.
This may results in gaps on the graph. So increase the allowed jitter to 1/4
of scrape interval in order to reduce the probability of gaps on the graphs
over time series with high jitter for scrape_interval.
 2019-08-02 20:10:23 +03:00
Aliaksandr Valialkin
a9f39168d2 app/vminsert/influx: round automatically generated timestamp according to the given precision arg 2019-08-02 00:24:06 +03:00
Aliaksandr Valialkin
f090b2e917 app/vmselect/promql: tolerate higher jitter in scrape interval 
Allow jitter for up to 1/8 instead of 1/16 for the scrape interval.
This should imrpove graphs when `step` is smaller than the `scrape_interval`.
 2019-08-01 23:26:00 +03:00
Aliaksandr Valialkin
3a38b23fa3 app/vmselect/promql: add vm_slow_queries_total metric for counting slow queries 
The query is slow if its execution time exceeds `-search.logSlowQueryDuration`
 2019-07-31 03:36:37 +03:00
Aliaksandr Valialkin
8bd6f1f6df app/vmselect/promql: return NaN from histogram_quantile if at least a single bucket is broken 2019-07-31 01:18:07 +03:00
Aliaksandr Valialkin
4aaa5c2efc app/vmselect/promql: allow adjusting window for default rollup function 
Default rollup function is `last_over_time`. It must support adjusting
the provided window in order to prevent from gaps on the graph
for window values smaller than scrape interval.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/134
 2019-07-31 00:45:54 +03:00
Aliaksandr Valialkin
10f5a26bec app/vmselect/promql: return NaN values if invalid bucket counts are passed to histogram_quantile 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/136
 2019-07-30 22:05:10 +03:00
Aliaksandr Valialkin
afcc7fb167 app/vmselect/netstorage: improve error message when reading data blocks from storage 
Mention the block number in the error. This should simplify troubleshooting in this code.
 2019-07-28 12:12:35 +03:00
Aliaksandr Valialkin
1eade9b358 app/vminsert: add vm_rows_per_insert summary metric 
This metric should help tuning batch sizes on clients writing data to VictoriaMetrics
 2019-07-27 13:21:46 +03:00
Aliaksandr Valialkin
0168a1b658 package: various fixes 
- Use `-prod` binaries instead of development binaries for both deb and rpm packages.
- Fix binary directory from /usr/sbin to /usr/local/bin as outlined in package/victoria-metrics.service
- Fix binary name from `victoriametrics` to `victoria-metrics-prod` in package/victoria-metrics.service
 2019-07-26 22:31:04 +03:00
Aliaksandr Valialkin
bf6cbb762c app/vminsert: improve error messages for Influx, OpenTSDB and Graphite parsing 
Include in the error message the line which failed to parse.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/127
 2019-07-26 22:08:52 +03:00
Aliaksandr Valialkin
c98725db55 app/vmstorage: consistency renaming for ignored rows metrics 
vm_too_big_timestamp_rows_total -> vm_rows_ignored_total{reason="big_timestamp"}
  vm_too_small_timestamp_rows_total -> vm_rows_ignored_total{reason="small_timestamp"}
 2019-07-26 20:02:06 +03:00
Aliaksandr Valialkin
f586e1f83c lib/storage: add metrics for calculating skipped rows outside the retention 
The metrics are:

    - vm_too_big_timestamp_rows_total
    - vm_too_small_timestamp_rows_total
 2019-07-26 14:11:01 +03:00
Kostya Vasilyev
89eb6d78a4 RPM packaging (#122) 2019-07-25 23:47:41 +03:00
Aliaksandr Valialkin
17096b5750 app/vmselect/promql: return NaN from count() over zero time series 
This aligns `count` behavior with Prometheus.
 2019-07-25 22:02:30 +03:00
Aliaksandr Valialkin
66efa5745f app/vmselect/promql: properly calculate incremental aggregations grouped by __name__ 
Previously the following query may fail on multiple distinct metric names match:

    sum(count_over_time{__name__!=''}) by (__name__)
 2019-07-25 21:53:20 +03:00
Aliaksandr Valialkin
162f1fb1b7 all: small updates after PR #114 2019-07-23 19:54:50 +03:00
Roman Khavronenko
5bf4e5ffb5 all: add Pure Go build (pull request #114) 
Updates #94
 2019-07-23 19:26:39 +03:00
Kostya Vasilyev
8c3629a892 Debian packaging (#116) 
* initial commit of deb packaging

* Incorporated feedback from @valyala:
- Put data directory under /var/lib
- More beef in systemd file
- Packaging for arm64
- Package all target which builds and packages both amd64 and arm64

* Remove PIDFile from systemd unit, useless

* per PR feedback, move debian specific files into deb subdirectory

Updates #107 .
 2019-07-22 17:12:48 +03:00
Aliaksandr Valialkin
575d2f0a91 app/vminsert: use netutil.TCPListener for collecting network-related metrics for Graphite and OpenTSDB TCP traffic 2019-07-15 22:58:00 +03:00
Aliaksandr Valialkin
6a943a6a58 app/vmselect/promql: remove empty time series after applying filters like q > 0 
This should reduce CPU and RAM usage for queries over high number of time series.
 2019-07-12 19:59:27 +03:00
Aliaksandr Valialkin
ab88890523 app/vmselect/promql: parallelize incremental aggregation to multiple CPU cores 
This may reduce response times for aggregation over big number of time series
with small step between output data points.
 2019-07-12 15:52:22 +03:00
Aliaksandr Valialkin
62a7353479 app/vmselect/prometheus: set start arg in /api/v1/series to the minimum allowed time by default as Prometheus does 
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/91
 2019-07-11 17:10:14 +03:00
Aliaksandr Valialkin
54bd21eb4a app/vmselect/prometheus: convert negative times to 0, since they arent supported by the storage 2019-07-11 17:07:20 +03:00
Artem Navoiev
cd4833d3d0 integration tests 2019-07-11 15:48:08 +03:00
Aliaksandr Valialkin
101fa258e5 app/vmstorage: prepare for integration tests with multiple Init / Stop cycles 2019-07-11 15:34:50 +03:00
Aliaksandr Valialkin
a336bb4e22 app/vmselect/promql: reduce RAM usage for aggregates over big number of time series 
Calculate incremental aggregates for `aggr(metric_selector)` function instead of
keeping all the time series matching the given `metric_selector` in memory.
 2019-07-10 13:04:39 +03:00
Aliaksandr Valialkin
1fe6d784d8 all: consistency renaming: bytesSize -> sizeBytes 2019-07-10 00:47:36 +03:00
Aliaksandr Valialkin
55fe36149c app/vmselect/promql: mention -search.logSlowQueryDuration flag value in the slow query log message 2019-07-10 00:41:24 +03:00
Aliaksandr Valialkin
9203170eb2 app/vmselect/promql: extract rmoeveGroupTags function for removing unneeded tags from MetricName according to the given modifierExpr 2019-07-09 23:20:48 +03:00
Aliaksandr Valialkin
2db685c19c app/vmselect/promql: properly preserve metric name after applying functions in any case from transformFuncsKeepMetricGroup 2019-07-09 23:10:35 +03:00
Aliaksandr Valialkin
40a6c0d672 app/vmselect/prometheus: typo fix 2019-07-07 23:34:23 +03:00
Aliaksandr Valialkin
1371024747 app/vmselect/prometheus: handle minTime and maxTime values that may be set by Promxy or Prometheus client 
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/88
 2019-07-07 21:53:48 +03:00
Aliaksandr Valialkin
4dbd642c86 app/vmselect/promql: remove empty timeseries left after topk call 2019-07-04 19:42:39 +03:00
Aliaksandr Valialkin
56c154f45b all: add vm_data_size_bytes metrics for easy monitoring of on-disk data size and on-disk inverted index size 2019-07-04 19:42:30 +03:00
Aliaksandr Valialkin
9a4b2b8315 app/vmselect/prometheus: update adjustLastPoints function 
- Do not overwrite last points by the previous NaNs, since this may result in empty time series.
- Overwrite the last 2 points instead of 3. This should be enough in most cases.
 2019-07-04 09:14:18 +03:00
Aliaksandr Valialkin
e06866005d app/vmselect/promql: gracefully handle duplicate timestamps in irate and rollup_rate funcs 
Previously such timestamps result in `+Inf` results. Now the previous timestamp is used
for the calculations.
 2019-07-03 12:39:55 +03:00
Aliaksandr Valialkin
b9166a60ff app/vmselect: do not return empty time series in /api/v1/query result 2019-07-01 17:16:34 +03:00
Aliaksandr Valialkin
74c0fb04f3 app/vmselect/promql: consistency renaming: candlestick -> rollup_candlestick 2019-06-29 03:13:02 +03:00
Aliaksandr Valialkin
55bcf60ea6 app/vmselect: fix 32bit arm build 
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/83
 2019-06-27 19:36:17 +03:00
Aliaksandr Valialkin
796b010139 app/vmselect: add candlestick(m[d]) func for returning open, close, low and high rollups on the given time range d 
This function is frequently used in financial apps. See https://en.wikipedia.org/wiki/Candlestick_chart
 2019-06-27 18:46:13 +03:00
Jiri Tyr
e0fccc6c60 Change the default influxMeasurementFieldSeparator 2019-06-26 13:22:03 +03:00
Aliaksandr Valialkin
1f7d9a213a app/vminsert: fix inifinite loop when reading two lines without newline in the end 
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/82
 2019-06-26 02:51:56 +03:00