Commit graph

9128 commits

Author SHA1 Message Date
Aliaksandr Valialkin
a4b4db9bf6 README.md: add a chapter about downsampling 2019-06-12 01:32:26 +03:00
Aliaksandr Valialkin
c1276edab5 lib/fs: panic with fatal error when directories cannot be removed
Unremoved directories may lead to inconsistent data directory,
so VictoriaMetrics will fail to start next time.

So panic on the first error when trying to remove directory in order
to simplify recover process.
2019-06-12 01:20:54 +03:00
Aliaksandr Valialkin
2322c9a45a lib/fs: attempt #2 to work around NFS issue with directory removal
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
2019-06-12 01:07:05 +03:00
Aliaksandr Valialkin
89b928ff24 vendor: update github.com/VictoriaMetrics/fastcache to v1.5.1 2019-06-11 23:56:08 +03:00
Aliaksandr Valialkin
935bfd7a18 lib/fs: consistency renaming SyncPath -> MustSyncPath, since it doesnt return error 2019-06-11 23:13:49 +03:00
Aliaksandr Valialkin
3dd36b8088 lib/fs: make sure the created directory remains visible in the fs in the event of power loss
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/63
2019-06-11 23:08:09 +03:00
Aliaksandr Valialkin
afb964670a lib/fs: use filepath.Dir instead of filepath.Split, since the filename is unused 2019-06-11 22:54:26 +03:00
Aliaksandr Valialkin
20fc0e0e54 lib/{storage,mergeset}: sync filenames inside part when finalizing the part
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/63
2019-06-11 21:51:13 +03:00
Aliaksandr Valialkin
4d9f088526 README.md: add examples on how to write data with Graphite and OpenTSDB protocols 2019-06-11 21:24:32 +03:00
Aliaksandr Valialkin
82d1707861 README.md: add missing port to example urls 2019-06-11 21:05:24 +03:00
Aliaksandr Valialkin
70d20ce8de README.md: use proper urls for single-node version in examples 2019-06-11 20:33:52 +03:00
Aliaksandr Valialkin
723bf1af7f README.md: add example on how to write data with Influx line protocol to VictoriaMetrics 2019-06-11 20:31:25 +03:00
Aliaksandr Valialkin
ac7b186f13 all: try hard removing directory with contents
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
2019-06-11 01:57:59 +03:00
Roman Khavronenko
cd1bc32158 convert dashboard for provisioning (#62) 2019-06-11 01:07:09 +03:00
Aliaksandr Valialkin
1c33b5937e app/vmselect/promql: prevent from count_values explosion of timeseries, which could result in OOM 2019-06-11 01:03:13 +03:00
Aliaksandr Valialkin
8bb6bc986d app/vmselect/promql: skip superflouos timestamps copying in count_values 2019-06-11 00:44:01 +03:00
Aliaksandr Valialkin
d2be567482 app/vmselect/promql: remove superflouos timeseries copy in histogram_quantile func 2019-06-11 00:39:41 +03:00
Aliaksandr Valialkin
7e7d4d5275 app/vmselect/promql: remove superflouos timeseries copy in union func 2019-06-11 00:35:20 +03:00
Aliaksandr Valialkin
bf9782eaf6 app/vmselect/promql: skip NaN values in count_values func 2019-06-10 22:42:32 +03:00
Aliaksandr Valialkin
cbe692f0e2 app/vmselect: add /api/v1/labels/count handler for quick detection of labels with the maximum number of distinct values 2019-06-10 19:55:38 +03:00
Aliaksandr Valialkin
7b6623558f lib/storage: skip adaptive searching for tag filter matching the minimum number of metrics if the identical previous search didn't found such filter
This should improve speed for searching metrics among high number of time series
with high churn rate like in big Kubernetes clusters with frequent deployments.
2019-06-10 14:07:39 +03:00
Aliaksandr Valialkin
a1351bbaee lib/storage: factor out getTagFilterWithMinMetricIDsCountAdaptive from updateMetricIDsForTagFilters 2019-06-10 13:26:44 +03:00
Aliaksandr Valialkin
b4d707d9bb lib/storage: give clearer names to more functions 2019-06-10 13:01:23 +03:00
Aliaksandr Valialkin
bee7298f81 lib/storage: give more clear names to functions 2019-06-10 12:50:44 +03:00
Aliaksandr Valialkin
dbd217b8f0 lib/storage: test GetSeriesCount 2019-06-10 12:43:34 +03:00
Aliaksandr Valialkin
4d936b1524 lib/storage: make getSeriesCount func indexSearch method 2019-06-10 12:29:11 +03:00
Aliaksandr Valialkin
7354090aad app/vmstorage: add missing _total suffixes to newly added metrics 2019-06-09 22:11:36 +03:00
Aliaksandr Valialkin
d37924900b lib/storage: optimize time series lookup for recent hours when the db contains many millions of time series with high churn rate (aka frequent deployments in Kubernetes) 2019-06-09 19:13:56 +03:00
Aliaksandr Valialkin
c0baa977cf app/vminsert/concurrencylimiter: typo fix in the error message 2019-06-08 22:43:33 +03:00
Aliaksandr Valialkin
f4252f87e6 app/vminsert: really fix #60
ReadLinesBlock may accept dstBuf with non-zero length. In this case the last line without trailing newline isn't read.
Fix this by comparing len(dstBuf) to 0 instead of its original length.
2019-06-07 23:37:03 +03:00
Aliaksandr Valialkin
0b78d228d2 app/vminsert: properly read trailing line without newline in the end
This fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/60
2019-06-07 23:17:59 +03:00
Aliaksandr Valialkin
0371c216a7 deployment/docker: move victoriametrics single-node docker image from valyala/victoria-metrics to victoriametrics/victoria-metrics docker hub path 2019-06-07 11:52:53 +03:00
Aliaksandr Valialkin
c1f18ee48d app/vmselect/promql: properly handle {__name__ op "string"} queries
This has been broken in 7294ef333ad26f4f6578b783e97649e58b1f8945 .
2019-06-07 02:02:04 +03:00
Roman Khavronenko
fbd7044b2b Dashboard update (#57)
* split "pending datapoints" by storage and index pending entities

* update provisioned dVM dashboard
2019-06-07 01:31:45 +03:00
Roman Khavronenko
2afe511d80 Setup Grafana provisioning for docker-compose setup (#50)
* setup Grafana provisioning for docker-compose setup

* review fixes
2019-06-06 23:37:44 +03:00
Seua Polyakov
f4e63cd070 Add SIGINT as stopsignal to docker file (#54)
Add sigint as stopsignal to docker file. You can find more here: https://docs.docker.com/engine/reference/builder/#usage
With this change, the main process inside the container will receive SIGINT, and after a grace period, SIGKILL.
2019-06-06 22:36:21 +03:00
Aliaksandr Valialkin
667115a5c7 app/vmselect/prometheus: report about incorrect time or duration instead of silently using the default value
This should prevent from incorrect usage of the querying API.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/52
2019-06-06 22:18:18 +03:00
Aliaksandr Valialkin
1458450dba app/vmselect/promql: return the correct time series from quantile
Previously arbitrary time series could be returned from `quantile`
depending on sort order for the last data point in the selected range.

Fix this by returning the calculated time series.

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/55
2019-06-06 17:07:31 +03:00
Aliaksandr Valialkin
5a5ba749f2 README.md: add an example on how Influx line protocol is converted into Prometheus data points 2019-06-06 16:08:29 +03:00
Aliaksandr Valialkin
a3e26de45e lib/procutil: typo fix in comment to WaitForSigterm 2019-06-04 17:31:47 +03:00
Aliaksandr Valialkin
53ea90865d app/vmselect/promql: add -search.disableCache flag for disabling response caching
This may be useful for data back-filling, when the response caching
could interfere badly with newly added data points with timestamps
in the past.
2019-06-04 17:30:45 +03:00
Aliaksandr Valialkin
17f0a53068 app/vminsert: explain that /query request emulation is required for TSBS benchmark 2019-06-03 18:40:27 +03:00
Anton Patsev
b03bdb32ff Prettify Table of contents (#47) 2019-06-03 17:31:15 +03:00
Aliaksandr Valialkin
15f59c6df9 deployment/docker: remove trailing whitespace 2019-06-03 14:53:08 +03:00
Artem Navoiev
da45a20491 docker compose for VM 2019-06-03 09:57:33 +02:00
Roman Khavronenko
5859bb9556 Add grafana dashboard for VM (#46) 2019-06-03 00:25:07 +03:00
Aliaksandr Valialkin
28f6c36ab4 lib/storage: tune updating a map with today`s metric ids
- Increase update iterval from 1s to 10s. This should reduce CPU usage
  for large amounts of metric ids with constant churn.
- Reduce pendingTodayMetricIDsLock lock duration during the update.
2019-06-02 21:58:16 +03:00
Aliaksandr Valialkin
4794f894a4 lib/storage: speed up checking metricID existence in the list for the current date 2019-06-02 18:34:08 +03:00
Aliaksandr Valialkin
c7280ba61a vendor: update deps with make vendor-update 2019-06-01 23:39:58 +03:00
Aliaksandr Valialkin
fbd8b03f15 README.md: fixed the link to yum repository source codes 2019-06-01 13:55:44 +03:00