Commit graph

2798 commits

Author SHA1 Message Date
Aliaksandr Valialkin
3838d224d5 app/vminsert/opentsdb: remove unused const maxReadPacketSize; update https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69 2019-06-20 14:30:02 +03:00
Aliaksandr Valialkin
a3a53647ba app/vmselect/prometheus: return better error messages on missing args to /api/v1/* 2019-06-20 14:07:44 +03:00
Aliaksandr Valialkin
a0c22a6830 app/vmstorage: add vm_cache_entries{type="storage/hour_metric_ids"} metric for tracking active time series count 2019-06-19 18:37:38 +03:00
Aliaksandr Valialkin
08e255a206 README.md: add link to source codes for cluster branch 2019-06-19 17:56:56 +03:00
Aliaksandr Valialkin
24ae3ef532 lib/prompb: remove superflouos bytes copying in ReadSnappy 2019-06-18 21:02:02 +03:00
Aliaksandr Valialkin
d4ed6189d4 app/vminsert/graphite: allow skipping timestamps in Graphite plaintext protocol
In this case VictoriaMetrics uses the ingestion time as a timestamp.
2019-06-18 19:05:46 +03:00
Aliaksandr Valialkin
7b93da5b57 vendor: update golang.org/x/sys 2019-06-18 16:20:09 +03:00
Aliaksandr Valialkin
2ebcd0c98b deployment: update docker images 2019-06-18 13:36:42 +03:00
Aliaksandr Valialkin
e40224d5de lib/flagutil: add NewArray helper func 2019-06-18 10:44:09 +03:00
Aliaksandr Valialkin
02417071cd README.md: use link to Wikipedia about broken gossip protocol instead of a link to document about Gossip protocol removal from Thanos
Thanos removed non-working gossip protocol a few months ago - https://github.com/improbable-eng/thanos/issues/734 ,
so the link to the design document https://github.com/improbable-eng/thanos/blob/master/docs/proposals/approved/201809_gossip-removal.md
became unavailable. So use a link to Wikipedia article instead.

Closes https://github.com/VictoriaMetrics/VictoriaMetrics/pull/68
2019-06-17 19:10:29 +03:00
Aliaksandr Valialkin
3b16d49514 app/vminsert/influx: add -influxSkipSingleField flag for using {measurement} instead of {measurement}{separator}{field_name} for Influx lines with a single field
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/66
2019-06-17 19:05:46 +03:00
Aliaksandr Valialkin
5f0b3589b2 app/vminsert/influx: add -influxMeasurementFieldSeparator flag for the ability to change separator for {measurement}{separator}{field_name} metric name 2019-06-14 09:57:13 +03:00
Aliaksandr Valialkin
14edd122a6 deployment/docker: switch builder from go1.12.5 to go1.12.6 2019-06-14 09:31:56 +03:00
Aliaksandr Valialkin
f9e1d32168 lib/storage: persist metric ids for the current and the previous hour on graceful shutdown
This should improve performance after restart when the db contains a lot of time series
with high time series churn (i.e. metrics from Kubernetes with many pods and frequent deployments)
2019-06-14 07:55:09 +03:00
Aliaksandr Valialkin
ba3cccd471 deployment: update docker images 2019-06-12 23:31:06 +03:00
Aliaksandr Valialkin
947bc16f8c app/vmselect/promql: use dynamic limit on memory for concurrent queries 2019-06-12 23:18:23 +03:00
Aliaksandr Valialkin
fe1b33ef1a README.md: mention that accountID is known as tenant 2019-06-12 21:32:10 +03:00
Aliaksandr Valialkin
8567e3463d app/vmselect/promql: merge non-overlapping duplicate time series in group_left and group_right joins 2019-06-12 20:33:01 +03:00
Aliaksandr Valialkin
345ecc37b6 deployment: update docker images 2019-06-12 18:36:17 +03:00
Aliaksandr Valialkin
88005237f4 app/vmselect/promql: swap binary operation with modifier in the error message for improved readability 2019-06-12 17:14:33 +03:00
Aliaksandr Valialkin
a71381ad2a app/vmselect/promql: list a sample of duplicate time series in the error message for group_left or group_right
This should improve troubleshooting for complex queries involving `group_left` and `group_right` modifiers.
2019-06-12 16:57:34 +03:00
Aliaksandr Valialkin
b0b93e3d50 lib/fs: sync parent dir in MustRemoveAll only if it exists
The parent directory may be non-existing when the deleted directory
didn't exist before the MustRemoveAll call
2019-06-12 02:16:15 +03:00
Aliaksandr Valialkin
18d6f293f7 lib/fs: consolidate *RemoveAll* funcs into a single MustRemoveAll func
The func syncs parent dir in order to persist directory removal
in the event of power loss
2019-06-12 01:55:18 +03:00
Aliaksandr Valialkin
28d9904efc lib/fs: panic with fatal error when directories cannot be removed
Unremoved directories may lead to inconsistent data directory,
so VictoriaMetrics will fail to start next time.

So panic on the first error when trying to remove directory in order
to simplify recover process.
2019-06-12 01:20:10 +03:00
Aliaksandr Valialkin
d897bc3f08 lib/fs: attempt #2 to work around NFS issue with directory removal
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
2019-06-12 01:07:29 +03:00
Aliaksandr Valialkin
f165500225 vendor: update github.com/VictoriaMetrics/fastcache to v1.5.1 2019-06-11 23:57:15 +03:00
Aliaksandr Valialkin
d1ca2e5a2d deployment: update docker images 2019-06-11 23:22:51 +03:00
Aliaksandr Valialkin
51e2e255a6 lib/fs: consistency renaming SyncPath -> MustSyncPath, since it doesnt return error 2019-06-11 23:13:45 +03:00
Aliaksandr Valialkin
3fa4c28f6b lib/fs: make sure the created directory remains visible in the fs in the event of power loss
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/63
2019-06-11 23:08:17 +03:00
Aliaksandr Valialkin
0b7f751f60 lib/fs: use filepath.Dir instead of filepath.Split, since the filename is unused 2019-06-11 22:54:23 +03:00
Aliaksandr Valialkin
cb9e746484 deployment: update docker images 2019-06-11 22:02:08 +03:00
Aliaksandr Valialkin
b491045a4b lib/{storage,mergeset}: sync filenames inside part when finalizing the part
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/63
2019-06-11 21:51:19 +03:00
Aliaksandr Valialkin
3437c30180 all: try hard removing directory with contents
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
2019-06-11 01:58:08 +03:00
Aliaksandr Valialkin
f2a8599908 deployment: update docker images 2019-06-11 01:09:52 +03:00
Aliaksandr Valialkin
eea7da8e0c app/vmselect/promql: prevent from count_values explosion of timeseries, which could result in OOM 2019-06-11 01:03:18 +03:00
Aliaksandr Valialkin
e87a602209 app/vmselect/promql: skip superflouos timestamps copying in count_values 2019-06-11 00:44:09 +03:00
Aliaksandr Valialkin
ec84febc1c app/vmselect/promql: remove superflouos timeseries copy in histogram_quantile func 2019-06-11 00:39:35 +03:00
Aliaksandr Valialkin
1fab34fb5c app/vmselect/promql: remove superflouos timeseries copy in union func 2019-06-11 00:35:09 +03:00
Aliaksandr Valialkin
a6f368499d app/vmselect/promql: skip NaN values in count_values func 2019-06-10 22:42:41 +03:00
Aliaksandr Valialkin
2d7165033a deployment: update docker images 2019-06-10 20:38:18 +03:00
Aliaksandr Valialkin
945894e049 app/vmselect: properly handle empty label (aka __name__) in LabelEntries handler 2019-06-10 19:55:02 +03:00
Aliaksandr Valialkin
75a0acf72d app/vmselect: add /api/v1/labels/count handler for quick detection of labels with the maximum number of distinct values 2019-06-10 19:54:55 +03:00
Aliaksandr Valialkin
547bcdce63 app/vmstorage: enable compression of responses to vmselect by default
This should save vmstorage => vmselect network bandwidth in common case
when recently added data is queried.
2019-06-10 14:54:59 +03:00
Aliaksandr Valialkin
0ccedbdfd2 lib/storage: mention the accountID and projectID in error message when filtering out other (accountID, projectID) entries 2019-06-10 14:43:53 +03:00
Aliaksandr Valialkin
d54f5fec0b lib/storage: skip adaptive searching for tag filter matching the minimum number of metrics if the identical previous search didn't found such filter
This should improve speed for searching metrics among high number of time series
with high churn rate like in big Kubernetes clusters with frequent deployments.
2019-06-10 14:07:47 +03:00
Aliaksandr Valialkin
27e50e86f4 lib/storage: factor out getTagFilterWithMinMetricIDsCountAdaptive from updateMetricIDsForTagFilters 2019-06-10 13:26:00 +03:00
Aliaksandr Valialkin
b69d3dbd0c lib/storage: filter out metricIDs from another (AccountID, ProjectID) in getMetricIDsForRecentHours 2019-06-10 13:05:16 +03:00
Aliaksandr Valialkin
3059ae7be0 lib/storage: give clearer names to more functions 2019-06-10 12:59:33 +03:00
Aliaksandr Valialkin
d3a024d2d6 lib/storage: give more clear names to functions 2019-06-10 12:50:22 +03:00
Aliaksandr Valialkin
00e0760608 lib/storage: test GetSeriesCount 2019-06-10 12:40:33 +03:00