Aliaksandr Valialkin
3304dc1e85
lib/mergeset: detect whether we are in test by executable suffix
2019-09-22 23:12:35 +03:00
Aliaksandr Valialkin
d2ed8cb0b2
lib/storage: generate the first tag->metricIDs item in a mergeset block with a single metricID
...
The first item from each mergeset block goes into index (lib/mergeset.blockHeader),
so it must be short in order to reduce index size.
2019-09-22 19:37:50 +03:00
Aliaksandr Valialkin
0a9cb6368e
lib/workingsetcache: remove data race when resetting c.misses
2019-09-22 19:37:09 +03:00
Aliaksandr Valialkin
7d13c31566
lib/{storage,mergeset}: merge tag->metricID
rows into tag->metricIDs
rows for common tag
values
...
This should improve lookup performance if the same `label=value` pair exists
in big number of time series.
This should also reduce memory usage for mergeset data cache, since `tag->metricIDs` rows
occupy less space than the original `tag->metricID` rows.
2019-09-20 22:06:23 +03:00
Aliaksandr Valialkin
272e2f77c9
lib/encoding: optimize UnmarshalUint* and UnmarshalInt*
2019-09-20 13:08:24 +03:00
Aliaksandr Valialkin
7e0c6d4ca6
lib/storage: optimize selecting all the metricIDs by scanning MetricID->TSID entries instead of tag->MetricID entries
...
The number of MetricID->TSID entries is smaller than the number of tag->MetricID entries
and MetricID->TSID entries are usually shorter than tag->MetricID entries.
This should improve performance when selecting all the metricIDs.
2019-09-20 11:57:57 +03:00
Aliaksandr Valialkin
b0c738ae8b
app/vminsert/opentsdbhttp: remove FATAL prefix from logger.Fatalf errors for the sake of consistency with other logger.Fatalf calls
2019-09-19 22:16:11 +03:00
Aliaksandr Valialkin
bf8505353a
lib/mergeset: rename misleading mergeSmallParts to mergeExistingParts
2019-09-19 21:48:36 +03:00
Aliaksandr Valialkin
ebbef20535
lib/mergeset: use sort.IsSorted instead of sort.SliceIsSorted in inmemoryBlock.isSorted in order to reduce memory allocations
2019-09-19 20:13:54 +03:00
Aliaksandr Valialkin
89234f395d
lib/storage: use sort.Sort instead of sort.slice in getSortedMetricIDs
2019-09-19 20:08:13 +03:00
Aliaksandr Valialkin
6e586fa09c
lib/storage: skip duplicate call to intersectMetricIDsWithTagFilter on zero successful intersects
2019-09-19 17:51:10 +03:00
Aliaksandr Valialkin
410f993bf6
lib/mergeset: fill partHeader.firstItem on first block flush
2019-09-19 17:48:22 +03:00
Aliaksandr Valialkin
c05885fb5f
lib/storage: mark tag filter returning errFallbackToMetricNameMatch as useless
...
This will save CPU on subsequent calls for this filter
2019-09-18 19:11:44 +03:00
Aliaksandr Valialkin
e041a196a7
deployment/docker/docker-compose.yml: update Prometheus from v2.3.2 to v2.12.0
2019-09-18 18:30:02 +03:00
Aliaksandr Valialkin
db71c940ea
lib/storage: properly construct keys for uselessTagFiltersCache and register useless negative tag filters there
2019-09-17 23:18:37 +03:00
Artem Navoiev
ccb6dc6925
[deployment] clean up helm ( #185 )
2019-09-16 21:58:16 +03:00
Aliaksandr Valialkin
491b1317f4
vendor: update github.com/valyala/gozstd from v1.6.1 to v1.6.2
2019-09-16 21:50:02 +03:00
Aliaksandr Valialkin
5666112de2
deployment: switch docker image tag from v1.27.2-cluster to v1.27.3-cluster
2019-09-14 11:33:18 +03:00
Aliaksandr Valialkin
ba21622b78
vendor: make vendor-update
2019-09-13 22:49:34 +03:00
Aliaksandr Valialkin
020341d13a
deployment/docker: remove file system paths from the compiled binary
2019-09-13 22:46:07 +03:00
Aliaksandr Valialkin
550a12415a
app/vminsert/netstorage: log network errors when sending data to vmstorage nodes
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/175
2019-09-13 22:26:24 +03:00
Aliaksandr Valialkin
41ef6b060e
lib/mergeset: properly check for sorted block headers
...
Fix a typo for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/181
2019-09-13 21:59:38 +03:00
Aliaksandr Valialkin
ee4585db33
app/vmselect/promql: properly handle subqueries like aggr_func(rollup_func(metric[window:step]))
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/184
2019-09-13 21:42:11 +03:00
hanzai
08cde5e3f4
lib/workingsetcache: adjust switching from mode=split
to mode=whole
smoothly and load cachefile successfully
2019-09-13 19:13:16 +03:00
Aliaksandr Valialkin
828e5f6d26
app/vmselect/promql: binary operation fixes according to Prometheus behaviour
...
The follosing issues were fixed:
- VictoriaMetrics could leave superflouos labels when using `on` or `ignoring` modifiers
- VictoriaMetrics could return `duplicate timeseries` error when using `group_left` or `group_right` with non-empty label list
2019-09-13 17:43:09 +03:00
Artem Navoiev
62b424bc4c
[ci] github actions - run pipeline on pull request. Fix running of test in external PR from forks
2019-09-11 14:54:45 +03:00
Aliaksandr Valialkin
ed50b8792b
app/vminsert/netstorage: reduce the maximum buffer size for rerouted rows, so it occupies less RAM
2019-09-11 14:50:30 +03:00
Aliaksandr Valialkin
b101064f8b
all: report the number of bytes read on io.ReadFull error
...
This should simplify error investigation similar to https://github.com/VictoriaMetrics/VictoriaMetrics/issues/175
2019-09-11 14:50:24 +03:00
Aliaksandr Valialkin
2f4c950fe9
app/vminsert/netstorage: send per-storageNode bufs to vmstorage nodes in parallel
...
This should improve the maximum ingestion throughput.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/175
2019-09-11 14:50:19 +03:00
Aliaksandr Valialkin
694cc59ed1
app/vminsert/netstorage: dynamically adjust timeouts for sending packets from vminsert to vmstorage depending on packet size
...
Bigger packets will have more chances to be sent to vmstorage.
2019-09-11 14:50:14 +03:00
Aliaksandr Valialkin
568ff61dcf
lib/mergeset: dynamically calculate the maximum number of items per part, which can be cached in OS page cache
2019-09-09 11:42:45 +03:00
Artem Navoiev
dc6e4151b0
[ci] bump version of go to 1.13 in github actions config
2019-09-08 19:52:05 +03:00
Aliaksandr Valialkin
9b8af27786
vendor: update github.com/klauspost/compress from v1.7.6 to v1.8.2
2019-09-06 00:49:57 +03:00
Aliaksandr Valialkin
b71d828e84
vendor: update golang.org/x/sys
2019-09-06 00:49:57 +03:00
Aliaksandr Valialkin
1f4e0b722d
deployment: switch docker image tag from v1.27.1-cluster to v1.27.2-cluster
2019-09-05 12:30:03 +03:00
Aliaksandr Valialkin
2c654258ef
lib/fs: add MustStopDirRemover for waiting until pending directories are removed on graceful shutdown
...
This patch is mainly required for laggy NFS. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/162
2019-09-05 11:17:17 +03:00
Aliaksandr Valialkin
d0953e9f02
app/vmselect/promql: ignore grouping by destination label in count_values
, since such a grouping is performed automatically
2019-09-04 19:59:02 +03:00
Aliaksandr Valialkin
2c2bd897dd
lib/storage: remove duplicate tag keys on MetricName.Marshal
call
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/172
2019-09-04 18:13:51 +03:00
Aliaksandr Valialkin
5a9b1d85bb
deployment/docker: switch Go builder from Go 1.12.9 to Go 1.13.0
2019-09-04 17:17:52 +03:00
Aliaksandr Valialkin
f78ffe565f
app/vmselect/promql: do not return artificial points beyond the last point in time series
2019-09-04 16:34:29 +03:00
Aliaksandr Valialkin
a7d5d611fe
app/vmselect/prometheus: do not adjust start
and end
args in /api/v1/query_range
if nocache=1
arg is set
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/171
2019-09-04 13:10:17 +03:00
Aliaksandr Valialkin
82bfe818d0
lib/fs: try harder with directory removal on NFS in the event of temporary lock
...
Do not give up after 11 attempts of directory removal on laggy NFS.
Add `vm_nfs_dir_remove_failed_attempts_total` metric for counting the number of failed attempts
on directory removal.
Log failed attempts on directory removal after long sleep times.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/162
2019-09-04 12:24:41 +03:00
Alfred Krohmer
7cde25bac4
Don't render extraLabels in ServiceMonitor if they are not given
...
This produced invalid YAML before.
2019-09-03 22:11:24 +03:00
Aliaksandr Valialkin
3182e2a66b
deployment: update docker images from v1.27.0-cluster to v1.27.1-cluster
2019-09-03 21:05:50 +03:00
Aliaksandr Valialkin
b08f085082
app/vmselect/promql: reset timeseries name on group_left and group_right as Prometheus does
2019-09-03 20:43:29 +03:00
Aliaksandr Valialkin
458d412bb6
app/vmselect/netstorage: adaptively adjust the maximum inmemory file size for storing temporary blocks
...
The maximum inmemory file size now depends on `-memory.allowedPercent`.
This should improve performance and reduce the number of filesystem calls
on machines with big amounts of RAM when performing heavy queries
over big number of samples and time series.
2019-09-03 13:32:18 +03:00
Aliaksandr Valialkin
0b0153ba3d
lib/storage: invalidate tagFilters -> TSIDS
cache when newly added index data becomes visible to search
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/163
2019-08-29 15:08:44 +03:00
Aliaksandr Valialkin
8504a38214
lib/prombp: apply ba06b47c16
...
The following commands used:
gofmt -r '(uint64(x)&0x7F)<<shift -> uint64(x&0x7F)<<shift' -w ./lib/prompb/
gofmt -r '(int64(x)&0x7F)<<shift -> int64(x&0x7F)<<shift' -w ./lib/prompb/
2019-08-29 13:35:54 +03:00
Aliaksandr Valialkin
fb719bfb23
deployment: update docker images from v1.26.0-cluster to v1.27.0-cluster
2019-08-29 00:09:51 +03:00
Aliaksandr Valialkin
8f81908b1f
.github/workflows: added GitHub actions
2019-08-28 23:11:26 +03:00