Aliaksandr Valialkin
ecddba30fe
app/vminsert/netstorage: increase timeout for pushing data from vminsert to vmstorage by 3x
...
Our clients report that the previous timeout could lead to frequent errors when
vmstorage starts background merge for big parts on slow HDD.
2020-01-21 18:21:49 +02:00
Aliaksandr Valialkin
9eaa2ab871
app/vmselect/promql: add label_match(q, label, regexp)
and label_mismatch(q, label, regexp)
functions for filtering out time series with labels matching the given regexp
2020-01-21 15:00:35 +02:00
Aliaksandr Valialkin
62b041e90a
lib/{mergeset,storage}: properly update lastAccessTime
in index and data block cache entries
2020-01-20 15:00:10 +02:00
Aliaksandr Valialkin
b297fec515
README.md: mention that delete API
shouldnt be used on a regular basis due to non-zero overhead
2020-01-20 13:28:58 +02:00
Aliaksandr Valialkin
d3b4b0f492
docs/FAQ.md: typo fix according to comment from https://www.reddit.com/message/messages/lezkmo
2020-01-18 18:05:22 +02:00
Aliaksandr Valialkin
179c7db4c9
docs/CaseStudies.md: add links to COLOPL talk about VictoriaMetrics
2020-01-18 17:24:00 +02:00
Aliaksandr Valialkin
cbd0452317
app/vminsert: increase default value for -insert.maxQueueDuration
from 30s to 60s
...
This should help catching up with high ingestion rate after VictoriaMetrics restart.
2020-01-18 14:39:30 +02:00
Aliaksandr Valialkin
607d4418b8
lib/uint64set: add missing bucket32.b16his values
2020-01-18 14:26:23 +02:00
Aliaksandr Valialkin
e3379537cd
lib/uint64set: optimize Set.Union
...
This should improve performance for queries over big number of time series
2020-01-18 13:47:34 +02:00
Aliaksandr Valialkin
5077efd3f7
lib/uint64set: add benchmarks for Set.Union
2020-01-18 13:47:33 +02:00
Aliaksandr Valialkin
a851c75703
lib/storage: skip recovering timestamps order for lossless compression (PrecisionBits=64)
2020-01-17 23:59:19 +02:00
Aliaksandr Valialkin
2084921e64
all: use github.com/klauspost/compress/gzip
instead of compress/gzip
...
`github.com/klauspost/compress/gzip` is more optimized than `compress/gzip`.
This gives better gzip compression and decompression speeds.
2020-01-17 23:59:17 +02:00
Aliaksandr Valialkin
ab4d5d72eb
lib/uint64set: reduce memory allocations in Set.AppendTo
2020-01-17 22:33:00 +02:00
Aliaksandr Valialkin
476c7fb109
lib/storage: reduce memory allocations when merging metricID sets
2020-01-17 22:10:56 +02:00
Aliaksandr Valialkin
29d21259f0
lib/uint64set: typo fix in Set.Intersect
2020-01-17 18:11:46 +02:00
Aliaksandr Valialkin
54db08a60f
app/vmselect/netstorage: make fmt
2020-01-17 17:46:20 +02:00
Aliaksandr Valialkin
d21cc2d16a
app/vmselect/netstorage: limit the maximum size for in-memory buffer for temporary blocks file
...
This should reduce memory usage on systems with more than 8GB RAM.
2020-01-17 16:28:28 +02:00
Aliaksandr Valialkin
ed1d259b10
lib/uint64set: optimize Intersect, Subtract and Union functions
...
This should improve performance for queries over big number of time series.
2020-01-17 16:16:43 +02:00
Aliaksandr Valialkin
68d35357b1
lib/uint64set: improve benchmark for Set.Intersect
2020-01-17 16:16:43 +02:00
Aliaksandr Valialkin
b05f6cf11c
app/vmselect: limit the default value for -search.maxConcurrentRequests
, so it plays well on systems with more than 16 vCPUs
...
A single heavy request can saturate all the available CPUs, so let's limit the number of concurrent requests to lower value.
This will give more chances for executing insert path.
2020-01-17 16:16:43 +02:00
Aliaksandr Valialkin
a9f683423c
app/{vminsert,vmselect}: improve error messages when VictoriaMetrics cannot handle too high number of concurrent inserts / selects
2020-01-17 16:16:43 +02:00
Aliaksandr Valialkin
ffe352ad31
lib/uint64set: add benchmark for Set.Intersect
2020-01-17 16:16:43 +02:00
Aliaksandr Valialkin
bdfb219992
make vendor-update
2020-01-16 14:17:48 +02:00
Aliaksandr Valialkin
4b16b7fd11
all: mention command-line flags used for limiting the incoming request size in error messages
...
This should improve error logs usability.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/287
2020-01-16 13:06:43 +02:00
Aliaksandr Valialkin
ce0b602405
app/vmselect/promql: fix panic on sum(aggr_over_time(...))
with incorrect number of args
2020-01-15 16:26:16 +02:00
Aliaksandr Valialkin
7d429e2806
lib/uint64set: reduce memory usage in Union, Intersect and Subtract methods
...
Iterate items with newly added Set.ForEach method instead of allocating `[]uint64`
slice for all the items before the iteration.
2020-01-15 12:15:48 +02:00
Aliaksandr Valialkin
4ecb7f15b6
docs/FAQ.md: add bullet comparison with Cortex and Thanos
2020-01-15 10:47:46 +02:00
Aliaksandr Valialkin
caffb0cd01
lib/{mergeset,storage}: fix uint64 counters alignment for 32-bit architectures (GOARCH=386, GOARCH=arm)
2020-01-14 22:47:42 +02:00
Aliaksandr Valialkin
b03ccbf6f7
lib/{storage,mergeset}: gradually remove stale entries from block cache and index caches
...
This should reduce memory usage in the long run when old blocks and indexes
aren't accessed anymore.
2020-01-14 21:38:29 +02:00
Aliaksandr Valialkin
8a4d4978a3
deployment/docker: update Prometheus from v2.14.0 to v2.15.2 and Grafana from v6.5.0 to v6.5.2
2020-01-12 23:14:56 +02:00
Aliaksandr Valialkin
cbafb7ae59
docs: add a link to VictoriaMetrics subreddit - https://www.reddit.com/r/VictoriaMetrics/
2020-01-12 00:07:06 +02:00
Aliaksandr Valialkin
bcd3f0c5bd
app/vmselect/promql: add hoeffding_bound_upper(phi, m[d])
and hoeffding_bound_lower(phi, m[d])
functions
...
These functions can be used for calculating Hoeffding bounds
for `m` over `d` time range and for the given `phi` in the range `[0..1]`.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/283
2020-01-11 14:47:13 +02:00
Aliaksandr Valialkin
fc01b11ddc
app/vmselect/promql: return continuous values for min_over_time
and max_over_time
when step
is smaller than scrape_interval
2020-01-11 12:47:57 +02:00
Aliaksandr Valialkin
92e00779fa
deployment/docker: switch Go builder from v1.13.5 to v1.13.6
2020-01-11 11:06:12 +02:00
Aliaksandr Valialkin
2cacea8c64
README.md: mention about Prometheus->VictoriaMetrics exporter https://github.com/ryotarai/prometheus-tsdb-dump
2020-01-11 01:28:50 +02:00
Aliaksandr Valialkin
16fb128bbc
app/vmselect/promql: do not take into account the previous point before time window in square brackets for min_over_time
, max_over_time
, rollup_first
and rollup_last
functions
...
This makes the behaviour for these functions similar to Prometheus when processing broken time series with irregular data points
like `gitlab_runner_jobs`. See https://gitlab.com/gitlab-org/gitlab-exporter/issues/50 for details.
2020-01-11 00:26:11 +02:00
Aliaksandr Valialkin
1c445bf7eb
vendor: update github.com/valyala/fastjson from v1.4.2 to v1.4.5
...
This should fix parsing Inf values in `/api/v1/import`. The previous attempt to fix this in VictoriaMetrics v1.32.1 was unsuccessful.
2020-01-10 23:14:34 +02:00
Aliaksandr Valialkin
adc36d00b7
app/vmselect/promql: properly handle aggr(aggr_over_time(...))
2020-01-10 21:57:11 +02:00
Aliaksandr Valialkin
87a106702b
app/vmselect/promql: add aggr_over_time(("aggr_func1", "aggr_func2", ...), m[d])
function
...
This function can be used for simultaneous calculating of multiple `aggr_func*` functions
that accept range vector. For example, `aggr_over_time(("min_over_time", "max_over_time"), m[d])`
would calculate `min_over_time` and `max_over_time` for `m[d]`.
2020-01-10 21:18:12 +02:00
Aliaksandr Valialkin
c314d9a219
app/vmselect/promql: add tmin_over_time(m[d])
and tmax_over_time(m[d])
functions
...
These functions return timestamp in seconds for the minimum and maximum value for `m` over time range `d`
2020-01-10 19:39:34 +02:00
Aliaksandr Valialkin
706b33dc82
docs: fix spelling typos
2020-01-09 23:43:45 +02:00
Aliaksandr Valialkin
1029b6ab34
lib/backup/s3remote: check whether the file exists before deleting it
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/284
2020-01-09 23:20:51 +02:00
Aliaksandr Valialkin
705af61587
app/{vmbackup,vmrestore}: add backup complete
file to backup when it is complete and check for this file before restoring from backup
...
This should prevent from restoring from incomplete backups.
Add `-skipBackupCompleteCheck` command-line flag to `vmrestore` in order to be able restoring from old backups without `backup complete` file.
2020-01-09 15:35:45 +02:00
Aliaksandr Valialkin
7edbd930d5
vendor: update github.com/valyala/fastjson from v1.4.1 to v1.4.2
...
This fixes parsing of `inf` and `nan` values in json lines passed to `/api/v1/import`
2020-01-08 20:48:08 +02:00
Aliaksandr Valialkin
da62e894dd
README.md: remove height="200px"
from logo image, since it is improperly displayed on smartphones
2020-01-08 20:29:37 +02:00
Aliaksandr Valialkin
24a852f900
README.md: typo fix
2020-01-08 14:44:28 +02:00
Aliaksandr Valialkin
7c6df1e51d
app/vmselect/promql: skip rate
calculation for the first point on time series
2020-01-08 14:42:44 +02:00
Aliaksandr Valialkin
7d8d921db9
docs: add references to Remote Write Storage Wars
...
Also mention than VictoriaMetrics uses less RAM than Thanos Store Gateway - see https://github.com/thanos-io/thanos/issues/448 for details.
2020-01-04 23:58:27 +02:00
Aliaksandr Valialkin
53e176ed67
lib/storage: limit maxRaRowsPerPartition by 500K for any number of rawRowsShardsPerPartition
...
This should reduce write amplification for high ingestion rate on multi-CPU systems
2020-01-04 23:58:23 +02:00
Aliaksandr Valialkin
1f941875db
docs/CaseStudies.md: add link to Remote Write Storage Wars
talk from Adidas at PromCon 2019
2020-01-04 16:51:02 +02:00