github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
778 commits 150 branches 748 tags 347 MiB
Commit graph

300 commits

Author SHA1 Message Date
Aliaksandr Valialkin
846d7fa7e9 app/vmselect: add sort_by_label(q, label) and sort_by_label_desc(q, label) functions 
This is implementation of https://github.com/prometheus/prometheus/pull/1533 for VictoriaMetrics.
 2020-02-13 17:01:50 +02:00
Aliaksandr Valialkin
e3b18ca1ab lib/mergeset: skip createing temporary part objects when merging source inmemory parts 
This should reduce CPU usage when adding new entries to inverted index.
This should alos prevent from creating stalled cleaner goroutines for the created temporary parts,
since they were never closed.

This should fix the following issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/316 .
 2020-02-13 14:09:13 +02:00
Aliaksandr Valialkin
347aaba79d lib/{storage,mergeset}: use time.Ticker instead of time.Timer where appropriate 
It has been appeared that time.Timer was used in places where time.Ticker must be used instead.
This could result in blocked goroutines as in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/316 .
 2020-02-13 13:21:48 +02:00
Aliaksandr Valialkin
e7d1037210 docs: migrate ExtendedPromQL->MetricsQL in order to be more consistent 2020-02-10 23:03:31 +02:00
Aliaksandr Valialkin
fcdd95a6ef lib/envflag: check for incorrect flag values read from environment vars 2020-02-10 16:09:03 +02:00
Aliaksandr Valialkin
9c5db9400c lib/envflag: add -envflag.enable command-line flag for enabling reading flags from environment vars 
By default flags are read only from command line. They can be read from environment vars if `-envflag.enable` is set.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/311
 2020-02-10 16:09:01 +02:00
Aliaksandr Valialkin
1010a57882 all: allow setting flags via environment vars 
Now flags can be set via environment vars with the same names as flags.
Command-line flags override flags set via env vars.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/311
 2020-02-10 13:31:21 +02:00
Aliaksandr Valialkin
ea66212c93 lib/storage: move -dedup.minScrapeInterval flag outside lib/storage, so it doesnt show up in vminsert in cluster version 2020-02-10 13:07:25 +02:00
Aliaksandr Valialkin
8b360a25e9 lib/logger: initialize output to os.Stderr by default 2020-02-04 22:43:26 +02:00
Aliaksandr Valialkin
1f271a9815 lib/logger: add -loggerOutput command-line flag 
This flag allows changing log output from `stderr` to `stdout` if `-loggerOutput=stdout` is set.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/306
 2020-02-04 21:48:24 +02:00
Aliaksandr Valialkin
49ab3fa076 lib/logger: do not clutter -loggerFormat=json output with stack trace 
This should improve json parsing
 2020-02-04 21:40:20 +02:00
Aliaksandr Valialkin
56d6b8ed0a lib/storage: do not deduplicate blocks with less than 32 samples during merge 
This should improve deduplication accuracy for blocks with higher number of samples.
 2020-02-04 18:41:37 +02:00
Aliaksandr Valialkin
7cde594696 all: do not clash flag description with back-quoted flag types 
See https://golang.org/pkg/flag/#PrintDefaults for more details.
 2020-02-04 15:56:01 +02:00
Aliaksandr Valialkin
9b25a2fb67 lib/fs: remove unused readerAt interface 2020-01-31 15:13:00 +02:00
Aliaksandr Valialkin
e3adc095bd all: add -dedup.minScrapeInterval command-line flag for data de-duplication 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/86
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/278
 2020-01-31 01:18:54 +02:00
Aliaksandr Valialkin
a45f25699c lib/storage: re-use indexSearch inside Storage.prefetchMetricNames 2020-01-31 01:18:53 +02:00
Aliaksandr Valialkin
cb5c39ee70 lib/fs: optimize small reads for ReaderAt.MustReadAt by reading from memory-mapped space instead of reading from file descriptor 
This should improve performance when reading many small blocks.
 2020-01-30 15:16:16 +02:00
Aliaksandr Valialkin
da19fffa08 all: rename ReadAt* to MustReadAt* in order to dont clash with io.ReaderAt 2020-01-30 15:16:16 +02:00
Aliaksandr Valialkin
1332ddc15e lib/storage: pass missing AccountID and ProjectID to searchMetricName 2020-01-30 15:16:16 +02:00
Aliaksandr Valialkin
4ed5e9a7ce lib/storage: pre-fetch metricNames for the found metricIDs in Search.Init 
This should speed up Search.NextMetricBlock loop for big number of found time series.
 2020-01-30 15:16:16 +02:00
Aliaksandr Valialkin
cb2a2f281f lib/mergeset: properly update lastAccesstime in indexBlockCache entries 
This is a follow-up for 6665f10e7b
 2020-01-29 21:21:01 +02:00
Aliaksandr Valialkin
170c1c3a4e app/vmselect/promql: add keep_next_value(q) for filling gaps with the next non-empty value 2020-01-29 00:48:14 +02:00
Aliaksandr Valialkin
a9c1d5b351 app/vminsert: moved -maxInsertRequestSize command-line flag out of lib/prompb in order to prevent its inclusion in vmselect and vmstorage apps 2020-01-28 22:53:50 +02:00
Aliaksandr Valialkin
81ba371eaf lib/logger: fix improperly set skipframes for all the logging functions 2020-01-26 18:34:58 +02:00
Aliaksandr Valialkin
9f595cb2b1 lib/httpserver: log the caller of httpserver.Errorf 
Previously log message contained `httpserver.Errorf`, not it contains the caller of `httpserver.Errorf`, which is more useful.
 2020-01-25 20:18:06 +02:00
Aliaksandr Valialkin
36a1a21d6e lib/protoparser: add parser for Prometheus exposition text format 
This parser will be used by vmagent
 2020-01-24 20:11:19 +02:00
Aliaksandr Valialkin
0cda6afa8e app/vminsert: move ingestion protocol parsers to lib/protoparser, so they could be re-used in the upcoming vmagent 2020-01-24 16:55:18 +02:00
Aliaksandr Valialkin
ea53a21b02 all: consistently log durations in seconds with millisecond precision 
This should improve logs readability
 2020-01-22 18:35:24 +02:00
Aliaksandr Valialkin
40e564eb9c app/vmselect/promql: add range_over_time(m[d]) function for calculating value range for m over d 2020-01-21 19:05:29 +02:00
Aliaksandr Valialkin
9eaa2ab871 app/vmselect/promql: add label_match(q, label, regexp) and label_mismatch(q, label, regexp) functions for filtering out time series with labels matching the given regexp 2020-01-21 15:00:35 +02:00
Aliaksandr Valialkin
62b041e90a lib/{mergeset,storage}: properly update lastAccessTime in index and data block cache entries 2020-01-20 15:00:10 +02:00
Aliaksandr Valialkin
607d4418b8 lib/uint64set: add missing bucket32.b16his values 2020-01-18 14:26:23 +02:00
Aliaksandr Valialkin
e3379537cd lib/uint64set: optimize Set.Union 
This should improve performance for queries over big number of time series
 2020-01-18 13:47:34 +02:00
Aliaksandr Valialkin
5077efd3f7 lib/uint64set: add benchmarks for Set.Union 2020-01-18 13:47:33 +02:00
Aliaksandr Valialkin
a851c75703 lib/storage: skip recovering timestamps order for lossless compression (PrecisionBits=64) 2020-01-17 23:59:19 +02:00
Aliaksandr Valialkin
2084921e64 all: use github.com/klauspost/compress/gzip instead of compress/gzip 
`github.com/klauspost/compress/gzip` is more optimized than `compress/gzip`.
This gives better gzip compression and decompression speeds.
 2020-01-17 23:59:17 +02:00
Aliaksandr Valialkin
ab4d5d72eb lib/uint64set: reduce memory allocations in Set.AppendTo 2020-01-17 22:33:00 +02:00
Aliaksandr Valialkin
476c7fb109 lib/storage: reduce memory allocations when merging metricID sets 2020-01-17 22:10:56 +02:00
Aliaksandr Valialkin
29d21259f0 lib/uint64set: typo fix in Set.Intersect 2020-01-17 18:11:46 +02:00
Aliaksandr Valialkin
ed1d259b10 lib/uint64set: optimize Intersect, Subtract and Union functions 
This should improve performance for queries over big number of time series.
 2020-01-17 16:16:43 +02:00
Aliaksandr Valialkin
68d35357b1 lib/uint64set: improve benchmark for Set.Intersect 2020-01-17 16:16:43 +02:00
Aliaksandr Valialkin
ffe352ad31 lib/uint64set: add benchmark for Set.Intersect 2020-01-17 16:16:43 +02:00
Aliaksandr Valialkin
4b16b7fd11 all: mention command-line flags used for limiting the incoming request size in error messages 
This should improve error logs usability.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/287
 2020-01-16 13:06:43 +02:00
Aliaksandr Valialkin
7d429e2806 lib/uint64set: reduce memory usage in Union, Intersect and Subtract methods 
Iterate items with newly added Set.ForEach method instead of allocating `[]uint64`
slice for all the items before the iteration.
 2020-01-15 12:15:48 +02:00
Aliaksandr Valialkin
caffb0cd01 lib/{mergeset,storage}: fix uint64 counters alignment for 32-bit architectures (GOARCH=386, GOARCH=arm) 2020-01-14 22:47:42 +02:00
Aliaksandr Valialkin
b03ccbf6f7 lib/{storage,mergeset}: gradually remove stale entries from block cache and index caches 
This should reduce memory usage in the long run when old blocks and indexes
aren't accessed anymore.
 2020-01-14 21:38:29 +02:00
Aliaksandr Valialkin
bcd3f0c5bd app/vmselect/promql: add hoeffding_bound_upper(phi, m[d]) and hoeffding_bound_lower(phi, m[d]) functions 
These functions can be used for calculating Hoeffding bounds
for `m` over `d` time range and for the given `phi` in the range `[0..1]`.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/283
 2020-01-11 14:47:13 +02:00
Aliaksandr Valialkin
87a106702b app/vmselect/promql: add aggr_over_time(("aggr_func1", "aggr_func2", ...), m[d]) function 
This function can be used for simultaneous calculating of multiple `aggr_func*` functions
that accept range vector. For example, `aggr_over_time(("min_over_time", "max_over_time"), m[d])`
would calculate `min_over_time` and `max_over_time` for `m[d]`.
 2020-01-10 21:18:12 +02:00
Aliaksandr Valialkin
c314d9a219 app/vmselect/promql: add tmin_over_time(m[d]) and tmax_over_time(m[d]) functions 
These functions return timestamp in seconds for the minimum and maximum value for `m` over time range `d`
 2020-01-10 19:39:34 +02:00
Aliaksandr Valialkin
1029b6ab34 lib/backup/s3remote: check whether the file exists before deleting it 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/284
 2020-01-09 23:20:51 +02:00