Aliaksandr Valialkin
18d6f293f7
lib/fs: consolidate *RemoveAll* funcs into a single MustRemoveAll func
...
The func syncs parent dir in order to persist directory removal
in the event of power loss
2019-06-12 01:55:18 +03:00
Aliaksandr Valialkin
3437c30180
all: try hard removing directory with contents
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
2019-06-11 01:58:08 +03:00
Aliaksandr Valialkin
eea7da8e0c
app/vmselect/promql: prevent from count_values explosion of timeseries, which could result in OOM
2019-06-11 01:03:18 +03:00
Aliaksandr Valialkin
e87a602209
app/vmselect/promql: skip superflouos timestamps copying in count_values
2019-06-11 00:44:09 +03:00
Aliaksandr Valialkin
ec84febc1c
app/vmselect/promql: remove superflouos timeseries copy in histogram_quantile func
2019-06-11 00:39:35 +03:00
Aliaksandr Valialkin
1fab34fb5c
app/vmselect/promql: remove superflouos timeseries copy in union func
2019-06-11 00:35:09 +03:00
Aliaksandr Valialkin
a6f368499d
app/vmselect/promql: skip NaN values in count_values func
2019-06-10 22:42:41 +03:00
Aliaksandr Valialkin
945894e049
app/vmselect: properly handle empty label (aka __name__) in LabelEntries handler
2019-06-10 19:55:02 +03:00
Aliaksandr Valialkin
75a0acf72d
app/vmselect: add /api/v1/labels/count handler for quick detection of labels with the maximum number of distinct values
2019-06-10 19:54:55 +03:00
Aliaksandr Valialkin
547bcdce63
app/vmstorage: enable compression of responses to vmselect by default
...
This should save vmstorage => vmselect network bandwidth in common case
when recently added data is queried.
2019-06-10 14:54:59 +03:00
Aliaksandr Valialkin
d54f5fec0b
lib/storage: skip adaptive searching for tag filter matching the minimum number of metrics if the identical previous search didn't found such filter
...
This should improve speed for searching metrics among high number of time series
with high churn rate like in big Kubernetes clusters with frequent deployments.
2019-06-10 14:07:47 +03:00
Aliaksandr Valialkin
4c3913290a
app/vmstorage: add missing _total suffixes to newly added metrics
2019-06-09 22:11:41 +03:00
Aliaksandr Valialkin
d882afa905
lib/storage: optimize time series lookup for recent hours when the db contains many millions of time series with high churn rate (aka frequent deployments in Kubernetes)
2019-06-09 19:14:04 +03:00
Aliaksandr Valialkin
5fcdb4a59a
app/vminsert: improve handling of unhealthy vmstorage nodes
...
* Spread load evenly among remaining healthy nodes instead of hammering
the next node after the unhealthy node.
* Make sure that the packet is flushed to storage node before returning success.
Previously packets could stay in local buffers and thus lost on connection errors.
* Keep rows in the limited memory when all the storage nodes are unhealthy.
2019-06-09 00:42:36 +03:00
Aliaksandr Valialkin
0f64673327
app/vminsert/concurrencylimiter: typo fix in the error message
2019-06-08 22:43:56 +03:00
Aliaksandr Valialkin
89a113cb5d
app/vminsert: really fix #60
...
ReadLinesBlock may accept dstBuf with non-zero length. In this case the last line without trailing newline isn't read.
Fix this by comparing len(dstBuf) to 0 instead of its original length.
2019-06-07 23:40:10 +03:00
Aliaksandr Valialkin
e1c45b314a
app/vminsert: properly read trailing line without newline in the end
...
This fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/60
2019-06-07 23:18:34 +03:00
Aliaksandr Valialkin
8cf0a0e59c
app/vminsert: split vm_rows_inserted_total into per-(accountID, projectID) metrics
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/59
2019-06-07 22:11:20 +03:00
Aliaksandr Valialkin
913f888d0c
app/vmselect/promql: properly handle {__name__ op "string"} queries
...
This has been broken in 7294ef333ad26f4f6578b783e97649e58b1f8945 .
2019-06-07 02:02:09 +03:00
Aliaksandr Valialkin
11979e4d85
app/vmselect/prometheus: report about incorrect time or duration instead of silently using the default value
...
This should prevent from incorrect usage of the querying API.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/52
2019-06-06 22:17:15 +03:00
Aliaksandr Valialkin
5f2aa4539a
app/vminsert: add multi-tenancy support for OpenTSDB and Graphite ingestion via custom tags
...
* VictoriaMetrics_AccountID tag may be used for setting AccountID
* VictoriaMetrics_ProjectID tag may be used for setting ProjectID
2019-06-06 18:07:30 +03:00
Aliaksandr Valialkin
8f4790625d
app/vmselect/promql: return the correct time series from quantile
...
Previously arbitrary time series could be returned from `quantile`
depending on sort order for the last data point in the selected range.
Fix this by returning the calculated time series.
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/55
2019-06-06 17:33:53 +03:00
Aliaksandr Valialkin
2ff0d595b0
app/vmselect/promql: add -search.disableCache flag for disabling response caching
...
This may be useful for data back-filling, when the response caching
could interfere badly with newly added data points with timestamps
in the past.
2019-06-04 17:30:41 +03:00
Aliaksandr Valialkin
ba58af9d8c
app/vminsert/influx: take into account all the tags for consistent hash calculations
2019-06-03 22:54:21 +03:00
Aliaksandr Valialkin
db21d46417
app/vminsert: emulate influx/query request, which is required for TSBS benchmark
2019-06-03 18:39:46 +03:00
Aliaksandr Valialkin
31d6566aff
app/vminsert: accept data on /insert/<accountID>/prometheus/api/v1/write
2019-06-03 18:18:09 +03:00
Aliaksandr Valialkin
a06b7f7f84
app/vmselect/netstorage: remove spammy error message when certain vmstorage nodes are unavailable during query execution
...
The amount of partial responses may be tracked by `vm_partial_search_results_total` metric.
2019-06-03 17:09:50 +03:00
Aliaksandr Valialkin
53242105fb
app/vmselect/promql: allow escaping identifiers with \ and \xXX
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/42
2019-05-31 17:35:54 +03:00
Aliaksandr Valialkin
ee776ca8fc
app/vminsert: add -maxConcurrentInserts command-line flag for limiting the number of concurrent inserts
2019-05-29 12:40:22 +03:00
Aliaksandr Valialkin
a4ec139a4a
app/vminsert: reduce memory usage for Influx, Graphite and OpenTSDB protocols
...
Do not buffer per-connection data and just store it as it arrives
2019-05-28 18:47:52 +03:00
Aliaksandr Valialkin
a6d02ff275
lib/timerpool: use timer pool in concurrency limiters
...
This should reduce the number of memory allocations in highly loaded system
2019-05-28 17:30:10 +03:00
Aliaksandr Valialkin
b7a91d6ba7
app/vmselect: update comment according to the updated code
2019-05-26 22:39:09 +03:00
Aliaksandr Valialkin
15d1e15ae6
app/vminsert/influx: try converting string values to numeric values, since Influx agents may send numeric values as strings
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/34
2019-05-26 22:12:55 +03:00
Aliaksandr Valialkin
a2c71f18a3
app/vmselect/promql: misspeling fix
2019-05-25 21:53:48 +03:00
Aliaksandr Valialkin
bdf696ef18
all: fix misspellings
2019-05-25 21:51:24 +03:00
Aliaksandr Valialkin
121a920a18
Makefile: add -s flag to go fmt in make fmt command
2019-05-25 21:44:36 +03:00
Aliaksandr Valialkin
2ff996e276
app/vmselect: log slow queries if their execution time exceeds -search.logSlowQueryDuration
2019-05-24 16:14:46 +03:00
Aliaksandr Valialkin
628708ad76
app/vmselect: consume resultsCh data in exportHandler if writeResponseFunc failed to consume it
2019-05-24 14:54:54 +03:00
Aliaksandr Valialkin
364f4ec3bb
all: remove -p XXXX:XXXX from docker run options, since it is unnesessary if --net=host is set
2019-05-24 12:53:12 +03:00
Aliaksandr Valialkin
f37903adb3
app/vminsert: add -rpc.disableCompression command-line flag for reducing CPU usage at the cost of higher network bandwidth usage
2019-05-24 12:51:07 +03:00
Aliaksandr Valialkin
8e3eb5b39d
app/vmselect/promql: add alias(q, name) function that sets the given name to all the time series in q
2019-05-24 02:42:10 +03:00
Aliaksandr Valialkin
bb048937bc
app/vmselect/promql: add label_transform(q, label, regexp, replacement) function for replacing all the occurences of regexp with replacement in the given label for q
2019-05-23 16:26:07 +03:00
Aliaksandr Valialkin
24578b4bb1
all: open-sourcing cluster version
2019-05-23 00:25:38 +03:00
Aliaksandr Valialkin
1836c415e6
all: open-sourcing single-node version
2019-05-23 00:18:06 +03:00
