github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
1327 commits 150 branches 748 tags 347 MiB
Commit graph

507 commits

Author SHA1 Message Date
Seva Poliakov
1ae0334e17 Fix graphite minus one timestamp (#609) 
* fix graphite -1 timestamp

* format the graphite fix -1 timestamp
 2020-07-08 14:13:01 +03:00
Aliaksandr Valialkin
fad008df7e lib/storage: clarify out of retention period error message by mentioning -retentionPeriod command-line flag 2020-07-08 13:54:13 +03:00
Aliaksandr Valialkin
fe58462bef lib/storage: reset MetricName->TSID cache after deleting time series 
This should prevent from adding new data points to deleted time series
without the need to check for the deleted time series.

This improves ingestion performance a bit when the `deleted time series ids` aka `dmis` set
contains big number of time series.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/596

Based on the idea from @n4mine at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/604
 2020-07-06 22:01:24 +03:00
Aliaksandr Valialkin
77bb0e6595 lib/fs: clarify description for -fs.disableMmap command-line flag 2020-07-06 14:28:57 +03:00
Aliaksandr Valialkin
0bff96fe4b lib/storage: prioritize data ingestion over heavy queries 
Heavy queries could result in the lack of CPU resources for processing the current data ingestion stream.
Prevent this by delaying queries' execution until free resources are available for data ingestion.

Expose `vm_search_delays_total` metric, which may be used in for alerting when there is no enough CPU resources
for data ingestion and/or for executing heavy queries.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/291
 2020-07-05 19:44:04 +03:00
Aliaksandr Valialkin
6f1d926698 lib/promscrape: use HostClient.DoDeadline instead of HostClient.Do in order to guarantee strict deadline across multiple scrape attempts 2020-07-03 21:33:48 +03:00
Aliaksandr Valialkin
ee03b4ccbd lib/promscrape: prevent from too big deadline misses on scrape retries 
The maximum deadline miss duration is reduced to 2x scrape_interval in the worst case.
By default it is limited to scrape_interval configured for the given scrape target.
 2020-07-03 20:42:09 +03:00
Aliaksandr Valialkin
dfa83a4a35 lib/promscrape: check for nil error before checking for the returned status code when scraping targets 2020-07-03 18:37:25 +03:00
Aliaksandr Valialkin
8bb3622e9d app/vminsert: prevent from adding and/or selecting labels with empty values 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/600
 2020-07-02 23:17:12 +03:00
Aliaksandr Valialkin
6ebac3ab63 app/vminsert: add ability to apply relabeling to all the incoming metrics if -relabelConfig command-line arg points to a file with a list of relabel_config entries 
See https://victoriametrics.github.io/#relabeling
 2020-07-02 20:36:33 +03:00
Aliaksandr Valialkin
2361ad8ab4 lib/promscrape: add ability to set disable_compression and disable_keepalive options in scrape_config section of the config passed to -promscrape.config 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/580
 2020-07-02 14:19:34 +03:00
Aliaksandr Valialkin
0f754bea49 lib/promscrape: add -promscrape.disableKeepAlive command-line flag for disabling http keep-alive connections when scraping targets 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/580
 2020-07-01 02:20:46 +03:00
Aliaksandr Valialkin
4cb3e7595c app/vmstorage: add -denyQueriesOutsideRetention command-line flag for denying queries outside the configured retention 2020-07-01 00:58:42 +03:00
Aliaksandr Valialkin
81e3d4305f lib/httpserver: add Unwrap method to ErrorWithStatusCode, so As and Is functions in standard errors package may properly unwrap the error inside ErrorWithStatusCode 2020-07-01 00:53:49 +03:00
Aliaksandr Valialkin
fe77d661b3 all: use errors.As instead of type assertion for detecting net.Error 2020-07-01 00:16:13 +03:00
Aliaksandr Valialkin
0c4e8aeb2b all: use errors.As for inspecting errors that implement httpserver.ErrorWithStatusCode 2020-07-01 00:03:11 +03:00
Aliaksandr Valialkin
d962568e93 all: use %w instead of %s for wrapping errors in fmt.Errorf 
This will simplify examining the returned errors such as httpserver.ErrorWithStatusCode .
See https://blog.golang.org/go1.13-errors for details.
 2020-06-30 23:33:46 +03:00
Aliaksandr Valialkin
5a43842bd3 lib/promscrape: add missing label sorting for autogenerated metrics 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/592
 2020-06-29 22:39:40 +03:00
Ween
b42cf33c4d Fix Auto metrics relabeled errors (#593) 
* Fix Auto metrics relabeled errors

* Finalize auto-genenated  Labels

* Fix Test Errors

Co-authored-by: xinyulong <xinyulong@kuaishou.com>
 2020-06-29 22:39:39 +03:00
Aliaksandr Valialkin
aad38c8283 lib/promrelabel: properly apply ^ and $ anchors to regex value in Prometheus relabeling rules 2020-06-25 17:19:02 +03:00
Aliaksandr Valialkin
fd7a3d880e lib/fs: go fmt 2020-06-23 23:03:08 +03:00
Aliaksandr Valialkin
08edb90814 lib/fs: fall back to cgo copy for copying the last 4KB of mmaped data 
This probably should fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/581
 2020-06-23 22:55:56 +03:00
Aliaksandr Valialkin
3a444bb7bb lib/promrelabel: add support for keep_if_equal and drop_if_equal actions to relabel configs 
These actions may be useful for filtering out unneeded targets and/or metrics if they contain equal label values.
For example, the following rule would leave the target only if __meta_kubernetes_annotation_prometheus_io_port
equals __meta_kubernetes_pod_container_port_number:

  - action: keep_if_equal
    source_labels: [__meta_kubernetes_annotation_prometheus_io_port, __meta_kubernetes_pod_container_port_number]
 2020-06-23 17:29:19 +03:00
Aliaksandr Valialkin
de7e585ac8 lib/promscrape: preserve the previously discovered targets on discovery errors per each job_name 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/582
 2020-06-23 15:42:46 +03:00
Aliaksandr Valialkin
521c657f8d lib/fs: an attempt to fix SIGBUS error by rounding mmap`ed region to multiple of 4KB pages 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/581
 2020-06-23 13:40:20 +03:00
Aliaksandr Valialkin
5fb60dd647 lib/logger: add -loggerErrorsPerSecondLimit for limiting the rate of ERROR messages 2020-06-23 12:42:59 +03:00
Aliaksandr Valialkin
a80e852aab lib/promscrape: retry performing the request to the server for up to 3 times before giving up when it closes keep-alive connections 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/580
 2020-06-23 12:34:12 +03:00
Aliaksandr Valialkin
70bf8218bb app/vmselect/promql: properly override label values from group_left and group_right lists like Prometheus does 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/577
 2020-06-21 16:32:27 +03:00
Aliaksandr Valialkin
50aa34bcbe lib/promscrape/discovery/consul: reduce load on Consul when discovering big number of targets by using background caching 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
 2020-06-20 18:20:07 +03:00
Aliaksandr Valialkin
62e1908986 lib/promscrape: reduce default value for -promscrape.discovery.concurrency from 500 to 100 
This should reduce load on Kubernetes API server and Consul when big number of targets are discovered

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
 2020-06-20 17:53:48 +03:00
Aliaksandr Valialkin
1f2826bae2 lib/promscrape/discovery/ec2: expose __meta_ec2_ami like the next Prometheus release will do 
See b5d61fb66c for details
 2020-06-20 17:45:30 +03:00
Tristan Su
c254b683fd lib/storage: set big/small merge concurrency (#568) 
fixed #567

Co-authored-by: Tristan Su <suqing.sq@alibaba-inc.com>
 2020-06-19 02:21:55 +03:00
Aliaksandr Valialkin
2e5b6220a4 lib/promrelabel: allows regex capture groups in target_label like Prometheus does 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/569
 2020-06-19 02:20:58 +03:00
Aliaksandr Valialkin
4f673a5201 app/vminsert: export metrics for determining ingested rows with dropped or truncated labels 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/565
 2020-06-19 01:12:44 +03:00
Aliaksandr Valialkin
5f3a895c23 lib/storage: add key!=".+" filter additionally to negative filter matching empty value such as key!~"|foo" 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/546
 2020-06-18 20:05:45 +03:00
Aliaksandr Valialkin
c40f29f783 lib/storage: properly match {tag!="|foo"} filters 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/546
 2020-06-10 19:34:37 +03:00
Aliaksandr Valialkin
9f55dea162 lib/httpserver: do not flush and do not close gzip writer if response compression is disabled 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/535
 2020-06-05 21:37:46 +03:00
Aliaksandr Valialkin
cf91a94daf lib/backup: properly create missing parent directories in fs.CreateFile 2020-06-05 19:28:25 +03:00
Aliaksandr Valialkin
ba1f764b29 lib/fs: optimize queries that read recent samples for big number of time series 
Use standard copy() func instead of mmap-aware copy func for reading recently touched mmap-ed data.
This improves read performance by up to 4x.
 2020-06-05 19:10:22 +03:00
Aliaksandr Valialkin
2358d9e41d lib/fs: add a benchmark for ReaderAt.MustReadAt 2020-06-05 19:10:21 +03:00
Aliaksandr Valialkin
8b0d9df51d lib/backup/fsremote: create all the parent directories before creating file in CreateFile 2020-06-05 10:25:28 +03:00
Aliaksandr Valialkin
3d0a0b3785 lib/fs: optimize MustGetFreeSpace performance by caching the results for up to 2 seconds 2020-06-04 13:14:04 +03:00
Vyacheslav Mitrofanov
89a922fb19 allow to use values lower than 10 with the flag -memory.allowedPercent (#531) 
Co-authored-by: Vyacheslav Mitrofanov <vmitrofanov@mfms.ru>
 2020-06-03 23:40:13 +03:00
Aliaksandr Valialkin
304f9499cf lib/bytesutil: prevent from garbage collecting s before returning from ToUnsafeBytes 2020-06-03 00:23:27 +03:00
Aliaksandr Valialkin
eca1afdc20 lib/storage: fix Graphite wildcard matching, which has been broken in v1.36.0 2020-05-28 11:58:47 +03:00
Aliaksandr Valialkin
b0131c79b6 lib/storage: improve search speed for time series matching Graphite whildcards such as foo.*.bar.baz 
Add index for reverse Graphite-like metric names with dots. Use this index during search for filters
like `__name__=~"foo\\.[^.]*\\.bar\\.baz"` which end with non-empty suffix with dots, i.e. `.bar.baz` in this case.

This change may "hide" historical time series during queries. The workaround is to add `[.]*` to the end of regexp label filter,
i.e. "foo\\.[^.]*\\.bar\\.baz" should be substituted with "foo\\.[^.]*\\.bar\\.baz[.]*".
 2020-05-27 21:48:08 +03:00
Aliaksandr Valialkin
301838e7b1 lib/httpserver: properly set status code for empty response 2020-05-24 23:55:55 +03:00
Aliaksandr Valialkin
64bec11c91 lib/httpserver: fix compression for static files 2020-05-24 22:16:51 +03:00
Aliaksandr Valialkin
b747362936 lib/promscrape: mention about -promscrape.maxScrapeSize in the error message when target returns too big response 2020-05-24 14:41:24 +03:00
Aliaksandr Valialkin
be7253c084 lib/httpserver: do not recompress already compressed response 
This shoud help with vmauth issue - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/514
 2020-05-22 16:45:20 +03:00