github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
2105 commits 150 branches 748 tags 347 MiB
Commit graph

768 commits

Author SHA1 Message Date
Aliaksandr Valialkin
d65c03c004 lib/storage: properly determine max rows for output part when merging small parts 2020-12-18 23:14:38 +02:00
Aliaksandr Valialkin
ebf8da3730 lib/{storage,mergeset}: tune background merge process in order to reduce CPU usage and disk IO usage 2020-12-18 20:01:08 +02:00
Aliaksandr Valialkin
2dfa746c91 lib/promscrape: remove ID field from ScrapeWork struct. Use a pointer to ScrapeWork as a key in targetStatusMap 
This simplifies the code a bit.
 2020-12-17 14:32:56 +02:00
Aliaksandr Valialkin
9abb2d6c74 lib/protoparser/prometheus: follow-up commit after 7d38627b9f6f212ae602aea6a72f469fe3c70ba2 
Document the bugfix in docs/CHANGELOG.md and add a test for the bugfix.
 2020-12-16 23:40:17 +02:00
BigFish
27f0261257
lib/protoparser/prometheus/parser.go (#970) 
fix parse timestamp error if there are some whitespaces after timestamp
 2020-12-16 23:36:20 +02:00
Aliaksandr Valialkin
a4c7fcb5e1 lib/promscrape: properly remove deleted target from /targets page 
Previously `sw` variable wasn't captured correctly by the started goroutine.
 2020-12-15 20:57:09 +02:00
Aliaksandr Valialkin
b730fc2667 lib/promscrape: properly handle scrape errors when stream parsing is enabled 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/967
 2020-12-15 14:08:28 +02:00
Nikolay
95c9b630cc
adds new Array Flags (#965) 
* adds ArrayDuration and ArrayBool flags,
makes sendTimeout and tlsInsecure configurable per remoteWrite url

* added backward compatibility testcases for ArrayDuration and ArrayBool

* fixes bool flag

* fixes test cases
 2020-12-15 12:51:12 +02:00
Aliaksandr Valialkin
60fcac4878 lib/promscrape: add bootstrap styles to /targets html page 2020-12-15 12:37:56 +02:00
Aliaksandr Valialkin
5af2a9ca0e lib/promscrape: formatting fixes for /tarets page 2020-12-15 11:59:04 +02:00
Aliaksandr Valialkin
020917949b lib/promscrape: formatting fixes for /targets page 2020-12-15 11:24:18 +02:00
Aliaksandr Valialkin
d006b41eff lib/persistentqueue: verify that ReaderOffset doesnt exceed WriterOffset when opening the persistent queue 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/964
 2020-12-14 19:25:25 +02:00
Aliaksandr Valialkin
ae972429c7 lib/promscrape: add missing whitespace between duration and ago word at /targets page 2020-12-14 14:19:58 +02:00
Aliaksandr Valialkin
069c9ade52 app/{vmagent,vminsert}: follow-up for ce8c2dd1f1: return /targets page in HTML when requested via web browser 2020-12-14 14:06:00 +02:00
Nikolay
ce8c2dd1f1
Changes targets api (#961) 
* changes /targets api
adds html response if requester accepts text/html,
adds quick template for /targets api,
fixes pathPrefix for / requests

* changes namings

* renamed targets file

* Update app/victoria-metrics/main.go

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>

* adds trimspace to qtpl,
moves content-type for targets response closer to writer

* fixes bug with prefix

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-12-14 13:36:48 +02:00
Aliaksandr Valialkin
5f9d88a3cb lib/promscrape/discovery/consul: reduce load on Consul API server by increasing timeout for blocking requests from 50 seconds to 9 minutes 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
 2020-12-11 17:24:13 +02:00
Aliaksandr Valialkin
fd9fd191b9 lib/promscrape/discovery/consul: properly pass Datacenter filter to Consul API server 
Previously it has been passed as `sdc` query arg, while it should be passed as `dc` query arg.
See https://www.consul.io/api-docs/health#list-nodes-for-service for details.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574#issuecomment-740454170
 2020-12-08 21:52:42 +02:00
Aliaksandr Valialkin
4146fc4668 all: properly handle CPU limits set on the host system/container 
This can reduce memory usage on systems with enabled CPU limits.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
 2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin
364f30a6e7 lib/promscrape: store ScrapeWork items by pointer in the slice returned from get*ScrapeWork() 
This should prevent from possible 'memory leaks' when a pointer to ScrapeWork item stored in the slice
could prevent from releasing memory occupied by all the ScrapeWork items stored in the slice when they
are no longer used.

See the related commit e205975716 and the related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825
 2020-12-08 17:50:05 +02:00
Aliaksandr Valialkin
08b71d2067 lib/promscrape: re-use strings for labels stored in ScrapeWork 
This should reduce memory usage when working with big number of scrape targets.
 2020-12-08 12:22:59 +02:00
Aliaksandr Valialkin
0f1b969aa6 lib/promscrape: export vm_promscrape_scrapers_{started|stopped}_total metrics for monitoring target churn rate 2020-12-08 11:57:52 +02:00
Aliaksandr Valialkin
c7ac7c1807 lib/promscrape: store targetStatus entries in targetStatusMap by pointer instead of by value 
This guarantees that GC frees memory occupied by targetStatus after it is unregistered from targetStatusMap.
 2020-12-08 11:50:48 +02:00
Aliaksandr Valialkin
05813259dc lib/promscrape: export vm_promscrape_active_scrapers{type="<sd_type>"} metric for tracking the number of active scrapers per each service discovery type 2020-12-08 01:54:23 +02:00
Aliaksandr Valialkin
9c1c9d8e76 lib/promscrape: do not enable strict config parsing when -promscrape.config.dryRun command-line flag is passed 
Strict parsing for -promscrape.config can be enabled by passing `-promscrape.config.strictParse` command-line flag.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/944
 2020-12-07 13:18:50 +02:00
Aliaksandr Valialkin
82972a8f2a lib/promscrape: mention in scrape error message that scrape errors can be disabled by -promscrape.suppressScrapeErrors command-line flag 2020-12-06 23:27:58 +02:00
Aliaksandr Valialkin
299a35948c lib/promscrape: clarify error message on failed connection to scrape target when -enableTCP6 command-line flag isn't set 2020-12-06 13:18:39 +02:00
Aliaksandr Valialkin
b0e4b234cb lib/protoparser/influx: allow multiple whitespace chars between measurement, fields and timestamp in Influx line protocol 2020-12-06 12:01:27 +02:00
Aliaksandr Valialkin
52915c8f7e lib/promscrape/discoveryutils: remove limit on the number of concurrently running blocking queries 
Too low limit could result in unexpected errors when performing big number of blocking queries.
 2020-12-05 12:15:52 +02:00
Aliaksandr Valialkin
eb27dbde13 lib/flagutil: make golangci-lint happy by using strings.TrimPrefix instead of manual prefix removal via strings.HasPrefix 2020-12-03 22:07:57 +02:00
Aliaksandr Valialkin
9d787f9edd all: do not print usage info for all the flags when incorrect command-line flag is passed 
This should improve usability for VictoriaMetrics apps that have big number of command-line flags,
i.e. all the apps.
 2020-12-03 21:47:37 +02:00
Aliaksandr Valialkin
de0643fab5 lib/promscrape/discovery/consul: log the time needed for stoppig Consul service watcher 2020-12-03 20:14:55 +02:00
Aliaksandr Valialkin
9cd8eb92f1 lib/promscrape/discovery/consul: make sure that block response contains X-Consul-Index header 2020-12-03 20:05:23 +02:00
Aliaksandr Valialkin
5009b25a03 lib/promscrape: code cleanup after c6dee6c52d 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
 2020-12-03 19:50:53 +02:00
Nikolay
c6dee6c52d
Changes consul discovery api (#921) 
* adds consul watch api,
it must reduce load on consul service with blocking wait requests,
changed discoveryClient api with fetchResponseMeta callback.

* small fix

* fix after master merge

* adds watch client at discovery utils

* fixes consul watcher,
changes namings,
fixes data race

* small typo fix

* sanity fix

* fix naming and service node update
 2020-12-03 19:47:40 +02:00
Aliaksandr Valialkin
357f886f97 Revert "lib/storage: add missing (AccountID, ProjectID) in MetricName.String() test" 
This reverts commit f3e80eb70e493620e42a0cc22a62c9af75076c77, since it isn't needed for single-node version of VictoriaMetrics
 2020-11-29 01:27:25 +02:00
Aliaksandr Valialkin
ace969d595 lib/storage: add missing (AccountID, ProjectID) in MetricName.String() test 2020-11-29 01:26:04 +02:00
Aliaksandr Valialkin
32869e4c0f lib/promscrape: fix failing tests after a906b3862f 2020-11-29 01:26:03 +02:00
Aliaksandr Valialkin
a906b3862f lib/protoparser/prometheus: properly parse OpenMetrics timestamps 
OpenMetrics timestamps are floating-point numbers, that represent Unix timestamp in seconds.
This differs from Prometheus exposition format, where timestamps are integer numbers representing Unix timestamp in milliseconds.
 2020-11-27 14:54:29 +02:00
Aliaksandr Valialkin
3a32789352 lib/promscrape: reduce memory allocations when unpacking gzipped responses received from scrape targets 2020-11-26 18:32:06 +02:00
Aliaksandr Valialkin
2cea4d403f all: typo fix: thouthand->thousand 2020-11-26 13:33:46 +02:00
Aliaksandr Valialkin
b0a5c382ee lib/promscrape: release http response non-200 status code 2020-11-26 13:25:17 +02:00
Aliaksandr Valialkin
b7fcdb528d app/{vmagent,victoria-metrics}: add -dryRun option and make more clear handling for -promscrape.config.dryRun 2020-11-25 22:59:13 +02:00
Aliaksandr Valialkin
1c669a69a8 lib/mergeset: tune the number of rawItemsBlocks to merge at once 
512 blocks give higher ingestion performance and slightly lower memory usage
 2020-11-25 21:52:52 +02:00
Aliaksandr Valialkin
7119f294f3 lib/mergeset: help GC by removing refereces to slices in inmemoryBlock.Reset 2020-11-25 21:19:43 +02:00
Aliaksandr Valialkin
8a057e705a lib/storage: log metric name plus all its labels when the metric timestamp is outside the configured retention 
This should simplify debugging when the source of the metric with unexpected timestamp must be found.
 2020-11-25 14:41:37 +02:00
Aliaksandr Valialkin
b65236530c lib/storage: typo fix in error message: allowd->allowed 2020-11-25 14:15:42 +02:00
Aliaksandr Valialkin
ae04378424 lib/protoparser/prometheus: properly parse "infinity" values in OpenMetrics format 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/924
 2020-11-24 19:03:38 +02:00
Aliaksandr Valialkin
bf95fbfc1d lib/logger: disable rate limiting for error and warn logs by default 2020-11-24 12:42:25 +02:00
Aliaksandr Valialkin
78d2715d04 all: spelling fix: superflouos->superfluous. This is a follow-up for 0acdab3ab9 2020-11-24 12:42:22 +02:00
Aliaksandr Valialkin
b7f4fc6e0d lib/protoparser/prometheus: properly parse metrics with exemplars 
Examplars have been introduced in OpenMetrics - see https://github.com/OpenObservability/OpenMetrics/blob/master/OpenMetrics.md#exemplars-1
Previously VictoriaMetrics couldn't parse the following metric

    foo{bar="baz"} 123 # exemplar here

This commit fixes this. Note that VictoriaMetrics ignores the exemplar as for now.
 2020-11-24 12:34:56 +02:00