github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
2254 commits 150 branches 748 tags 347 MiB
Commit graph

816 commits

Author SHA1 Message Date
Aliaksandr Valialkin
cdecf83ce5 app/vmstorage: export vm_composite_index_min_timestamp metric 2021-02-10 17:14:08 +02:00
Aliaksandr Valialkin
553016ea99 lib/storage: disable composite index usage when querying old data 2021-02-10 14:57:50 +02:00
Aliaksandr Valialkin
fcb7655d1e lib/storage: fix metric name match for composite filter 2021-02-10 01:27:45 +02:00
Aliaksandr Valialkin
c7dccebaef lib/storage: optimize search by label filters matching big number of time series 2021-02-10 00:44:54 +02:00
Aliaksandr Valialkin
6b4e6c229c lib/storage: reduce lock contention in dateMetricIDCache when registering new time series for the current day 
This should help systems with multiple CPU cores
 2021-02-10 00:01:13 +02:00
Aliaksandr Valialkin
31f6b9c977 lib/fs: remove the code for tracking whether the given memory region is in page cache 
This code didn't give performance gains under production workload, so let's remove it in order to simplify the code.
 2021-02-09 16:49:03 +02:00
Aliaksandr Valialkin
0a69122d81 lib/mergeset: remove dead code left after a4140de9e6 2021-02-09 16:33:52 +02:00
Aliaksandr Valialkin
d56390b925 optimize Storage.updatePerDateData() 2021-02-09 02:55:36 +02:00
Aliaksandr Valialkin
fda61e8e96 lib/storage: skip deduplication when creating inmemory data blocks 
The deduplication will be performed later during merging such blocks.
 2021-02-09 02:25:32 +02:00
Aliaksandr Valialkin
a4140de9e6 lib/mergeset: unconditionally cache indexdb blocks 
Production workloads show that indexdb blocks must be cached unconditionally for reducing CPU usage.
This shouldn't increase memory usage too much, since unused blocks are removed from the cache every two minutes.
 2021-02-09 00:47:50 +02:00
Aliaksandr Valialkin
cb96a1865b app/vmstorage: export missing vm_cache_size_bytes metrics for indexdb and data caches 2021-02-09 00:47:00 +02:00
Aliaksandr Valialkin
c5770600a2 lib/cgroup: follow-up after b9bf3cbe3e 2021-02-08 15:54:38 +02:00
Nikolay
b9bf3cbe3e
refactored cgroups limits, (#1061) 
adds tests, remove os.Exec
 2021-02-08 15:46:22 +02:00
Aliaksandr Valialkin
2242647a04 lib/storage: optimize data ingestion in the beginning of every hour 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1046
 2021-02-08 12:01:12 +02:00
Aliaksandr Valialkin
8f28a578d3 lib/logger: exit the app if unsupported timezone value has been passed to -loggerTimezone 
While at it, clarify descrption for `-loggerTimezone` command-line flag.
 2021-02-07 23:35:37 +02:00
Aliaksandr Valialkin
83d3e582ab lib/storage: check for prevHourMetricIDs cache before falling back to checking for (date, metricID) entries during data ingestion 
This should reduce possible CPU usage spikes at the beginning of every hour.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1046
 2021-02-04 18:48:13 +02:00
Aliaksandr Valialkin
9fb38569eb lib/httpserver: expose process_open_fds and process_max_fds metrics 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/402
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1037
 2021-02-04 16:40:50 +02:00
Nikolay
48c8c5093b
fixes dockerswarm (#1053) 
fixes improper usage of host network services
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1028
 2021-02-04 15:56:42 +02:00
Aliaksandr Valialkin
d16f22f3a1 app/vmselect,lib/storage: properly parse Graphite selectors with inner wildcards 
Example: foo{bar{x,yz},a[b-c],*de}
 2021-02-03 20:14:22 +02:00
Aliaksandr Valialkin
a5a1b9bd66 lib/storage: fix a bug, which breaks searching by Graphite wildcard filters 2021-02-03 20:14:22 +02:00
Aliaksandr Valialkin
6123aa3e75 sort orSuffixes in tagFilter.InitFromGraphiteQuery for faster seeks 2021-02-03 20:14:22 +02:00
Aliaksandr Valialkin
157c02622b app/vmselect: add ability to set Graphite-compatible filter via {__graphite__="foo.*.bar"} syntax 2021-02-03 01:21:54 +02:00
Aliaksandr Valialkin
4068f8d590 lib/promscrape: add vm_promscrape_service_discovery_duration_seconds metric 2021-02-02 16:15:25 +02:00
Aliaksandr Valialkin
bd11fd8f1d lib/promscrape: add vm_promscrape_scrape_retries_total, vm_promscrape_discovery_retries_total and vm_promscrape_discovery_requests_total metrics 2021-02-01 20:06:27 +02:00
Aliaksandr Valialkin
f0087f0dbb lib/flagutil: typo fix in comment to ArrayInt.GetOptionalArgOrDefault() func 2021-02-01 14:35:39 +02:00
Aliaksandr Valialkin
b2aa80e74b app/vmagent: add -remoteWrite.roundDigits command-line option for limiting the number of digits after the point for stored values 
This commit also adds --vm-round-digits command-line option to vmctl tool.
 2021-02-01 14:27:09 +02:00
Aliaksandr Valialkin
d6347a3e56 lib/logger: initialize timezone by UTC in order to fix failing tests 2021-01-27 00:59:12 +02:00
Aliaksandr Valialkin
fc5b26d856 lib/promscrape: export vm_promscrape_scrapes_failed_per_url_total and vm_promscrape_scrapes_skipped_by_sample_limit_per_url_total metrics 
These metrics could be useful for determining imporperly working scrape targets.
Note that these metrics are exported only for failing scrape targets. They aren't exposed for normally working targets.
 2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin
de3c662e8a all: consistently use timers from timerpool 2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin
3149ac7a7e lib/fs: properly initialize cleaner for pageCache bitmaps 
Previously it wasnt working because the timer was fired only once
 2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin
3fe848cdd7 lib/logger: add -loggerTimezone command-line flag for adjusting timezone for timestamps in log messages 2021-01-26 22:51:54 +02:00
Aliaksandr Valialkin
8cea3c3cc4 lib/promscrape: retry scrape and service discovery requests when the remote server closes http keep-alive connection 2021-01-22 13:22:33 +02:00
faceair
b638c1eed5
lib/mergeset: add missing shouldCacheBlock (#1019) 2021-01-15 11:46:01 +02:00
Aliaksandr Valialkin
bdd0a1cdb2 lib/backup: increase backup chunk size from 128MB to 1GB 
This should reduce costs for object storage API calls by 8x. See https://cloud.google.com/storage/pricing#operations-pricing
 2021-01-13 12:16:35 +02:00
Nikolay
7bf5d48315
bumps minimal tls version (#1012) 2021-01-13 00:35:47 +02:00
Aliaksandr Valialkin
31ec79eaf6 lib/storage: inline marshalTags function and remove the code for handling duplicate tags from here 
This is a follow-up commit after c8ea697db8
 2021-01-12 15:13:30 +02:00
Aliaksandr Valialkin
c8ea697db8 lib/storage: de-duplicate tags in MetricName.sortTags 
Leave only the last tag among tags with duplicate keys. This is needed for reliable addition of extra_labels
during data ingestion. See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1007 for details.
 2021-01-12 15:03:42 +02:00
Nikolay
7976c22797
Fixes error handling for promscrape.streamParse (#1009) 
properly return error if client cannot read data,
properly suppress scraper errors
 2021-01-12 13:31:47 +02:00
Aliaksandr Valialkin
2c44f9989a lib/promscrape: properly show scrape duration on /targets page 
Previously it has been shown as 0.000s for any scrape duration.
 2021-01-11 21:14:46 +02:00
Aliaksandr Valialkin
24ffad74c1 all: use net.Dial instead of fasthttp.Dial, because fasthttp.Dial limits the number of concurrent dials to 1000 2021-01-11 12:53:30 +02:00
Aliaksandr Valialkin
9dcb18e03d app/vmstorage: disable final merge by default, since it may result in high disk IO and CPU usage without measurable benefits such as increased query performance and reduced disk space usage 2021-01-08 00:16:05 +02:00
Aliaksandr Valialkin
490c69c64e lib/storage: wait for pending transactions before closing and dropping the partition 
This deflakes `make test-full-386` test
 2020-12-25 11:45:53 +02:00
Aliaksandr Valialkin
cab7e936a3 lib/storage: physically remove stale parts 
Previously they were removed from partition struct, but the corresponding directories weren't removed.

This is a follow-up for 46dba00756
 2020-12-24 16:51:36 +02:00
Nikolay
b21e16ad0c
fixes kubernetes_sd (#983) 
* fixes kubernetes_sd,
adds missing service metadata for pod ports without endpoint
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/982

* fix test
 2020-12-24 11:26:14 +02:00
Aliaksandr Valialkin
820669da69 lib/promscrape: code prettifying for 8dd03ecf19 2020-12-24 10:56:10 +02:00
Nikolay
8dd03ecf19
adds proxy_url support, (#980) 
* adds proxy_url support,
adds proxy_url to the dockerswarm, eureka, kubernetes and consul service discovery,
adds proxy_url to the scrape_config for targets scrapping,
http based proxy is supported atm,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/503

* fixes imports
 2020-12-24 10:52:37 +02:00
Aliaksandr Valialkin
9e4ed5e591 lib/storage: do not remove parts outside the configured retention if they are currently merged 
These parts are automatically removed after the merge is complete.
 2020-12-24 08:51:28 +02:00
Aliaksandr Valialkin
46dba00756 lib/storage: remove stale parts as soon as they go outside the configured retention 
Previously such parts could remain undeleted for long durations until they are merged with other parts.
This should help for `-retentionPeriod` values smaller than one month.
 2020-12-22 19:54:31 +02:00
Aliaksandr Valialkin
d65c03c004 lib/storage: properly determine max rows for output part when merging small parts 2020-12-18 23:14:38 +02:00
Aliaksandr Valialkin
ebf8da3730 lib/{storage,mergeset}: tune background merge process in order to reduce CPU usage and disk IO usage 2020-12-18 20:01:08 +02:00