Aliaksandr Valialkin
daefb64f38
app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload
...
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792
2022-06-28 20:18:31 +03:00
Aliaksandr Valialkin
9adff0b686
app/vmselect/promql: show the number of scanned samples in the query trace
2022-06-28 19:32:03 +03:00
Aliaksandr Valialkin
c21f5a508f
app/vmselect/prometheus: reduce the default value for -search.maxSeries from 100k to 30k
...
Production experience shows that 100k is too big for /api/v1/series .
It leads to increased CPU usage when Grafana queries /api/v1/series over VictoriaMetrics
with big number of time series during auto-completion and when modifying template variables.
2022-06-28 18:22:24 +03:00
Aliaksandr Valialkin
5e22e6046d
docs/Single-server-VictoriaMetrics.md: mention about -search.maxTagValueSuffixesPerSearch command-line flag in resource limits docs
2022-06-28 18:17:49 +03:00
Aliaksandr Valialkin
7d5d33fd71
lib/storage: return marshaled metric names from SearchMetricNames
...
Previously SearchMetricNames was returning unmarshaled metric names.
This wasn't great for vmstorage, which should spend additional CPU time
for marshaling the metric names before sending them to vmselect.
While at it, remove possible duplicate metric names, which could occur when
multiple samples for new time series are ingested via concurrent requests.
Also sort the metric names before returning them to the client.
This simplifies debugging of the returned metric names across repeated requests to /api/v1/series
2022-06-28 18:16:32 +03:00
Aliaksandr Valialkin
69bbdf7304
vendor: make vendor-update
2022-06-28 14:55:41 +03:00
Aliaksandr Valialkin
15da802f5f
lib/storage: put into query trace the number of found entries in SearchMetricNames
2022-06-28 14:52:39 +03:00
Aliaksandr Valialkin
36edb1912b
app/vmstorage: rename "transport" package to "servers" package for better clarity
2022-06-28 14:04:14 +03:00
Aliaksandr Valialkin
399d4c36ae
app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day
2022-06-28 12:55:20 +03:00
Aliaksandr Valialkin
64505e924d
app/vmstorage: extract vmselect api server into a separate package - lib/vmselectapi
...
This opens doors for implementing vmselect api server at vmselect level,
so top-level vmselect could query lower-level vmselect nodes in the same way
as it queries vmstorage nodes.
This will create the ability to create highly available querying architecture
when multiple independent VictoriaMetrics clusters with the same data
are located in distinct availability zones. In this case we can use top-level
vmselect instead of Promxy for simultaneous querying of all the clusters
in all the AZs.
2022-06-27 14:20:41 +03:00
Aliaksandr Valialkin
1af6efd737
docs/Single-server-VictoriaMetrics.md: mention about -search.maxTagValueSuffixesPerSearch command-line flag in resource limits docs
2022-06-27 14:02:01 +03:00
Aliaksandr Valialkin
a667d339be
app/vmselect/netstorage/netstorage.go: group metrics in order to improve readability a bit
2022-06-27 14:00:24 +03:00
Roman Khavronenko
50c0eb4c4e
vmalert: make __name__ available for templating in alerts ( #2783 )
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-27 13:53:55 +03:00
Aliaksandr Valialkin
a38568ddfe
docs/CHANGELOG.md: document 45f20ad1aa
2022-06-27 13:53:26 +03:00
Aliaksandr Valialkin
6386f117c8
all: show timeRange in traces in human-readable format instead of timestamps in milliseconds
2022-06-27 13:42:57 +03:00
Aliaksandr Valialkin
4b41a05ca7
app/vmalert: load static js and css from proper paths if -http.pathPrefix command-line flag is set
...
This is a follow-up for b104f67beb
2022-06-27 13:12:57 +03:00
Aliaksandr Valialkin
926fccbb8d
lib/storage: add querytracer to more contexts
...
querytracer has been added to the following storage.Storage methods:
- RegisterMetricNames
- DeleteMetrics
- SearchTagValueSuffixes
- SearchGraphitePaths
2022-06-27 12:53:49 +03:00
Aliaksandr Valialkin
6c66804fd3
all: locate throttled loggers via logger.WithThrottler() only once and then use them
...
This reduces the contention on logThrottlerRegistryMu mutex when logger.WithThrottler()
is called frequently from concurrent goroutines.
2022-06-27 12:34:30 +03:00
Aliaksandr Valialkin
08de733924
app/vmselect/netstorage: assume the response is full if up to -replicationFactor-1 vmstorage nodes are unavailable
...
This is a follow-up for ee5c502446https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1767
2022-06-27 12:21:26 +03:00
Roman Khavronenko
572db17857
vmalert: use absolute path for assets ( #2784 )
...
Using relative path breaks assets loading on alert view page.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-27 00:47:36 +03:00
Aliaksandr Valialkin
bc9d704ef4
app/vmselect/netstorage: remove Get prefix from netstorage functions
...
This makes these function names more consistent with the server side
2022-06-27 00:37:49 +03:00
Aliaksandr Valialkin
ea2aa617e5
docs/CHANGELOG.md: update after e40d015e9a
2022-06-24 18:03:56 +03:00
Aliaksandr Valialkin
94b81165d0
all: limit the maximum memory usage for regexp cache, which stores parsed regular expressions in MetricsQL queries
...
Previously the cache could store 10K unique regexps. When every regexp is huge (e.g. hundreds of kilobytes),
then the total cache size could grow to multiples of gigabytes. Now the cache size is limited by the total length
of all cached regexps. So huge regexps won't result in high memory usage for the cache.
2022-06-24 17:58:47 +03:00
hagen1778
e40d015e9a
vmselect: make vm_partial_results_total consistent
...
Metrics `vm_partial_results_total` and `vm_requests_total` serving
the similar purpose, but contain inconsistent set of labels.
This change updates `vm_partial_results_total` labels to be consistent
with `vm_requests_total`.
The change breaks backward compatibility with assumption that
`vm_partial_results_total` wasn't widely used, since it is
not documented and absent in the alerts and dashboards.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-24 13:50:26 +02:00
Dmytro Kozlov
f28cbcc7b5
vmui: added query tracing ( #2748 )
...
* vmui: added query tracing
* vmui: updated ui
* vmui: update tracing logic, fix bugs, disable tracing by default
* vmui: use empty message as props
* vmui: fixed ui, added delete for each tacing data, show query in header
* vmui: added timelines
* vmui: speedup render
* vmui: use memo for sorting
* vmui: use Trace model, remove unused functions, simplify part of code
* vmui: update recursive logic
* vmui: fix set query to header
* vmui: code cleanup, remove unused code
* vmui: remove unused type, rename component
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-23 23:00:45 +03:00
Nikolay
ee5c502446
app/vmselect: fixes partial response with replicationFactor ( #2777 )
...
* app/vmselect: fixes partial response with replicationFactor
Allow partial response if it meets replicationFactor configured at vmselect
https://t.me/VictoriaMetrics_ru1/38490
* docs/CHANGELOG.md: document this change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-23 20:17:24 +03:00
Yurii Kravets
f0c1edb175
docs: Update CHANGELOG Update notes ( #2776 )
...
* docs: Update CHANGELOG Update notes
Specified the reason why `vmselect` and `vmstorage` nodes may log communication errors.
2022-06-23 19:40:20 +03:00
Aliaksandr Valialkin
e0ce6c0ff8
app/vmstorage/transport: refactoring: split Server into VMInsertServer and VMStorageServer
...
This makes the code more clear
2022-06-23 19:20:09 +03:00
Aliaksandr Valialkin
71b0dfdefa
lib/promscrape: always send stale markers with the real scrape timestamp
...
This guarantees that query won't return data just after the series is disappeared.
2022-06-23 11:49:13 +03:00
Roman Khavronenko
fc03950efa
dashboards: update cluster dashboard ( #2773 )
...
* dashboards: update cluster dashboard
* add assisted merges panel https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2754
* add mem panel per each component
* remove lines filling for some panels for clarity
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* Update dashboards/victoriametrics.json
2022-06-23 09:46:28 +02:00
Denys Holius
d7c4b01472
Adds a list of supported architectures ( #2769 )
...
* add list of supported architectures
* Update docs/BestPractices.md
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2022-06-22 22:01:29 +03:00
Aliaksandr Valialkin
3ae6300497
lib/promauth: add ability to send additional http headers in requests to scrape targets
...
This solves https://stackoverflow.com/questions/66032498/prometheus-scrape-metric-with-custom-header
2022-06-22 20:40:50 +03:00
Aliaksandr Valialkin
7429dfbe9e
app/vmselect: add -search.setLookbackToStep command-line flag for making the gap filling algorithm similar to InfluxDB data model
...
This option should override `-search.maxStalenessInterval` for most cases when users migrate from InfluxDB to VictoriaMetrics
2022-06-22 14:20:02 +03:00
Aliaksandr Valialkin
3e0f364489
docs/Cluster-VictoriaMetrics.md: small fixes
2022-06-22 13:42:07 +03:00
Aliaksandr Valialkin
4f64da874d
app/vmselect: typo fix in the exported metric name: vm_http_request_total -> vm_http_requests_total
2022-06-22 13:18:03 +03:00
Roman Khavronenko
54f0f2d384
docs: follow-up for 197d3cdd74 ( #2766 )
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-22 13:18:03 +03:00
云原生驿站
67e5833ced
docs: supplement vmalert downsampling docs ( #2765 )
...
Co-authored-by: 吴典秋 <muti_kube@163.com>
2022-06-22 13:18:03 +03:00
Aliaksandr Valialkin
dceca7e864
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package
...
This is a follow-up for fe2269b999
2022-06-21 20:27:30 +03:00
Aliaksandr Valialkin
fe2269b999
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package
...
This package already has the same name, so there is no need in explicit name
2022-06-21 20:24:28 +03:00
Denys Holius
432b261c13
url-examples: added curl output after deleting metrics ( #2764 )
...
docs: add more details to url-examples for series deleting
2022-06-21 17:56:00 +03:00
Loki's Wager
ca4730c00f
BugFix part_header.go ( #2763 )
...
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2757
Co-authored-by: haotingyi <haotingyi@corp.netease.com>
2022-06-21 15:59:11 +03:00
Aliaksandr Valialkin
9362da2e7f
docs: update -help output for vmbackup, vmbackupmanager, vmgateway and vmrestore components
2022-06-21 15:49:26 +03:00
Aliaksandr Valialkin
597bce4f55
docs: update docs after e4d6b750f6
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2753
2022-06-21 14:01:25 +03:00
Aliaksandr Valialkin
288d13af8d
lib/netutil: parallelize background pings for remote addresses
...
This should improve the time needed for determining unavailale remote addresses
across big numer of ConnPool's.
This is a follow-up for a1629bd3behttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2022-06-21 13:32:27 +03:00
Aliaksandr Valialkin
24097f2417
docs/CHANGELOG.md: cut v1.78.0
2022-06-20 18:11:04 +03:00
Yurii Kravets
14397ba23e
Changed the level type in alerts.yml for TooManyLogs alert ( #2759 )
...
alerts: filter out non error log messages for `TooManyLogs`
Info and Warn error levels aren't always a result of malfunctioning
or faulty state. So we filter them out.
2022-06-20 16:45:52 +02:00
Aliaksandr Valialkin
a1629bd3be
lib/netutil.ConnPool: skip dialing remote address if the previous dial attempt was unsuccessful
...
If the previous dial attempt was unsuccessful, then all the new dial attempts are skipped
until the background goroutine determines that the given address can be successfully dialed.
This reduces query latency when some of vmstorage nodes are unavailable and dialing them is slow.
This should help with https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
This commit is based on ideas from the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2756
The main differences are:
- The check for healthy/unhealthy storage nodes is moved one level lower from app/vmselect/netstorage to lib/netutil.ConnPool.
This makes possible re-using this feature everywhere lib/netutil.ConnPool is used.
- The check doesn't take into account handshake errors for already established connections.
Handshake errors usually mean improperly configured VictoriaMetrics cluster, so they shouldn't be ignored.
2022-06-20 17:33:54 +03:00
Aliaksandr Valialkin
45e9732764
docs: follow-up after e4d6b750f6
2022-06-20 17:15:52 +03:00
Nikolay
15662c0f29
lib/httpserver: adds flagsAuthKey command-line flag ( #2758 )
...
* lib/httpserver: adds flagsAuthKey command-line flag
It protects /flags endpoint with authKey.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2753O
* Apply suggestions from code review
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-20 17:15:51 +03:00
Aliaksandr Valialkin
b28c6febf9
app/{vminsert,vmselect}: add -vmstorageDialTimeout command-line flag for tuning the maximum time needed for establishing connections to vmstorage
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2022-06-20 15:17:34 +03:00
