mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-11-21 14:44:00 +00:00
lib/promscrape: follow-up after 6aa50ca954
- Improve docs - Hide `debug relabeling` column when -promscrape.dropOriginalLabels command-line flag is set - Inline the code from the added template functions, since the code is harder to follow with the template functions, especially when these functions have misleading names. Also, these functions are used only in one place, e.g. they do not reduce the amounts of code. - Hide `click to show original labels` title at `labels` column when original labels aren't available. - Show the reason on whey original labels aren't available at /service-discovery page. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4597
This commit is contained in:
Aliaksandr Valialkin
parent
b8ba2d5f1a
commit
49bd2905fa
6 changed files with 456 additions and 621 deletions
|
|
@ -463,8 +463,9 @@ with [additional enhancements](#relabeling-enhancements). The relabeling can be
|
|||
This relabeling is used for modifying labels in discovered targets and for dropping unneeded targets.
|
||||
See [relabeling cookbook](https://docs.victoriametrics.com/relabeling.html) for details.
|
||||
|
||||
This relabeling can be debugged by clicking the `debug` link (the link is available only if `--promscrape.dropOriginalLabels=false`) at the corresponding target on the `http://vmagent:8429/targets` page
|
||||
This relabeling can be debugged by clicking the `debug` link at the corresponding target on the `http://vmagent:8429/targets` page
|
||||
or on the `http://vmagent:8429/service-discovery` page. See [these docs](#relabel-debug) for details.
|
||||
The link is unavailable if `vmagent` runs with `-promscrape.dropOriginalLabels` command-line flag.
|
||||
|
||||
* At the `scrape_config -> metric_relabel_configs` section in `-promscrape.config` file.
|
||||
This relabeling is used for modifying labels in scraped metrics and for dropping unneeded metrics.
|
||||
|
|
@ -644,19 +645,21 @@ provide the following tools for debugging target-level and metric-level relabeli
|
|||
|
||||
- Target-level debugging (e.g. `relabel_configs` section at [scrape_configs](https://docs.victoriametrics.com/sd_configs.html#scrape_configs))
|
||||
can be performed by navigating to `http://vmagent:8429/targets` page (`http://victoriametrics:8428/targets` page for single-node VictoriaMetrics)
|
||||
and clicking the `debug target relabeling` link (the link is available only if `--promscrape.dropOriginalLabels=false`) at the target, which must be debugged.
|
||||
and clicking the `debug target relabeling` link at the target, which must be debugged.
|
||||
The link is unavailable if `vmagent` runs with `-promscrape.dropOriginalLabels` command-line flag.
|
||||
The opened page shows step-by-step results for the actual target relabeling rules applied to the discovered target labels.
|
||||
The page shows also the target URL generated after applying all the relabeling rules.
|
||||
|
||||
The `http://vmagent:8429/targets` page shows only active targets. If you need to understand why some target
|
||||
is dropped during the relabeling, then navigate to `http://vmagent:8428/service-discovery` page
|
||||
(`http://victoriametrics:8428/service-discovery` for single-node VictoriaMetrics), find the dropped target
|
||||
and click the `debug` link (the link is available only if `--promscrape.dropOriginalLabels=false`) there. The opened page shows step-by-step results for the actual relabeling rules,
|
||||
which result to target drop.
|
||||
and click the `debug` link there. The link is unavailable if `vmagent` runs with `-promscrape.dropOriginalLabels` command-line flag.
|
||||
The opened page shows step-by-step results for the actual relabeling rules, which result to target drop.
|
||||
|
||||
- Metric-level debugging (e.g. `metric_relabel_configs` section at [scrape_configs](https://docs.victoriametrics.com/sd_configs.html#scrape_configs)
|
||||
can be performed by navigating to `http://vmagent:8429/targets` page (`http://victoriametrics:8428/targets` page for single-node VictoriaMetrics)
|
||||
and clicking the `debug metrics relabeling` link (the link is available only if `--promscrape.dropOriginalLabels=false`) at the target, which must be debugged.
|
||||
and clicking the `debug metrics relabeling` link at the target, which must be debugged.
|
||||
The link is unavailable if `vmagent` runs with `-promscrape.dropOriginalLabels` command-line flag.
|
||||
The opened page shows step-by-step results for the actual metric relabeling rules applied to the given target labels.
|
||||
|
||||
## Prometheus staleness markers
|
||||
|
|
|
|||
|
|
@ -68,7 +68,7 @@ The following `tip` changes can be tested by building VictoriaMetrics components
|
|||
* BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert.html): correctly calculate evaluation time for rules. Before, there was a low probability for discrepancy between actual time and rules evaluation time if evaluation interval was lower than the execution time for rules within the group.
|
||||
* BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert.html): reset evaluation timestamp after modifying group interval. Before, there could have latency on rule evaluation time.
|
||||
* BUGFIX: vmselect: fix timestamp alignment for Prometheus querying API if time argument is less than 10m from the beginning of Unix epoch.
|
||||
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent.html): hide `debug` links on the `/targets` page if `--promscrape.dropOriginalLabels` is enabled. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4597).
|
||||
* BUGFIX: [vmagent](https://docs.victoriametrics.com/vmagent.html): do not show [relabel debug](https://docs.victoriametrics.com/vmagent.html#relabel-debug) links at the `/targets` page when `vmagent` runs with `-promscrape.dropOriginalLabels` command-line flag, since it has no the original labels needed for relabel debug. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4597).
|
||||
* BUGFIX: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): properly parse binary operations with reserved words on the right side such as `foo + (on{bar="baz"})`. Previously such queries could lead to panic. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4422).
|
||||
* BUGFIX: [Official Grafana dashboards for VictoriaMetrics](https://grafana.com/orgs/victoriametrics): display cache usage for all components on panel `Cache usage % by type` for cluster dashboard. Before, only vmstorage caches were shown.
|
||||
|
||||
|
|
|
|||
|
|
@ -474,8 +474,9 @@ with [additional enhancements](#relabeling-enhancements). The relabeling can be
|
|||
This relabeling is used for modifying labels in discovered targets and for dropping unneeded targets.
|
||||
See [relabeling cookbook](https://docs.victoriametrics.com/relabeling.html) for details.
|
||||
|
||||
This relabeling can be debugged by clicking the `debug` link (the link is available only if `--promscrape.dropOriginalLabels=false`) at the corresponding target on the `http://vmagent:8429/targets` page
|
||||
This relabeling can be debugged by clicking the `debug` link at the corresponding target on the `http://vmagent:8429/targets` page
|
||||
or on the `http://vmagent:8429/service-discovery` page. See [these docs](#relabel-debug) for details.
|
||||
The link is unavailable if `vmagent` runs with `-promscrape.dropOriginalLabels` command-line flag.
|
||||
|
||||
* At the `scrape_config -> metric_relabel_configs` section in `-promscrape.config` file.
|
||||
This relabeling is used for modifying labels in scraped metrics and for dropping unneeded metrics.
|
||||
|
|
@ -655,19 +656,21 @@ provide the following tools for debugging target-level and metric-level relabeli
|
|||
|
||||
- Target-level debugging (e.g. `relabel_configs` section at [scrape_configs](https://docs.victoriametrics.com/sd_configs.html#scrape_configs))
|
||||
can be performed by navigating to `http://vmagent:8429/targets` page (`http://victoriametrics:8428/targets` page for single-node VictoriaMetrics)
|
||||
and clicking the `debug target relabeling` link (the link is available only if `--promscrape.dropOriginalLabels=false`) at the target, which must be debugged.
|
||||
and clicking the `debug target relabeling` link at the target, which must be debugged.
|
||||
The link is unavailable if `vmagent` runs with `-promscrape.dropOriginalLabels` command-line flag.
|
||||
The opened page shows step-by-step results for the actual target relabeling rules applied to the discovered target labels.
|
||||
The page shows also the target URL generated after applying all the relabeling rules.
|
||||
|
||||
The `http://vmagent:8429/targets` page shows only active targets. If you need to understand why some target
|
||||
is dropped during the relabeling, then navigate to `http://vmagent:8428/service-discovery` page
|
||||
(`http://victoriametrics:8428/service-discovery` for single-node VictoriaMetrics), find the dropped target
|
||||
and click the `debug` link (the link is available only if `--promscrape.dropOriginalLabels=false`) there. The opened page shows step-by-step results for the actual relabeling rules,
|
||||
which result to target drop.
|
||||
and click the `debug` link there. The link is unavailable if `vmagent` runs with `-promscrape.dropOriginalLabels` command-line flag.
|
||||
The opened page shows step-by-step results for the actual relabeling rules, which result to target drop.
|
||||
|
||||
- Metric-level debugging (e.g. `metric_relabel_configs` section at [scrape_configs](https://docs.victoriametrics.com/sd_configs.html#scrape_configs)
|
||||
can be performed by navigating to `http://vmagent:8429/targets` page (`http://victoriametrics:8428/targets` page for single-node VictoriaMetrics)
|
||||
and clicking the `debug metrics relabeling` link (the link is available only if `--promscrape.dropOriginalLabels=false`) at the target, which must be debugged.
|
||||
and clicking the `debug metrics relabeling` link at the target, which must be debugged.
|
||||
The link is unavailable if `vmagent` runs with `-promscrape.dropOriginalLabels` command-line flag.
|
||||
The opened page shows step-by-step results for the actual metric relabeling rules applied to the given target labels.
|
||||
|
||||
## Prometheus staleness markers
|
||||
|
|
|
|||
|
|
@ -156,11 +156,6 @@ func (tsm *targetStatusMap) getScrapeWorkByTargetID(targetID string) *scrapeWork
|
|||
}
|
||||
|
||||
func getLabelsID(labels *promutils.Labels) string {
|
||||
// return empty string if OriginalLabels was dropped
|
||||
// via flag --promscrape.dropOriginalLabels
|
||||
if labels == nil {
|
||||
return ""
|
||||
}
|
||||
return fmt.Sprintf("%016x", uintptr(unsafe.Pointer(labels)))
|
||||
}
|
||||
|
||||
|
|
@ -397,6 +392,7 @@ func (tsm *targetStatusMap) getTargetsStatusByJob(filter *requestFilter) *target
|
|||
}
|
||||
dts := droppedTargetsMap.getTargetsList()
|
||||
return &targetsStatusResult{
|
||||
hasOriginalLabels: !*dropOriginalLabels,
|
||||
jobTargetsStatuses: jts,
|
||||
droppedTargets: dts,
|
||||
emptyJobs: emptyJobs,
|
||||
|
|
@ -507,6 +503,7 @@ func getRequestFilter(r *http.Request) *requestFilter {
|
|||
}
|
||||
|
||||
type targetsStatusResult struct {
|
||||
hasOriginalLabels bool
|
||||
jobTargetsStatuses []*jobTargetsStatuses
|
||||
droppedTargets []droppedTarget
|
||||
emptyJobs []string
|
||||
|
|
|
|||
|
|
@ -180,7 +180,7 @@
|
|||
<div class="row mt-4">
|
||||
<div class="col-12">
|
||||
{% for i, jts := range tsr.jobTargetsStatuses %}
|
||||
{%= scrapeJobTargets(i, jts) %}
|
||||
{%= scrapeJobTargets(i, jts, tsr.hasOriginalLabels) %}
|
||||
{% endfor %}
|
||||
{% for i, jobName := range tsr.emptyJobs %}
|
||||
{% code
|
||||
|
|
@ -189,13 +189,13 @@
|
|||
jobName: jobName,
|
||||
}
|
||||
%}
|
||||
{%= scrapeJobTargets(num, jts) %}
|
||||
{%= scrapeJobTargets(num, jts, tsr.hasOriginalLabels) %}
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
{% endfunc %}
|
||||
|
||||
{% func scrapeJobTargets(num int, jts *jobTargetsStatuses) %}
|
||||
{% func scrapeJobTargets(num int, jts *jobTargetsStatuses, hasOriginalLabels bool) %}
|
||||
<div class="row mb-4">
|
||||
<div class="col-12">
|
||||
<h4>
|
||||
|
|
@ -209,7 +209,9 @@
|
|||
<th scope="col">Endpoint</th>
|
||||
<th scope="col">State</th>
|
||||
<th scope="col" title="target labels">Labels</th>
|
||||
{% if hasOriginalLabels %}
|
||||
<th scope="col" title="debug relabeling">Debug relabeling</th>
|
||||
{% endif %}
|
||||
<th scope="col" title="total scrapes">Scrapes</th>
|
||||
<th scope="col" title="total scrape errors">Errors</th>
|
||||
<th scope="col" title="the time of the last scrape">Last Scrape</th>
|
||||
|
|
@ -222,13 +224,20 @@
|
|||
{% for _, ts := range jts.targetsStatus %}
|
||||
{% code
|
||||
endpoint := ts.sw.Config.ScrapeURL
|
||||
originalLabels := ts.sw.Config.OriginalLabels
|
||||
|
||||
// The target is uniquely identified by a pointer to its original labels.
|
||||
targetID := getLabelsID(ts.sw.Config.OriginalLabels)
|
||||
targetID := getLabelsID(originalLabels)
|
||||
lastScrapeDuration := ts.getDurationFromLastScrape()
|
||||
%}
|
||||
<tr {% if !ts.up %}{%space%}class="alert alert-danger" role="alert" {% endif %}>
|
||||
<td class="endpoint">
|
||||
{%= endpointContent(endpoint, targetID)%}
|
||||
<a href="{%s endpoint %}" target="_blank">{%s endpoint %}</a>
|
||||
{% if hasOriginalLabels %}
|
||||
{% space %}
|
||||
(<a href="target_response?id={%s targetID %}" target="_blank"
|
||||
title="click to fetch target response on behalf of the scraper">response</a>)
|
||||
{% endif %}
|
||||
</td>
|
||||
<td>
|
||||
{% if ts.up %}
|
||||
|
|
@ -238,11 +247,26 @@
|
|||
{% endif %}
|
||||
</td>
|
||||
<td class="labels">
|
||||
{%= labelsContent(targetID, ts.sw.Config) %}
|
||||
<div
|
||||
{% if hasOriginalLabels %}
|
||||
{% space %}title="click to show original labels"
|
||||
onclick="document.getElementById('original-labels-{%s targetID %}').style.display='block'"
|
||||
{% endif %}
|
||||
>
|
||||
{%= formatLabels(ts.sw.Config.Labels) %}
|
||||
</div>
|
||||
{% if hasOriginalLabels %}
|
||||
<div style="display:none" id="original-labels-{%s targetID %}">
|
||||
{%= formatLabels(originalLabels) %}
|
||||
</div>
|
||||
{% endif %}
|
||||
</td>
|
||||
{% if hasOriginalLabels %}
|
||||
<td>
|
||||
{%= debugRelabelingContent(targetID)%}
|
||||
<a href="target-relabel-debug?id={%s targetID %}" target="_blank">target</a>{% space %}
|
||||
<a href="metric-relabel-debug?id={%s targetID %}" target="_blank">metrics</a>
|
||||
</td>
|
||||
{% endif %}
|
||||
<td>{%d ts.scrapesTotal %}</td>
|
||||
<td>{%d ts.scrapesFailed %}</td>
|
||||
<td>
|
||||
|
|
@ -264,6 +288,11 @@
|
|||
{% endfunc %}
|
||||
|
||||
{% func discoveredTargets(tsr *targetsStatusResult) %}
|
||||
{% if !tsr.hasOriginalLabels %}
|
||||
Discovered targets are unavailable when -promscrape.dropOriginalLabels command-line flag is set
|
||||
{% return %}
|
||||
{% endif %}
|
||||
|
||||
{% code tljs := tsr.getTargetLabelsByJob() %}
|
||||
<div class="row mt-4">
|
||||
<div class="col-12">
|
||||
|
|
@ -317,7 +346,8 @@
|
|||
{%= formatLabels(t.labels) %}
|
||||
</td>
|
||||
<td>
|
||||
{%= discoveredJobsRelabelContent(t.originalLabels) %}
|
||||
{% code targetID := getLabelsID(t.originalLabels) %}
|
||||
<a href="target-relabel-debug?id={%s targetID %}" target="_blank">debug</a>
|
||||
</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
|
|
@ -370,37 +400,3 @@
|
|||
{% endfunc %}
|
||||
|
||||
{% endstripspace %}
|
||||
|
||||
{% func endpointContent(endpoint, targetID string) %}
|
||||
<a href="{%s endpoint %}" target="_blank">{%s endpoint %}</a>{% space %}
|
||||
{% if targetID != "" %}
|
||||
(<a href="target_response?id={%s targetID %}" target="_blank"
|
||||
title="click to fetch target response on behalf of the scraper">response</a>)
|
||||
{% endif %}
|
||||
{% endfunc %}
|
||||
|
||||
{% func labelsContent(targetID string, config *ScrapeWork) %}
|
||||
<div title="click to show original labels"
|
||||
{% if targetID != "" %}onclick="document.getElementById('original-labels-{%s targetID %}').style.display='block'" {% endif %}>
|
||||
{%= formatLabels(config.Labels) %}
|
||||
</div>
|
||||
{% if config.OriginalLabels != nil %}
|
||||
<div style="display:none" id="original-labels-{%s targetID %}">
|
||||
{%= formatLabels(config.OriginalLabels) %}
|
||||
</div>
|
||||
{% endif %}
|
||||
{% endfunc %}
|
||||
|
||||
{% func debugRelabelingContent(targetID string) %}
|
||||
{% if targetID != "" %}
|
||||
<a href="target-relabel-debug?id={%s targetID %}" target="_blank">target</a>{% space %}
|
||||
<a href="metric-relabel-debug?id={%s targetID %}" target="_blank">metrics</a>
|
||||
{% endif %}
|
||||
{% endfunc %}
|
||||
|
||||
{% func discoveredJobsRelabelContent(originalLabels *promutils.Labels) %}
|
||||
{% if originalLabels != nil %}
|
||||
{% code targetID := getLabelsID(originalLabels) %}
|
||||
<a href="target-relabel-debug?id={%s targetID %}" target="_blank">debug</a>
|
||||
{% endif %}
|
||||
{% endfunc %}
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
Loading…
Reference in a new issue