github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
VictoriaMetrics/app/vmselect/prometheus/expand-with-exprs.qtpl

269 lines
6.2 KiB
Text
Raw Normal View History

app/vmselect: add /expand-with-exprs page
2022-12-15 00:01:33 +00:00
{% import (
vmui: Integrate WITH template playground (#3831) * feat: add WithTemplate page * app/vmselect/prometheus: enable json mode for expand with expr API * app/vmselect/prometheus: enable CORS and add content type * feat: add api for expand with templates * fix: remove console from useExpandWithExprs * app/vmselect/prometheus: fix escaping * vmui: integrate WITH template * app/vmctl: check content type instead of form param * fix: add content-type for fetch with-exprs * fix: add a header to the server's response that allows the "Content-Type" header * app/vmctl: added comment and cleanup * app/vmctl: use format query param --------- Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>
2023-04-25 08:40:01 +00:00
"fmt"
app/vmselect: add /expand-with-exprs page
2022-12-15 00:01:33 +00:00
"github.com/VictoriaMetrics/metricsql"
) %}
{% stripspace %}
// ExpandWithExprsResponse returns a webpage, which expands with templates in q MetricsQL.
{% func ExpandWithExprsResponse(q string) %}
<html>
<head>
<title>Expand WITH expressions</title>
<style>
p { font-weight: bold }
textarea { margin: 1em }
</style>
</head>
<body>
<div>
<form method="get">
<div>
<p>
all: replace old https://docs.victoriametrics.com/MetricsQL.html url with the new one - https://docs.victoriametrics.com/metricsql/
2024-04-18 00:14:53 +00:00
<a href="https://docs.victoriametrics.com/metricsql/">MetricsQL</a> query with optional WITH expressions:
app/vmselect: add /expand-with-exprs page
2022-12-15 00:01:33 +00:00
</p>
<textarea name="query" style="height: 15em; width: 90%">{%s q %}</textarea><br/>
<input type="submit" value="Expand" />
<p>
all: replace old https://docs.victoriametrics.com/MetricsQL.html url with the new one - https://docs.victoriametrics.com/metricsql/
2024-04-18 00:14:53 +00:00
<a href="https://docs.victoriametrics.com/metricsql/">MetricsQL</a> query after expanding WITH expressions and applying other optimizations:
app/vmselect: add /expand-with-exprs page
2022-12-15 00:01:33 +00:00
</p>
<textarea style="height: 5em; width: 90%" readonly="readonly">{%= expandWithExprs(q) %}</textarea>
</div>
</form>
</div>
<div>{%= withExprsTutorial() %}</div>
</body>
</html>
{% endfunc %}
{% func expandWithExprs(q string) %}
{% if len(q) == 0 %}
{% return %}
{% endif %}
{% code expr, err := metricsql.Parse(q) %}
{% if err != nil %}
Cannot parse query: {%v err %}
{% else %}
{% code expr = metricsql.Optimize(expr) %}
{%z expr.AppendString(nil) %}
{% endif %}
{% endfunc %}
app/vmselect: small cleanup after 68e31a6000cd7dc59a6a5f5851437954f235458f Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3811
2023-05-08 21:34:34 +00:00
{% func ExpandWithExprsJSONResponse(q string) %}
{% if len(q) == 0 %}
{
"status": "error",
"error": "query string cannot be empty"
}
{% return %}
{% endif %}
{
{% code expr, err := metricsql.Parse(q) %}
{% if err != nil %}
"status": "error",
app/vmselect: small cleanup after 4f3f9950d0b297369d9f66a862089e16887d771f Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3807
2023-05-08 21:57:09 +00:00
"error": {%q= fmt.Sprintf("Cannot parse query: %s", err) %}
app/vmselect: small cleanup after 68e31a6000cd7dc59a6a5f5851437954f235458f Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3811
2023-05-08 21:34:34 +00:00
{% else %}
{% code expr = metricsql.Optimize(expr) %}
"status": "success",
app/vmselect: small cleanup after 4f3f9950d0b297369d9f66a862089e16887d771f Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3807
2023-05-08 21:57:09 +00:00
"expr": {%qz= expr.AppendString(nil) %}
app/vmselect: small cleanup after 68e31a6000cd7dc59a6a5f5851437954f235458f Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3811
2023-05-08 21:34:34 +00:00
{% endif %}
}
{% endfunc %}
app/vmselect: add /expand-with-exprs page
2022-12-15 00:01:33 +00:00
{% endstripspace %}
{% func withExprsTutorial() %}
all: replace old https://docs.victoriametrics.com/MetricsQL.html url with the new one - https://docs.victoriametrics.com/metricsql/
2024-04-18 00:14:53 +00:00
<h3>Tutorial for WITH expressions in <a href="https://docs.victoriametrics.com/metricsql/">MetricsQL</a></h3>
app/vmselect: add /expand-with-exprs page
2022-12-15 00:01:33 +00:00
<p>
docs: use persistent links to Grafana dashboards These links do not depend on the dashboard name, so they do not break after the renaming of the dashboard. This is a follow-up for ff33e60a3d46b50bc585a66c433fd62ec00ec7ca
2024-01-21 22:09:19 +00:00
Let's look at the following real query from <a href="https://grafana.com/grafana/dashboards/1860">Node Exporter Full</a> dashboard:
app/vmselect: add /expand-with-exprs page
2022-12-15 00:01:33 +00:00
</p>
<pre>
(
(
node_memory_MemTotal_bytes{instance=~"$node:$port", job=~"$job"}
-
node_memory_MemFree_bytes{instance=~"$node:$port", job=~"$job"}
)
/
node_memory_MemTotal_bytes{instance=~"$node:$port", job=~"$job"}
)
*
100
</pre>
<p>
It is clear the query calculates the percentage of used memory
for the given $node, $port and $job. Isn't it? :)
</p>
<p>
What's wrong with this query? Copy-pasted label filters for distinct timeseries
which makes it easy to mistype these filters during modification.
Let's simplify the query with WITH expressions:
</p>
<pre>
WITH (
commonFilters = {instance=~"$node:$port",job=~"$job"}
)
(
node_memory_MemTotal_bytes{commonFilters}
-
node_memory_MemFree_bytes{commonFilters}
)
/
node_memory_MemTotal_bytes{commonFilters} * 100
</pre>
<p>
Now label filters are located in a single place instead of three distinct places.
The query mentions node_memory_MemTotal_bytes metric twice and {commonFilters}
three times. WITH expressions may improve this:
</p>
<pre>
WITH (
my_resource_utilization(free, limit, filters) = (limit{filters} - free{filters}) / limit{filters} * 100
)
my_resource_utilization(
node_memory_MemFree_bytes,
node_memory_MemTotal_bytes,
{instance=~"$node:$port",job=~"$job"},
)
</pre>
<p>
Now the template function my_resource_utilization() may be used for monitoring arbitrary
resources - memory, CPU, network, storage, you name it.
</p>
<p>
docs: use persistent links to Grafana dashboards These links do not depend on the dashboard name, so they do not break after the renaming of the dashboard. This is a follow-up for ff33e60a3d46b50bc585a66c433fd62ec00ec7ca
2024-01-21 22:09:19 +00:00
Let's take another nice query from <a href="https://grafana.com/grafana/dashboards/1860">Node Exporter Full</a> dashboard:
app/vmselect: add /expand-with-exprs page
2022-12-15 00:01:33 +00:00
</p>
<pre>
(
(
(
count(
count(node_cpu_seconds_total{instance=~"$node:$port",job=~"$job"}) by (cpu)
)
)
-
avg(
sum by (mode) (rate(node_cpu_seconds_total{mode='idle',instance=~"$node:$port",job=~"$job"}[5m]))
)
)
*
100
)
/
count(
count(node_cpu_seconds_total{instance=~"$node:$port",job=~"$job"}) by (cpu)
)
</pre>
<p>
Do you understand what does this mess do? Is it manageable? :) WITH expressions are happy to help in a few iterations.
<br/>
<br/>
1. Extract common filters used in multiple places into a commonFilters variable:
</p>
<pre>
WITH (
commonFilters = {instance=~"$node:$port",job=~"$job"}
)
(
(
(
count(
count(node_cpu_seconds_total{commonFilters}) by (cpu)
)
)
-
avg(
sum by (mode) (rate(node_cpu_seconds_total{mode='idle',commonFilters}[5m]))
)
)
*
100
)
/
count(
count(node_cpu_seconds_total{commonFilters}) by (cpu)
)
</pre>
<p>
2. Extract "count(count(...) by (cpu))" into cpuCount variable:
</p>
<pre>
WITH (
commonFilters = {instance=~"$node:$port",job=~"$job"},
cpuCount = count(count(node_cpu_seconds_total{commonFilters}) by (cpu))
)
(
(
cpuCount
-
avg(
sum by (mode) (rate(node_cpu_seconds_total{mode='idle',commonFilters}[5m]))
)
)
*
100
) / cpuCount
</pre>
<p>
3. Extract rate(...) part into cpuIdle variable, since it is clear now that this part calculates the number of idle CPUs:
</p>
<pre>
WITH (
commonFilters = {instance=~"$node:$port",job=~"$job"},
cpuCount = count(count(node_cpu_seconds_total{commonFilters}) by (cpu)),
cpuIdle = sum(rate(node_cpu_seconds_total{mode='idle',commonFilters}[5m]))
)
((cpuCount - cpuIdle) * 100) / cpuCount
</pre>
<p>
4. Put node_cpu_seconds_total{commonFilters} into its own varialbe with the name cpuSeconds:
</p>
<pre>
WITH (
cpuSeconds = node_cpu_seconds_total{instance=~"$node:$port",job=~"$job"},
cpuCount = count(count(cpuSeconds) by (cpu)),
cpuIdle = sum(rate(cpuSeconds{mode='idle'}[5m]))
)
((cpuCount - cpuIdle) * 100) / cpuCount
</pre>
<p>
Now the query became more clear comparing to the initial query.
</p>
<p>
WITH expressions may be nested and may be put anywhere. Try expanding the following query:
</p>
<pre>
WITH (
f(a, b) = WITH (
f1(x) = b-x,
f2(x) = x+x
) f1(a)*f2(b)
) f(foo, with(x=bar) x)
</pre>
{% endfunc %}
Reference in a new issue Copy permalink