vendor: update github.com/VictoriaMetrics/metricsql from v0.72.1 to v0.73.0

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5383
This commit is contained in:
Aliaksandr Valialkin 2024-02-18 18:41:39 +02:00
parent 149d83e596
commit 2e30842582
No known key found for this signature in database
GPG key ID: 52C003EE2BCDB9EB
7 changed files with 377 additions and 122 deletions

View file

@ -30,9 +30,11 @@ See also [LTS releases](https://docs.victoriametrics.com/LTS-releases.html).
## tip ## tip
* FEATURE: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): propagate [label filters](https://docs.victoriametrics.com/keyconcepts/#filtering) via all the [label manipulation functions](https://docs.victoriametrics.com/metricsql/#label-manipulation-functions). For example, `label_del(some_metric{job="foo"}, "instance") + other_metric{pod="bar"}` is now transformed to `label_del(some_metric{job="foo",pod="bar"}, "instance") + other_metric{job="foo",pod="bar"}`. This should reduce the amounts of time series processed during query execution.
* FEATURE: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/): expose `vm_last_partition_parts` [metrics](https://docs.victoriametrics.com/#monitoring), which show the number of [parts in the latest partition](https://docs.victoriametrics.com/#storage). These metrics may help debugging query performance slowdown related to the increased number of parts in the last partition, since usually all the ingested data is written to the last partition and all the queries are performed over the recently ingested data, e.g. the last partition. * FEATURE: [Single-node VictoriaMetrics](https://docs.victoriametrics.com/) and `vmstorage` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/): expose `vm_last_partition_parts` [metrics](https://docs.victoriametrics.com/#monitoring), which show the number of [parts in the latest partition](https://docs.victoriametrics.com/#storage). These metrics may help debugging query performance slowdown related to the increased number of parts in the last partition, since usually all the ingested data is written to the last partition and all the queries are performed over the recently ingested data, e.g. the last partition.
* FEATURE: [vmctl](https://docs.victoriametrics.com/vmctl.html): support client-side TLS configuration for [InfluxDB](https://docs.victoriametrics.com/vmctl/#migrating-data-from-influxdb-1x). See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5748). Thanks to @khushijain21 for [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5783). * FEATURE: [vmctl](https://docs.victoriametrics.com/vmctl.html): support client-side TLS configuration for [InfluxDB](https://docs.victoriametrics.com/vmctl/#migrating-data-from-influxdb-1x). See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5748). Thanks to @khushijain21 for [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5783).
* FEATURE: [vmctl](https://docs.victoriametrics.com/vmctl.html): support client-side TLS configuration for [Remote Read protocol](https://docs.victoriametrics.com/vmctl/#migrating-data-by-remote-read-protocol). See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5748). Thanks to @khushijain21 for [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5798). * FEATURE: [vmctl](https://docs.victoriametrics.com/vmctl.html): support client-side TLS configuration for [Remote Read protocol](https://docs.victoriametrics.com/vmctl/#migrating-data-by-remote-read-protocol). See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5748). Thanks to @khushijain21 for [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5798).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): preserve [`WITH` templates](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/expand-with-exprs) when clicking the `prettify query` button at the right side of query input field. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5383).
* BUGFIX: fix the misleading error `0ms is out of allowed range [0 ...` when passing `step=0` to [/api/v1/query](https://docs.victoriametrics.com/keyconcepts/#instant-query) * BUGFIX: fix the misleading error `0ms is out of allowed range [0 ...` when passing `step=0` to [/api/v1/query](https://docs.victoriametrics.com/keyconcepts/#instant-query)
or [/api/v1/query_range](https://docs.victoriametrics.com/keyconcepts/#range-query). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5795). or [/api/v1/query_range](https://docs.victoriametrics.com/keyconcepts/#range-query). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5795).

2
go.mod
View file

@ -9,7 +9,7 @@ require (
github.com/VictoriaMetrics/easyproto v0.1.4 github.com/VictoriaMetrics/easyproto v0.1.4
github.com/VictoriaMetrics/fastcache v1.12.2 github.com/VictoriaMetrics/fastcache v1.12.2
github.com/VictoriaMetrics/metrics v1.32.0 github.com/VictoriaMetrics/metrics v1.32.0
github.com/VictoriaMetrics/metricsql v0.72.1 github.com/VictoriaMetrics/metricsql v0.73.0
github.com/aws/aws-sdk-go-v2 v1.25.0 github.com/aws/aws-sdk-go-v2 v1.25.0
github.com/aws/aws-sdk-go-v2/config v1.27.0 github.com/aws/aws-sdk-go-v2/config v1.27.0
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.16.0 github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.16.0

4
go.sum
View file

@ -71,8 +71,8 @@ github.com/VictoriaMetrics/fastcache v1.12.2/go.mod h1:AmC+Nzz1+3G2eCPapF6UcsnkT
github.com/VictoriaMetrics/metrics v1.24.0/go.mod h1:eFT25kvsTidQFHb6U0oa0rTrDRdz4xTYjpL8+UPohys= github.com/VictoriaMetrics/metrics v1.24.0/go.mod h1:eFT25kvsTidQFHb6U0oa0rTrDRdz4xTYjpL8+UPohys=
github.com/VictoriaMetrics/metrics v1.32.0 h1:r9JK2zndYv0TIxFXLEHwhQqRdnu8/O3cwJiCBX4vJCM= github.com/VictoriaMetrics/metrics v1.32.0 h1:r9JK2zndYv0TIxFXLEHwhQqRdnu8/O3cwJiCBX4vJCM=
github.com/VictoriaMetrics/metrics v1.32.0/go.mod h1:r7hveu6xMdUACXvB8TYdAj8WEsKzWB0EkpJN+RDtOf8= github.com/VictoriaMetrics/metrics v1.32.0/go.mod h1:r7hveu6xMdUACXvB8TYdAj8WEsKzWB0EkpJN+RDtOf8=
github.com/VictoriaMetrics/metricsql v0.72.1 h1:fLIHgzezXgD4NjY5ksF4lRkHILW88uI5Lz0Q+N2ucnY= github.com/VictoriaMetrics/metricsql v0.73.0 h1:MvYnUIZHWD+Kj+sKuBSI1asR1fw1BxQPGshs32C7FIk=
github.com/VictoriaMetrics/metricsql v0.72.1/go.mod h1:k4UaP/+CjuZslIjd+kCigNG9TQmUqh5v0TP/nMEy90I= github.com/VictoriaMetrics/metricsql v0.73.0/go.mod h1:k4UaP/+CjuZslIjd+kCigNG9TQmUqh5v0TP/nMEy90I=
github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1ow= github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1ow=
github.com/VividCortex/ewma v1.2.0/go.mod h1:nz4BbCtbLyFDeC9SUHbtcT5644juEuWfUAUnGx7j5l4= github.com/VividCortex/ewma v1.2.0/go.mod h1:nz4BbCtbLyFDeC9SUHbtcT5644juEuWfUAUnGx7j5l4=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=

View file

@ -82,14 +82,24 @@ func getCommonLabelFilters(e Expr) []LabelFilter {
case *RollupExpr: case *RollupExpr:
return getCommonLabelFilters(t.Expr) return getCommonLabelFilters(t.Expr)
case *FuncExpr: case *FuncExpr:
if strings.ToLower(t.Name) == "label_set" { switch strings.ToLower(t.Name) {
case "label_set":
return getCommonLabelFiltersForLabelSet(t.Args) return getCommonLabelFiltersForLabelSet(t.Args)
} case "label_replace", "label_join", "label_map", "label_match", "label_mismatch", "label_transform":
return getCommonLabelFiltersForLabelReplace(t.Args)
case "label_copy", "label_move":
return getCommonLabelFiltersForLabelCopy(t.Args)
case "label_del", "label_uppercase", "label_lowercase", "labels_equal":
return getCommonLabelFiltersForLabelDel(t.Args)
case "label_keep":
return getCommonLabelFiltersForLabelKeep(t.Args)
default:
arg := getFuncArgForOptimization(t.Name, t.Args) arg := getFuncArgForOptimization(t.Name, t.Args)
if arg == nil { if arg == nil {
return nil return nil
} }
return getCommonLabelFilters(arg) return getCommonLabelFilters(arg)
}
case *AggrFuncExpr: case *AggrFuncExpr:
args := t.Args args := t.Args
if len(args) > 0 && canAcceptMultipleArgsForAggrFunc(t.Name) { if len(args) > 0 && canAcceptMultipleArgsForAggrFunc(t.Name) {
@ -164,6 +174,49 @@ func getCommonLabelFilters(e Expr) []LabelFilter {
} }
} }
func getCommonLabelFiltersForLabelKeep(args []Expr) []LabelFilter {
if len(args) == 0 {
return nil
}
lfs := getCommonLabelFilters(args[0])
lfs = keepLabelFiltersForLabelNames(lfs, args[1:])
return lfs
}
func getCommonLabelFiltersForLabelDel(args []Expr) []LabelFilter {
if len(args) == 0 {
return nil
}
lfs := getCommonLabelFilters(args[0])
lfs = dropLabelFiltersForLabelNames(lfs, args[1:])
return lfs
}
func getCommonLabelFiltersForLabelCopy(args []Expr) []LabelFilter {
if len(args) == 0 {
return nil
}
lfs := getCommonLabelFilters(args[0])
args = args[1:]
var labelNames []Expr
for i := 0; i < len(args); i += 2 {
if i+1 >= len(args) {
return nil
}
labelNames = append(labelNames, args[i+1])
}
lfs = dropLabelFiltersForLabelNames(lfs, labelNames)
return lfs
}
func getCommonLabelFiltersForLabelReplace(args []Expr) []LabelFilter {
if len(args) < 2 {
return nil
}
lfs := getCommonLabelFilters(args[0])
return dropLabelFiltersForLabelName(lfs, args[1])
}
func getCommonLabelFiltersForLabelSet(args []Expr) []LabelFilter { func getCommonLabelFiltersForLabelSet(args []Expr) []LabelFilter {
if len(args) == 0 { if len(args) == 0 {
return nil return nil
@ -190,13 +243,8 @@ func getCommonLabelFiltersForLabelSet(args []Expr) []LabelFilter {
continue continue
} }
lfsDst := lfs[:0] lfs = dropLabelFiltersForLabelName(lfs, labelName)
for _, lf := range lfs { lfs = append(lfs, LabelFilter{
if lf.Label != seLabelName.S {
lfsDst = append(lfsDst, lf)
}
}
lfs = append(lfsDst, LabelFilter{
Label: seLabelName.S, Label: seLabelName.S,
Value: seLabelValue.S, Value: seLabelValue.S,
}) })
@ -288,11 +336,18 @@ func pushdownBinaryOpFiltersInplace(e Expr, lfs []LabelFilter) {
case *RollupExpr: case *RollupExpr:
pushdownBinaryOpFiltersInplace(t.Expr, lfs) pushdownBinaryOpFiltersInplace(t.Expr, lfs)
case *FuncExpr: case *FuncExpr:
if strings.ToLower(t.Name) == "label_set" && len(t.Args) > 0 { switch strings.ToLower(t.Name) {
arg := t.Args[0] case "label_set":
lfs = getPushdownLabelFiltersForLabelSet(t.Args[1:], lfs) pushdownLabelFiltersForLabelSet(t.Args, lfs)
pushdownBinaryOpFiltersInplace(arg, lfs) case "label_replace", "label_join", "label_map", "label_match", "label_mismatch", "label_transform":
} else { pushdownLabelFiltersForLabelReplace(t.Args, lfs)
case "label_copy", "label_move":
pushdownLabelFiltersForLabelCopy(t.Args, lfs)
case "label_del", "label_uppercase", "label_lowercase", "labels_equal":
pushdownLabelFiltersForLabelDel(t.Args, lfs)
case "label_keep":
pushdownLabelFiltersForLabelKeep(t.Args, lfs)
default:
arg := getFuncArgForOptimization(t.Name, t.Args) arg := getFuncArgForOptimization(t.Name, t.Args)
if arg != nil { if arg != nil {
pushdownBinaryOpFiltersInplace(arg, lfs) pushdownBinaryOpFiltersInplace(arg, lfs)
@ -318,24 +373,59 @@ func pushdownBinaryOpFiltersInplace(e Expr, lfs []LabelFilter) {
} }
} }
func getPushdownLabelFiltersForLabelSet(args []Expr, lfs []LabelFilter) []LabelFilter { func pushdownLabelFiltersForLabelKeep(args []Expr, lfs []LabelFilter) {
m := make(map[string]struct{}) if len(args) == 0 {
for i := 0; i < len(args); i += 2 { return
labelName := args[i]
seLabelName, ok := labelName.(*StringExpr)
if !ok {
return nil
}
m[seLabelName.S] = struct{}{}
} }
lfs = keepLabelFiltersForLabelNames(lfs, args[1:])
pushdownBinaryOpFiltersInplace(args[0], lfs)
}
var lfsDst []LabelFilter func pushdownLabelFiltersForLabelDel(args []Expr, lfs []LabelFilter) {
for _, lf := range lfs { if len(args) == 0 {
if _, ok := m[lf.Label]; !ok { return
lfsDst = append(lfsDst, lf)
} }
lfs = dropLabelFiltersForLabelNames(lfs, args[1:])
pushdownBinaryOpFiltersInplace(args[0], lfs)
}
func pushdownLabelFiltersForLabelCopy(args []Expr, lfs []LabelFilter) {
if len(args) == 0 {
return
} }
return lfsDst arg := args[0]
args = args[1:]
var labelNames []Expr
for i := 0; i < len(args); i += 2 {
if i+1 >= len(args) {
return
}
labelNames = append(labelNames, args[i+1])
}
lfs = dropLabelFiltersForLabelNames(lfs, labelNames)
pushdownBinaryOpFiltersInplace(arg, lfs)
}
func pushdownLabelFiltersForLabelReplace(args []Expr, lfs []LabelFilter) {
if len(args) < 2 {
return
}
lfs = dropLabelFiltersForLabelName(lfs, args[1])
pushdownBinaryOpFiltersInplace(args[0], lfs)
}
func pushdownLabelFiltersForLabelSet(args []Expr, lfs []LabelFilter) {
if len(args) == 0 {
return
}
arg := args[0]
args = args[1:]
var labelNames []Expr
for i := 0; i < len(args); i += 2 {
labelNames = append(labelNames, args[i])
}
lfs = dropLabelFiltersForLabelNames(lfs, labelNames)
pushdownBinaryOpFiltersInplace(arg, lfs)
} }
func intersectLabelFilters(lfsA, lfsB []LabelFilter) []LabelFilter { func intersectLabelFilters(lfsA, lfsB []LabelFilter) []LabelFilter {
@ -354,6 +444,48 @@ func intersectLabelFilters(lfsA, lfsB []LabelFilter) []LabelFilter {
return lfs return lfs
} }
func keepLabelFiltersForLabelNames(lfs []LabelFilter, labelNames []Expr) []LabelFilter {
m := make(map[string]struct{}, len(labelNames))
for _, labelName := range labelNames {
seLabelName, ok := labelName.(*StringExpr)
if !ok {
return nil
}
m[seLabelName.S] = struct{}{}
}
var lfsDst []LabelFilter
for _, lf := range lfs {
if _, ok := m[lf.Label]; ok {
lfsDst = append(lfsDst, lf)
}
}
return lfsDst
}
func dropLabelFiltersForLabelNames(lfs []LabelFilter, labelNames []Expr) []LabelFilter {
for _, labelName := range labelNames {
lfs = dropLabelFiltersForLabelName(lfs, labelName)
}
return lfs
}
func dropLabelFiltersForLabelName(lfs []LabelFilter, labelName Expr) []LabelFilter {
seLabelName, ok := labelName.(*StringExpr)
if !ok {
return nil
}
lfsDst := make([]LabelFilter, 0, len(lfs))
for _, lf := range lfs {
if lf.Label != seLabelName.S {
lfsDst = append(lfsDst, lf)
}
}
return lfsDst
}
func unionLabelFilters(lfsA, lfsB []LabelFilter) []LabelFilter { func unionLabelFilters(lfsA, lfsB []LabelFilter) []LabelFilter {
if len(lfsA) == 0 { if len(lfsA) == 0 {
return lfsB return lfsB
@ -498,7 +630,13 @@ func getRollupArgIdxForOptimization(funcName string, args []Expr) int {
func getTransformArgIdxForOptimization(funcName string, args []Expr) int { func getTransformArgIdxForOptimization(funcName string, args []Expr) int {
switch strings.ToLower(funcName) { switch strings.ToLower(funcName) {
case "", "absent", "scalar", "union", "vector", "range_normalize": case "label_copy", "label_del", "label_join", "label_keep", "label_lowercase", "label_map",
"label_match", "label_mismatch", "label_move", "label_replace", "label_set", "label_transform",
"label_uppercase", "labels_equal":
panic(fmt.Errorf("BUG: unexpected funcName passed to getTransformArgIdxForOptimization: %s", funcName))
case "drop_common_labels", "range_normalize":
return -1
case "", "absent", "scalar", "union", "vector":
return -1 return -1
case "end", "now", "pi", "ru", "start", "step", "time": case "end", "now", "pi", "ru", "start", "step", "time":
return -1 return -1
@ -509,9 +647,8 @@ func getTransformArgIdxForOptimization(funcName string, args []Expr) int {
return 1 return 1
case "histogram_quantiles": case "histogram_quantiles":
return len(args) - 1 return len(args) - 1
case "drop_common_labels", "label_copy", "label_del", "label_graphite_group", "label_join", "label_keep", "label_lowercase", case "label_graphite_group":
"label_map", "label_move", "label_replace", "label_set", "label_transform", "label_uppercase": return 0
return -1
default: default:
return 0 return 0
} }

View file

@ -13,6 +13,26 @@ import (
// //
// MetricsQL is backwards-compatible with PromQL. // MetricsQL is backwards-compatible with PromQL.
func Parse(s string) (Expr, error) { func Parse(s string) (Expr, error) {
// Parse s
e, err := parseInternal(s)
if err != nil {
return nil, err
}
// Expand `WITH` expressions.
was := getDefaultWithArgExprs()
if e, err = expandWithExpr(was, e); err != nil {
return nil, fmt.Errorf(`cannot expand WITH expressions: %s`, err)
}
e = removeParensExpr(e)
e = simplifyConstants(e)
if err := checkSupportedFunctions(e); err != nil {
return nil, err
}
return e, nil
}
func parseInternal(s string) (Expr, error) {
var p parser var p parser
p.lex.Init(s) p.lex.Init(s)
if err := p.lex.Next(); err != nil { if err := p.lex.Next(); err != nil {
@ -25,15 +45,6 @@ func Parse(s string) (Expr, error) {
if !isEOF(p.lex.Token) { if !isEOF(p.lex.Token) {
return nil, fmt.Errorf(`unparsed data left: %q`, p.lex.Context()) return nil, fmt.Errorf(`unparsed data left: %q`, p.lex.Context())
} }
was := getDefaultWithArgExprs()
if e, err = expandWithExpr(was, e); err != nil {
return nil, fmt.Errorf(`cannot expand WITH expressions: %s`, err)
}
e = removeParensExpr(e)
e = simplifyConstants(e)
if err := checkSupportedFunctions(e); err != nil {
return nil, err
}
return e, nil return e, nil
} }
@ -104,36 +115,33 @@ func mustParseWithArgExpr(s string) *withArgExpr {
// removeParensExpr removes parensExpr for (Expr) case. // removeParensExpr removes parensExpr for (Expr) case.
func removeParensExpr(e Expr) Expr { func removeParensExpr(e Expr) Expr {
if re, ok := e.(*RollupExpr); ok { switch t := e.(type) {
re.Expr = removeParensExpr(re.Expr) case *RollupExpr:
if re.At != nil { t.Expr = removeParensExpr(t.Expr)
re.At = removeParensExpr(re.At) if t.At != nil {
t.At = removeParensExpr(t.At)
} }
return re return t
case *BinaryOpExpr:
t.Left = removeParensExpr(t.Left)
t.Right = removeParensExpr(t.Right)
return t
case *AggrFuncExpr:
for i, arg := range t.Args {
t.Args[i] = removeParensExpr(arg)
} }
if be, ok := e.(*BinaryOpExpr); ok { return t
be.Left = removeParensExpr(be.Left) case *FuncExpr:
be.Right = removeParensExpr(be.Right) for i, arg := range t.Args {
return be t.Args[i] = removeParensExpr(arg)
} }
if ae, ok := e.(*AggrFuncExpr); ok { return t
for i, arg := range ae.Args { case *parensExpr:
ae.Args[i] = removeParensExpr(arg) args := *t
}
return ae
}
if fe, ok := e.(*FuncExpr); ok {
for i, arg := range fe.Args {
fe.Args[i] = removeParensExpr(arg)
}
return fe
}
if pe, ok := e.(*parensExpr); ok {
args := *pe
for i, arg := range args { for i, arg := range args {
args[i] = removeParensExpr(arg) args[i] = removeParensExpr(arg)
} }
if len(*pe) == 1 { if len(*t) == 1 {
return args[0] return args[0]
} }
// Treat parensExpr as a function with empty name, i.e. union() // Treat parensExpr as a function with empty name, i.e. union()
@ -142,38 +150,43 @@ func removeParensExpr(e Expr) Expr {
Args: args, Args: args,
} }
return fe return fe
case *withExpr:
for _, arg := range t.Was {
arg.Expr = removeParensExpr(arg.Expr)
} }
t.Expr = removeParensExpr(t.Expr)
return t
default:
return e return e
}
} }
func simplifyConstants(e Expr) Expr { func simplifyConstants(e Expr) Expr {
if re, ok := e.(*RollupExpr); ok { switch t := e.(type) {
re.Expr = simplifyConstants(re.Expr) case *withExpr:
if re.At != nil { panic(fmt.Errorf("BUG: withExpr shouldn't be passed to simplifyConstants"))
re.At = simplifyConstants(re.At) case *parensExpr:
panic(fmt.Errorf("BUG: parensExpr shouldn't be passed to simplifyConstants"))
case *RollupExpr:
t.Expr = simplifyConstants(t.Expr)
if t.At != nil {
t.At = simplifyConstants(t.At)
} }
return re return t
} case *AggrFuncExpr:
if ae, ok := e.(*AggrFuncExpr); ok { simplifyConstantsInplace(t.Args)
simplifyConstantsInplace(ae.Args) return t
return ae case *FuncExpr:
} simplifyConstantsInplace(t.Args)
if fe, ok := e.(*FuncExpr); ok { return t
simplifyConstantsInplace(fe.Args) case *BinaryOpExpr:
return fe return simplifyConstantsInBinaryExpr(t)
} default:
if pe, ok := e.(*parensExpr); ok {
if len(*pe) == 1 {
return simplifyConstants((*pe)[0])
}
simplifyConstantsInplace(*pe)
return pe
}
be, ok := e.(*BinaryOpExpr)
if !ok {
return e return e
} }
}
func simplifyConstantsInBinaryExpr(be *BinaryOpExpr) Expr {
be.Left = simplifyConstants(be.Left) be.Left = simplifyConstants(be.Left)
be.Right = simplifyConstants(be.Right) be.Right = simplifyConstants(be.Right)
@ -202,7 +215,7 @@ func simplifyConstants(e Expr) Expr {
return be return be
} }
// Perform string comparisons. // Perform string comparisons.
ok = false ok := false
switch be.Op { switch be.Op {
case "==": case "==":
ok = lse.S == rse.S ok = lse.S == rse.S
@ -1715,7 +1728,13 @@ type StringExpr struct {
// AppendString appends string representation of se to dst and returns the result. // AppendString appends string representation of se to dst and returns the result.
func (se *StringExpr) AppendString(dst []byte) []byte { func (se *StringExpr) AppendString(dst []byte) []byte {
if len(se.tokens) > 0 { if len(se.tokens) > 0 {
panic(fmt.Errorf("BUG: StringExpr=%q must be already parsed with expandWithExpr()", se.tokens)) for i, token := range se.tokens {
dst = append(dst, token...)
if i+1 < len(se.tokens) {
dst = append(dst, '+')
}
}
return dst
} }
return strconv.AppendQuote(dst, se.S) return strconv.AppendQuote(dst, se.S)
} }
@ -2182,15 +2201,24 @@ type MetricExpr struct {
} }
func appendLabelFilterss(dst []byte, lfss [][]*labelFilterExpr) []byte { func appendLabelFilterss(dst []byte, lfss [][]*labelFilterExpr) []byte {
offset := 0
metricName := getMetricNameFromLabelFilterss(lfss)
if metricName != "" {
offset = 1
dst = appendEscapedIdent(dst, metricName)
}
if isOnlyMetricNameInLabelFilterss(lfss) {
return dst
}
dst = append(dst, '{') dst = append(dst, '{')
for i, lfs := range lfss { for i, lfs := range lfss {
for j, lf := range lfs { lfs = lfs[offset:]
dst = lf.AppendString(dst) if len(lfs) == 0 {
if j+1 < len(lfs) { continue
dst = append(dst, ',')
} }
} dst = appendLabelFilterExprs(dst, lfs)
if i+1 < len(lfss) { if i+1 < len(lfss) && len(lfss[i+1]) > offset {
dst = append(dst, " or "...) dst = append(dst, " or "...)
} }
} }
@ -2198,11 +2226,59 @@ func appendLabelFilterss(dst []byte, lfss [][]*labelFilterExpr) []byte {
return dst return dst
} }
func appendLabelFilterExprs(dst []byte, lfs []*labelFilterExpr) []byte {
for i, lf := range lfs {
dst = lf.AppendString(dst)
if i+1 < len(lfs) {
dst = append(dst, ',')
}
}
return dst
}
func isOnlyMetricNameInLabelFilterss(lfss [][]*labelFilterExpr) bool {
if getMetricNameFromLabelFilterss(lfss) == "" {
return false
}
for _, lfs := range lfss {
if len(lfs) > 1 {
return false
}
}
return true
}
func getMetricNameFromLabelFilterss(lfss [][]*labelFilterExpr) string {
if len(lfss) == 0 || len(lfss[0]) == 0 || lfss[0][0].Label != "__name__" || len(lfss[0][0].Value.tokens) != 1 {
return ""
}
metricName := mustExtractMetricNameFromToken(lfss[0][0].Value.tokens[0])
for _, lfs := range lfss[1:] {
if len(lfs) == 0 || lfs[0].Label != "__name__" || len(lfs[0].Value.tokens) != 1 {
return ""
}
metricNameLocal := mustExtractMetricNameFromToken(lfs[0].Value.tokens[0])
if metricNameLocal != metricName {
return ""
}
}
return metricName
}
func mustExtractMetricNameFromToken(token string) string {
metricName, err := extractStringValue(token)
if err != nil {
panic(fmt.Errorf("BUG: cannot obtain metric name: %w", err))
}
return metricName
}
// AppendString appends string representation of me to dst and returns the result. // AppendString appends string representation of me to dst and returns the result.
func (me *MetricExpr) AppendString(dst []byte) []byte { func (me *MetricExpr) AppendString(dst []byte) []byte {
if len(me.labelFilterss) > 0 { if len(me.labelFilterss) > 0 {
return appendLabelFilterss(dst, me.labelFilterss) return appendLabelFilterss(dst, me.labelFilterss)
} }
lfss := me.LabelFilterss lfss := me.LabelFilterss
if len(lfss) == 0 { if len(lfss) == 0 {
dst = append(dst, "{}"...) dst = append(dst, "{}"...)
@ -2214,15 +2290,19 @@ func (me *MetricExpr) AppendString(dst []byte) []byte {
offset = 1 offset = 1
dst = appendEscapedIdent(dst, metricName) dst = appendEscapedIdent(dst, metricName)
} }
if len(lfss) == 1 && len(lfss[0]) == offset { if me.isOnlyMetricName() {
return dst return dst
} }
dst = append(dst, '{') dst = append(dst, '{')
lfs := lfss[0] for i, lfs := range lfss {
dst = appendLabelFilters(dst, lfs[offset:]) lfs = lfs[offset:]
for _, lfs := range lfss[1:] { if len(lfs) == 0 {
continue
}
dst = appendLabelFilters(dst, lfs)
if i+1 < len(lfss) && len(lfss[i+1]) > offset {
dst = append(dst, " or "...) dst = append(dst, " or "...)
dst = appendLabelFilters(dst, lfs[offset:]) }
} }
dst = append(dst, '}') dst = append(dst, '}')
return dst return dst

View file

@ -2,10 +2,11 @@ package metricsql
// Prettify returns prettified representation of MetricsQL query q. // Prettify returns prettified representation of MetricsQL query q.
func Prettify(q string) (string, error) { func Prettify(q string) (string, error) {
e, err := Parse(q) e, err := parseInternal(q)
if err != nil { if err != nil {
return "", err return "", err
} }
e = removeParensExpr(e)
b := appendPrettifiedExpr(nil, e, 0, false) b := appendPrettifiedExpr(nil, e, 0, false)
return string(b), nil return string(b), nil
} }
@ -28,11 +29,11 @@ func appendPrettifiedExpr(dst []byte, e Expr, indent int, needParens bool) []byt
dst = append(dst, ')') dst = append(dst, ')')
} }
if len(dst)-dstLen <= maxPrettifiedLineLen { if len(dst)-dstLen <= maxPrettifiedLineLen {
// There is no need in splitting the e string representation, since its' length doesn't exceed. // There is no need in splitting the e string representation, since its' length doesn't exceed maxPrettifiedLineLen.
return dst return dst
} }
// The e string representation exceeds maxPrettifiedLineLen. Split it into multiple lines // The e string representation exceeds maxPrettifiedLineLen. Split it into multiple lines.
dst = dst[:dstLen] dst = dst[:dstLen]
if needParens { if needParens {
dst = appendIndent(dst, indent) dst = appendIndent(dst, indent)
@ -40,6 +41,37 @@ func appendPrettifiedExpr(dst []byte, e Expr, indent int, needParens bool) []byt
indent++ indent++
} }
switch t := e.(type) { switch t := e.(type) {
case *withExpr:
// Put every WITH expression on a separate line
dst = appendIndent(dst, indent)
dst = append(dst, "WITH (\n"...)
indent++
for _, wa := range t.Was {
dst = appendPrettifiedExpr(dst, wa, indent, false)
dst = append(dst, ",\n"...)
}
indent--
dst = appendIndent(dst, indent)
dst = append(dst, ")\n"...)
dst = appendPrettifiedExpr(dst, t.Expr, indent, false)
case *withArgExpr:
// Wrap long withArgExpr into `(...)`
dst = appendIndent(dst, indent)
dst = appendEscapedIdent(dst, t.Name)
if len(t.Args) > 0 {
dst = append(dst, '(')
dst = appendEscapedIdent(dst, t.Args[0])
for _, arg := range t.Args[1:] {
dst = append(dst, ", "...)
dst = appendEscapedIdent(dst, arg)
}
dst = append(dst, ')')
}
dst = append(dst, " = (\n"...)
dst = appendPrettifiedExpr(dst, t.Expr, indent+1, false)
dst = append(dst, '\n')
dst = appendIndent(dst, indent)
dst = append(dst, ')')
case *BinaryOpExpr: case *BinaryOpExpr:
// Split: // Split:
// //
@ -125,20 +157,24 @@ func appendPrettifiedExpr(dst []byte, e Expr, indent int, needParens bool) []byt
// or // or
// filtersN // filtersN
// } // }
lfss := t.labelFilterss
offset := 0 offset := 0
metricName := t.getMetricName() metricName := getMetricNameFromLabelFilterss(lfss)
if metricName != "" { if metricName != "" {
offset = 1 offset = 1
} }
dst = appendIndent(dst, indent) dst = appendIndent(dst, indent)
dst = appendEscapedIdent(dst, metricName) dst = appendEscapedIdent(dst, metricName)
if !t.isOnlyMetricName() { if !isOnlyMetricNameInLabelFilterss(lfss) {
dst = append(dst, "{\n"...) dst = append(dst, "{\n"...)
lfss := t.LabelFilterss
for i, lfs := range lfss { for i, lfs := range lfss {
dst = appendPrettifiedLabelFilters(dst, indent+1, lfs[offset:]) lfs = lfs[offset:]
if len(lfs) == 0 {
continue
}
dst = appendPrettifiedLabelFilters(dst, indent+1, lfs)
dst = append(dst, '\n') dst = append(dst, '\n')
if i+1 < len(lfss) { if i+1 < len(lfss) && len(lfss[i+1]) > offset {
dst = appendIndent(dst, indent+2) dst = appendIndent(dst, indent+2)
dst = append(dst, "or\n"...) dst = append(dst, "or\n"...)
} }
@ -173,12 +209,12 @@ func appendPrettifiedFuncArgs(dst []byte, indent int, args []Expr) []byte {
return dst return dst
} }
func appendPrettifiedLabelFilters(dst []byte, indent int, lfs []LabelFilter) []byte { func appendPrettifiedLabelFilters(dst []byte, indent int, lfs []*labelFilterExpr) []byte {
dstLen := len(dst) dstLen := len(dst)
// Try marshaling lfs into a single line // Try marshaling lfs into a single line
dst = appendIndent(dst, indent) dst = appendIndent(dst, indent)
dst = appendLabelFilters(dst, lfs) dst = appendLabelFilterExprs(dst, lfs)
if len(dst)-dstLen <= maxPrettifiedLineLen { if len(dst)-dstLen <= maxPrettifiedLineLen {
return dst return dst
} }

2
vendor/modules.txt vendored
View file

@ -102,7 +102,7 @@ github.com/VictoriaMetrics/fastcache
# github.com/VictoriaMetrics/metrics v1.32.0 # github.com/VictoriaMetrics/metrics v1.32.0
## explicit; go 1.17 ## explicit; go 1.17
github.com/VictoriaMetrics/metrics github.com/VictoriaMetrics/metrics
# github.com/VictoriaMetrics/metricsql v0.72.1 # github.com/VictoriaMetrics/metricsql v0.73.0
## explicit; go 1.13 ## explicit; go 1.13
github.com/VictoriaMetrics/metricsql github.com/VictoriaMetrics/metricsql
github.com/VictoriaMetrics/metricsql/binaryop github.com/VictoriaMetrics/metricsql/binaryop