github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2025-03-11 15:34:56 +00:00
Code Issues Projects Releases Packages Wiki Activity

vmalert: enable configuring explicit path (#1536)

Browse source 
* vmalert: allow to disable automatically added path to remote write address via disablePathAppend flag
* docs: update docs to include remoteWrite.disablePathAppend
This commit is contained in:
Alexander Rickardsson 2021-08-16 13:20:57 +02:00 committed by GitHub
parent 9af8c71975
commit 2400f85761
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 68 additions and 56 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

34
app/vmalert/README.md
View file

@ -107,11 +107,11 @@ expression and then act according to the Rule type.
There are two types of Rules: There are two types of Rules:
* [alerting](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) - * [alerting](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) -
Alerting rules allows to define alert conditions via `expr` field and to send notifications Alerting rules allows to define alert conditions via `expr` field and to send notifications
[Alertmanager](https://github.com/prometheus/alertmanager) if execution result is not empty. [Alertmanager](https://github.com/prometheus/alertmanager) if execution result is not empty.
* [recording](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) - * [recording](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) -
Recording rules allows to define `expr` which result will be than backfilled to configured Recording rules allows to define `expr` which result will be than backfilled to configured
`-remoteWrite.url`. Recording rules are used to precompute frequently needed or computationally `-remoteWrite.url`. Recording rules are used to precompute frequently needed or computationally
expensive expressions and save their result as a new set of time series. expensive expressions and save their result as a new set of time series.
`vmalert` forbids to define duplicates - rules with the same combination of name, expression and labels `vmalert` forbids to define duplicates - rules with the same combination of name, expression and labels
@ -189,26 +189,26 @@ The state stored to the configured address on every rule evaluation.
from configured address by querying time series with name `ALERTS_FOR_STATE`. from configured address by querying time series with name `ALERTS_FOR_STATE`.
Both flags are required for the proper state restoring. Restore process may fail if time series are missing Both flags are required for the proper state restoring. Restore process may fail if time series are missing
in configured `-remoteRead.url`, weren't updated in the last `1h` (controlled by `-remoteRead.lookback`) in configured `-remoteRead.url`, weren't updated in the last `1h` (controlled by `-remoteRead.lookback`)
or received state doesn't match current `vmalert` rules configuration. or received state doesn't match current `vmalert` rules configuration.
### Multitenancy ### Multitenancy
There are the following approaches for alerting and recording rules across There are the following approaches for alerting and recording rules across
[multiple tenants](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multitenancy): [multiple tenants](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multitenancy):
* To run a separate `vmalert` instance per each tenant. * To run a separate `vmalert` instance per each tenant.
The corresponding tenant must be specified in `-datasource.url` command-line flag The corresponding tenant must be specified in `-datasource.url` command-line flag
according to [these docs](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#url-format). according to [these docs](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#url-format).
For example, `/path/to/vmalert -datasource.url=http://vmselect:8481/select/123/prometheus` For example, `/path/to/vmalert -datasource.url=http://vmselect:8481/select/123/prometheus`
would run alerts against `AccountID=123`. For recording rules the `-remoteWrite.url` command-line would run alerts against `AccountID=123`. For recording rules the `-remoteWrite.url` command-line
flag must contain the url for the specific tenant as well. flag must contain the url for the specific tenant as well.
For example, `-remoteWrite.url=http://vminsert:8480/insert/123/prometheus` would write recording For example, `-remoteWrite.url=http://vminsert:8480/insert/123/prometheus` would write recording
rules to `AccountID=123`. rules to `AccountID=123`.
* To specify `tenant` parameter per each alerting and recording group if * To specify `tenant` parameter per each alerting and recording group if
[enterprise version of vmalert](https://victoriametrics.com/enterprise.html) is used [enterprise version of vmalert](https://victoriametrics.com/enterprise.html) is used
with `-clusterMode` command-line flag. For example: with `-clusterMode` command-line flag. For example:
```yaml ```yaml
@ -224,12 +224,12 @@ groups:
# Rules for accountID=456, projectID=789 # Rules for accountID=456, projectID=789
``` ```
If `-clusterMode` is enabled, then `-datasource.url`, `-remoteRead.url` and `-remoteWrite.url` must If `-clusterMode` is enabled, then `-datasource.url`, `-remoteRead.url` and `-remoteWrite.url` must
contain only the hostname without tenant id. For example: `-datasource.url=http://vmselect:8481`. contain only the hostname without tenant id. For example: `-datasource.url=http://vmselect:8481`.
`vmselect` automatically adds the specified tenant to urls per each recording rule in this case. `vmselect` automatically adds the specified tenant to urls per each recording rule in this case.
The enterprise version of vmalert is available in `vmutils-*-enterprise.tar.gz` files The enterprise version of vmalert is available in `vmutils-*-enterprise.tar.gz` files
at [release page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) and in `*-enterprise` at [release page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) and in `*-enterprise`
tags at [Docker Hub](https://hub.docker.com/r/victoriametrics/vmalert/tags). tags at [Docker Hub](https://hub.docker.com/r/victoriametrics/vmalert/tags).
@ -502,6 +502,8 @@ The shortlist of configuration flags is the following:
Optional TLS server name to use for connections to -remoteWrite.url. By default the server name from -remoteWrite.url is used Optional TLS server name to use for connections to -remoteWrite.url. By default the server name from -remoteWrite.url is used
-remoteWrite.url string -remoteWrite.url string
Optional URL to VictoriaMetrics or vminsert where to persist alerts state and recording rules results in form of timeseries. E.g. http://127.0.0.1:8428 Optional URL to VictoriaMetrics or vminsert where to persist alerts state and recording rules results in form of timeseries. E.g. http://127.0.0.1:8428
-remoteWrite.disablePathAppend
Whether to disable automatic appending of '/api/v1/write' path to the configured -remoteWrite.url.
-replay.maxDatapointsPerQuery int -replay.maxDatapointsPerQuery int
Max number of data points expected in one request. The higher the value, the less requests will be made during replay. (default 1000) Max number of data points expected in one request. The higher the value, the less requests will be made during replay. (default 1000)
-replay.ruleRetryAttempts int -replay.ruleRetryAttempts int

18
app/vmalert/remotewrite/init.go
View file

@ -27,6 +27,7 @@ var (
"By default system CA is used") "By default system CA is used")
tlsServerName = flag.String("remoteWrite.tlsServerName", "", "Optional TLS server name to use for connections to -remoteWrite.url. "+ tlsServerName = flag.String("remoteWrite.tlsServerName", "", "Optional TLS server name to use for connections to -remoteWrite.url. "+
"By default the server name from -remoteWrite.url is used") "By default the server name from -remoteWrite.url is used")
disablePathAppend = flag.Bool("remoteWrite.disablePathAppend", false, "Whether to disable automatic appending of '/api/v1/write' path to the configured -remoteWrite.url.")
) )
// Init creates Client object from given flags. // Init creates Client object from given flags.
@ -42,13 +43,14 @@ func Init(ctx context.Context) (*Client, error) {
} }
return NewClient(ctx, Config{ return NewClient(ctx, Config{
Addr: *addr, Addr: *addr,
Concurrency: *concurrency, Concurrency: *concurrency,
MaxQueueSize: *maxQueueSize, MaxQueueSize: *maxQueueSize,
MaxBatchSize: *maxBatchSize, MaxBatchSize: *maxBatchSize,
FlushInterval: *flushInterval, FlushInterval: *flushInterval,
BasicAuthUser: *basicAuthUsername, BasicAuthUser: *basicAuthUsername,
BasicAuthPass: *basicAuthPassword, BasicAuthPass: *basicAuthPassword,
Transport: t, DisablePathAppend: *disablePathAppend,
Transport: t,
}) })
} }

38
app/vmalert/remotewrite/remotewrite.go
View file

@ -19,13 +19,14 @@ import (
// Client is an asynchronous HTTP client for writing // Client is an asynchronous HTTP client for writing
// timeseries via remote write protocol. // timeseries via remote write protocol.
type Client struct { type Client struct {
addr string addr string
c *http.Client c *http.Client
input chan prompbmarshal.TimeSeries input chan prompbmarshal.TimeSeries
baUser, baPass string baUser, baPass string
flushInterval time.Duration flushInterval time.Duration
maxBatchSize int maxBatchSize int
maxQueueSize int maxQueueSize int
disablePathAppend bool
wg sync.WaitGroup wg sync.WaitGroup
doneCh chan struct{} doneCh chan struct{}
@ -56,6 +57,8 @@ type Config struct {
WriteTimeout time.Duration WriteTimeout time.Duration
// Transport will be used by the underlying http.Client // Transport will be used by the underlying http.Client
Transport *http.Transport Transport *http.Transport
// DisablePathAppend can be used to not automatically append '/api/v1/write' to the remote write url
DisablePathAppend bool
} }
const ( const (
@ -94,14 +97,15 @@ func NewClient(ctx context.Context, cfg Config) (*Client, error) {
Timeout: cfg.WriteTimeout, Timeout: cfg.WriteTimeout,
Transport: cfg.Transport, Transport: cfg.Transport,
}, },
addr: strings.TrimSuffix(cfg.Addr, "/"), addr: strings.TrimSuffix(cfg.Addr, "/"),
baUser: cfg.BasicAuthUser, baUser: cfg.BasicAuthUser,
baPass: cfg.BasicAuthPass, baPass: cfg.BasicAuthPass,
flushInterval: cfg.FlushInterval, flushInterval: cfg.FlushInterval,
maxBatchSize: cfg.MaxBatchSize, maxBatchSize: cfg.MaxBatchSize,
maxQueueSize: cfg.MaxQueueSize, maxQueueSize: cfg.MaxQueueSize,
doneCh: make(chan struct{}), doneCh: make(chan struct{}),
input: make(chan prompbmarshal.TimeSeries, cfg.MaxQueueSize), input: make(chan prompbmarshal.TimeSeries, cfg.MaxQueueSize),
disablePathAppend: cfg.DisablePathAppend,
} }
cc := defaultConcurrency cc := defaultConcurrency
if cfg.Concurrency > 0 { if cfg.Concurrency > 0 {
@ -231,7 +235,9 @@ func (c *Client) send(ctx context.Context, data []byte) error {
if c.baPass != "" { if c.baPass != "" {
req.SetBasicAuth(c.baUser, c.baPass) req.SetBasicAuth(c.baUser, c.baPass)
} }
req.URL.Path += writePath if !c.disablePathAppend {
req.URL.Path += writePath
}
resp, err := c.c.Do(req.WithContext(ctx)) resp, err := c.c.Do(req.WithContext(ctx))
if err != nil { if err != nil {
return fmt.Errorf("error while sending request to %s: %w; Data len %d(%d)", return fmt.Errorf("error while sending request to %s: %w; Data len %d(%d)",

34
docs/vmalert.md
View file

@ -111,11 +111,11 @@ expression and then act according to the Rule type.
There are two types of Rules: There are two types of Rules:
* [alerting](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) - * [alerting](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) -
Alerting rules allows to define alert conditions via `expr` field and to send notifications Alerting rules allows to define alert conditions via `expr` field and to send notifications
[Alertmanager](https://github.com/prometheus/alertmanager) if execution result is not empty. [Alertmanager](https://github.com/prometheus/alertmanager) if execution result is not empty.
* [recording](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) - * [recording](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) -
Recording rules allows to define `expr` which result will be than backfilled to configured Recording rules allows to define `expr` which result will be than backfilled to configured
`-remoteWrite.url`. Recording rules are used to precompute frequently needed or computationally `-remoteWrite.url`. Recording rules are used to precompute frequently needed or computationally
expensive expressions and save their result as a new set of time series. expensive expressions and save their result as a new set of time series.
`vmalert` forbids to define duplicates - rules with the same combination of name, expression and labels `vmalert` forbids to define duplicates - rules with the same combination of name, expression and labels
@ -193,26 +193,26 @@ The state stored to the configured address on every rule evaluation.
from configured address by querying time series with name `ALERTS_FOR_STATE`. from configured address by querying time series with name `ALERTS_FOR_STATE`.
Both flags are required for the proper state restoring. Restore process may fail if time series are missing Both flags are required for the proper state restoring. Restore process may fail if time series are missing
in configured `-remoteRead.url`, weren't updated in the last `1h` (controlled by `-remoteRead.lookback`) in configured `-remoteRead.url`, weren't updated in the last `1h` (controlled by `-remoteRead.lookback`)
or received state doesn't match current `vmalert` rules configuration. or received state doesn't match current `vmalert` rules configuration.
### Multitenancy ### Multitenancy
There are the following approaches for alerting and recording rules across There are the following approaches for alerting and recording rules across
[multiple tenants](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multitenancy): [multiple tenants](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multitenancy):
* To run a separate `vmalert` instance per each tenant. * To run a separate `vmalert` instance per each tenant.
The corresponding tenant must be specified in `-datasource.url` command-line flag The corresponding tenant must be specified in `-datasource.url` command-line flag
according to [these docs](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#url-format). according to [these docs](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#url-format).
For example, `/path/to/vmalert -datasource.url=http://vmselect:8481/select/123/prometheus` For example, `/path/to/vmalert -datasource.url=http://vmselect:8481/select/123/prometheus`
would run alerts against `AccountID=123`. For recording rules the `-remoteWrite.url` command-line would run alerts against `AccountID=123`. For recording rules the `-remoteWrite.url` command-line
flag must contain the url for the specific tenant as well. flag must contain the url for the specific tenant as well.
For example, `-remoteWrite.url=http://vminsert:8480/insert/123/prometheus` would write recording For example, `-remoteWrite.url=http://vminsert:8480/insert/123/prometheus` would write recording
rules to `AccountID=123`. rules to `AccountID=123`.
* To specify `tenant` parameter per each alerting and recording group if * To specify `tenant` parameter per each alerting and recording group if
[enterprise version of vmalert](https://victoriametrics.com/enterprise.html) is used [enterprise version of vmalert](https://victoriametrics.com/enterprise.html) is used
with `-clusterMode` command-line flag. For example: with `-clusterMode` command-line flag. For example:
```yaml ```yaml
@ -228,12 +228,12 @@ groups:
# Rules for accountID=456, projectID=789 # Rules for accountID=456, projectID=789
``` ```
If `-clusterMode` is enabled, then `-datasource.url`, `-remoteRead.url` and `-remoteWrite.url` must If `-clusterMode` is enabled, then `-datasource.url`, `-remoteRead.url` and `-remoteWrite.url` must
contain only the hostname without tenant id. For example: `-datasource.url=http://vmselect:8481`. contain only the hostname without tenant id. For example: `-datasource.url=http://vmselect:8481`.
`vmselect` automatically adds the specified tenant to urls per each recording rule in this case. `vmselect` automatically adds the specified tenant to urls per each recording rule in this case.
The enterprise version of vmalert is available in `vmutils-*-enterprise.tar.gz` files The enterprise version of vmalert is available in `vmutils-*-enterprise.tar.gz` files
at [release page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) and in `*-enterprise` at [release page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) and in `*-enterprise`
tags at [Docker Hub](https://hub.docker.com/r/victoriametrics/vmalert/tags). tags at [Docker Hub](https://hub.docker.com/r/victoriametrics/vmalert/tags).
@ -506,6 +506,8 @@ The shortlist of configuration flags is the following:
Optional TLS server name to use for connections to -remoteWrite.url. By default the server name from -remoteWrite.url is used Optional TLS server name to use for connections to -remoteWrite.url. By default the server name from -remoteWrite.url is used
-remoteWrite.url string -remoteWrite.url string
Optional URL to VictoriaMetrics or vminsert where to persist alerts state and recording rules results in form of timeseries. E.g. http://127.0.0.1:8428 Optional URL to VictoriaMetrics or vminsert where to persist alerts state and recording rules results in form of timeseries. E.g. http://127.0.0.1:8428
-remoteWrite.disablePathAppend
Whether to disable automatic appending of '/api/v1/write' path to the configured -remoteWrite.url.
-replay.maxDatapointsPerQuery int -replay.maxDatapointsPerQuery int
Max number of data points expected in one request. The higher the value, the less requests will be made during replay. (default 1000) Max number of data points expected in one request. The higher the value, the less requests will be made during replay. (default 1000)
-replay.ruleRetryAttempts int -replay.ruleRetryAttempts int