mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2025-01-10 15:14:09 +00:00
vmalert: enable configuring explicit path (#1536)
* vmalert: allow to disable automatically added path to remote write address via disablePathAppend flag * docs: update docs to include remoteWrite.disablePathAppend
This commit is contained in:
Alexander Rickardsson
Aliaksandr Valialkin
parent
a9813fa032
commit
d27dc3721b
4 changed files with 68 additions and 56 deletions
|
|
@ -107,11 +107,11 @@ expression and then act according to the Rule type.
|
|||
|
||||
There are two types of Rules:
|
||||
* [alerting](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) -
|
||||
Alerting rules allows to define alert conditions via `expr` field and to send notifications
|
||||
Alerting rules allows to define alert conditions via `expr` field and to send notifications
|
||||
[Alertmanager](https://github.com/prometheus/alertmanager) if execution result is not empty.
|
||||
* [recording](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) -
|
||||
Recording rules allows to define `expr` which result will be than backfilled to configured
|
||||
`-remoteWrite.url`. Recording rules are used to precompute frequently needed or computationally
|
||||
`-remoteWrite.url`. Recording rules are used to precompute frequently needed or computationally
|
||||
expensive expressions and save their result as a new set of time series.
|
||||
|
||||
`vmalert` forbids to define duplicates - rules with the same combination of name, expression and labels
|
||||
|
|
@ -189,26 +189,26 @@ The state stored to the configured address on every rule evaluation.
|
|||
from configured address by querying time series with name `ALERTS_FOR_STATE`.
|
||||
|
||||
Both flags are required for the proper state restoring. Restore process may fail if time series are missing
|
||||
in configured `-remoteRead.url`, weren't updated in the last `1h` (controlled by `-remoteRead.lookback`)
|
||||
in configured `-remoteRead.url`, weren't updated in the last `1h` (controlled by `-remoteRead.lookback`)
|
||||
or received state doesn't match current `vmalert` rules configuration.
|
||||
|
||||
|
||||
### Multitenancy
|
||||
|
||||
There are the following approaches for alerting and recording rules across
|
||||
There are the following approaches for alerting and recording rules across
|
||||
[multiple tenants](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multitenancy):
|
||||
|
||||
* To run a separate `vmalert` instance per each tenant.
|
||||
The corresponding tenant must be specified in `-datasource.url` command-line flag
|
||||
according to [these docs](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#url-format).
|
||||
* To run a separate `vmalert` instance per each tenant.
|
||||
The corresponding tenant must be specified in `-datasource.url` command-line flag
|
||||
according to [these docs](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#url-format).
|
||||
For example, `/path/to/vmalert -datasource.url=http://vmselect:8481/select/123/prometheus`
|
||||
would run alerts against `AccountID=123`. For recording rules the `-remoteWrite.url` command-line
|
||||
flag must contain the url for the specific tenant as well.
|
||||
For example, `-remoteWrite.url=http://vminsert:8480/insert/123/prometheus` would write recording
|
||||
would run alerts against `AccountID=123`. For recording rules the `-remoteWrite.url` command-line
|
||||
flag must contain the url for the specific tenant as well.
|
||||
For example, `-remoteWrite.url=http://vminsert:8480/insert/123/prometheus` would write recording
|
||||
rules to `AccountID=123`.
|
||||
|
||||
* To specify `tenant` parameter per each alerting and recording group if
|
||||
[enterprise version of vmalert](https://victoriametrics.com/enterprise.html) is used
|
||||
* To specify `tenant` parameter per each alerting and recording group if
|
||||
[enterprise version of vmalert](https://victoriametrics.com/enterprise.html) is used
|
||||
with `-clusterMode` command-line flag. For example:
|
||||
|
||||
```yaml
|
||||
|
|
@ -224,12 +224,12 @@ groups:
|
|||
# Rules for accountID=456, projectID=789
|
||||
```
|
||||
|
||||
If `-clusterMode` is enabled, then `-datasource.url`, `-remoteRead.url` and `-remoteWrite.url` must
|
||||
contain only the hostname without tenant id. For example: `-datasource.url=http://vmselect:8481`.
|
||||
If `-clusterMode` is enabled, then `-datasource.url`, `-remoteRead.url` and `-remoteWrite.url` must
|
||||
contain only the hostname without tenant id. For example: `-datasource.url=http://vmselect:8481`.
|
||||
`vmselect` automatically adds the specified tenant to urls per each recording rule in this case.
|
||||
|
||||
The enterprise version of vmalert is available in `vmutils-*-enterprise.tar.gz` files
|
||||
at [release page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) and in `*-enterprise`
|
||||
The enterprise version of vmalert is available in `vmutils-*-enterprise.tar.gz` files
|
||||
at [release page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) and in `*-enterprise`
|
||||
tags at [Docker Hub](https://hub.docker.com/r/victoriametrics/vmalert/tags).
|
||||
|
||||
|
||||
|
|
@ -502,6 +502,8 @@ The shortlist of configuration flags is the following:
|
|||
Optional TLS server name to use for connections to -remoteWrite.url. By default the server name from -remoteWrite.url is used
|
||||
-remoteWrite.url string
|
||||
Optional URL to VictoriaMetrics or vminsert where to persist alerts state and recording rules results in form of timeseries. E.g. http://127.0.0.1:8428
|
||||
-remoteWrite.disablePathAppend
|
||||
Whether to disable automatic appending of '/api/v1/write' path to the configured -remoteWrite.url.
|
||||
-replay.maxDatapointsPerQuery int
|
||||
Max number of data points expected in one request. The higher the value, the less requests will be made during replay. (default 1000)
|
||||
-replay.ruleRetryAttempts int
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@ var (
|
|||
"By default system CA is used")
|
||||
tlsServerName = flag.String("remoteWrite.tlsServerName", "", "Optional TLS server name to use for connections to -remoteWrite.url. "+
|
||||
"By default the server name from -remoteWrite.url is used")
|
||||
disablePathAppend = flag.Bool("remoteWrite.disablePathAppend", false, "Whether to disable automatic appending of '/api/v1/write' path to the configured -remoteWrite.url.")
|
||||
)
|
||||
|
||||
// Init creates Client object from given flags.
|
||||
|
|
@ -42,13 +43,14 @@ func Init(ctx context.Context) (*Client, error) {
|
|||
}
|
||||
|
||||
return NewClient(ctx, Config{
|
||||
Addr: *addr,
|
||||
Concurrency: *concurrency,
|
||||
MaxQueueSize: *maxQueueSize,
|
||||
MaxBatchSize: *maxBatchSize,
|
||||
FlushInterval: *flushInterval,
|
||||
BasicAuthUser: *basicAuthUsername,
|
||||
BasicAuthPass: *basicAuthPassword,
|
||||
Transport: t,
|
||||
Addr: *addr,
|
||||
Concurrency: *concurrency,
|
||||
MaxQueueSize: *maxQueueSize,
|
||||
MaxBatchSize: *maxBatchSize,
|
||||
FlushInterval: *flushInterval,
|
||||
BasicAuthUser: *basicAuthUsername,
|
||||
BasicAuthPass: *basicAuthPassword,
|
||||
DisablePathAppend: *disablePathAppend,
|
||||
Transport: t,
|
||||
})
|
||||
}
|
||||
|
|
|
|||
|
|
@ -19,13 +19,14 @@ import (
|
|||
// Client is an asynchronous HTTP client for writing
|
||||
// timeseries via remote write protocol.
|
||||
type Client struct {
|
||||
addr string
|
||||
c *http.Client
|
||||
input chan prompbmarshal.TimeSeries
|
||||
baUser, baPass string
|
||||
flushInterval time.Duration
|
||||
maxBatchSize int
|
||||
maxQueueSize int
|
||||
addr string
|
||||
c *http.Client
|
||||
input chan prompbmarshal.TimeSeries
|
||||
baUser, baPass string
|
||||
flushInterval time.Duration
|
||||
maxBatchSize int
|
||||
maxQueueSize int
|
||||
disablePathAppend bool
|
||||
|
||||
wg sync.WaitGroup
|
||||
doneCh chan struct{}
|
||||
|
|
@ -56,6 +57,8 @@ type Config struct {
|
|||
WriteTimeout time.Duration
|
||||
// Transport will be used by the underlying http.Client
|
||||
Transport *http.Transport
|
||||
// DisablePathAppend can be used to not automatically append '/api/v1/write' to the remote write url
|
||||
DisablePathAppend bool
|
||||
}
|
||||
|
||||
const (
|
||||
|
|
@ -94,14 +97,15 @@ func NewClient(ctx context.Context, cfg Config) (*Client, error) {
|
|||
Timeout: cfg.WriteTimeout,
|
||||
Transport: cfg.Transport,
|
||||
},
|
||||
addr: strings.TrimSuffix(cfg.Addr, "/"),
|
||||
baUser: cfg.BasicAuthUser,
|
||||
baPass: cfg.BasicAuthPass,
|
||||
flushInterval: cfg.FlushInterval,
|
||||
maxBatchSize: cfg.MaxBatchSize,
|
||||
maxQueueSize: cfg.MaxQueueSize,
|
||||
doneCh: make(chan struct{}),
|
||||
input: make(chan prompbmarshal.TimeSeries, cfg.MaxQueueSize),
|
||||
addr: strings.TrimSuffix(cfg.Addr, "/"),
|
||||
baUser: cfg.BasicAuthUser,
|
||||
baPass: cfg.BasicAuthPass,
|
||||
flushInterval: cfg.FlushInterval,
|
||||
maxBatchSize: cfg.MaxBatchSize,
|
||||
maxQueueSize: cfg.MaxQueueSize,
|
||||
doneCh: make(chan struct{}),
|
||||
input: make(chan prompbmarshal.TimeSeries, cfg.MaxQueueSize),
|
||||
disablePathAppend: cfg.DisablePathAppend,
|
||||
}
|
||||
cc := defaultConcurrency
|
||||
if cfg.Concurrency > 0 {
|
||||
|
|
@ -231,7 +235,9 @@ func (c *Client) send(ctx context.Context, data []byte) error {
|
|||
if c.baPass != "" {
|
||||
req.SetBasicAuth(c.baUser, c.baPass)
|
||||
}
|
||||
req.URL.Path += writePath
|
||||
if !c.disablePathAppend {
|
||||
req.URL.Path += writePath
|
||||
}
|
||||
resp, err := c.c.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return fmt.Errorf("error while sending request to %s: %w; Data len %d(%d)",
|
||||
|
|
|
|||
|
|
@ -111,11 +111,11 @@ expression and then act according to the Rule type.
|
|||
|
||||
There are two types of Rules:
|
||||
* [alerting](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) -
|
||||
Alerting rules allows to define alert conditions via `expr` field and to send notifications
|
||||
Alerting rules allows to define alert conditions via `expr` field and to send notifications
|
||||
[Alertmanager](https://github.com/prometheus/alertmanager) if execution result is not empty.
|
||||
* [recording](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) -
|
||||
Recording rules allows to define `expr` which result will be than backfilled to configured
|
||||
`-remoteWrite.url`. Recording rules are used to precompute frequently needed or computationally
|
||||
`-remoteWrite.url`. Recording rules are used to precompute frequently needed or computationally
|
||||
expensive expressions and save their result as a new set of time series.
|
||||
|
||||
`vmalert` forbids to define duplicates - rules with the same combination of name, expression and labels
|
||||
|
|
@ -193,26 +193,26 @@ The state stored to the configured address on every rule evaluation.
|
|||
from configured address by querying time series with name `ALERTS_FOR_STATE`.
|
||||
|
||||
Both flags are required for the proper state restoring. Restore process may fail if time series are missing
|
||||
in configured `-remoteRead.url`, weren't updated in the last `1h` (controlled by `-remoteRead.lookback`)
|
||||
in configured `-remoteRead.url`, weren't updated in the last `1h` (controlled by `-remoteRead.lookback`)
|
||||
or received state doesn't match current `vmalert` rules configuration.
|
||||
|
||||
|
||||
### Multitenancy
|
||||
|
||||
There are the following approaches for alerting and recording rules across
|
||||
There are the following approaches for alerting and recording rules across
|
||||
[multiple tenants](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multitenancy):
|
||||
|
||||
* To run a separate `vmalert` instance per each tenant.
|
||||
The corresponding tenant must be specified in `-datasource.url` command-line flag
|
||||
according to [these docs](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#url-format).
|
||||
* To run a separate `vmalert` instance per each tenant.
|
||||
The corresponding tenant must be specified in `-datasource.url` command-line flag
|
||||
according to [these docs](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#url-format).
|
||||
For example, `/path/to/vmalert -datasource.url=http://vmselect:8481/select/123/prometheus`
|
||||
would run alerts against `AccountID=123`. For recording rules the `-remoteWrite.url` command-line
|
||||
flag must contain the url for the specific tenant as well.
|
||||
For example, `-remoteWrite.url=http://vminsert:8480/insert/123/prometheus` would write recording
|
||||
would run alerts against `AccountID=123`. For recording rules the `-remoteWrite.url` command-line
|
||||
flag must contain the url for the specific tenant as well.
|
||||
For example, `-remoteWrite.url=http://vminsert:8480/insert/123/prometheus` would write recording
|
||||
rules to `AccountID=123`.
|
||||
|
||||
* To specify `tenant` parameter per each alerting and recording group if
|
||||
[enterprise version of vmalert](https://victoriametrics.com/enterprise.html) is used
|
||||
* To specify `tenant` parameter per each alerting and recording group if
|
||||
[enterprise version of vmalert](https://victoriametrics.com/enterprise.html) is used
|
||||
with `-clusterMode` command-line flag. For example:
|
||||
|
||||
```yaml
|
||||
|
|
@ -228,12 +228,12 @@ groups:
|
|||
# Rules for accountID=456, projectID=789
|
||||
```
|
||||
|
||||
If `-clusterMode` is enabled, then `-datasource.url`, `-remoteRead.url` and `-remoteWrite.url` must
|
||||
contain only the hostname without tenant id. For example: `-datasource.url=http://vmselect:8481`.
|
||||
If `-clusterMode` is enabled, then `-datasource.url`, `-remoteRead.url` and `-remoteWrite.url` must
|
||||
contain only the hostname without tenant id. For example: `-datasource.url=http://vmselect:8481`.
|
||||
`vmselect` automatically adds the specified tenant to urls per each recording rule in this case.
|
||||
|
||||
The enterprise version of vmalert is available in `vmutils-*-enterprise.tar.gz` files
|
||||
at [release page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) and in `*-enterprise`
|
||||
The enterprise version of vmalert is available in `vmutils-*-enterprise.tar.gz` files
|
||||
at [release page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) and in `*-enterprise`
|
||||
tags at [Docker Hub](https://hub.docker.com/r/victoriametrics/vmalert/tags).
|
||||
|
||||
|
||||
|
|
@ -506,6 +506,8 @@ The shortlist of configuration flags is the following:
|
|||
Optional TLS server name to use for connections to -remoteWrite.url. By default the server name from -remoteWrite.url is used
|
||||
-remoteWrite.url string
|
||||
Optional URL to VictoriaMetrics or vminsert where to persist alerts state and recording rules results in form of timeseries. E.g. http://127.0.0.1:8428
|
||||
-remoteWrite.disablePathAppend
|
||||
Whether to disable automatic appending of '/api/v1/write' path to the configured -remoteWrite.url.
|
||||
-replay.maxDatapointsPerQuery int
|
||||
Max number of data points expected in one request. The higher the value, the less requests will be made during replay. (default 1000)
|
||||
-replay.ruleRetryAttempts int
|
||||
|
|
|
|||
Loading…
Reference in a new issue