mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2025-03-11 15:34:56 +00:00
Merge branch 'master' into streaming-aggregation-ui
This commit is contained in:
Alexander Marshalov
GitHub
commit
cecee0e05f
37 changed files with 645 additions and 225 deletions
23
README.md
23
README.md
|
|
@ -1722,9 +1722,10 @@ See also [resource usage limits at VictoriaMetrics cluster](https://docs.victori
|
|||
|
||||
The general approach for achieving high availability is the following:
|
||||
|
||||
- to run two identically configured VictoriaMetrics instances in distinct datacenters (availability zones)
|
||||
- to store the collected data simultaneously into these instances via [vmagent](https://docs.victoriametrics.com/vmagent.html) or Prometheus
|
||||
- to query the first VictoriaMetrics instance and to fail over to the second instance when the first instance becomes temporarily unavailable.
|
||||
- To run two identically configured VictoriaMetrics instances in distinct datacenters (availability zones);
|
||||
- To store the collected data simultaneously into these instances via [vmagent](https://docs.victoriametrics.com/vmagent.html) or Prometheus.
|
||||
- To query the first VictoriaMetrics instance and to fail over to the second instance when the first instance becomes temporarily unavailable.
|
||||
This can be done via [vmauth](https://docs.victoriametrics.com/vmauth.html) according to [these docs](https://docs.victoriametrics.com/vmauth.html#high-availability).
|
||||
|
||||
Such a setup guarantees that the collected data isn't lost when one of VictoriaMetrics instance becomes unavailable.
|
||||
The collected data continues to be written to the available VictoriaMetrics instance, so it should be available for querying.
|
||||
|
|
@ -2287,11 +2288,14 @@ The following command-line flags are related to pushing metrics from VictoriaMet
|
|||
The `-pushmetrics.url` can be specified multiple times. In this case metrics are pushed to all the specified urls.
|
||||
The url can contain basic auth params in the form `http://user:pass@hostname/api/v1/import/prometheus`.
|
||||
Metrics are pushed to the provided `-pushmetrics.url` in a compressed form with `Content-Encoding: gzip` request header.
|
||||
This allows reducing the required network bandwidth for metrics push.
|
||||
* `-pushmetrics.extraLabel` - labels to add to all the metrics before sending them to `-pushmetrics.url`. Each label must be specified in the format `label="value"`.
|
||||
This allows reducing the required network bandwidth for metrics push. The compression can be disabled by passing `-pushmetrics.disableCompression` command-line flag.
|
||||
* `-pushmetrics.extraLabel` - labels to add to all the metrics before sending them to every `-pushmetrics.url`. Each label must be specified in the format `label="value"`.
|
||||
It is OK to specify multiple `-pushmetrics.extraLabel` command-line flags. In this case all the specified labels
|
||||
are added to all the metrics before sending them to all the configured `-pushmetrics.url` addresses.
|
||||
* `-pushmetrics.interval` - the interval between pushes. By default it is set to 10 seconds.
|
||||
* `-pushmetrics.header` - an optional HTTP header to send to every `-pushmetrics.url`. For example, `-pushmetrics.header='Authorization: Basic foo'` instructs to send
|
||||
`Authorization: Basic foo` HTTP header with every request to every `-pushmetrics.url`. It is possible to set multiple `-pushmetrics.header` command-line flags
|
||||
for sending multiple different HTTP headers to `-pushmetrics.url`.
|
||||
|
||||
For example, the following command instructs VictoriaMetrics to push metrics from `/metrics` page to `https://maas.victoriametrics.com/api/v1/import/prometheus`
|
||||
with `user:pass` [Basic auth](https://en.wikipedia.org/wiki/Basic_access_authentication). The `instance="foobar"` and `job="vm"` labels
|
||||
|
|
@ -2837,11 +2841,16 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
|
|||
The delay for suppressing repeated scrape errors logging per each scrape targets. This may be used for reducing the number of log lines related to scrape errors. See also -promscrape.suppressScrapeErrors
|
||||
-promscrape.yandexcloudSDCheckInterval duration
|
||||
Interval for checking for changes in Yandex Cloud API. This works only if yandexcloud_sd_configs is configured in '-promscrape.config' file. See https://docs.victoriametrics.com/sd_configs.html#yandexcloud_sd_configs for details (default 30s)
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
|
|||
|
|
@ -118,9 +118,9 @@ func main() {
|
|||
return
|
||||
}
|
||||
|
||||
eu, err := getExternalURL(*externalURL, *httpListenAddr, httpserver.IsTLS())
|
||||
eu, err := getExternalURL(*externalURL)
|
||||
if err != nil {
|
||||
logger.Fatalf("failed to init `external.url`: %s", err)
|
||||
logger.Fatalf("failed to init `-external.url`: %s", err)
|
||||
}
|
||||
|
||||
alertURLGeneratorFn, err = getAlertURLGenerator(eu, *externalAlertSource, *validateTemplates)
|
||||
|
|
@ -243,14 +243,26 @@ func newManager(ctx context.Context) (*manager, error) {
|
|||
return manager, nil
|
||||
}
|
||||
|
||||
func getExternalURL(externalURL, httpListenAddr string, isSecure bool) (*url.URL, error) {
|
||||
if externalURL != "" {
|
||||
return url.Parse(externalURL)
|
||||
func getExternalURL(customURL string) (*url.URL, error) {
|
||||
if customURL == "" {
|
||||
// use local hostname as external URL
|
||||
return getHostnameAsExternalURL(*httpListenAddr, httpserver.IsTLS())
|
||||
}
|
||||
hname, err := os.Hostname()
|
||||
u, err := url.Parse(customURL)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if u.Scheme != "http" && u.Scheme != "https" {
|
||||
return nil, fmt.Errorf("invalid scheme %q in url %q, only 'http' and 'https' are supported", u.Scheme, u.String())
|
||||
}
|
||||
return u, nil
|
||||
}
|
||||
|
||||
func getHostnameAsExternalURL(httpListenAddr string, isSecure bool) (*url.URL, error) {
|
||||
hname, err := os.Hostname()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get hostname: %w", err)
|
||||
}
|
||||
port := ""
|
||||
if ipport := strings.Split(httpListenAddr, ":"); len(ipport) > 1 {
|
||||
port = ":" + ipport[1]
|
||||
|
|
|
|||
|
|
@ -22,22 +22,29 @@ func init() {
|
|||
}
|
||||
|
||||
func TestGetExternalURL(t *testing.T) {
|
||||
expURL := "https://vicotriametrics.com/path"
|
||||
u, err := getExternalURL(expURL, "", false)
|
||||
invalidURL := "victoriametrics.com/path"
|
||||
_, err := getExternalURL(invalidURL)
|
||||
if err == nil {
|
||||
t.Errorf("expected error, got nil")
|
||||
}
|
||||
|
||||
expURL := "https://victoriametrics.com/path"
|
||||
u, err := getExternalURL(expURL)
|
||||
if err != nil {
|
||||
t.Errorf("unexpected error %s", err)
|
||||
}
|
||||
if u.String() != expURL {
|
||||
t.Errorf("unexpected url want %s, got %s", expURL, u.String())
|
||||
t.Errorf("unexpected url: want %q, got %s", expURL, u.String())
|
||||
}
|
||||
|
||||
h, _ := os.Hostname()
|
||||
expURL = fmt.Sprintf("https://%s:4242", h)
|
||||
u, err = getExternalURL("", "0.0.0.0:4242", true)
|
||||
expURL = fmt.Sprintf("http://%s:8880", h)
|
||||
u, err = getExternalURL("")
|
||||
if err != nil {
|
||||
t.Errorf("unexpected error %s", err)
|
||||
}
|
||||
if u.String() != expURL {
|
||||
t.Errorf("unexpected url want %s, got %s", expURL, u.String())
|
||||
t.Errorf("unexpected url: want %s, got %s", expURL, u.String())
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -56,7 +56,7 @@ type UserInfo struct {
|
|||
DefaultURL *URLPrefix `yaml:"default_url,omitempty"`
|
||||
RetryStatusCodes []int `yaml:"retry_status_codes,omitempty"`
|
||||
LoadBalancingPolicy string `yaml:"load_balancing_policy,omitempty"`
|
||||
DropSrcPathPrefixParts int `yaml:"drop_src_path_prefix_parts,omitempty"`
|
||||
DropSrcPathPrefixParts *int `yaml:"drop_src_path_prefix_parts,omitempty"`
|
||||
TLSInsecureSkipVerify *bool `yaml:"tls_insecure_skip_verify,omitempty"`
|
||||
TLSCAFile string `yaml:"tls_ca_file,omitempty"`
|
||||
|
||||
|
|
@ -126,16 +126,30 @@ func (h *Header) MarshalYAML() (interface{}, error) {
|
|||
|
||||
// URLMap is a mapping from source paths to target urls.
|
||||
type URLMap struct {
|
||||
SrcPaths []*SrcPath `yaml:"src_paths,omitempty"`
|
||||
URLPrefix *URLPrefix `yaml:"url_prefix,omitempty"`
|
||||
HeadersConf HeadersConf `yaml:",inline"`
|
||||
RetryStatusCodes []int `yaml:"retry_status_codes,omitempty"`
|
||||
LoadBalancingPolicy string `yaml:"load_balancing_policy,omitempty"`
|
||||
DropSrcPathPrefixParts int `yaml:"drop_src_path_prefix_parts,omitempty"`
|
||||
// SrcHosts is the list of regular expressions, which match the request hostname.
|
||||
SrcHosts []*Regex `yaml:"src_hosts,omitempty"`
|
||||
|
||||
// SrcPaths is the list of regular expressions, which match the request path.
|
||||
SrcPaths []*Regex `yaml:"src_paths,omitempty"`
|
||||
|
||||
// UrlPrefix contains backend url prefixes for the proxied request url.
|
||||
URLPrefix *URLPrefix `yaml:"url_prefix,omitempty"`
|
||||
|
||||
// HeadersConf is the config for augumenting request and response headers.
|
||||
HeadersConf HeadersConf `yaml:",inline"`
|
||||
|
||||
// RetryStatusCodes is the list of response status codes used for retrying requests.
|
||||
RetryStatusCodes []int `yaml:"retry_status_codes,omitempty"`
|
||||
|
||||
// LoadBalancingPolicy is load balancing policy among UrlPrefix backends.
|
||||
LoadBalancingPolicy string `yaml:"load_balancing_policy,omitempty"`
|
||||
|
||||
// DropSrcPathPrefixParts is the number of `/`-delimited request path prefix parts to drop before proxying the request to backend.
|
||||
DropSrcPathPrefixParts *int `yaml:"drop_src_path_prefix_parts,omitempty"`
|
||||
}
|
||||
|
||||
// SrcPath represents an src path
|
||||
type SrcPath struct {
|
||||
// Regex represents a regex
|
||||
type Regex struct {
|
||||
sOriginal string
|
||||
re *regexp.Regexp
|
||||
}
|
||||
|
|
@ -152,6 +166,9 @@ type URLPrefix struct {
|
|||
|
||||
// load balancing policy used
|
||||
loadBalancingPolicy string
|
||||
|
||||
// how many request path prefix parts to drop before routing the request to backendURL.
|
||||
dropSrcPathPrefixParts int
|
||||
}
|
||||
|
||||
func (up *URLPrefix) setLoadBalancingPolicy(loadBalancingPolicy string) error {
|
||||
|
|
@ -333,8 +350,8 @@ func (up *URLPrefix) MarshalYAML() (interface{}, error) {
|
|||
return string(b), nil
|
||||
}
|
||||
|
||||
func (sp *SrcPath) match(s string) bool {
|
||||
prefix, ok := sp.re.LiteralPrefix()
|
||||
func (r *Regex) match(s string) bool {
|
||||
prefix, ok := r.re.LiteralPrefix()
|
||||
if ok {
|
||||
// Fast path - literal match
|
||||
return s == prefix
|
||||
|
|
@ -342,11 +359,11 @@ func (sp *SrcPath) match(s string) bool {
|
|||
if !strings.HasPrefix(s, prefix) {
|
||||
return false
|
||||
}
|
||||
return sp.re.MatchString(s)
|
||||
return r.re.MatchString(s)
|
||||
}
|
||||
|
||||
// UnmarshalYAML implements yaml.Unmarshaler
|
||||
func (sp *SrcPath) UnmarshalYAML(f func(interface{}) error) error {
|
||||
func (r *Regex) UnmarshalYAML(f func(interface{}) error) error {
|
||||
var s string
|
||||
if err := f(&s); err != nil {
|
||||
return err
|
||||
|
|
@ -356,14 +373,14 @@ func (sp *SrcPath) UnmarshalYAML(f func(interface{}) error) error {
|
|||
if err != nil {
|
||||
return fmt.Errorf("cannot build regexp from %q: %w", s, err)
|
||||
}
|
||||
sp.sOriginal = s
|
||||
sp.re = re
|
||||
r.sOriginal = s
|
||||
r.re = re
|
||||
return nil
|
||||
}
|
||||
|
||||
// MarshalYAML implements yaml.Marshaler.
|
||||
func (sp *SrcPath) MarshalYAML() (interface{}, error) {
|
||||
return sp.sOriginal, nil
|
||||
func (r *Regex) MarshalYAML() (interface{}, error) {
|
||||
return r.sOriginal, nil
|
||||
}
|
||||
|
||||
var (
|
||||
|
|
@ -592,17 +609,22 @@ func parseAuthConfigUsers(ac *AuthConfig) (map[string]*UserInfo, error) {
|
|||
func (ui *UserInfo) initURLs() error {
|
||||
retryStatusCodes := defaultRetryStatusCodes.Values()
|
||||
loadBalancingPolicy := *defaultLoadBalancingPolicy
|
||||
dropSrcPathPrefixParts := 0
|
||||
if ui.URLPrefix != nil {
|
||||
if err := ui.URLPrefix.sanitize(); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(ui.RetryStatusCodes) > 0 {
|
||||
if ui.RetryStatusCodes != nil {
|
||||
retryStatusCodes = ui.RetryStatusCodes
|
||||
}
|
||||
if ui.LoadBalancingPolicy != "" {
|
||||
loadBalancingPolicy = ui.LoadBalancingPolicy
|
||||
}
|
||||
if ui.DropSrcPathPrefixParts != nil {
|
||||
dropSrcPathPrefixParts = *ui.DropSrcPathPrefixParts
|
||||
}
|
||||
ui.URLPrefix.retryStatusCodes = retryStatusCodes
|
||||
ui.URLPrefix.dropSrcPathPrefixParts = dropSrcPathPrefixParts
|
||||
if err := ui.URLPrefix.setLoadBalancingPolicy(loadBalancingPolicy); err != nil {
|
||||
return err
|
||||
}
|
||||
|
|
@ -613,8 +635,8 @@ func (ui *UserInfo) initURLs() error {
|
|||
}
|
||||
}
|
||||
for _, e := range ui.URLMaps {
|
||||
if len(e.SrcPaths) == 0 {
|
||||
return fmt.Errorf("missing `src_paths` in `url_map`")
|
||||
if len(e.SrcPaths) == 0 && len(e.SrcHosts) == 0 {
|
||||
return fmt.Errorf("missing `src_paths` and `src_hosts` in `url_map`")
|
||||
}
|
||||
if e.URLPrefix == nil {
|
||||
return fmt.Errorf("missing `url_prefix` in `url_map`")
|
||||
|
|
@ -624,16 +646,21 @@ func (ui *UserInfo) initURLs() error {
|
|||
}
|
||||
rscs := retryStatusCodes
|
||||
lbp := loadBalancingPolicy
|
||||
if len(e.RetryStatusCodes) > 0 {
|
||||
dsp := dropSrcPathPrefixParts
|
||||
if e.RetryStatusCodes != nil {
|
||||
rscs = e.RetryStatusCodes
|
||||
}
|
||||
if e.LoadBalancingPolicy != "" {
|
||||
lbp = e.LoadBalancingPolicy
|
||||
}
|
||||
if e.DropSrcPathPrefixParts != nil {
|
||||
dsp = *e.DropSrcPathPrefixParts
|
||||
}
|
||||
e.URLPrefix.retryStatusCodes = rscs
|
||||
if err := e.URLPrefix.setLoadBalancingPolicy(lbp); err != nil {
|
||||
return err
|
||||
}
|
||||
e.URLPrefix.dropSrcPathPrefixParts = dsp
|
||||
}
|
||||
if len(ui.URLMaps) == 0 && ui.URLPrefix == nil {
|
||||
return fmt.Errorf("missing `url_prefix`")
|
||||
|
|
|
|||
|
|
@ -145,6 +145,12 @@ users:
|
|||
url_map:
|
||||
- src_paths: ["/foo/bar"]
|
||||
`)
|
||||
f(`
|
||||
users:
|
||||
- username: a
|
||||
url_map:
|
||||
- src_hosts: ["foobar"]
|
||||
`)
|
||||
|
||||
// Invalid url_prefix in url_map
|
||||
f(`
|
||||
|
|
@ -154,6 +160,13 @@ users:
|
|||
- src_paths: ["/foo/bar"]
|
||||
url_prefix: foo.bar
|
||||
`)
|
||||
f(`
|
||||
users:
|
||||
- username: a
|
||||
url_map:
|
||||
- src_hosts: ["foobar"]
|
||||
url_prefix: foo.bar
|
||||
`)
|
||||
|
||||
// empty url_prefix in url_map
|
||||
f(`
|
||||
|
|
@ -163,8 +176,15 @@ users:
|
|||
- src_paths: ['/foo/bar']
|
||||
url_prefix: []
|
||||
`)
|
||||
f(`
|
||||
users:
|
||||
- username: a
|
||||
url_map:
|
||||
- src_phosts: ['foobar']
|
||||
url_prefix: []
|
||||
`)
|
||||
|
||||
// Missing src_paths in url_map
|
||||
// Missing src_paths and src_hosts in url_map
|
||||
f(`
|
||||
users:
|
||||
- username: a
|
||||
|
|
@ -181,6 +201,15 @@ users:
|
|||
url_prefix: http://foobar
|
||||
`)
|
||||
|
||||
// Invalid regexp in src_hosts
|
||||
f(`
|
||||
users:
|
||||
- username: a
|
||||
url_map:
|
||||
- src_hosts: ['fo[obar']
|
||||
url_prefix: http://foobar
|
||||
`)
|
||||
|
||||
// Invalid headers in url_map (missing ':')
|
||||
f(`
|
||||
users:
|
||||
|
|
@ -263,7 +292,7 @@ users:
|
|||
TLSInsecureSkipVerify: &insecureSkipVerifyFalse,
|
||||
RetryStatusCodes: []int{500, 501},
|
||||
LoadBalancingPolicy: "first_available",
|
||||
DropSrcPathPrefixParts: 1,
|
||||
DropSrcPathPrefixParts: intp(1),
|
||||
},
|
||||
})
|
||||
|
||||
|
|
@ -293,6 +322,7 @@ users:
|
|||
- src_paths: ["/api/v1/query","/api/v1/query_range","/api/v1/label/[^./]+/.+"]
|
||||
url_prefix: http://vmselect/select/0/prometheus
|
||||
- src_paths: ["/api/v1/write"]
|
||||
src_hosts: ["foo\\.bar", "baz:1234"]
|
||||
url_prefix: ["http://vminsert1/insert/0/prometheus","http://vminsert2/insert/0/prometheus"]
|
||||
headers:
|
||||
- "foo: bar"
|
||||
|
|
@ -302,11 +332,12 @@ users:
|
|||
BearerToken: "foo",
|
||||
URLMaps: []URLMap{
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/query", "/api/v1/query_range", "/api/v1/label/[^./]+/.+"}),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/query", "/api/v1/query_range", "/api/v1/label/[^./]+/.+"}),
|
||||
URLPrefix: mustParseURL("http://vmselect/select/0/prometheus"),
|
||||
},
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/write"}),
|
||||
SrcHosts: getRegexs([]string{"foo\\.bar", "baz:1234"}),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/write"}),
|
||||
URLPrefix: mustParseURLs([]string{
|
||||
"http://vminsert1/insert/0/prometheus",
|
||||
"http://vminsert2/insert/0/prometheus",
|
||||
|
|
@ -330,11 +361,12 @@ users:
|
|||
BearerToken: "foo",
|
||||
URLMaps: []URLMap{
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/query", "/api/v1/query_range", "/api/v1/label/[^./]+/.+"}),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/query", "/api/v1/query_range", "/api/v1/label/[^./]+/.+"}),
|
||||
URLPrefix: mustParseURL("http://vmselect/select/0/prometheus"),
|
||||
},
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/write"}),
|
||||
SrcHosts: getRegexs([]string{"foo\\.bar", "baz:1234"}),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/write"}),
|
||||
URLPrefix: mustParseURLs([]string{
|
||||
"http://vminsert1/insert/0/prometheus",
|
||||
"http://vminsert2/insert/0/prometheus",
|
||||
|
|
@ -396,11 +428,11 @@ users:
|
|||
BearerToken: "foo",
|
||||
URLMaps: []URLMap{
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/query", "/api/v1/query_range", "/api/v1/label/[^./]+/.+"}),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/query", "/api/v1/query_range", "/api/v1/label/[^./]+/.+"}),
|
||||
URLPrefix: mustParseURL("http://vmselect/select/0/prometheus"),
|
||||
},
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/write"}),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/write"}),
|
||||
URLPrefix: mustParseURLs([]string{
|
||||
"http://vminsert1/insert/0/prometheus",
|
||||
"http://vminsert2/insert/0/prometheus",
|
||||
|
|
@ -428,11 +460,11 @@ users:
|
|||
BearerToken: "foo",
|
||||
URLMaps: []URLMap{
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/query", "/api/v1/query_range", "/api/v1/label/[^./]+/.+"}),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/query", "/api/v1/query_range", "/api/v1/label/[^./]+/.+"}),
|
||||
URLPrefix: mustParseURL("http://vmselect/select/0/prometheus"),
|
||||
},
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/write"}),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/write"}),
|
||||
URLPrefix: mustParseURLs([]string{
|
||||
"http://vminsert1/insert/0/prometheus",
|
||||
"http://vminsert2/insert/0/prometheus",
|
||||
|
|
@ -501,10 +533,10 @@ func isSetBool(boolP *bool, expectedValue bool) bool {
|
|||
return *boolP == expectedValue
|
||||
}
|
||||
|
||||
func getSrcPaths(paths []string) []*SrcPath {
|
||||
var sps []*SrcPath
|
||||
func getRegexs(paths []string) []*Regex {
|
||||
var sps []*Regex
|
||||
for _, path := range paths {
|
||||
sps = append(sps, &SrcPath{
|
||||
sps = append(sps, &Regex{
|
||||
sOriginal: path,
|
||||
re: regexp.MustCompile("^(?:" + path + ")$"),
|
||||
})
|
||||
|
|
@ -552,3 +584,7 @@ func mustParseURLs(us []string) *URLPrefix {
|
|||
bus: bus,
|
||||
}
|
||||
}
|
||||
|
||||
func intp(n int) *int {
|
||||
return &n
|
||||
}
|
||||
|
|
|
|||
|
|
@ -92,7 +92,7 @@ users:
|
|||
# - to http://default1:8888/unsupported_url_handler?request_path=/non/existing/path
|
||||
# - or http://default2:8888/unsupported_url_handler?request_path=/non/existing/path
|
||||
#
|
||||
# Regular expressions are allowed in `src_paths` entries.
|
||||
# Regular expressions are allowed in `src_paths` and `src_hosts` entries.
|
||||
- username: "foobar"
|
||||
url_map:
|
||||
- src_paths:
|
||||
|
|
|
|||
|
|
@ -164,7 +164,7 @@ func processUserRequest(w http.ResponseWriter, r *http.Request, ui *UserInfo) {
|
|||
|
||||
func processRequest(w http.ResponseWriter, r *http.Request, ui *UserInfo) {
|
||||
u := normalizeURL(r.URL)
|
||||
up, hc, dropSrcPathPrefixParts := ui.getURLPrefixAndHeaders(u)
|
||||
up, hc := ui.getURLPrefixAndHeaders(u)
|
||||
isDefault := false
|
||||
if up == nil {
|
||||
if ui.DefaultURL == nil {
|
||||
|
|
@ -198,7 +198,7 @@ func processRequest(w http.ResponseWriter, r *http.Request, ui *UserInfo) {
|
|||
query.Set("request_path", u.String())
|
||||
targetURL.RawQuery = query.Encode()
|
||||
} else { // Update path for regular routes.
|
||||
targetURL = mergeURLs(targetURL, u, dropSrcPathPrefixParts)
|
||||
targetURL = mergeURLs(targetURL, u, up.dropSrcPathPrefixParts)
|
||||
}
|
||||
ok := tryProcessingRequest(w, r, targetURL, hc, up.retryStatusCodes, ui.httpTransport)
|
||||
bu.put()
|
||||
|
|
|
|||
|
|
@ -49,18 +49,28 @@ func dropPrefixParts(path string, parts int) string {
|
|||
return path
|
||||
}
|
||||
|
||||
func (ui *UserInfo) getURLPrefixAndHeaders(u *url.URL) (*URLPrefix, HeadersConf, int) {
|
||||
func (ui *UserInfo) getURLPrefixAndHeaders(u *url.URL) (*URLPrefix, HeadersConf) {
|
||||
for _, e := range ui.URLMaps {
|
||||
for _, sp := range e.SrcPaths {
|
||||
if sp.match(u.Path) {
|
||||
return e.URLPrefix, e.HeadersConf, e.DropSrcPathPrefixParts
|
||||
}
|
||||
if matchAnyRegex(e.SrcHosts, u.Host) && matchAnyRegex(e.SrcPaths, u.Path) {
|
||||
return e.URLPrefix, e.HeadersConf
|
||||
}
|
||||
}
|
||||
if ui.URLPrefix != nil {
|
||||
return ui.URLPrefix, ui.HeadersConf, ui.DropSrcPathPrefixParts
|
||||
return ui.URLPrefix, ui.HeadersConf
|
||||
}
|
||||
return nil, HeadersConf{}, 0
|
||||
return nil, HeadersConf{}
|
||||
}
|
||||
|
||||
func matchAnyRegex(rs []*Regex, s string) bool {
|
||||
if len(rs) == 0 {
|
||||
return true
|
||||
}
|
||||
for _, r := range rs {
|
||||
if r.match(s) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func normalizeURL(uOrig *url.URL) *url.URL {
|
||||
|
|
|
|||
|
|
@ -89,12 +89,12 @@ func TestCreateTargetURLSuccess(t *testing.T) {
|
|||
t.Fatalf("cannot parse %q: %s", requestURI, err)
|
||||
}
|
||||
u = normalizeURL(u)
|
||||
up, hc, dropSrcPathPrefixParts := ui.getURLPrefixAndHeaders(u)
|
||||
up, hc := ui.getURLPrefixAndHeaders(u)
|
||||
if up == nil {
|
||||
t.Fatalf("cannot determie backend: %s", err)
|
||||
}
|
||||
bu := up.getLeastLoadedBackendURL()
|
||||
target := mergeURLs(bu.url, u, dropSrcPathPrefixParts)
|
||||
target := mergeURLs(bu.url, u, up.dropSrcPathPrefixParts)
|
||||
bu.put()
|
||||
if target.String() != expectedTarget {
|
||||
t.Fatalf("unexpected target; got %q; want %q", target, expectedTarget)
|
||||
|
|
@ -109,8 +109,8 @@ func TestCreateTargetURLSuccess(t *testing.T) {
|
|||
if up.loadBalancingPolicy != expectedLoadBalancingPolicy {
|
||||
t.Fatalf("unexpected loadBalancingPolicy; got %q; want %q", up.loadBalancingPolicy, expectedLoadBalancingPolicy)
|
||||
}
|
||||
if dropSrcPathPrefixParts != expectedDropSrcPathPrefixParts {
|
||||
t.Fatalf("unexpected dropSrcPathPrefixParts; got %d; want %d", dropSrcPathPrefixParts, expectedDropSrcPathPrefixParts)
|
||||
if up.dropSrcPathPrefixParts != expectedDropSrcPathPrefixParts {
|
||||
t.Fatalf("unexpected dropSrcPathPrefixParts; got %d; want %d", up.dropSrcPathPrefixParts, expectedDropSrcPathPrefixParts)
|
||||
}
|
||||
}
|
||||
// Simple routing with `url_prefix`
|
||||
|
|
@ -127,7 +127,7 @@ func TestCreateTargetURLSuccess(t *testing.T) {
|
|||
},
|
||||
RetryStatusCodes: []int{503, 501},
|
||||
LoadBalancingPolicy: "first_available",
|
||||
DropSrcPathPrefixParts: 2,
|
||||
DropSrcPathPrefixParts: intp(2),
|
||||
}, "/a/b/c", "http://foo.bar/c", `[{"bb" "aaa"}]`, `[]`, []int{503, 501}, "first_available", 2)
|
||||
f(&UserInfo{
|
||||
URLPrefix: mustParseURL("http://foo.bar/federate"),
|
||||
|
|
@ -149,7 +149,8 @@ func TestCreateTargetURLSuccess(t *testing.T) {
|
|||
ui := &UserInfo{
|
||||
URLMaps: []URLMap{
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/vmsingle/api/v1/query"}),
|
||||
SrcHosts: getRegexs([]string{"host42"}),
|
||||
SrcPaths: getRegexs([]string{"/vmsingle/api/v1/query"}),
|
||||
URLPrefix: mustParseURL("http://vmselect/0/prometheus"),
|
||||
HeadersConf: HeadersConf{
|
||||
RequestHeaders: []Header{
|
||||
|
|
@ -171,11 +172,13 @@ func TestCreateTargetURLSuccess(t *testing.T) {
|
|||
},
|
||||
RetryStatusCodes: []int{503, 500, 501},
|
||||
LoadBalancingPolicy: "first_available",
|
||||
DropSrcPathPrefixParts: 1,
|
||||
DropSrcPathPrefixParts: intp(1),
|
||||
},
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/write"}),
|
||||
URLPrefix: mustParseURL("http://vminsert/0/prometheus"),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/write"}),
|
||||
URLPrefix: mustParseURL("http://vminsert/0/prometheus"),
|
||||
RetryStatusCodes: []int{},
|
||||
DropSrcPathPrefixParts: intp(0),
|
||||
},
|
||||
},
|
||||
URLPrefix: mustParseURL("http://default-server"),
|
||||
|
|
@ -190,23 +193,30 @@ func TestCreateTargetURLSuccess(t *testing.T) {
|
|||
}},
|
||||
},
|
||||
RetryStatusCodes: []int{502},
|
||||
DropSrcPathPrefixParts: 2,
|
||||
DropSrcPathPrefixParts: intp(2),
|
||||
}
|
||||
f(ui, "/vmsingle/api/v1/query?query=up", "http://vmselect/0/prometheus/api/v1/query?query=up", `[{"xx" "aa"} {"yy" "asdf"}]`, `[{"qwe" "rty"}]`, []int{503, 500, 501}, "first_available", 1)
|
||||
f(ui, "/api/v1/write", "http://vminsert/0/prometheus/api/v1/write", "[]", "[]", []int{502}, "least_loaded", 0)
|
||||
f(ui, "/foo/bar/api/v1/query_range", "http://default-server/api/v1/query_range", `[{"bb" "aaa"}]`, `[{"x" "y"}]`, []int{502}, "least_loaded", 2)
|
||||
f(ui, "http://host42/vmsingle/api/v1/query?query=up", "http://vmselect/0/prometheus/api/v1/query?query=up",
|
||||
`[{"xx" "aa"} {"yy" "asdf"}]`, `[{"qwe" "rty"}]`, []int{503, 500, 501}, "first_available", 1)
|
||||
f(ui, "http://host123/vmsingle/api/v1/query?query=up", "http://default-server/v1/query?query=up",
|
||||
`[{"bb" "aaa"}]`, `[{"x" "y"}]`, []int{502}, "least_loaded", 2)
|
||||
f(ui, "https://foo-host/api/v1/write", "http://vminsert/0/prometheus/api/v1/write", "[]", "[]", []int{}, "least_loaded", 0)
|
||||
f(ui, "https://foo-host/foo/bar/api/v1/query_range", "http://default-server/api/v1/query_range", `[{"bb" "aaa"}]`, `[{"x" "y"}]`, []int{502}, "least_loaded", 2)
|
||||
|
||||
// Complex routing regexp paths in `url_map`
|
||||
ui = &UserInfo{
|
||||
URLMaps: []URLMap{
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/query(_range)?", "/api/v1/label/[^/]+/values"}),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/query(_range)?", "/api/v1/label/[^/]+/values"}),
|
||||
URLPrefix: mustParseURL("http://vmselect/0/prometheus"),
|
||||
},
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/write"}),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/write"}),
|
||||
URLPrefix: mustParseURL("http://vminsert/0/prometheus"),
|
||||
},
|
||||
{
|
||||
SrcHosts: getRegexs([]string{"vmui\\..+"}),
|
||||
URLPrefix: mustParseURL("http://vmui.host:1234/vmui/"),
|
||||
},
|
||||
},
|
||||
URLPrefix: mustParseURL("http://default-server"),
|
||||
}
|
||||
|
|
@ -215,6 +225,8 @@ func TestCreateTargetURLSuccess(t *testing.T) {
|
|||
f(ui, "/api/v1/label/foo/values", "http://vmselect/0/prometheus/api/v1/label/foo/values", "[]", "[]", nil, "least_loaded", 0)
|
||||
f(ui, "/api/v1/write", "http://vminsert/0/prometheus/api/v1/write", "[]", "[]", nil, "least_loaded", 0)
|
||||
f(ui, "/api/v1/foo/bar", "http://default-server/api/v1/foo/bar", "[]", "[]", nil, "least_loaded", 0)
|
||||
f(ui, "https://vmui.foobar.com/a/b?c=d", "http://vmui.host:1234/vmui/a/b?c=d", "[]", "[]", nil, "least_loaded", 0)
|
||||
|
||||
f(&UserInfo{
|
||||
URLPrefix: mustParseURL("http://foo.bar?extra_label=team=dev"),
|
||||
}, "/api/v1/query", "http://foo.bar/api/v1/query?extra_label=team=dev", "[]", "[]", nil, "least_loaded", 0)
|
||||
|
|
@ -231,7 +243,7 @@ func TestCreateTargetURLFailure(t *testing.T) {
|
|||
t.Fatalf("cannot parse %q: %s", requestURI, err)
|
||||
}
|
||||
u = normalizeURL(u)
|
||||
up, hc, dropSrcPathPrefixParts := ui.getURLPrefixAndHeaders(u)
|
||||
up, hc := ui.getURLPrefixAndHeaders(u)
|
||||
if up != nil {
|
||||
t.Fatalf("unexpected non-empty up=%#v", up)
|
||||
}
|
||||
|
|
@ -241,15 +253,12 @@ func TestCreateTargetURLFailure(t *testing.T) {
|
|||
if hc.ResponseHeaders != nil {
|
||||
t.Fatalf("unexpected non-empty response headers=%q", hc.ResponseHeaders)
|
||||
}
|
||||
if dropSrcPathPrefixParts != 0 {
|
||||
t.Fatalf("unexpected non-zero dropSrcPathPrefixParts=%d", dropSrcPathPrefixParts)
|
||||
}
|
||||
}
|
||||
f(&UserInfo{}, "/foo/bar")
|
||||
f(&UserInfo{
|
||||
URLMaps: []URLMap{
|
||||
{
|
||||
SrcPaths: getSrcPaths([]string{"/api/v1/query"}),
|
||||
SrcPaths: getRegexs([]string{"/api/v1/query"}),
|
||||
URLPrefix: mustParseURL("http://foobar/baz"),
|
||||
},
|
||||
},
|
||||
|
|
|
|||
|
|
@ -326,9 +326,9 @@ const (
|
|||
vmNativeFilterTimeReverse = "vm-native-filter-time-reverse"
|
||||
vmNativeStepInterval = "vm-native-step-interval"
|
||||
|
||||
vmNativeDisableBinaryProtocol = "vm-native-disable-binary-protocol"
|
||||
vmNativeDisableHTTPKeepAlive = "vm-native-disable-http-keep-alive"
|
||||
vmNativeDisableRetries = "vm-native-disable-retries"
|
||||
vmNativeDisableBinaryProtocol = "vm-native-disable-binary-protocol"
|
||||
vmNativeDisableHTTPKeepAlive = "vm-native-disable-http-keep-alive"
|
||||
vmNativeDisablePerMetricMigration = "vm-native-disable-per-metric-migration"
|
||||
|
||||
vmNativeSrcAddr = "vm-native-src-addr"
|
||||
vmNativeSrcUser = "vm-native-src-user"
|
||||
|
|
@ -454,8 +454,8 @@ var (
|
|||
Value: 2,
|
||||
},
|
||||
&cli.BoolFlag{
|
||||
Name: vmNativeDisableRetries,
|
||||
Usage: "Defines whether to disable retries with backoff policy for migration process",
|
||||
Name: vmNativeDisablePerMetricMigration,
|
||||
Usage: "Defines whether to disable per-metric migration and migrate all data via one connection. In this mode, vmctl makes less export/import requests, but can't provide a progress bar or retry failed requests.",
|
||||
Value: false,
|
||||
},
|
||||
&cli.BoolFlag{
|
||||
|
|
|
|||
|
|
@ -254,11 +254,11 @@ func main() {
|
|||
ExtraLabels: dstExtraLabels,
|
||||
HTTPClient: dstHTTPClient,
|
||||
},
|
||||
backoff: backoff.New(),
|
||||
cc: c.Int(vmConcurrency),
|
||||
disableRetries: c.Bool(vmNativeDisableRetries),
|
||||
isSilent: c.Bool(globalSilent),
|
||||
isNative: !c.Bool(vmNativeDisableBinaryProtocol),
|
||||
backoff: backoff.New(),
|
||||
cc: c.Int(vmConcurrency),
|
||||
disablePerMetricRequests: c.Bool(vmNativeDisablePerMetricMigration),
|
||||
isSilent: c.Bool(globalSilent),
|
||||
isNative: !c.Bool(vmNativeDisableBinaryProtocol),
|
||||
}
|
||||
return p.run(ctx)
|
||||
},
|
||||
|
|
|
|||
|
|
@ -29,13 +29,14 @@ type vmNativeProcessor struct {
|
|||
src *native.Client
|
||||
backoff *backoff.Backoff
|
||||
|
||||
s *stats
|
||||
rateLimit int64
|
||||
interCluster bool
|
||||
cc int
|
||||
disableRetries bool
|
||||
isSilent bool
|
||||
isNative bool
|
||||
s *stats
|
||||
rateLimit int64
|
||||
interCluster bool
|
||||
cc int
|
||||
isSilent bool
|
||||
isNative bool
|
||||
|
||||
disablePerMetricRequests bool
|
||||
}
|
||||
|
||||
const (
|
||||
|
|
@ -119,19 +120,16 @@ func (p *vmNativeProcessor) runSingle(ctx context.Context, f native.Filter, srcU
|
|||
return fmt.Errorf("failed to init export pipe: %w", err)
|
||||
}
|
||||
|
||||
if p.disableRetries && bar != nil {
|
||||
if p.disablePerMetricRequests && bar != nil {
|
||||
fmt.Printf("Continue import process with filter %s:\n", f.String())
|
||||
reader = bar.NewProxyReader(reader)
|
||||
}
|
||||
|
||||
pr, pw := io.Pipe()
|
||||
done := make(chan struct{})
|
||||
importCh := make(chan error)
|
||||
go func() {
|
||||
defer func() { close(done) }()
|
||||
if err := p.dst.ImportPipe(ctx, dstURL, pr); err != nil {
|
||||
logger.Errorf("error initialize import pipe: %s", err)
|
||||
return
|
||||
}
|
||||
importCh <- p.dst.ImportPipe(ctx, dstURL, pr)
|
||||
close(importCh)
|
||||
}()
|
||||
|
||||
w := io.Writer(pw)
|
||||
|
|
@ -153,9 +151,8 @@ func (p *vmNativeProcessor) runSingle(ctx context.Context, f native.Filter, srcU
|
|||
if err := pw.Close(); err != nil {
|
||||
return err
|
||||
}
|
||||
<-done
|
||||
|
||||
return nil
|
||||
return <-importCh
|
||||
}
|
||||
|
||||
func (p *vmNativeProcessor) runBackfilling(ctx context.Context, tenantID string, ranges [][]time.Time, silent bool) error {
|
||||
|
|
@ -193,7 +190,7 @@ func (p *vmNativeProcessor) runBackfilling(ctx context.Context, tenantID string,
|
|||
var foundSeriesMsg string
|
||||
|
||||
metrics := []string{p.filter.Match}
|
||||
if !p.disableRetries {
|
||||
if !p.disablePerMetricRequests {
|
||||
log.Printf("Exploring metrics...")
|
||||
metrics, err = p.src.Explore(ctx, p.filter, tenantID)
|
||||
if err != nil {
|
||||
|
|
@ -231,7 +228,7 @@ func (p *vmNativeProcessor) runBackfilling(ctx context.Context, tenantID string,
|
|||
var bar *pb.ProgressBar
|
||||
if !silent {
|
||||
bar = barpool.NewSingleProgress(fmt.Sprintf(nativeWithBackoffTpl, barPrefix), len(metrics)*len(ranges))
|
||||
if p.disableRetries {
|
||||
if p.disablePerMetricRequests {
|
||||
bar = barpool.NewSingleProgress(nativeSingleProcessTpl, 0)
|
||||
}
|
||||
bar.Start()
|
||||
|
|
@ -247,7 +244,7 @@ func (p *vmNativeProcessor) runBackfilling(ctx context.Context, tenantID string,
|
|||
go func() {
|
||||
defer wg.Done()
|
||||
for f := range filterCh {
|
||||
if !p.disableRetries {
|
||||
if !p.disablePerMetricRequests {
|
||||
if err := p.do(ctx, f, srcURL, dstURL, nil); err != nil {
|
||||
errCh <- err
|
||||
return
|
||||
|
|
|
|||
|
|
@ -686,10 +686,7 @@ func tryGetArgRollupFuncWithMetricExpr(ae *metricsql.AggrFuncExpr) (*metricsql.F
|
|||
return nil, nil
|
||||
}
|
||||
// e = rollupFunc(metricExpr)
|
||||
return &metricsql.FuncExpr{
|
||||
Name: fe.Name,
|
||||
Args: []metricsql.Expr{me},
|
||||
}, nrf
|
||||
return fe, nrf
|
||||
}
|
||||
if re, ok := arg.(*metricsql.RollupExpr); ok {
|
||||
if me, ok := re.Expr.(*metricsql.MetricExpr); !ok || me.IsEmpty() || re.ForSubquery() {
|
||||
|
|
|
|||
|
|
@ -136,7 +136,11 @@ export const useFetchQuery = ({
|
|||
totalLength += resp.data.result.length;
|
||||
} else {
|
||||
tempData.push({ metric: {}, values: [], group: counter } as MetricBase);
|
||||
setQueryErrors(prev => [...prev, `${resp.errorType}\r\n${resp?.error}`]);
|
||||
const errorType = resp.errorType || ErrorTypes.unknownType;
|
||||
const errorMessage = resp?.error || resp?.message || "see console for more details";
|
||||
const error = [errorType, errorMessage].join(",\r\n");
|
||||
setQueryErrors(prev => [...prev, `${error}`]);
|
||||
console.error(`Fetch query error: ${errorType}`, resp);
|
||||
}
|
||||
counter++;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -47,7 +47,8 @@ export enum ErrorTypes {
|
|||
traceNotFound = "Not found the tracing information",
|
||||
emptyTitle = "Please enter title",
|
||||
positiveNumber = "Please enter positive number",
|
||||
validStep = "Please enter a valid step"
|
||||
validStep = "Please enter a valid step",
|
||||
unknownType = "Unknown server response format: must have 'errorType'",
|
||||
}
|
||||
|
||||
export interface PanelSettings {
|
||||
|
|
|
|||
|
|
@ -28,6 +28,15 @@ The sandbox cluster installation is running under the constant load generated by
|
|||
|
||||
## tip
|
||||
|
||||
* FEATURE: [vmauth](https://docs.victoriametrics.com/vmauth.html): add ability to proxy incoming requests to different backends based on the requested host via `src_hosts` option at `url_map`. See [these docs](https://docs.victoriametrics.com/vmauth.html#generic-http-proxy-for-different-backends).
|
||||
* FEATURE: [vmctl](https://docs.victoriametrics.com/vmctl.html): rename cmd-line flag `vm-native-disable-retries` to `vm-native-disable-per-metric-migration` to better reflect its meaning.
|
||||
* FEATURE: all VictoriaMetrics components: add ability to specify arbitrary HTTP headers to send with every request to `-pushmetrics.url`. See [`push metrics` docs](https://docs.victoriametrics.com/#push-metrics).
|
||||
|
||||
* BUGFIX: `vminsert`: properly accept samples via [OpenTelemetry data ingestion protocol](https://docs.victoriametrics.com/#sending-data-via-opentelemetry) when these samples have no [resource attributes](https://opentelemetry.io/docs/instrumentation/go/resources/). Previously such samples were silently skipped.
|
||||
* BUGFIX: `vmstorage`: added missing `-inmemoryDataFlushInterval` command-line flag, which was missing in [VictoriaMetrics cluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html) after implementing [this feature](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337) in [v1.85.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.85.0).
|
||||
* BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert.html): check `-external.url` schema when starting vmalert, must be `http` or `https`. Before, alertmanager could reject alert notifications if `-external.url` contained no or wrong schema.
|
||||
* BUGFIX: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): properly handle queries, which wrap [rollup functions](https://docs.victoriametrics.com/MetricsQL.html#rollup-functions) with multiple arguments without explicitly specified lookbehind window in square brackets into [aggregate functions](https://docs.victoriametrics.com/MetricsQL.html#aggregate-functions). For example, `sum(quantile_over_time(0.5, process_resident_memory_bytes))` was resulting to `expecting at least 2 args to ...; got 1 args` error. Thanks to @atykhyy for [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5414).
|
||||
* BUGFIX: [vmctl](https://docs.victoriametrics.com/vmctl.html): retry on import errors in `vm-native` mode. Before, retries happened only on writes into a network connection between source and destination. But errors returned by server after all the data was transmitted were logged, but not retried.
|
||||
|
||||
## [v1.96.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.96.0)
|
||||
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ where you can chat with VictoriaMetrics users to get additional references, revi
|
|||
- [Naver](#naver)
|
||||
- [Percona](#percona)
|
||||
- [Razorpay](#razorpay)
|
||||
- [RELEX Solutions](#relex-solutions)
|
||||
- [Roblox](#roblox)
|
||||
- [Sensedia](#sensedia)
|
||||
- [Smarkets](#smarkets)
|
||||
|
|
@ -434,6 +435,33 @@ Percona migrated from Prometheus to VictoriaMetrics in the [Percona Monitoring a
|
|||
|
||||
See [the full article](https://engineering.razorpay.com/scaling-to-trillions-of-metric-data-points-f569a5b654f2).
|
||||
|
||||
## RELEX Solutions
|
||||
|
||||
[RELEX Solutions](https://www.relexsolutions.com/), a global software company from Finland, is the market-leading supply chain and retail planning platform.
|
||||
|
||||
VictoriaMetrics is used as the central metrics storage for timeseries about applications and machines hosted both in the public cloud and in the private cloud. Metrics are remote-written by Prometheus, the OpenTelemetry collector and sometimes directly by custom Prometheus exporters.
|
||||
|
||||
Alerts are evaluated on vmalert when necessary, either because metrics from multiple sources are needed or because the source is Prometheus in agent mode (mostly for kubernetes clusters). Prometheus Alertmanager and Grafana+Promxy combine all sources together so that the end users of dashboards (and ideally the recipients of alert notifications) don't have to worry where to look for some information.
|
||||
|
||||
VictoriaMetrics has allowed us to extend data retention for our metrics effortlessly, while Prometheus tsdb would have required us to manage ever bigger disks or to plan aggregation and downsampling within a Prometheus hierarchical federation. It also allows for a separation of concerns: whereas alerting requires immediacy and precision in the metrics, visualizations of historical data can bear with delay and downsampling.
|
||||
|
||||
Across our production VictoriaMetrics clusters, in a 12 months period we go beyond the following figures.
|
||||
|
||||
- Active time series: 10M
|
||||
- Ingestion rate: 300k samples per second
|
||||
- Total number of datapoints: 4400G
|
||||
- Data size on disk: 3600 GiB
|
||||
- Available memory: 320 GiB as seen by kubernetes (160 GiB physical memory for the hosts)
|
||||
- CPU: 20 cores (AMD EPYC 7763), about 70 % idle
|
||||
- Retention period: ~1 year
|
||||
- Churn rate: 6M new time series per day (monthly average)
|
||||
- Query rate:
|
||||
- `/api/v1/query_range`: 10 queries per second
|
||||
- `/api/v1/query`: 10 queries per second
|
||||
- Query duration for `/api/v1/query_range` (weekly mean):
|
||||
- 99th percentile: 700 ms
|
||||
- median: 10 ms
|
||||
|
||||
## Roblox
|
||||
|
||||
[Roblox](https://roblox.com/) builds the tools and platform that empower people to create their own immersive experiences,
|
||||
|
|
|
|||
|
|
@ -1109,11 +1109,16 @@ Below is the output for `/path/to/vminsert -help`:
|
|||
Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-prevCacheRemovalPercent float
|
||||
Items in the previous caches are removed when the percent of requests it serves becomes lower than this value. Higher values reduce memory usage at the cost of higher CPU usage. See also -cacheExpireDuration (default 0.1)
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
@ -1289,11 +1294,16 @@ Below is the output for `/path/to/vmselect -help`:
|
|||
Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-prevCacheRemovalPercent float
|
||||
Items in the previous caches are removed when the percent of requests it serves becomes lower than this value. Higher values reduce memory usage at the cost of higher CPU usage. See also -cacheExpireDuration (default 0.1)
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
@ -1501,6 +1511,8 @@ Below is the output for `/path/to/vmstorage -help`:
|
|||
Address to listen for http connections. See also -httpListenAddr.useProxyProtocol (default ":8482")
|
||||
-httpListenAddr.useProxyProtocol
|
||||
Whether to use proxy protocol for connections accepted at -httpListenAddr . See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt . With enabled proxy protocol http server cannot serve regular /metrics endpoint. Use -pushmetrics.url for metrics pushing
|
||||
-inmemoryDataFlushInterval duration
|
||||
The interval for guaranteed saving of in-memory data to disk. The saved data survives unclean shutdowns such as OOM crash, hardware reset, SIGKILL, etc. Bigger intervals may help increase the lifetime of flash storage with limited write cycles (e.g. Raspberry PI). Smaller intervals increase disk IO load. Minimum supported value is 1s (default 5s)
|
||||
-insert.maxQueueDuration duration
|
||||
The maximum duration to wait in the queue when -maxConcurrentInserts concurrent insert requests are executed (default 1m0s)
|
||||
-internStringCacheExpireDuration duration
|
||||
|
|
@ -1550,11 +1562,16 @@ Below is the output for `/path/to/vmstorage -help`:
|
|||
The number of precision bits to store per each value. Lower precision bits improves data compression at the cost of precision loss (default 64)
|
||||
-prevCacheRemovalPercent float
|
||||
Items in the previous caches are removed when the percent of requests it serves becomes lower than this value. Higher values reduce memory usage at the cost of higher CPU usage. See also -cacheExpireDuration (default 0.1)
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
|
|||
|
|
@ -1725,9 +1725,10 @@ See also [resource usage limits at VictoriaMetrics cluster](https://docs.victori
|
|||
|
||||
The general approach for achieving high availability is the following:
|
||||
|
||||
- to run two identically configured VictoriaMetrics instances in distinct datacenters (availability zones)
|
||||
- to store the collected data simultaneously into these instances via [vmagent](https://docs.victoriametrics.com/vmagent.html) or Prometheus
|
||||
- to query the first VictoriaMetrics instance and to fail over to the second instance when the first instance becomes temporarily unavailable.
|
||||
- To run two identically configured VictoriaMetrics instances in distinct datacenters (availability zones);
|
||||
- To store the collected data simultaneously into these instances via [vmagent](https://docs.victoriametrics.com/vmagent.html) or Prometheus.
|
||||
- To query the first VictoriaMetrics instance and to fail over to the second instance when the first instance becomes temporarily unavailable.
|
||||
This can be done via [vmauth](https://docs.victoriametrics.com/vmauth.html) according to [these docs](https://docs.victoriametrics.com/vmauth.html#high-availability).
|
||||
|
||||
Such a setup guarantees that the collected data isn't lost when one of VictoriaMetrics instance becomes unavailable.
|
||||
The collected data continues to be written to the available VictoriaMetrics instance, so it should be available for querying.
|
||||
|
|
@ -2290,11 +2291,14 @@ The following command-line flags are related to pushing metrics from VictoriaMet
|
|||
The `-pushmetrics.url` can be specified multiple times. In this case metrics are pushed to all the specified urls.
|
||||
The url can contain basic auth params in the form `http://user:pass@hostname/api/v1/import/prometheus`.
|
||||
Metrics are pushed to the provided `-pushmetrics.url` in a compressed form with `Content-Encoding: gzip` request header.
|
||||
This allows reducing the required network bandwidth for metrics push.
|
||||
* `-pushmetrics.extraLabel` - labels to add to all the metrics before sending them to `-pushmetrics.url`. Each label must be specified in the format `label="value"`.
|
||||
This allows reducing the required network bandwidth for metrics push. The compression can be disabled by passing `-pushmetrics.disableCompression` command-line flag.
|
||||
* `-pushmetrics.extraLabel` - labels to add to all the metrics before sending them to every `-pushmetrics.url`. Each label must be specified in the format `label="value"`.
|
||||
It is OK to specify multiple `-pushmetrics.extraLabel` command-line flags. In this case all the specified labels
|
||||
are added to all the metrics before sending them to all the configured `-pushmetrics.url` addresses.
|
||||
* `-pushmetrics.interval` - the interval between pushes. By default it is set to 10 seconds.
|
||||
* `-pushmetrics.header` - an optional HTTP header to send to every `-pushmetrics.url`. For example, `-pushmetrics.header='Authorization: Basic foo'` instructs to send
|
||||
`Authorization: Basic foo` HTTP header with every request to every `-pushmetrics.url`. It is possible to set multiple `-pushmetrics.header` command-line flags
|
||||
for sending multiple different HTTP headers to `-pushmetrics.url`.
|
||||
|
||||
For example, the following command instructs VictoriaMetrics to push metrics from `/metrics` page to `https://maas.victoriametrics.com/api/v1/import/prometheus`
|
||||
with `user:pass` [Basic auth](https://en.wikipedia.org/wiki/Basic_access_authentication). The `instance="foobar"` and `job="vm"` labels
|
||||
|
|
@ -2840,11 +2844,16 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
|
|||
The delay for suppressing repeated scrape errors logging per each scrape targets. This may be used for reducing the number of log lines related to scrape errors. See also -promscrape.suppressScrapeErrors
|
||||
-promscrape.yandexcloudSDCheckInterval duration
|
||||
Interval for checking for changes in Yandex Cloud API. This works only if yandexcloud_sd_configs is configured in '-promscrape.config' file. See https://docs.victoriametrics.com/sd_configs.html#yandexcloud_sd_configs for details (default 30s)
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
|
|||
|
|
@ -1733,9 +1733,10 @@ See also [resource usage limits at VictoriaMetrics cluster](https://docs.victori
|
|||
|
||||
The general approach for achieving high availability is the following:
|
||||
|
||||
- to run two identically configured VictoriaMetrics instances in distinct datacenters (availability zones)
|
||||
- to store the collected data simultaneously into these instances via [vmagent](https://docs.victoriametrics.com/vmagent.html) or Prometheus
|
||||
- to query the first VictoriaMetrics instance and to fail over to the second instance when the first instance becomes temporarily unavailable.
|
||||
- To run two identically configured VictoriaMetrics instances in distinct datacenters (availability zones);
|
||||
- To store the collected data simultaneously into these instances via [vmagent](https://docs.victoriametrics.com/vmagent.html) or Prometheus.
|
||||
- To query the first VictoriaMetrics instance and to fail over to the second instance when the first instance becomes temporarily unavailable.
|
||||
This can be done via [vmauth](https://docs.victoriametrics.com/vmauth.html) according to [these docs](https://docs.victoriametrics.com/vmauth.html#high-availability).
|
||||
|
||||
Such a setup guarantees that the collected data isn't lost when one of VictoriaMetrics instance becomes unavailable.
|
||||
The collected data continues to be written to the available VictoriaMetrics instance, so it should be available for querying.
|
||||
|
|
@ -2298,11 +2299,14 @@ The following command-line flags are related to pushing metrics from VictoriaMet
|
|||
The `-pushmetrics.url` can be specified multiple times. In this case metrics are pushed to all the specified urls.
|
||||
The url can contain basic auth params in the form `http://user:pass@hostname/api/v1/import/prometheus`.
|
||||
Metrics are pushed to the provided `-pushmetrics.url` in a compressed form with `Content-Encoding: gzip` request header.
|
||||
This allows reducing the required network bandwidth for metrics push.
|
||||
* `-pushmetrics.extraLabel` - labels to add to all the metrics before sending them to `-pushmetrics.url`. Each label must be specified in the format `label="value"`.
|
||||
This allows reducing the required network bandwidth for metrics push. The compression can be disabled by passing `-pushmetrics.disableCompression` command-line flag.
|
||||
* `-pushmetrics.extraLabel` - labels to add to all the metrics before sending them to every `-pushmetrics.url`. Each label must be specified in the format `label="value"`.
|
||||
It is OK to specify multiple `-pushmetrics.extraLabel` command-line flags. In this case all the specified labels
|
||||
are added to all the metrics before sending them to all the configured `-pushmetrics.url` addresses.
|
||||
* `-pushmetrics.interval` - the interval between pushes. By default it is set to 10 seconds.
|
||||
* `-pushmetrics.header` - an optional HTTP header to send to every `-pushmetrics.url`. For example, `-pushmetrics.header='Authorization: Basic foo'` instructs to send
|
||||
`Authorization: Basic foo` HTTP header with every request to every `-pushmetrics.url`. It is possible to set multiple `-pushmetrics.header` command-line flags
|
||||
for sending multiple different HTTP headers to `-pushmetrics.url`.
|
||||
|
||||
For example, the following command instructs VictoriaMetrics to push metrics from `/metrics` page to `https://maas.victoriametrics.com/api/v1/import/prometheus`
|
||||
with `user:pass` [Basic auth](https://en.wikipedia.org/wiki/Basic_access_authentication). The `instance="foobar"` and `job="vm"` labels
|
||||
|
|
@ -2848,11 +2852,16 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
|
|||
The delay for suppressing repeated scrape errors logging per each scrape targets. This may be used for reducing the number of log lines related to scrape errors. See also -promscrape.suppressScrapeErrors
|
||||
-promscrape.yandexcloudSDCheckInterval duration
|
||||
Interval for checking for changes in Yandex Cloud API. This works only if yandexcloud_sd_configs is configured in '-promscrape.config' file. See https://docs.victoriametrics.com/sd_configs.html#yandexcloud_sd_configs for details (default 30s)
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
|
|||
|
|
@ -10,7 +10,6 @@ aliases:
|
|||
---
|
||||
# Multi Retention Setup within VictoriaMetrics Cluster
|
||||
|
||||
|
||||
**Objective**
|
||||
|
||||
Setup Victoria Metrics Cluster with support of multiple retention periods within one installation.
|
||||
|
|
@ -29,7 +28,9 @@ A multi-retention setup can be implemented by dividing a [victoriametrics cluste
|
|||
|
||||
Example:
|
||||
Setup should handle 3 different retention groups 3months, 1year and 3 years.
|
||||
Solution contains 3 groups of vmstorages + vminserts and one group of vmselects. Routing is done by [vmagent](https://docs.victoriametrics.com/vmagent.html) and [relabeling configuration](https://docs.victoriametrics.com/vmagent.html#relabeling). The [-retentionPeriod](https://docs.victoriametrics.com/#retention) sets how long to keep the metrics.
|
||||
Solution contains 3 groups of vmstorages + vminserts and one group of vmselects. Routing is done by [vmagent](https://docs.victoriametrics.com/vmagent.html)
|
||||
by [splitting data streams](https://docs.victoriametrics.com/vmagent.html#splitting-data-streams-among-multiple-systems).
|
||||
The [-retentionPeriod](https://docs.victoriametrics.com/#retention) sets how long to keep the metrics.
|
||||
|
||||
The diagram below shows a proposed solution
|
||||
|
||||
|
|
|
|||
|
|
@ -207,6 +207,34 @@ which is applied independently for each configured `-remoteWrite.url` destinatio
|
|||
data among long-term remote storage, short-term remote storage and a real-time analytical system [built on top of Kafka](https://github.com/Telefonica/prometheus-kafka-adapter).
|
||||
Note that each destination can receive its own subset of the collected data due to per-destination relabeling via `-remoteWrite.urlRelabelConfig`.
|
||||
|
||||
For example, let's assume all the scraped or received metrics by `vmagent` have label `env` with values `dev` or `prod`.
|
||||
To route metrics `env=dev` to destination `dev` and metrics with `env=prod` to destination `prod` apply the following config:
|
||||
1. Create relabeling config file `relabelDev.yml` to drop all metrics that don't have label `env=dev`:
|
||||
```yaml
|
||||
- action: keep
|
||||
source_labels: [env]
|
||||
regex: "dev"
|
||||
```
|
||||
1. Create relabeling config file `relabelProd.yml` to drop all metrics that don't have label `env=prod`:
|
||||
```yaml
|
||||
- action: keep
|
||||
source_labels: [env]
|
||||
regex: "prod"
|
||||
```
|
||||
1. Configure `vmagent` with 2 `-remoteWrite.url` flags pointing to destinations `dev` and `prod` with corresponding
|
||||
`-remoteWrite.urlRelabelConfig` configs:
|
||||
```console
|
||||
./vmagent \
|
||||
-remoteWrite.url=http://<dev-url> -remoteWrite.urlRelabelConfig=relabelDev.yml \
|
||||
-remoteWrite.url=http://<prod-url> -remoteWrite.urlRelabelConfig=relabelProd.yml
|
||||
```
|
||||
With this configuration `vmagent` will forward to `http://<dev-url>` only metrics that have `env=dev` label.
|
||||
And to `http://<prod-url>` it will forward only metrics that have `env=prod` label.
|
||||
|
||||
Please note, order of flags is important: 1st mentioned `-remoteWrite.urlRelabelConfig` will be applied to the
|
||||
1st mentioned `-remoteWrite.url`, and so on.
|
||||
|
||||
|
||||
### Prometheus remote_write proxy
|
||||
|
||||
`vmagent` can be used as a proxy for Prometheus data sent via Prometheus `remote_write` protocol. It can accept data via the `remote_write` API
|
||||
|
|
@ -1794,11 +1822,16 @@ See the docs at https://docs.victoriametrics.com/vmagent.html .
|
|||
The delay for suppressing repeated scrape errors logging per each scrape targets. This may be used for reducing the number of log lines related to scrape errors. See also -promscrape.suppressScrapeErrors
|
||||
-promscrape.yandexcloudSDCheckInterval duration
|
||||
Interval for checking for changes in Yandex Cloud API. This works only if yandexcloud_sd_configs is configured in '-promscrape.config' file. See https://docs.victoriametrics.com/sd_configs.html#yandexcloud_sd_configs for details (default 30s)
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
|
|||
|
|
@ -533,7 +533,7 @@ In case when you want to spread the load on these components - add balancers bef
|
|||
|
||||
#### HA vmalert
|
||||
|
||||
For HA user can run multiple identically configured `vmalert` instances.
|
||||
For High Availability(HA) user can run multiple identically configured `vmalert` instances.
|
||||
It means all of them will execute the same rules, write state and results to
|
||||
the same destinations, and send alert notifications to multiple configured
|
||||
Alertmanagers.
|
||||
|
|
@ -1194,11 +1194,16 @@ The shortlist of configuration flags is the following:
|
|||
The maximum duration for waiting to perform API requests if more than -promscrape.discovery.concurrency requests are simultaneously performed (default 1m0s)
|
||||
-promscrape.dnsSDCheckInterval duration
|
||||
Interval for checking for changes in dns. This works only if dns_sd_configs is configured in '-promscrape.config' file. See https://docs.victoriametrics.com/sd_configs.html#dns_sd_configs for details (default 30s)
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
|
|||
|
|
@ -74,13 +74,13 @@ unauthorized_user:
|
|||
|
||||
### Generic HTTP proxy for different backends
|
||||
|
||||
`vmauth` can proxy requests to different backends depending on the requested path.
|
||||
`vmauth` can proxy requests to different backends depending on the requested host and/or path.
|
||||
For example, the following [`-auth.config`](#auth-config) instructs `vmauth` to make the following:
|
||||
|
||||
- Requests starting with `/app1/` are proxied to `http://app1-backend/`. For example, the request to `http://vmauth:8427/app1/foo/bar?baz=qwe`
|
||||
is proxied to `http://app1-backend/foo/bar?baz=qwe`.
|
||||
- Requests starting with `/app2/` are proxied to `http://app2-backend/`. For example, the request to `http://vmauth:8427/app2/index.html`
|
||||
is proxied to `http://app2-backend/index.html`.
|
||||
- Requests starting with `/app1/` are proxied to `http://app1-backend/`, while the `/app1/` path prefix is dropped according to [`drop_src_path_prefix_parts`](#dropping-request-path-prefix).
|
||||
For example, the request to `http://vmauth:8427/app1/foo/bar?baz=qwe` is proxied to `http://app1-backend/foo/bar?baz=qwe`.
|
||||
- Requests starting with `/app2/` are proxied to `http://app2-backend/`, while the `/app2/` path prefix is dropped according to [`drop_src_path_prefix_parts`](#dropping-request-path-prefix).
|
||||
For example, the request to `http://vmauth:8427/app2/index.html` is proxied to `http://app2-backend/index.html`.
|
||||
- Other requests are proxied to `http://some-backend/404-page.html`, while the requested path is passed via `request_path` query arg.
|
||||
For example, the request to `http://vmauth:8427/foo/bar?baz=qwe` is proxied to `http://some-backend/404-page.html?request_path=%2Ffoo%2Fbar%3Fbaz%3Dqwe`.
|
||||
|
||||
|
|
@ -98,8 +98,23 @@ unauthorized_user:
|
|||
default_url: http://some-backend/404-page.html
|
||||
```
|
||||
|
||||
See [these docs](#dropping-request-path-prefix) for more details.
|
||||
The following config routes requests to host `app1.my-host.com` to `http://app1-backend`, while routing requests to `app2.my-host.com` to `http://app2-backend`:
|
||||
|
||||
```yml
|
||||
unauthorized_user:
|
||||
url_map:
|
||||
- src_hosts:
|
||||
- "app1\\.my-host\\.com"
|
||||
url_prefix: "http://app1-backend/"
|
||||
- src_paths:
|
||||
- "app2\\.my-host\\.com"
|
||||
url_prefix: "http://app2-backend/"
|
||||
```
|
||||
|
||||
`src_paths` and `src_hosts` accept a list of [regular expressions](https://github.com/google/re2/wiki/Syntax). The incoming request is routed to the given `url_prefix`
|
||||
if the whole request path matches at least one `src_paths` entry. The incoming request is routed to the given `url_prefix` if the whole request host matches at least one `src_hosts` entry.
|
||||
If both `src_paths` and `src_hosts` lists are specified, then the request is routed to the given `url_prefix` when both request path and request host match at least one entry
|
||||
in the corresponding lists.
|
||||
|
||||
### Generic HTTP load balancer
|
||||
|
||||
|
|
@ -603,7 +618,7 @@ users:
|
|||
# - to http://default1:8888/unsupported_url_handler?request_path=/non/existing/path
|
||||
# - or http://default2:8888/unsupported_url_handler?request_path=/non/existing/path
|
||||
#
|
||||
# Regular expressions are allowed in `src_paths` entries.
|
||||
# Regular expressions are allowed in `src_paths` and `src_hosts` entries.
|
||||
- username: "foobar"
|
||||
url_map:
|
||||
- src_paths:
|
||||
|
|
@ -870,11 +885,16 @@ See the docs at https://docs.victoriametrics.com/vmauth.html .
|
|||
Auth key for /metrics endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-pprofAuthKey string
|
||||
Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
|
|||
|
|
@ -391,11 +391,16 @@ Run `vmbackup -help` in order to see all the available options:
|
|||
Optional origin directory on the remote storage with old backup for server-side copying when performing full backup. This speeds up full backups
|
||||
-pprofAuthKey string
|
||||
Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
|
|||
|
|
@ -530,11 +530,16 @@ command-line flags:
|
|||
Auth key for /metrics endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-pprofAuthKey string
|
||||
Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
|
|||
|
|
@ -821,7 +821,7 @@ Requests to make: 9 / 9 [██████████████████
|
|||
```
|
||||
|
||||
_To disable explore phase and switch to the old way of data migration via single connection use
|
||||
`--vm-native-disable-retries` cmd-line flag. Please note, in this mode vmctl won't be able to retry failed requests._
|
||||
`--vm-native-disable-per-metric-migration` cmd-line flag. Please note, in this mode vmctl won't be able to retry failed requests._
|
||||
|
||||
Importing tips:
|
||||
|
||||
|
|
@ -831,14 +831,17 @@ Importing tips:
|
|||
[here](https://docs.victoriametrics.com/#how-to-export-data-in-native-format).
|
||||
If hitting `the number of matching timeseries exceeds...` error, adjust filters to match less time series or
|
||||
update `-search.maxSeries` command-line flag on vmselect/vmsingle;
|
||||
1. Using smaller intervals via `--vm-native-step-interval` cmd-line flag can reduce the number of matched series per-request
|
||||
for sources with [high churn rate](https://docs.victoriametrics.com/FAQ.html#what-is-high-churn-rate).
|
||||
See more about [step interval here](#using-time-based-chunking-of-migration).
|
||||
1. Migrating all the metrics from one VM to another may collide with existing application metrics
|
||||
(prefixed with `vm_`) at destination and lead to confusion when using
|
||||
[official Grafana dashboards](https://grafana.com/orgs/victoriametrics/dashboards).
|
||||
To avoid such situation try to filter out VM process metrics via `--vm-native-filter-match='{__name__!~"vm_.*"}'` flag.
|
||||
1. Migrating data with overlapping time range or via unstable network can produce duplicates series at destination.
|
||||
To avoid duplicates set `-dedup.minScrapeInterval=1ms` for `vmselect`/`vmstorage` at the destination.
|
||||
This will instruct `vmselect`/`vmstorage` to ignore duplicates with identical timestamps.
|
||||
1. When migrating large volumes of data use `--vm-native-step-interval` flag to split migration [into steps](#using-time-based-chunking-of-migration).
|
||||
This will instruct `vmselect`/`vmstorage` to ignore duplicates with identical timestamps. Ignore this recommendation
|
||||
if you already have `-dedup.minScrapeInterval` set to 1ms or higher values at destination.
|
||||
1. When migrating data from one VM cluster to another, consider using [cluster-to-cluster mode](#cluster-to-cluster-migration-mode).
|
||||
Or manually specify addresses according to [URL format](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#url-format):
|
||||
```console
|
||||
|
|
|
|||
|
|
@ -425,11 +425,16 @@ The shortlist of configuration flags include the following:
|
|||
Auth key for /metrics endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-pprofAuthKey string
|
||||
Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
|
|||
|
|
@ -186,11 +186,16 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q
|
|||
Auth key for /metrics endpoint. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-pprofAuthKey string
|
||||
Auth key for /debug/pprof/* endpoints. It must be passed via authKey query arg. It overrides httpAuth.* settings
|
||||
-pushmetrics.disableCompression
|
||||
Whether to disable request body compression when pushing metrics to every -pushmetrics.url
|
||||
-pushmetrics.extraLabel array
|
||||
Optional labels to add to metrics pushed to -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url
|
||||
Optional labels to add to metrics pushed to every -pushmetrics.url . For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.header array
|
||||
Optional HTTP request header to send to every -pushmetrics.url . For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
-pushmetrics.interval duration
|
||||
Interval for pushing metrics to -pushmetrics.url (default 10s)
|
||||
Interval for pushing metrics to every -pushmetrics.url (default 10s)
|
||||
-pushmetrics.url array
|
||||
Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . By default, metrics exposed at /metrics page aren't pushed to any remote storage
|
||||
Supports an array of values separated by comma or specified via multiple flags.
|
||||
|
|
|
|||
2
go.mod
2
go.mod
|
|
@ -11,7 +11,7 @@ require (
|
|||
// Do not use the original github.com/valyala/fasthttp because of issues
|
||||
// like https://github.com/valyala/fasthttp/commit/996610f021ff45fdc98c2ce7884d5fa4e7f9199b
|
||||
github.com/VictoriaMetrics/fasthttp v1.2.0
|
||||
github.com/VictoriaMetrics/metrics v1.25.3
|
||||
github.com/VictoriaMetrics/metrics v1.26.1
|
||||
github.com/VictoriaMetrics/metricsql v0.70.0
|
||||
github.com/aws/aws-sdk-go-v2 v1.24.0
|
||||
github.com/aws/aws-sdk-go-v2/config v1.26.1
|
||||
|
|
|
|||
4
go.sum
4
go.sum
|
|
@ -63,8 +63,8 @@ github.com/VictoriaMetrics/fastcache v1.12.2/go.mod h1:AmC+Nzz1+3G2eCPapF6UcsnkT
|
|||
github.com/VictoriaMetrics/fasthttp v1.2.0 h1:nd9Wng4DlNtaI27WlYh5mGXCJOmee/2c2blTJwfyU9I=
|
||||
github.com/VictoriaMetrics/fasthttp v1.2.0/go.mod h1:zv5YSmasAoSyv8sBVexfArzFDIGGTN4TfCKAtAw7IfE=
|
||||
github.com/VictoriaMetrics/metrics v1.24.0/go.mod h1:eFT25kvsTidQFHb6U0oa0rTrDRdz4xTYjpL8+UPohys=
|
||||
github.com/VictoriaMetrics/metrics v1.25.3 h1:Zcxyj8JbAB6CQU51Er3D7RBRupcP55DevVQi9cFqo2Q=
|
||||
github.com/VictoriaMetrics/metrics v1.25.3/go.mod h1:ZKmlI+QN6b9LUC0OiHNp2LiGQGlBy4U1re6Slooln1o=
|
||||
github.com/VictoriaMetrics/metrics v1.26.1 h1:zJmbXMED+mnirSpJeVJKxXNaFKJaX3Bj/VuV2j2uKKY=
|
||||
github.com/VictoriaMetrics/metrics v1.26.1/go.mod h1:EQ7JHshszbJYGuC1IIqkL+K73bTTTurJMQvUT5NMT5Q=
|
||||
github.com/VictoriaMetrics/metricsql v0.70.0 h1:G0k/m1yAF6pmk0dM3VT9/XI5PZ8dL7EbcLhREf4bgeI=
|
||||
github.com/VictoriaMetrics/metricsql v0.70.0/go.mod h1:k4UaP/+CjuZslIjd+kCigNG9TQmUqh5v0TP/nMEy90I=
|
||||
github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1ow=
|
||||
|
|
|
|||
18
lib/fs/fs.go
18
lib/fs/fs.go
|
|
@ -190,24 +190,6 @@ func mustSyncParentDirIfExists(path string) {
|
|||
MustSyncPath(parentDirPath)
|
||||
}
|
||||
|
||||
// IsEmptyDir returns true if path points to empty directory.
|
||||
func IsEmptyDir(path string) bool {
|
||||
// See https://stackoverflow.com/a/30708914/274937
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
logger.Panicf("FATAL: cannot open dir: %s", err)
|
||||
}
|
||||
_, err = f.Readdirnames(1)
|
||||
MustClose(f)
|
||||
if err != nil {
|
||||
if err == io.EOF {
|
||||
return true
|
||||
}
|
||||
logger.Panicf("FATAL: unexpected error when reading directory %q: %s", path, err)
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// MustRemoveDirAtomic removes the given dir atomically.
|
||||
//
|
||||
// It uses the following algorithm:
|
||||
|
|
|
|||
|
|
@ -6,6 +6,8 @@ import (
|
|||
"strconv"
|
||||
"sync"
|
||||
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/bytesutil"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/decimal"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fasttime"
|
||||
|
|
@ -14,7 +16,6 @@ import (
|
|||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/common"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/protoparser/opentelemetry/pb"
|
||||
"github.com/VictoriaMetrics/VictoriaMetrics/lib/writeconcurrencylimiter"
|
||||
"github.com/VictoriaMetrics/metrics"
|
||||
)
|
||||
|
||||
// ParseStream parses OpenTelemetry protobuf or json data from r and calls callback for the parsed rows.
|
||||
|
|
@ -271,11 +272,11 @@ func (wr *writeContext) readAndUnpackRequest(r io.Reader) (*pb.ExportMetricsServ
|
|||
|
||||
func (wr *writeContext) parseRequestToTss(req *pb.ExportMetricsServiceRequest) {
|
||||
for _, rm := range req.ResourceMetrics {
|
||||
if rm.Resource == nil {
|
||||
// skip metrics without resource part.
|
||||
continue
|
||||
var attributes []*pb.KeyValue
|
||||
if rm.Resource != nil {
|
||||
attributes = rm.Resource.Attributes
|
||||
}
|
||||
wr.baseLabels = appendAttributesToPromLabels(wr.baseLabels[:0], rm.Resource.Attributes)
|
||||
wr.baseLabels = appendAttributesToPromLabels(wr.baseLabels[:0], attributes)
|
||||
for _, sc := range rm.ScopeMetrics {
|
||||
wr.appendSamplesFromScopeMetrics(sc)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
package pushmetrics
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"strings"
|
||||
"time"
|
||||
|
|
@ -14,9 +15,12 @@ import (
|
|||
var (
|
||||
pushURL = flagutil.NewArrayString("pushmetrics.url", "Optional URL to push metrics exposed at /metrics page. See https://docs.victoriametrics.com/#push-metrics . "+
|
||||
"By default, metrics exposed at /metrics page aren't pushed to any remote storage")
|
||||
pushInterval = flag.Duration("pushmetrics.interval", 10*time.Second, "Interval for pushing metrics to -pushmetrics.url")
|
||||
pushExtraLabel = flagutil.NewArrayString("pushmetrics.extraLabel", "Optional labels to add to metrics pushed to -pushmetrics.url . "+
|
||||
`For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to -pushmetrics.url`)
|
||||
pushInterval = flag.Duration("pushmetrics.interval", 10*time.Second, "Interval for pushing metrics to every -pushmetrics.url")
|
||||
pushExtraLabel = flagutil.NewArrayString("pushmetrics.extraLabel", "Optional labels to add to metrics pushed to every -pushmetrics.url . "+
|
||||
`For example, -pushmetrics.extraLabel='instance="foo"' adds instance="foo" label to all the metrics pushed to every -pushmetrics.url`)
|
||||
pushHeader = flagutil.NewArrayString("pushmetrics.header", "Optional HTTP request header to send to every -pushmetrics.url . "+
|
||||
"For example, -pushmetrics.header='Authorization: Basic foobar' adds 'Authorization: Basic foobar' header to every request to every -pushmetrics.url")
|
||||
disableCompression = flag.Bool("pushmetrics.disableCompression", false, "Whether to disable request body compression when pushing metrics to every -pushmetrics.url")
|
||||
)
|
||||
|
||||
func init() {
|
||||
|
|
@ -28,7 +32,12 @@ func init() {
|
|||
func Init() {
|
||||
extraLabels := strings.Join(*pushExtraLabel, ",")
|
||||
for _, pu := range *pushURL {
|
||||
if err := metrics.InitPushExt(pu, *pushInterval, extraLabels, appmetrics.WritePrometheusMetrics); err != nil {
|
||||
opts := &metrics.PushOptions{
|
||||
ExtraLabels: extraLabels,
|
||||
Headers: *pushHeader,
|
||||
DisableCompression: *disableCompression,
|
||||
}
|
||||
if err := metrics.InitPushExtWithOptions(context.Background(), pu, *pushInterval, appmetrics.WritePrometheusMetrics, opts); err != nil {
|
||||
logger.Fatalf("cannot initialize pushmetrics: %s", err)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
31
vendor/github.com/VictoriaMetrics/metrics/go_metrics.go
generated
vendored
31
vendor/github.com/VictoriaMetrics/metrics/go_metrics.go
generated
vendored
|
|
@ -3,6 +3,7 @@ package metrics
|
|||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"math"
|
||||
"runtime"
|
||||
runtimemetrics "runtime/metrics"
|
||||
|
|
@ -22,6 +23,25 @@ var runtimeMetrics = [][2]string{
|
|||
{"/gc/gomemlimit:bytes", "go_memlimit_bytes"},
|
||||
}
|
||||
|
||||
var supportedRuntimeMetrics = initSupportedRuntimeMetrics(runtimeMetrics)
|
||||
|
||||
func initSupportedRuntimeMetrics(rms [][2]string) [][2]string {
|
||||
exposedMetrics := make(map[string]struct{})
|
||||
for _, d := range runtimemetrics.All() {
|
||||
exposedMetrics[d.Name] = struct{}{}
|
||||
}
|
||||
var supportedMetrics [][2]string
|
||||
for _, rm := range rms {
|
||||
metricName := rm[0]
|
||||
if _, ok := exposedMetrics[metricName]; ok {
|
||||
supportedMetrics = append(supportedMetrics, rm)
|
||||
} else {
|
||||
log.Printf("github.com/VictoriaMetrics/metrics: do not expose %s metric, since the corresponding metric %s isn't supported in the current Go runtime", rm[1], metricName)
|
||||
}
|
||||
}
|
||||
return supportedMetrics
|
||||
}
|
||||
|
||||
func writeGoMetrics(w io.Writer) {
|
||||
writeRuntimeMetrics(w)
|
||||
|
||||
|
|
@ -81,18 +101,19 @@ func writeGoMetrics(w io.Writer) {
|
|||
}
|
||||
|
||||
func writeRuntimeMetrics(w io.Writer) {
|
||||
samples := make([]runtimemetrics.Sample, len(runtimeMetrics))
|
||||
for i, rm := range runtimeMetrics {
|
||||
samples := make([]runtimemetrics.Sample, len(supportedRuntimeMetrics))
|
||||
for i, rm := range supportedRuntimeMetrics {
|
||||
samples[i].Name = rm[0]
|
||||
}
|
||||
runtimemetrics.Read(samples)
|
||||
for i, rm := range runtimeMetrics {
|
||||
for i, rm := range supportedRuntimeMetrics {
|
||||
writeRuntimeMetric(w, rm[1], &samples[i])
|
||||
}
|
||||
}
|
||||
|
||||
func writeRuntimeMetric(w io.Writer, name string, sample *runtimemetrics.Sample) {
|
||||
switch sample.Value.Kind() {
|
||||
kind := sample.Value.Kind()
|
||||
switch kind {
|
||||
case runtimemetrics.KindBad:
|
||||
panic(fmt.Errorf("BUG: unexpected runtimemetrics.KindBad for sample.Name=%q", sample.Name))
|
||||
case runtimemetrics.KindUint64:
|
||||
|
|
@ -101,6 +122,8 @@ func writeRuntimeMetric(w io.Writer, name string, sample *runtimemetrics.Sample)
|
|||
fmt.Fprintf(w, "%s %g\n", name, sample.Value.Float64())
|
||||
case runtimemetrics.KindFloat64Histogram:
|
||||
writeRuntimeHistogramMetric(w, name, sample.Value.Float64Histogram())
|
||||
default:
|
||||
panic(fmt.Errorf("unexpected metric kind=%d", kind))
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
190
vendor/github.com/VictoriaMetrics/metrics/push.go
generated
vendored
190
vendor/github.com/VictoriaMetrics/metrics/push.go
generated
vendored
|
|
@ -2,17 +2,59 @@ package metrics
|
|||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"compress/gzip"
|
||||
)
|
||||
|
||||
// PushOptions is the list of options, which may be applied to InitPushWithOptions().
|
||||
type PushOptions struct {
|
||||
// ExtraLabels is an optional comma-separated list of `label="value"` labels, which must be added to all the metrics before pushing them to pushURL.
|
||||
ExtraLabels string
|
||||
|
||||
// Headers is an optional list of HTTP headers to add to every push request to pushURL.
|
||||
//
|
||||
// Every item in the list must have the form `Header: value`. For example, `Authorization: Custom my-top-secret`.
|
||||
Headers []string
|
||||
|
||||
// Whether to disable HTTP request body compression before sending the metrics to pushURL.
|
||||
//
|
||||
// By default the compression is enabled.
|
||||
DisableCompression bool
|
||||
}
|
||||
|
||||
// InitPushWithOptions sets up periodic push for globally registered metrics to the given pushURL with the given interval.
|
||||
//
|
||||
// The periodic push is stopped when ctx is canceled.
|
||||
//
|
||||
// If pushProcessMetrics is set to true, then 'process_*' and `go_*` metrics are also pushed to pushURL.
|
||||
//
|
||||
// opts may contain additional configuration options if non-nil.
|
||||
//
|
||||
// The metrics are pushed to pushURL in Prometheus text exposition format.
|
||||
// See https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#text-based-format
|
||||
//
|
||||
// It is recommended pushing metrics to /api/v1/import/prometheus endpoint according to
|
||||
// https://docs.victoriametrics.com/#how-to-import-data-in-prometheus-exposition-format
|
||||
//
|
||||
// It is OK calling InitPushWithOptions multiple times with different pushURL -
|
||||
// in this case metrics are pushed to all the provided pushURL urls.
|
||||
func InitPushWithOptions(ctx context.Context, pushURL string, interval time.Duration, pushProcessMetrics bool, opts *PushOptions) error {
|
||||
writeMetrics := func(w io.Writer) {
|
||||
WritePrometheus(w, pushProcessMetrics)
|
||||
}
|
||||
return InitPushExtWithOptions(ctx, pushURL, interval, writeMetrics, opts)
|
||||
}
|
||||
|
||||
// InitPushProcessMetrics sets up periodic push for 'process_*' metrics to the given pushURL with the given interval.
|
||||
//
|
||||
// extraLabels may contain comma-separated list of `label="value"` labels, which will be added
|
||||
|
|
@ -38,7 +80,7 @@ func InitPushProcessMetrics(pushURL string, interval time.Duration, extraLabels
|
|||
// extraLabels may contain comma-separated list of `label="value"` labels, which will be added
|
||||
// to all the metrics before pushing them to pushURL.
|
||||
//
|
||||
// If pushProcessMetrics is set to true, then 'process_*' metrics are also pushed to pushURL.
|
||||
// If pushProcessMetrics is set to true, then 'process_*' and `go_*` metrics are also pushed to pushURL.
|
||||
//
|
||||
// The metrics are pushed to pushURL in Prometheus text exposition format.
|
||||
// See https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#text-based-format
|
||||
|
|
@ -55,12 +97,33 @@ func InitPush(pushURL string, interval time.Duration, extraLabels string, pushPr
|
|||
return InitPushExt(pushURL, interval, extraLabels, writeMetrics)
|
||||
}
|
||||
|
||||
// InitPushWithOptions sets up periodic push for metrics from s to the given pushURL with the given interval.
|
||||
//
|
||||
// The periodic push is stopped when the ctx is canceled.
|
||||
//
|
||||
// opts may contain additional configuration options if non-nil.
|
||||
//
|
||||
// The metrics are pushed to pushURL in Prometheus text exposition format.
|
||||
// See https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#text-based-format
|
||||
//
|
||||
// It is recommended pushing metrics to /api/v1/import/prometheus endpoint according to
|
||||
// https://docs.victoriametrics.com/#how-to-import-data-in-prometheus-exposition-format
|
||||
//
|
||||
// It is OK calling InitPushWithOptions multiple times with different pushURL -
|
||||
// in this case metrics are pushed to all the provided pushURL urls.
|
||||
func (s *Set) InitPushWithOptions(ctx context.Context, pushURL string, interval time.Duration, opts *PushOptions) error {
|
||||
writeMetrics := func(w io.Writer) {
|
||||
s.WritePrometheus(w)
|
||||
}
|
||||
return InitPushExtWithOptions(ctx, pushURL, interval, writeMetrics, opts)
|
||||
}
|
||||
|
||||
// InitPush sets up periodic push for metrics from s to the given pushURL with the given interval.
|
||||
//
|
||||
// extraLabels may contain comma-separated list of `label="value"` labels, which will be added
|
||||
// to all the metrics before pushing them to pushURL.
|
||||
//
|
||||
// / The metrics are pushed to pushURL in Prometheus text exposition format.
|
||||
// The metrics are pushed to pushURL in Prometheus text exposition format.
|
||||
// See https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#text-based-format
|
||||
//
|
||||
// It is recommended pushing metrics to /api/v1/import/prometheus endpoint according to
|
||||
|
|
@ -90,14 +153,33 @@ func (s *Set) InitPush(pushURL string, interval time.Duration, extraLabels strin
|
|||
// in this case metrics are pushed to all the provided pushURL urls.
|
||||
//
|
||||
// It is OK calling InitPushExt multiple times with different writeMetrics -
|
||||
// in this case all the metrics generated by writeMetrics callbacks are writte to pushURL.
|
||||
// in this case all the metrics generated by writeMetrics callbacks are written to pushURL.
|
||||
func InitPushExt(pushURL string, interval time.Duration, extraLabels string, writeMetrics func(w io.Writer)) error {
|
||||
if interval <= 0 {
|
||||
return fmt.Errorf("interval must be positive; got %s", interval)
|
||||
}
|
||||
if err := validateTags(extraLabels); err != nil {
|
||||
return fmt.Errorf("invalid extraLabels=%q: %w", extraLabels, err)
|
||||
opts := &PushOptions{
|
||||
ExtraLabels: extraLabels,
|
||||
}
|
||||
return InitPushExtWithOptions(context.Background(), pushURL, interval, writeMetrics, opts)
|
||||
}
|
||||
|
||||
// InitPushExtWithOptions sets up periodic push for metrics obtained by calling writeMetrics with the given interval.
|
||||
//
|
||||
// The writeMetrics callback must write metrics to w in Prometheus text exposition format without timestamps and trailing comments.
|
||||
// See https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#text-based-format
|
||||
//
|
||||
// The periodic push is stopped when the ctx is canceled.
|
||||
//
|
||||
// opts may contain additional configuration options if non-nil.
|
||||
//
|
||||
// It is recommended pushing metrics to /api/v1/import/prometheus endpoint according to
|
||||
// https://docs.victoriametrics.com/#how-to-import-data-in-prometheus-exposition-format
|
||||
//
|
||||
// It is OK calling InitPushExtWithOptions multiple times with different pushURL -
|
||||
// in this case metrics are pushed to all the provided pushURL urls.
|
||||
//
|
||||
// It is OK calling InitPushExtWithOptions multiple times with different writeMetrics -
|
||||
// in this case all the metrics generated by writeMetrics callbacks are written to pushURL.
|
||||
func InitPushExtWithOptions(ctx context.Context, pushURL string, interval time.Duration, writeMetrics func(w io.Writer), opts *PushOptions) error {
|
||||
// validate pushURL
|
||||
pu, err := url.Parse(pushURL)
|
||||
if err != nil {
|
||||
return fmt.Errorf("cannot parse pushURL=%q: %w", pushURL, err)
|
||||
|
|
@ -108,22 +190,66 @@ func InitPushExt(pushURL string, interval time.Duration, extraLabels string, wri
|
|||
if pu.Host == "" {
|
||||
return fmt.Errorf("missing host in pushURL=%q", pushURL)
|
||||
}
|
||||
pushURLRedacted := pu.Redacted()
|
||||
c := &http.Client{
|
||||
Timeout: interval,
|
||||
|
||||
// validate interval
|
||||
if interval <= 0 {
|
||||
return fmt.Errorf("interval must be positive; got %s", interval)
|
||||
}
|
||||
|
||||
// validate ExtraLabels
|
||||
var extraLabels string
|
||||
if opts != nil {
|
||||
extraLabels = opts.ExtraLabels
|
||||
}
|
||||
if err := validateTags(extraLabels); err != nil {
|
||||
return fmt.Errorf("invalid extraLabels=%q: %w", extraLabels, err)
|
||||
}
|
||||
|
||||
// validate Headers
|
||||
headers := make(http.Header)
|
||||
if opts != nil {
|
||||
for _, h := range opts.Headers {
|
||||
n := strings.IndexByte(h, ':')
|
||||
if n < 0 {
|
||||
return fmt.Errorf("missing `:` delimiter in the header %q", h)
|
||||
}
|
||||
name := strings.TrimSpace(h[:n])
|
||||
value := strings.TrimSpace(h[n+1:])
|
||||
headers.Add(name, value)
|
||||
}
|
||||
}
|
||||
|
||||
// validate DisableCompression
|
||||
disableCompression := false
|
||||
if opts != nil {
|
||||
disableCompression = opts.DisableCompression
|
||||
}
|
||||
|
||||
// Initialize metrics for the given pushURL
|
||||
pushURLRedacted := pu.Redacted()
|
||||
pushesTotal := pushMetrics.GetOrCreateCounter(fmt.Sprintf(`metrics_push_total{url=%q}`, pushURLRedacted))
|
||||
pushErrorsTotal := pushMetrics.GetOrCreateCounter(fmt.Sprintf(`metrics_push_errors_total{url=%q}`, pushURLRedacted))
|
||||
bytesPushedTotal := pushMetrics.GetOrCreateCounter(fmt.Sprintf(`metrics_push_bytes_pushed_total{url=%q}`, pushURLRedacted))
|
||||
pushDuration := pushMetrics.GetOrCreateHistogram(fmt.Sprintf(`metrics_push_duration_seconds{url=%q}`, pushURLRedacted))
|
||||
pushBlockSize := pushMetrics.GetOrCreateHistogram(fmt.Sprintf(`metrics_push_block_size_bytes{url=%q}`, pushURLRedacted))
|
||||
pushMetrics.GetOrCreateFloatCounter(fmt.Sprintf(`metrics_push_interval_seconds{url=%q}`, pushURLRedacted)).Set(interval.Seconds())
|
||||
|
||||
c := &http.Client{
|
||||
Timeout: interval,
|
||||
}
|
||||
go func() {
|
||||
ticker := time.NewTicker(interval)
|
||||
var bb bytes.Buffer
|
||||
var tmpBuf []byte
|
||||
zw := gzip.NewWriter(&bb)
|
||||
for range ticker.C {
|
||||
stopCh := ctx.Done()
|
||||
for {
|
||||
select {
|
||||
case <-ticker.C:
|
||||
case <-stopCh:
|
||||
return
|
||||
}
|
||||
|
||||
bb.Reset()
|
||||
writeMetrics(&bb)
|
||||
if len(extraLabels) > 0 {
|
||||
|
|
@ -133,31 +259,47 @@ func InitPushExt(pushURL string, interval time.Duration, extraLabels string, wri
|
|||
panic(fmt.Errorf("BUG: cannot write %d bytes to bytes.Buffer: %s", len(tmpBuf), err))
|
||||
}
|
||||
}
|
||||
tmpBuf = append(tmpBuf[:0], bb.Bytes()...)
|
||||
bb.Reset()
|
||||
zw.Reset(&bb)
|
||||
if _, err := zw.Write(tmpBuf); err != nil {
|
||||
panic(fmt.Errorf("BUG: cannot write %d bytes to gzip writer: %s", len(tmpBuf), err))
|
||||
}
|
||||
if err := zw.Close(); err != nil {
|
||||
panic(fmt.Errorf("BUG: cannot flush metrics to gzip writer: %s", err))
|
||||
if !disableCompression {
|
||||
tmpBuf = append(tmpBuf[:0], bb.Bytes()...)
|
||||
bb.Reset()
|
||||
zw.Reset(&bb)
|
||||
if _, err := zw.Write(tmpBuf); err != nil {
|
||||
panic(fmt.Errorf("BUG: cannot write %d bytes to gzip writer: %s", len(tmpBuf), err))
|
||||
}
|
||||
if err := zw.Close(); err != nil {
|
||||
panic(fmt.Errorf("BUG: cannot flush metrics to gzip writer: %s", err))
|
||||
}
|
||||
}
|
||||
pushesTotal.Inc()
|
||||
blockLen := bb.Len()
|
||||
bytesPushedTotal.Add(blockLen)
|
||||
pushBlockSize.Update(float64(blockLen))
|
||||
req, err := http.NewRequest("GET", pushURL, &bb)
|
||||
req, err := http.NewRequestWithContext(ctx, "GET", pushURL, &bb)
|
||||
if err != nil {
|
||||
panic(fmt.Errorf("BUG: metrics.push: cannot initialize request for metrics push to %q: %w", pushURLRedacted, err))
|
||||
}
|
||||
|
||||
// Set the needed headers
|
||||
for name, values := range headers {
|
||||
for _, value := range values {
|
||||
req.Header.Add(name, value)
|
||||
}
|
||||
}
|
||||
req.Header.Set("Content-Type", "text/plain")
|
||||
req.Header.Set("Content-Encoding", "gzip")
|
||||
|
||||
if !disableCompression {
|
||||
req.Header.Set("Content-Encoding", "gzip")
|
||||
}
|
||||
|
||||
// Perform the request
|
||||
startTime := time.Now()
|
||||
resp, err := c.Do(req)
|
||||
pushDuration.UpdateDuration(startTime)
|
||||
if err != nil {
|
||||
log.Printf("ERROR: metrics.push: cannot push metrics to %q: %s", pushURLRedacted, err)
|
||||
pushErrorsTotal.Inc()
|
||||
if !errors.Is(err, context.Canceled) {
|
||||
log.Printf("ERROR: metrics.push: cannot push metrics to %q: %s", pushURLRedacted, err)
|
||||
pushErrorsTotal.Inc()
|
||||
}
|
||||
continue
|
||||
}
|
||||
if resp.StatusCode/100 != 2 {
|
||||
|
|
|
|||
4
vendor/modules.txt
vendored
4
vendor/modules.txt
vendored
|
|
@ -97,8 +97,8 @@ github.com/VictoriaMetrics/fastcache
|
|||
github.com/VictoriaMetrics/fasthttp
|
||||
github.com/VictoriaMetrics/fasthttp/fasthttputil
|
||||
github.com/VictoriaMetrics/fasthttp/stackless
|
||||
# github.com/VictoriaMetrics/metrics v1.25.3
|
||||
## explicit; go 1.20
|
||||
# github.com/VictoriaMetrics/metrics v1.26.1
|
||||
## explicit; go 1.16
|
||||
github.com/VictoriaMetrics/metrics
|
||||
# github.com/VictoriaMetrics/metricsql v0.70.0
|
||||
## explicit; go 1.13
|
||||
|
|
|
|||
Loading…
Reference in a new issue