github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-01 14:47:38 +00:00
Code Issues Projects Releases Packages Wiki Activity
VictoriaMetrics/docs/sd_configs.md
Aliaksandr Valialkin 9fbd45a22f
lib/promscrape/discovery/kuma: follow-up for 317fef95f9 
- Do not generate __meta_server label, since it is unavailable in Prometheus.
- Add a link to https://docs.victoriametrics.com/sd_configs.html#kuma_sd_configs to docs/CHANGELOG.md,
  so users could click it and read the docs without the need to search the corresponding docs.
- Remove kumaTarget struct, since it is easier generating labels for discovered targets
  directly from the response returned by Kuma. This simplifies the code.
- Store the generated labels for discovered targets inside atomic.Value. This allows reading them
  from concurrent goroutines without the need to use mutex.
- Use synchronouse requests to Kuma instead of long polling, since there is a little sense
  in the long polling when the Kuma server may return 304 Not Modified response every -promscrape.kumaSDCheckInterval.
- Remove -promscrape.kuma.waitTime command-line flag, since it is no longer needed when long polling isn't used.
- Set default value for -promscrape.kumaSDCheckInterval to 30s in order to be consistent with Prometheus.
- Remove unnecessary indirections for string literals, which are used only once, in order to improve code readability.
- Remove unused fields from discoveryRequest and discoveryResponse.
- Update tests.
- Document why fetch_timeout and refresh_interval options are missing in kuma_sd_config.
- Add docs to discoveryutils.RequestCallback and discoveryutils.ResponseCallback,
  since these are public types.

Side notes: it is weird that Prometheus implementation for kuma_sd_configs sets `instance` label,
since usually this label is set by the Prometheus itself to __address__ after the relabeling phase.
See https://www.robustperception.io/life-of-a-label/

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3389

See https://github.com/prometheus/prometheus/issues/7919
and https://github.com/prometheus/prometheus/pull/8844
as a reference implementation in Prometheus
2023-02-22 17:51:51 -08:00

1491 lines
72 KiB
Markdown
Raw Blame History

---
sort: 24
---
# Prometheus service discovery
# Supported service discovery configs
[vmagent](https://docs.victoriametrics.com/vmagent.html) and [single-node VictoriaMetrics](https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter) supports the following Prometheus-compatible service discovery options for Prometheus-compatible scrape targets in the file pointed by `-promscrape.config` command-line flag.
* `azure_sd_configs` is for scraping the targets registered in [Azure Cloud](https://azure.microsoft.com/en-us/). See [these docs](#azure_sd_configs).
* `consul_sd_configs` is for discovering and scraping targets registered in [Consul](https://www.consul.io/). See [these docs](#consul_sd_configs).
* `digitalocean_sd_configs` is for discovering and scraping targerts registered in [DigitalOcean](https://www.digitalocean.com/). See [these docs](#digitalocean_sd_configs).
* `dns_sd_configs` is for discovering and scraping targets from [DNS](https://it.wikipedia.org/wiki/Domain_Name_System) records (SRV, A and AAAA). See [these docs](#dns_sd_configs).
* `docker_sd_configs` is for discovering and scraping [Docker](https://www.docker.com/) targets. See [these docs](#docker_sd_configs).
* `dockerswarm_sd_configs` is for discovering and scraping [Docker Swarm](https://docs.docker.com/engine/swarm/) targets. See [these docs](#dockerswarm_sd_configs).
* `ec2_sd_configs` is for discovering and scraping [Amazon EC2](https://aws.amazon.com/ec2/) targets. See [these docs](#ec2_sd_configs).
* `eureka_sd_configs` is for discovering and scraping targets registered in [Netflix Eureka](https://github.com/Netflix/eureka). See [these docs](#eureka_sd_configs).
* `file_sd_configs` is for scraping targets defined in external files (aka file-based service discovery). See [these docs](#file_sd_configs).
* `gce_sd_configs` is for discovering and scraping [Google Compute Engine](https://cloud.google.com/compute) targets. See [these docs](#gce_sd_configs).
* `http_sd_configs` is for discovering and scraping targerts provided by external http-based service discovery. See [these docs](#http_sd_configs).
* `kubernetes_sd_configs` is for discovering and scraping [Kubernetes](https://kubernetes.io/) targets. See [these docs](#kubernetes_sd_configs).
* `kuma_sd_configs` is for discovering and scraping [Kuma](https://kuma.io) targets. See [these docs](#kuma_sd_configs).
* `nomad_sd_configs` is for discovering and scraping targets registered in [HashiCorp Nomad](https://www.nomadproject.io/). See [these docs](#nomad_sd_configs).
* `openstack_sd_configs` is for discovering and scraping OpenStack targets. See [these docs](#openstack_sd_configs).
* `static_configs` is for scraping statically defined targets. See [these docs](#static_configs).
* `yandexcloud_sd_configs` is for discoverying and scraping [Yandex Cloud](https://cloud.yandex.com/en/) targets. See [these docs](#yandexcloud_sd_configs).
Note that the `refresh_interval` option isn't supported for these scrape configs. Use the corresponding `-promscrape.*CheckInterval`
command-line flag instead. For example, `-promscrape.consulSDCheckInterval=60s` sets `refresh_interval` for all the `consul_sd_configs`
entries to 60s. Run `vmagent -help` or `victoria-metrics -help` in order to see default values for the `-promscrape.*CheckInterval` flags.
Please file feature requests to [our issue tracker](https://github.com/VictoriaMetrics/VictoriaMetrics/issues) if you need other service discovery mechanisms to be supported by VictoriaMetrics and `vmagent`.
## azure_sd_configs
Azure SD configuration discovers scrape targets from [Microsoft Azure](https://azure.microsoft.com/en-us/) VMs.
Configuration example:
```yaml
scrape_configs:
- job_name: azure
azure_sd_configs:
# subscription_id is a mandatory subscription ID.
- subscription_id: "..."
# environment is an optional Azure environment. By default "AzurePublicCloud" is used.
# environment: "..."
# authentication_method is an optional authentication method, either OAuth or ManagedIdentity.
# See https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview
# By default OAuth is used.
# authentication_method: "..."
# tenant_id is an optional tenant ID. Only required with authentication_method OAuth.
# tenant_id: "..."
# client_id is an optional client ID. Only required with authentication_method OAuth.
# client_id: "..."
# client_secret is an optional client secret. Only required with authentication_method OAuth.
# client_secret: "..."
# resource_group is an optional resource group name. Limits discovery to this resource group.
# resource_group: "..."
# port is an optional port to scrape metrics from.
# Port 80 is used by default.
# port: ...
# Additional HTTP API client options can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<private_ip>:<port>`, where `<private_ip>` is the machine's private IP and the `<port>` is the `port`
option specified in the `azure_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_azure_machine_id`: the machine ID
* `__meta_azure_machine_location`: the location the machine runs in
* `__meta_azure_machine_name`: the machine name
* `__meta_azure_machine_computer_name`: the machine computer name
* `__meta_azure_machine_os_type`: the machine operating system
* `__meta_azure_machine_private_ip`: the machine's private IP
* `__meta_azure_machine_public_ip`: the machine's public IP if it exists
* `__meta_azure_machine_resource_group`: the machine's resource group
* `__meta_azure_machine_scale_set`: the name of the scale set which the vm is part of (this value is only set if you are using a scale set)
* `__meta_azure_machine_size`: the machine size
* `__meta_azure_machine_tag_<tagname>`: each tag value of the machine
* `__meta_azure_subscription_id`: the subscription ID
* `__meta_azure_tenant_id`: the tenant ID
## consul_sd_configs
Consul SD configuration allows retrieving scrape targets from [Consul's Catalog API](https://www.consul.io/api-docs/catalog).
Configuration example:
```yaml
scrape_configs:
- job_name: consul
consul_sd_configs:
# server is an optional Consul server to connect to. By default localhost:8500 is used
- server: "localhost:8500"
# token is an optional Consul API token.
# If the token isn't specified, then it is read from a file pointed by CONSUL_HTTP_TOKEN_FILE
# environment var or from the CONSUL_HTTP_TOKEN environment var.
# token: "..."
# datacenter is an optional Consul API datacenter.
# If the datacenter isn't specified, then it is read from Consul server.
# See https://www.consul.io/api-docs/agent#read-configuration
# datacenter: "..."
# namespace is an optional Consul namespace.
# See https://developer.hashicorp.com/consul/docs/enterprise/namespaces
# If the namespace isn't specified, then it is read from CONSUL_NAMESPACE environment var.
# namespace: "..."
# partition is an optional Consul partition.
# See https://developer.hashicorp.com/consul/docs/enterprise/admin-partitions
# If partition isn't specified, then the default partition is used.
# partition: "..."
# scheme is an optional scheme (http or https) to use for connecting to Consul server.
# By default http scheme is used.
# scheme: "..."
# services is an optional list of services for which targets are retrieved.
# If omitted, all services are scraped.
# See https://www.consul.io/api-docs/catalog#list-nodes-for-service .
# services: ["...", "..."]
# tags is an optional list of tags used to filter nodes for a given service.
# Services must contain all tags in the list.
# tags: ["...", "..."]
# node_meta is an optional node metadata key/value pairs to filter nodes for a given service.
# node_meta:
# "...": "..."
# tag_separator is an optional string by which Consul tags are joined into the __meta_consul_tags label.
# By default "," is used as a tag separator.
# Individual tags are also available via __meta_consul_tag_<tagname> labels - see below.
# tag_separator: "..."
# allow_stale is an optional config, which allows stale Consul results.
# See https://www.consul.io/api/features/consistency.html
# Reduce load on Consul if set to true. By default is is set to true.
# allow_stale: ...
# Additional HTTP API client options can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<service_or_node_addr>:<service_port>`, where `<service_or_node_addr>` is the service address. If the service address is empty,
then the node address is used instead. The `<service_port>` is the service port.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_consul_address`: the address of the target
* `__meta_consul_dc`: the datacenter name for the target
* `__meta_consul_health`: the health status of the service
* `__meta_consul_metadata_<key>`: each node metadata key value of the target
* `__meta_consul_namespace`: namespace of the service - see [namespace docs](https://developer.hashicorp.com/consul/docs/enterprise/namespaces)
* `__meta_consul_node`: the node name defined for the target
* `__meta_consul_partition`: partition of the service - see [partition docs](https://developer.hashicorp.com/consul/docs/enterprise/admin-partitions)
* `__meta_consul_service_address`: the service address of the target
* `__meta_consul_service_id`: the service ID of the target
* `__meta_consul_service_metadata_<key>`: each service metadata key value of the target
* `__meta_consul_service_port`: the service port of the target
* `__meta_consul_service`: the name of the service the target belongs to
* `__meta_consul_tagged_address_<key>`: each node tagged address key value of the target
* `__meta_consul_tag_<tagname>`: the value for the given <tagname> tag of the target
* `__meta_consul_tagpresent_<tagname>`: "true" for every <tagname> tag of the target
* `__meta_consul_tags`: the list of tags of the target joined by the `tag_separator`
## digitalocean_sd_configs
DigitalOcean SD configuration allows retrieving scrape targets from [DigitalOcean's Droplets API](https://docs.digitalocean.com/reference/api/api-reference/#tag/Droplets).
Configuration example:
```yaml
scrape_configs:
- job_name: digitalocean
digitalocean_sd_configs:
# server is an optional DigitalOcean API server to query.
# By default https://api.digitalocean.com is used.
- server: "https://api.digitalocean.com"
# port is an optional port to scrape metrics from. By default port 80 is used.
# port: ...
# Additional HTTP API client options can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<public_ip>:<port>`, where `<public_ip>` is a public ipv4 address of the droplet, while `<port>` is the port specified in the `digitalocean_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_digitalocean_droplet_id`: the id of the droplet
* `__meta_digitalocean_droplet_name`: the name of the droplet
* `__meta_digitalocean_image`: the slug of the droplet's image
* `__meta_digitalocean_image_name`: the display name of the droplet's image
* `__meta_digitalocean_private_ipv4`: the private IPv4 of the droplet
* `__meta_digitalocean_public_ipv4`: the public IPv4 of the droplet
* `__meta_digitalocean_public_ipv6`: the public IPv6 of the droplet
* `__meta_digitalocean_region`: the region of the droplet
* `__meta_digitalocean_size`: the size of the droplet
* `__meta_digitalocean_status`: the status of the droplet
* `__meta_digitalocean_features`: the comma-separated list of features of the droplet
* `__meta_digitalocean_tags`: the comma-separated list of tags of the droplet
* `__meta_digitalocean_vpc`: the id of the droplet's VPC
## dns_sd_configs
DNS-based service discovery allows retrieving scrape tragets from the specified DNS domain names.
These specified names are periodically queried to discover a list of targets with the interval
configured via `-promscrape.dnsSDCheckInterval` command-line flag.
Configuration example:
```yaml
scrape_configs:
- job_name: dns
dns_sd_configs:
# names must contain a list of DNS names to query.
- names: ["...", "..."]
# type is an optional type of DNS query to perform.
# Supported values are: SRV, A, AAAA or MX.
# By default SRV is used.
# type: ...
# port is a port number to use if the query type is not SRV.
# port: ...
```
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to the `<addr>:<port>`, where `<addr>` is the discovered DNS address, while `<port>` is either the discovered port for SRV records or the port
specified in the `dns_sd_config`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_dns_name`: the record name that produced the discovered target.
* `__meta_dns_srv_record_target`: the target field of the SRV record
* `__meta_dns_srv_record_port`: the port field of the SRV record
* `__meta_dns_mx_record_target`: the target field of the MX record.
## docker_sd_configs
Docker SD configuration allows retrieving scrape targets from [Docker Engine](https://docs.docker.com/engine/) hosts.
Configuration example:
```yaml
scrape_configs:
- job_name: docker
docker_sd_configs:
# host must contain the address of the Docker daemon.
- host: "..."
# port is an optional port to scrape metrics from.
# By default port 80 is used.
# port: ...
# host_networking_host is an optional host to use if the container is in host networking mode.
# By default localhost is used.
# host_networking_host: "..."
# filters is an optional filters to limit the discovery process to a subset of available resources.
# See https://docs.docker.com/engine/api/v1.40/#operation/ContainerList
# filters:
# - name: "..."
# values: ["...", "..."]
# Additional HTTP API client options can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip_address>:<port>`, where `<ip_address>` is the exposed ip address of the docker container, while the `<port>` is either the exposed port
of the docker container or the port specified in the `docker_sd_configs` if the docker container has no exposed ports.
If a container exposes multiple ip addresses, then multiple targets will be discovered - one per each exposed ip address.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_docker_container_id`: the id of the container
* `__meta_docker_container_name`: the name of the container
* `__meta_docker_container_network_mode`: the network mode of the container
* `__meta_docker_container_label_<labelname>`: each label of the container
* `__meta_docker_network_id`: the ID of the network
* `__meta_docker_network_name`: the name of the network
* `__meta_docker_network_ingress`: whether the network is ingress
* `__meta_docker_network_internal`: whether the network is internal
* `__meta_docker_network_label_<labelname>`: each label of the network
* `__meta_docker_network_scope`: the scope of the network
* `__meta_docker_network_ip`: the IP of the container in this network
* `__meta_docker_port_private`: the port on the container
* `__meta_docker_port_public`: the external port if a port-mapping exists
* `__meta_docker_port_public_ip`: the public IP if a port-mapping exists
## dockerswarm_sd_configs
Docker Swarm SD configuration allows retrieving scrape targets from [Docker Swarm engine](https://docs.docker.com/engine/swarm/).
Configuration example:
```yaml
scrape_configs:
- job_name: dockerswarm
dockerswarm_sd_configs:
# host must contain the address of the Docker daemon.
- host: "..."
# role must contain `services`, `tasks` or `nodes` as described below.
role: ...
# port is an optional port to scrape metrics from, when `role` is nodes, and for discovered
# tasks and services that don't have published ports.
# By default port 80 is used.
# port: ...
# filters is an optional filters to limit the discovery process to a subset of available resources.
# The available filters are listed in the upstream documentation:
# Services: https://docs.docker.com/engine/api/v1.40/#operation/ServiceList
# Tasks: https://docs.docker.com/engine/api/v1.40/#operation/TaskList
# Nodes: https://docs.docker.com/engine/api/v1.40/#operation/NodeList
# filters:
# - name: "..."
# values: ["...", "..."]
# Additional HTTP API client options can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
One of the following roles can be configured to discover targets:
* `role: services`
The `services` role discovers all Swarm services.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip>:<port>`, where `<ip>` is the endpoint's virtual IP, while the `<port>` is the published port of the service.
If the service has multiple pulbished ports, then multiple targets are generated - one per each port.
If the service has no published ports, then the `<port>` is set to the `port` value obtained from `dockerswarm_sd_configs`.
Available meta labels for `role: services` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_dockerswarm_service_id`: the id of the service
* `__meta_dockerswarm_service_name`: the name of the service
* `__meta_dockerswarm_service_mode`: the mode of the service
* `__meta_dockerswarm_service_endpoint_port_name`: the name of the endpoint port, if available
* `__meta_dockerswarm_service_endpoint_port_publish_mode`: the publish mode of the endpoint port
* `__meta_dockerswarm_service_label_<labelname>`: each label of the service
* `__meta_dockerswarm_service_task_container_hostname`: the container hostname of the target, if available
* `__meta_dockerswarm_service_task_container_image`: the container image of the target
* `__meta_dockerswarm_service_updating_status`: the status of the service, if available
* `__meta_dockerswarm_network_id`: the ID of the network
* `__meta_dockerswarm_network_name`: the name of the network
* `__meta_dockerswarm_network_ingress`: whether the network is ingress
* `__meta_dockerswarm_network_internal`: whether the network is internal
* `__meta_dockerswarm_network_label_<labelname>`: each label of the network
* `__meta_dockerswarm_network_scope`: the scope of the network
* `role: tasks`
The `tasks` role discovers all Swarm tasks.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip>:<port>`, where the `<ip>` is the node IP, while the `<port>` is the published port of the task.
If the task has multiple published ports, then multiple targets are generated - one per each port.
If the task has no published ports, then the `<port>` is set to the `port` value obtained from `dockerswarm_sd_configs`.
Available meta labels for `role: tasks` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_dockerswarm_container_label_<labelname>`: each label of the container
* `__meta_dockerswarm_task_id`: the id of the task
* `__meta_dockerswarm_task_container_id`: the container id of the task
* `__meta_dockerswarm_task_desired_state`: the desired state of the task
* `__meta_dockerswarm_task_slot`: the slot of the task
* `__meta_dockerswarm_task_state`: the state of the task
* `__meta_dockerswarm_task_port_publish_mode`: the publish mode of the task port
* `__meta_dockerswarm_service_id`: the id of the service
* `__meta_dockerswarm_service_name`: the name of the service
* `__meta_dockerswarm_service_mode`: the mode of the service
* `__meta_dockerswarm_service_label_<labelname>`: each label of the service
* `__meta_dockerswarm_network_id`: the ID of the network
* `__meta_dockerswarm_network_name`: the name of the network
* `__meta_dockerswarm_network_ingress`: whether the network is ingress
* `__meta_dockerswarm_network_internal`: whether the network is internal
* `__meta_dockerswarm_network_label_<labelname>`: each label of the network
* `__meta_dockerswarm_network_label`: each label of the network
* `__meta_dockerswarm_network_scope`: the scope of the network
* `__meta_dockerswarm_node_id`: the ID of the node
* `__meta_dockerswarm_node_hostname`: the hostname of the node
* `__meta_dockerswarm_node_address`: the address of the node
* `__meta_dockerswarm_node_availability`: the availability of the node
* `__meta_dockerswarm_node_label_<labelname>`: each label of the node
* `__meta_dockerswarm_node_platform_architecture`: the architecture of the node
* `__meta_dockerswarm_node_platform_os`: the operating system of the node
* `__meta_dockerswarm_node_role`: the role of the node
* `__meta_dockerswarm_node_status`: the status of the node
The `__meta_dockerswarm_network_*` meta labels are not populated for ports which are published with `mode=host`.
* `role: nodes`
The `nodes` role is used to discover Swarm nodes.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip>:<port>`, where `<ip>` is the node IP, while the `<port>` is the `port` value obtained from the `dockerswarm_sd_configs`.
Available meta labels for `role: nodes` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_dockerswarm_node_address`: the address of the node
* `__meta_dockerswarm_node_availability`: the availability of the node
* `__meta_dockerswarm_node_engine_version`: the version of the node engine
* `__meta_dockerswarm_node_hostname`: the hostname of the node
* `__meta_dockerswarm_node_id`: the ID of the node
* `__meta_dockerswarm_node_label_<labelname>`: each label of the node
* `__meta_dockerswarm_node_manager_address`: the address of the manager component of the node
* `__meta_dockerswarm_node_manager_leader`: the leadership status of the manager component of the node (true or false)
* `__meta_dockerswarm_node_manager_reachability`: the reachability of the manager component of the node
* `__meta_dockerswarm_node_platform_architecture`: the architecture of the node
* `__meta_dockerswarm_node_platform_os`: the operating system of the node
* `__meta_dockerswarm_node_role`: the role of the node
* `__meta_dockerswarm_node_status`: the status of the node
## ec2_sd_configs
EC2 SD configuration allows retrieving scrape targets from [AWS EC2 instances](https://aws.amazon.com/ec2/).
Configuration example:
```yaml
scrape_configs:
- job_name: ec2
ec2_sd_configs:
# region is an optional config for AWS region.
# By default the region from the instance metadata is used.
- region: "..."
# endpoint is an optional custom AWS API endpoint to use.
# By default the standard endpoint for the given region is used.
# endpoint: "..."
# sts_endpoint is an optional custom STS API endpoint to use.
# By default the standard endpoint for the given region is used.
# sts_endpoint: "..."
# access_key is an optional AWS API access key.
# By default the access key is loaded from AWS_ACCESS_KEY_ID environment var.
# access_key: "..."
# secret_key is an optional AWS API secret key.
# By default the secret key is loaded from AWS_SECRET_ACCESS_KEY environment var.
# secret_key: "..."
# role_arn is an optional AWS Role ARN, an alternative to using AWS API keys.
# role_arn: "..."
# port is an optional port to scrape metrics from.
# By default port 80 is used.
# port: ...
# filters is an optional filters for the instance list.
# Available filter criteria can be found here:
# https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html
# Filter API documentation: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_Filter.html
# filters:
# - name: "..."
# values: ["...", "..."]
# az_filters is an optional filters for the availability zones list.
# Available filter criteria can be found here:
# https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeAvailabilityZones.html
# Filter API documentation: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_Filter.html
# az_filters:
# - name: "..."
# values: ["...", "..."]
```
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<instance_ip>:<port>`, where `<instance_ip>` is the private IP of the instance, while the `<port>` is set to the `port` value
obtaine from `ec2_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_ec2_ami`: the EC2 Amazon Machine Image
* `__meta_ec2_architecture`: the architecture of the instance
* `__meta_ec2_availability_zone`: the availability zone in which the instance is running
* `__meta_ec2_availability_zone_id`: the availability zone ID in which the instance is running (requires ec2:DescribeAvailabilityZones)
* `__meta_ec2_instance_id`: the EC2 instance ID
* `__meta_ec2_instance_lifecycle`: the lifecycle of the EC2 instance, set only for 'spot' or 'scheduled' instances, absent otherwise
* `__meta_ec2_instance_state`: the state of the EC2 instance
* `__meta_ec2_instance_type`: the type of the EC2 instance
* `__meta_ec2_ipv6_addresses`: comma separated list of IPv6 addresses assigned to the instance's network interfaces, if present
* `__meta_ec2_owner_id`: the ID of the AWS account that owns the EC2 instance
* `__meta_ec2_platform`: the Operating System platform, set to 'windows' on Windows servers, absent otherwise
* `__meta_ec2_primary_subnet_id`: the subnet ID of the primary network interface, if available
* `__meta_ec2_private_dns_name`: the private DNS name of the instance, if available
* `__meta_ec2_private_ip`: the private IP address of the instance, if present
* `__meta_ec2_public_dns_name`: the public DNS name of the instance, if available
* `__meta_ec2_public_ip`: the public IP address of the instance, if available
* `__meta_ec2_region`: EC2 region for the discovered instance
* `__meta_ec2_subnet_id`: comma separated list of subnets IDs in which the instance is running, if available
* `__meta_ec2_tag_<tagkey>`: each tag value of the instance
* `__meta_ec2_vpc_id`: the ID of the VPC in which the instance is running, if available
## eureka_sd_configs
Eureka SD configuration allows retrieving scrape targets using the [Eureka REST API](https://github.com/Netflix/eureka/wiki/Eureka-REST-operations).
Configuration example:
```yaml
scrape_configs:
- job_name: eureka
eureka_sd_configs:
# server is an optional URL to connect to the Eureka server.
# By default The http://localhost:8080/eureka/v2 is used.
- server: "..."
# Additional HTTP API client options can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<instance_host>:<instance_port>`, where `<instance_host>` is the discovered instance hostname, while the `<instance_port>`
is the discovered instance port. If the instance has no port, then port 80 is used.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_eureka_app_name`: the name of the app
* `__meta_eureka_app_instance_id`: the ID of the app instance
* `__meta_eureka_app_instance_hostname`: the hostname of the instance
* `__meta_eureka_app_instance_homepage_url`: the homepage url of the app instance
* `__meta_eureka_app_instance_statuspage_url`: the status page url of the app instance
* `__meta_eureka_app_instance_healthcheck_url`: the health check url of the app instance
* `__meta_eureka_app_instance_ip_addr`: the IP address of the app instance
* `__meta_eureka_app_instance_vip_address`: the VIP address of the app instance
* `__meta_eureka_app_instance_secure_vip_address`: the secure VIP address of the app instance
* `__meta_eureka_app_instance_status`: the status of the app instance
* `__meta_eureka_app_instance_port`: the port of the app instance
* `__meta_eureka_app_instance_port_enabled`: the port enabled of the app instance
* `__meta_eureka_app_instance_secure_port`: the secure port address of the app instance
* `__meta_eureka_app_instance_secure_port_enabled`: the secure port of the app instance
* `__meta_eureka_app_instance_country_id`: the country ID of the app instance
* `__meta_eureka_app_instance_metadata_<metadataname>`: app instance metadata
* `__meta_eureka_app_instance_datacenterinfo_name`: the datacenter name of the app instance
* `__meta_eureka_app_instance_datacenterinfo_metadata_<metadataname>`: the datacenter metadata
## file_sd_configs
File-based service discovery reads a set of files with lists of targets to scrape.
Scrape targets are automatically updated when the underlying files are changed with the interval
Configuration example:
```yaml
scrape_configs:
- job_name: file
file_sd_configs:
# files must contain a list of file patterns for files with scrape targets.
# The last path segment can contain `*`, which matches any number of chars in file name.
- files:
- "my/path/*.yaml"
- "another/path.json"
```
Files must contain a list of static configs in one of the following formats:
* JSON:
```json
[
{
"targets": ["<host>", ... ],
"labels": {
"<labelname>": "<labelvalue>",
...,
}
},
...
]
```
* YAML:
```yaml
- targets: ["<host>", ... ]
labels:
<labelname>: <labelvalue>
...
...
```
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to one of the `target` value specified in the target files.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_filepath`: the filepath from which the target was extracted
See the [list of integrations](https://prometheus.io/docs/operating/integrations/#file-service-discovery) with `file_sd_configs`.
## gce_sd_configs
GCE SD configuration allows retrieving scrape targets from [GCP GCE instances](https://cloud.google.com/compute).
Configuration example:
```yaml
scrape_configs:
- job_name: gce
gce_sd_configs:
# project is an optional GCE project where targets must be discovered.
# By default the local project is used.
- project: "..."
# zone is an optional zone where targets must be discovered.
# By default the local zone is used.
# If zone equals to '*', then targets in all the zones for the given project are discovered.
# The zone may contain a list of zones: zone["us-east1-a", "us-east1-b"]
# zone: "..."
# filter is an optional filter for the instance list.
# See https://cloud.google.com/compute/docs/reference/latest/instances/list
# filter: "..."
# port is an optional port to scrape metrics from.
# By default port 80 is used.
# port: ...
# tag_separator is an optional separator for tags in `__meta_gce_tags` label.
# By default "," is used.
# tag_separator: "..."
```
Credentials are discovered by looking in the following places, preferring the first location found:
1. a JSON file specified by the `GOOGLE_APPLICATION_CREDENTIALS` environment variable
2. a JSON file in the well-known path `$HOME/.config/gcloud/application_default_credentials.json`
3. fetched from the GCE metadata server
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<iface_ip>:<port>`, where `<iface_ip>` is private IP of the discovered instance, while `<port>` is the `port` value
specified in the `gce_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_gce_instance_id`: the numeric id of the instance
* `__meta_gce_instance_name`: the name of the instance
* `__meta_gce_label_<labelname>`: each GCE label of the instance
* `__meta_gce_machine_type`: full or partial URL of the machine type of the instance
* `__meta_gce_metadata_<name>`: each metadata item of the instance
* `__meta_gce_network`: the network URL of the instance
* `__meta_gce_private_ip`: the private IP address of the instance
* `__meta_gce_interface_ipv4_<name>`: IPv4 address of each named interface
* `__meta_gce_project`: the GCP project in which the instance is running
* `__meta_gce_public_ip`: the public IP address of the instance, if present
* `__meta_gce_subnetwork`: the subnetwork URL of the instance
* `__meta_gce_tags`: list of instance tags separated by tag_separator
* `__meta_gce_zone`: the GCE zone URL in which the instance is running
## http_sd_configs
HTTP-based service discovery fetches targets from the specified `url`.
Configuration example:
```yaml
scrape_configs:
- job_name: http
http_sd_configs:
# url must contain the URL from which the targets are fetched.
- url: "http://..."
# Additional HTTP API client options can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
The service at `url` must return JSON response in the following format:
```json
[
{
"targets": [ "<host>", ... ],
"labels": {
"<labelname>": "<labelvalue>",
...
}
},
...
]
```
The `url` is queried periodically with the interval specified in `-promscrape.httpSDCheckInterval` command-line flag.
Discovery errors are tracked in `promscrape_discovery_http_errors_total` metric.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to one of the targets returned by the http service.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_url`: the URL from which the target was extracted
## kubernetes_sd_configs
Kubernetes SD configuration allows retrieving scrape targets from [Kubernetes REST API](https://kubernetes.io/docs/reference/using-api/).
Configuration example:
```yaml
scrape_configs:
- job_name: kubernetes
kubernetes_sd_configs:
# role must contain the Kubernetes role of entities that should be discovered.
# It must have one of the following values:
# endpoints, endpointslice, service, pod, node or ingress.
# See docs below about each particular role.
- role: "..."
# api_server is an optional url for Kubernetes API server.
# By default it is read from /var/run/secrets/kubernetes.io/serviceaccount/
# api_server: "..."
# kubeconfig_file is an optional path to a kubeconfig file.
# Note that api_server and kubeconfig_file are mutually exclusive.
# kubeconfig_file: "..."
# namespaces is an optional namespace for service discovery.
# By default all namespaces are used.
# If own_namespace is set to true, then the current namespace is used for service discovery.
# namespaces:
# own_namespace: <boolean>
# names: ["...", "..."]
# selects is an optional label and field selectors to limit the discovery process to a subset of available resources.
# See https://kubernetes.io/docs/concepts/overview/working-with-objects/field-selectors/
# and https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
# The `role: endpoints` supports pod, service and endpoints selectors.
# The `role: pod` supports node selectors when configured with `attach_metadata: {node: true}`.
# Other roles only support selectors matching the role itself (e.g. node role can only contain node selectors).
# selectors:
# - role: "..."
# label: "..."
# field: "..."
# attach_metadata is an optional metadata to attach to discovered targets.
# When `node` is set to true, then node metadata is attached to discovered targets.
# Valid for roles: pod, endpoints, endpointslice.
# attach_metadata:
# node: <boolean>
# Additional HTTP API client options can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
One of the following `role` types can be configured to discover targets:
* `role: node`
The `role: node` discovers one target per cluster node.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip>:<port>`, where `<ip>` is to the first existing address of the Kubernetes node object in the address type order
of `NodeInternalIP`, `NodeExternalIP`, `NodeLegacyHostIP` and `NodeHostName`,
while `<port>` is the kubelet's port on the given node.
Available meta labels for `role: node` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_kubernetes_node_name`: The name of the node object.
* `__meta_kubernetes_node_provider_id`: The cloud provider's name for the node object.
* `__meta_kubernetes_node_label_<labelname>`: Each label from the node object.
* `__meta_kubernetes_node_labelpresent_<labelname>`: "true" for each label from the node object.
* `__meta_kubernetes_node_annotation_<annotationname>`: Each annotation from the node object.
* `__meta_kubernetes_node_annotationpresent_<annotationname>`: "true" for each annotation from the node object.
* `__meta_kubernetes_node_address_<address_type>`: The first address for each node address type, if it exists.
In addition, the `instance` label for the node will be set to the node name as retrieved from the API server.
* `role: service`
The `role: service` discovers Kubernetes services.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<service_name>.<namespace>:<port>`, where `<service_name>` is the service name, `<namespace>` is the service namespace
and `<port>` is the service port.
If the service has multiple ports, then multiple targets are discovered for the service - one per each port.
This is generally useful for blackbox monitoring of a service. The target address will be set to the Kubernetes DNS name
of the service and respective service port.
Available meta labels for `role: service` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_kubernetes_namespace`: The namespace of the service object.
* `__meta_kubernetes_service_annotation_<annotationname>`: Each annotation from the service object.
* `__meta_kubernetes_service_annotationpresent_<annotationname>`: "true" for each annotation of the service object.
* `__meta_kubernetes_service_cluster_ip`: The cluster IP address of the service. (Does not apply to services of type ExternalName)
* `__meta_kubernetes_service_external_name`: The DNS name of the service. (Applies to services of type ExternalName)
* `__meta_kubernetes_service_label_<labelname>`: Each label from the service object.
* `__meta_kubernetes_service_labelpresent_<labelname>`: "true" for each label of the service object.
* `__meta_kubernetes_service_name`: The name of the service object.
* `__meta_kubernetes_service_port_name`: Name of the service port for the target.
* `__meta_kubernetes_service_port_number`: Service port number for the target.
* `__meta_kubernetes_service_port_protocol`: Protocol of the service port for the target.
* `__meta_kubernetes_service_type`: The type of the service.
* `role: pod`
The `role: pod` discovers all pods and exposes their containers as targets.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip>:<port>`, where `<ip>` is pod IP, while `<port>` is the exposed container port.
If the pod has multiple container ports, then multiple targets are generated for the pod - one per each exposed container port.
If the pod has no exposed container ports, then the `__address__` for pod target is set to the pod IP.
Available meta labels for `role: pod` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_kubernetes_namespace`: The namespace of the pod object.
* `__meta_kubernetes_pod_name`: The name of the pod object.
* `__meta_kubernetes_pod_ip`: The pod IP of the pod object.
* `__meta_kubernetes_pod_label_<labelname>`: Each label from the pod object.
* `__meta_kubernetes_pod_labelpresent_<labelname>`: "true" for each label from the pod object.
* `__meta_kubernetes_pod_annotation_<annotationname>`: Each annotation from the pod object.
* `__meta_kubernetes_pod_annotationpresent_<annotationname>`: "true" for each annotation from the pod object.
* `__meta_kubernetes_pod_container_id`: ID of the container in the form `<type>://<container_id>`.
* `__meta_kubernetes_pod_container_image`: Container image the target address points to.
* `__meta_kubernetes_pod_container_init`: "true" if the container is an InitContainer.
* `__meta_kubernetes_pod_container_name`: Name of the container the target address points to.
* `__meta_kubernetes_pod_container_port_name`: Name of the container port.
* `__meta_kubernetes_pod_container_port_number`: Number of the container port.
* `__meta_kubernetes_pod_container_port_protocol`: Protocol of the container port.
* `__meta_kubernetes_pod_ready`: Set to true or false for the pod's ready state.
* `__meta_kubernetes_pod_phase`: Set to Pending, Running, Succeeded, Failed or Unknown in the lifecycle.
* `__meta_kubernetes_pod_node_name`: The name of the node the pod is scheduled onto.
* `__meta_kubernetes_pod_host_ip`: The current host IP of the pod object.
* `__meta_kubernetes_pod_uid`: The UID of the pod object.
* `__meta_kubernetes_pod_controller_kind`: Object kind of the pod controller.
* `__meta_kubernetes_pod_controller_name`: Name of the pod controller.
* `role: endpoints`
The `role: endpoints` discovers targets from listed endpoints of a service.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<addr>:<port>`, where `<addr>` is the endpoint address, while `<port>` is the endpoint port.
If the endpoint has multiple ports, then a single target per each port is generated.
If the endpoint is backed by a pod, all additional container ports of the pod, not bound to an endpoint port, are discovered as targets as well.
Available meta labels for `role: endpoints` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_kubernetes_namespace`: The namespace of the endpoints object.
* `__meta_kubernetes_endpoints_name`: The names of the endpoints object.
* `__meta_kubernetes_endpoints_label_<labelname>`: Each label from the endpoints object.
* `__meta_kubernetes_endpoints_labelpresent_<labelname>`: "true" for each label from the endpoints object.
For all targets discovered directly from the endpoints list (those not additionally inferred from underlying pods), the following labels are attached:
* `__meta_kubernetes_endpoint_hostname`: Hostname of the endpoint.
* `__meta_kubernetes_endpoint_node_name`: Name of the node hosting the endpoint.
* `__meta_kubernetes_endpoint_ready`: Set to true or false for the endpoint's ready state.
* `__meta_kubernetes_endpoint_port_name`: Name of the endpoint port.
* `__meta_kubernetes_endpoint_port_protocol`: Protocol of the endpoint port.
* `__meta_kubernetes_endpoint_address_target_kind`: Kind of the endpoint address target.
* `__meta_kubernetes_endpoint_address_target_name`: Name of the endpoint address target.
If the endpoints belong to a service, all labels of the `role: service` are attached.
For all targets backed by a pod, all labels of the `role: pod` are attached.
* `role: endpointslice`
The `role: endpointslice` discovers targets from existing endpointslices.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<addr>:<port>`, where `<addr>` is the endpoint address, while `<port>` is the endpoint port.
If the endpoint has multiple ports, then a single target per each port is generated.
If the endpoint is backed by a pod, all additional container ports of the pod, not bound to an endpoint port, are discovered as targets as well.
Available meta labels for `role: endpointslice` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_kubernetes_namespace`: The namespace of the endpoints object.
* `__meta_kubernetes_endpointslice_name`: The name of endpointslice object.
For all targets discovered directly from the endpointslice list (those not additionally inferred from underlying pods), the following labels are attached:
* `__meta_kubernetes_endpointslice_address_target_kind`: Kind of the referenced object.
* `__meta_kubernetes_endpointslice_address_target_name`: Name of referenced object.
* `__meta_kubernetes_endpointslice_address_type`: The ip protocol family of the address of the target.
* `__meta_kubernetes_endpointslice_endpoint_conditions_ready`: Set to true or false for the referenced endpoint's ready state.
* `__meta_kubernetes_endpointslice_endpoint_topology_kubernetes_io_hostname`: Name of the node hosting the referenced endpoint.
* `__meta_kubernetes_endpointslice_endpoint_topology_present_kubernetes_io_hostname`: Flag that shows if the referenced object has a kubernetes.io/hostname annotation.
* `__meta_kubernetes_endpointslice_port`: Port of the referenced endpoint.
* `__meta_kubernetes_endpointslice_port_name`: Named port of the referenced endpoint.
* `__meta_kubernetes_endpointslice_port_protocol`: Protocol of the referenced endpoint.
If the endpoints belong to a service, all labels of the `role: service` are attached.
For all targets backed by a pod, all labels of the `role: pod` are attached.
* `role: ingress`
The `role: ingress` discovers a target for each path of each ingress.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to the host obtained from ingress spec.
If the ingress has multiple specs with multiple hosts, then a target per each host is created.
This is generally useful for blackbox monitoring of an ingress.
Available meta labels for `role: ingress` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_kubernetes_namespace`: The namespace of the ingress object.
* `__meta_kubernetes_ingress_name`: The name of the ingress object.
* `__meta_kubernetes_ingress_label_<labelname>`: Each label from the ingress object.
* `__meta_kubernetes_ingress_labelpresent_<labelname>`: "true" for each label from the ingress object.
* `__meta_kubernetes_ingress_annotation_<annotationname>`: Each annotation from the ingress object.
* `__meta_kubernetes_ingress_annotationpresent_<annotationname>`: "true" for each annotation from the ingress object.
* `__meta_kubernetes_ingress_class_name`: Class name from ingress spec, if present.
* `__meta_kubernetes_ingress_scheme`: Protocol scheme of ingress, https if TLS config is set. Defaults to http.
* `__meta_kubernetes_ingress_path`: Path from ingress spec. Defaults to `/`.
## kuma_sd_configs
Kuma service discovery config allows to fetch targets from the specified control plane `server` of [Kuma Service Mesh](https://kuma.io).
It discovers "monitoring assignments" based on Kuma Dataplane Proxies,
via the [MADS (Monitoring Assignment Discovery Service)](https://kuma.io/docs/2.1.x/policies/traffic-metrics/#traffic-metrics)
[xDS RESP API](http://envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol).
Configuration example:
```yaml
scrape_configs:
- job_name: kuma
kuma_sd_configs:
# server must contain the URL of Kuma Control Plane's MADS xDS server.
- server: "http://localhost:5676"
# Additional HTTP API client options can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
The `server` is queried periodically with the interval specified in `-promscrape.kumaSDCheckInterval` command-line flag.
Discovery errors are tracked in `promscrape_discovery_kuma_errors_total` metric.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to one of the targets returned by the http service.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_kuma_mesh`: the name of the mesh
* `__meta_kuma_dataplane`: the name of the proxy
* `__meta_kuma_service`: the name of the service associated with the proxy
* `__meta_kuma_label_<label_name>`: each label of target given from Kuma Control Plane
## nomad_sd_configs
Nomad SD configuration allows retrieving scrape targets from [HashiCorp Nomad Services](https://www.hashicorp.com/blog/nomad-service-discovery).
Configuration example:
```yaml
scrape_configs:
- job_name: nomad
nomad_sd_configs:
# server is an optional Nomad server to connect to.
# If the server isn't specified, then it is read from NOMAD_ADDR environment var.
# If the NOMAD_ADDR environment var isn't set, then localhost:4646 is used.
- server: "localhost:4646"
# namespace is an optional Nomad namespace.
# If the namespace isn't specified, then it is read from NOMAD_NAMESPACE environment var.
# namespace: "..."
# region is an optional Nomad region.
# If the region isn't specified, then it is read from NOMAD_REGION environment var.
# If NOMAD_REGION environment var isn't set, then "global" region is used
# region: "..."
# tag_separator is an optional string by which Nomad tags are joined into the __meta_nomad_tags label.
# By default "," is used as a tag separator.
# Individual tags are also available via __meta_nomad_tag_<tagname> labels - see below.
# tag_separator: "..."
# allow_stale is an optional config, which allows stale Nomad results.
# See https://developer.hashicorp.com/nomad/api-docs#consistency-modes
# Reduces load on Nomad if set to true. By default is is set to true.
# allow_stale: ...
# Additional HTTP API client options can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<addr>:<port>`, where `<addr>` is the service address, while `<port>` is the service port.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_nomad_address`: the address of the target
* `__meta_nomad_dc`: the datacenter name for the target
* `__meta_nomad_namespace`: namespace of the service
* `__meta_nomad_node_id`: the node ID defined for the target
* `__meta_nomad_service`: the name of the service the target belongs to
* `__meta_nomad_service_address`: the service address of the target
* `__meta_nomad_service_alloc_id`: the AllocID of the target service
* `__meta_nomad_service_id`: the ID of the target service
* `__meta_nomad_service_job_id`: the JobID of the target service
* `__meta_nomad_service_port`: the service port of the target
* `__meta_nomad_tag_<tagname>`: the value for the given <tagname> tag of the target
* `__meta_nomad_tagpresent_<tagname>`: "true" for every <tagname> tag of the target
* `__meta_nomad_tags`: the list of tags of the target joined by the `tag_separator`
## openstack_sd_configs
OpenStack SD configuration allows retrieving scrape targets from [OpenStack Nova](https://docs.openstack.org/nova/latest/) instances.
[OpenStack identity API v3](https://docs.openstack.org/api-ref/identity/v3/) is supported only.
Configuration example:
```yaml
scrape_configs:
- job_name: openstack
openstack_sd_configs:
# role must contain either `hypervisor` or `instance`.
# See docs below for details.
- role: "..."
# region must contain OpenStack region for targets' discovery.
region: "..."
# identity_endpoint is an optional HTTP Identity API endpoint.
# By default it is read from OS_AUTH_URL environment variable.
# identity_endpoint: "..."
# username is an optional username to query Identity API.
# By default it is read from OS_USERNAME environment variable.
# username: "..."
# userid is an optional userid to query Identity API.
# By default it is read from OS_USERID environment variable.
# userid: "..."
# password is an optional password to query Identity API.
# By default it is read from OS_PASSWORD environment variable.
# password: "..."
# At most one of domain_id and domain_name must be provided.
# By default they are read from OS_DOMAIN_NAME and OS_DOMAIN_ID environment variables.
# domain_name: "..."
# domain_id: "..."
# project_name and project_id are optional project name and project id.
# By default is is read from OS_PROJECT_NAME and OS_PROJECT_ID environment variables.
# If these vars are emtpy, then the options are read
# from OS_TENANT_NAME and OS_TENANT_ID environment variables.
# project_name: "..."
# project_id: "..."
# By default these fields are read from OS_APPLICATION_CREDENTIAL_NAME
# and OS_APPLICATION_CREDENTIAL_ID environment variables
# application_credential_name: "..."
# application_credential_id: "..."
# By default this field is read from OS_APPLICATION_CREDENTIAL_SECRET
# application_credential_secret: "..."
# all_tenants can be set to true if all instances in all projects must be discovered.
# It is only relevant for the 'role: instance' and usually requires admin permissions.
# all_tenants: ...
# port is an optional port to scrape metrics from.
# Port 80 is used by default.
# port: ...
# availability is the availability of the endpoint to connect to.
# Must be one of public, admin or internal.
# By default it is set to public
# availability: "..."
# tls_config is an optional tls config.
# See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config
# tls_config:
# ...
```
One of the following `role` types can be configured to discover targets:
* `role: hypervisor`
The `role: hypervisor` discovers one target per Nova hypervisor node.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<host>:<port>`, where `<host>` is the discovered node IP, while `<port>` is the port specified in the `openstack_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_openstack_hypervisor_host_ip`: the hypervisor node's IP address.
* `__meta_openstack_hypervisor_hostname`: the hypervisor node's name.
* `__meta_openstack_hypervisor_id`: the hypervisor node's ID.
* `__meta_openstack_hypervisor_state`: the hypervisor node's state.
* `__meta_openstack_hypervisor_status`: the hypervisor node's status.
* `__meta_openstack_hypervisor_type`: the hypervisor node's type.
* `role: instance`
The `role: instance` discovers one target per network interface of Nova instance.
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<host>:<port>`, where `<host>` is the private IP address of the discovered instance, while `<port>` is the port specified in the `openstack_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_openstack_address_pool`: the pool of the private IP.
* `__meta_openstack_instance_flavor`: the flavor of the OpenStack instance.
* `__meta_openstack_instance_id`: the OpenStack instance ID.
* `__meta_openstack_instance_name`: the OpenStack instance name.
* `__meta_openstack_instance_status`: the status of the OpenStack instance.
* `__meta_openstack_private_ip`: the private IP of the OpenStack instance.
* `__meta_openstack_project_id`: the project (tenant) owning this instance.
* `__meta_openstack_public_ip`: the public IP of the OpenStack instance.
* `__meta_openstack_tag_<tagkey>`: each tag value of the instance.
* `__meta_openstack_user_id`: the user account owning the tenant.
## static_configs
A static config allows specifying a list of targets and a common label set for them.
Configuration example:
```yaml
scrape_configs:
- job_name: static
static_configs:
# targets must contain a list of `host:port` targets to scrape.
# The `http://host:port/metrics` endpoint is scraped per each configured traget then.
# The `http` scheme can be changed to `https` by setting it via `scheme` field at `scrape_config` level.
# The `/metrics` path can be changed to arbitrary path via `metrics_path` field at `scrape_config` level.
# See https://docs.victoriametrics.com/sd_configs.html#scrape_configs .
#
# Alternatively the scheme and path can be changed via `relabel_configs` section at `scrape_config` level.
# See https://docs.victoriametrics.com/vmagent.html#relabeling .
#
# It is also possible specifying full target urls here, e.g. "http://host:port/metrics/path?query_args"
- targets:
- "vmsingle1:8428"
- "vmsingleN:8428"
# labels is an optional labels to add to all the targets.
# labels:
# <labelname1>: "<labelvalue1>"
# ...
# <labelnameN>: "<labelvalueN>"
```
## yandexcloud_sd_configs
[Yandex Cloud](https://cloud.yandex.com/en/) SD configurations allow retrieving scrape targets from accessible folders.
Configuration example:
```yaml
scrape_configs:
- job_name: yandexcloud
yandexcloud_sd_configs:
# service is a mandatory option for yandexcloud service discovery
# currently only "compute" service is supported
- service: compute
# api_endpoint is an optional API endpoint for service discovery
# The https://api.cloud.yandex.net endpoint is used by default.
# api_endpoint: "https://api.cloud.yandex.net"
# yandex_passport_oauth_token is an optional OAuth token
# for querying yandexcloud API. See https://cloud.yandex.com/en-ru/docs/iam/concepts/authorization/oauth-token
# yandex_passport_oauth_token: "..."
# tls_config is an optional tls config.
# See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config
# tls_config:
# ...
```
Yandex Cloud SD support both user [OAuth token](https://cloud.yandex.com/en-ru/docs/iam/concepts/authorization/oauth-token)
and [instance service account](https://cloud.yandex.com/en-ru/docs/compute/operations/vm-connect/auth-inside-vm) if `yandex_passport_oauth_token` is omitted:
```yaml
scrape_configs:
- job_name: YC_with_oauth
yandexcloud_sd_configs:
- service: compute
yandex_passport_oauth_token: "AQAAAAAsfasah<...>7E10SaotuL0"
relabel_configs:
- source_labels: [__meta_yandexcloud_instance_public_ip_0]
target_label: __address__
replacement: "$1:9100"
- job_name: YC_with_Instance_service_account
yandexcloud_sd_configs:
- service: compute
relabel_configs:
- source_labels: [__meta_yandexcloud_instance_private_ip_0]
target_label: __address__
replacement: "$1:9100"
```
Each discovered target has an [`__address__`](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to the FQDN of the discovered instance.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_yandexcloud_instance_name`: the name of instance
* `__meta_yandexcloud_instance_id`: the id of instance
* `__meta_yandexcloud_instance_fqdn`: generated FQDN for instance
* `__meta_yandexcloud_instance_status`: the status of instance
* `__meta_yandexcloud_instance_platform_id`: instance platform ID (i.e. "standard-v3")
* `__meta_yandexcloud_instance_resources_cores`: instance vCPU cores
* `__meta_yandexcloud_instance_resources_core_fraction`: instance core fraction
* `__meta_yandexcloud_instance_resources_memory`: instance memory
* `__meta_yandexcloud_folder_id`: instance folder ID
* `__meta_yandexcloud_instance_label_<label name>`: each label from instance
* `__meta_yandexcloud_instance_private_ip_<interface index>`: private IP of <interface index> network interface
* `__meta_yandexcloud_instance_public_ip_<interface index>`: public (NAT) IP of <interface index> network interface
* `__meta_yandexcloud_instance_private_dns_<record number>`: if configured DNS records for private IP
* `__meta_yandexcloud_instance_public_dns_<record number>`: if configured DNS records for public IP
## scrape_configs
The `scrape_configs` section at file pointed by `-promscrape.config` command-line flag can contain [supported service discovery options](#supported-service-discovery-configs).
Additionally, it can contain the following options:
```yaml
scrape_configs:
# job_name must contain value for `job` label, which is added
# to all the metrics collected from the configured and discovered scrape targets.
# See https://prometheus.io/docs/concepts/jobs_instances/ .
- job_name: "..."
# scrape_interval is an optional interval to scrape targets.
# By default the scrape_interval sepcified in `global` section is used.
# See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#configuration-file
# If `global` section doesn't contain the `scrape_interval` option,
# then one minute interval is used.
# Example values:
# - "30s" - 30 seconds
# - "2m" - 2 minutes
# The scrape_interval can be set on a per-target basis by specifying `__scrape_interval__`
# label during target relabeling phase.
# See https://docs.victoriametrics.com/vmagent.html#relabeling
# scrape_interval: <duration>
# scrape_timeout is an optional timeout when scraping the targets.
# By default the scrape_timeout specified in `global` section is used.
# See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#configuration-file
# If `global` section doesn't contain the `scrape_timeout` option,
# then 10 seconds interval is used.
# Example values:
# - "30s" - 30 seconds
# - "2m" - 2 minutes
# The `scrape_timeout` cannot exceed the `scrape_interval`.
# The scrape_timeout can be set on a per-target basis by specifying `__scrape_timeout__`
# label during target relabeling phase.
# See https://docs.victoriametrics.com/vmagent.html#relabeling
# scrape_timeout: <duration>
# metrics_path is the path to fetch metrics from targets.
# By default metrics are fetched from "/metrics" path.
# metrics_path: "..."
# honor_labels controls how to handle conflicts between labels that are
# already present in scraped data and labels that would be attached
# server-side ("job" and "instance" labels, manually configured target
# labels, labels generated by service discovery, etc.
#
# If honor_labels is set to "true", label conflicts are resolved by keeping label
# values from the scraped data and ignoring the conflicting server-side labels.
#
# If honor_labels is set to "false", label conflicts are resolved by renaming
# conflicting labels in the scraped data to "exported_<original-label>" (for
# example "exported_instance", "exported_job") and then attaching server-side
# labels.
#
# Setting honor_labels to "true" is useful for use cases such as federation and
# scraping the Pushgateway, where all labels specified in the target should be
# preserved.
#
# By default honor_labels is set to false for security and consistency reasons.
# honor_labels: <boolean>
# honor_timestamps controls whether to respect the timestamps present in scraped data.
#
# If honor_timestamps is set to "true", the timestamps of the metrics exposed
# by the target will be used.
#
# If honor_timestamps is set to "false", the timestamps of the metrics exposed
# by the target will be ignored.
#
# By default honor_timestamps is set to true.
# honor_timestamps: <boolean>
# scheme configures the protocol scheme used for requests.
# Supported values: http and https.
# By default http is used.
# scheme: "..."
# Optional query arg parameters to add to scrape url.
# params:
# "param_name1": ["value1", ..., "valueN"]
# ...
# "param_nameM": ["valueM1", ..., "valueMN"]
# relabel_configs is an optional relabeling configurations
# for the specified and discovered scrape targets.
# See https://docs.victoriametrics.com/vmagent.html#relabeling
# relabel_configs:
# - <relabel_config> ...
# metric_relabel_configs is an optional relabeling configs
# for the collected metrics from active scrape targets.
# See https://docs.victoriametrics.com/vmagent.html#relabeling
# metric_relabel_configs:
# - <relabel_config> ...
# sample_limit is an optional per-scrape limit on number
# of scraped samples that will be accepted.
# If more than this number of samples are present after metric relabeling
# the entire scrape will be treated as failed.
# By default the limit is disabled.
# sample_limit: <int>
# disable_compression allows disabling HTTP compression for responses received from scrape targets.
# By default scrape targets are queried with `Accept-Encoding: gzip` http request header,
# so targets could send compressed responses in order to save network bandwidth.
# See https://docs.victoriametrics.com/vmagent.html#scrape_config-enhancements
# disable_compression: <boolean>
# disable_keepalive allows disabling HTTP keep-alive when scraping targets.
# By default HTTP keep-alive is enabled, so TCP connections to scrape targets
# could be re-used.
# See https://docs.victoriametrics.com/vmagent.html#scrape_config-enhancements
# disable_keepalive: <boolean>
# stream_parse allows enabling stream parsing mode when scraping targets.
# By default stream parsing mode is disabled for targets which return up to a few thosands samples.
# See https://docs.victoriametrics.com/vmagent.html#stream-parsing-mode .
# The stream_parse can be set on a per-target basis by specifying `__stream_parse__`
# label during target relabeling phase.
# See https://docs.victoriametrics.com/vmagent.html#relabeling
# stream_parse: <boolean>
# scrape_align_interval allows aligning scrapes to the given interval.
# Example values:
# - "5m" - align scrapes to every 5 minutes.
# - "1h" - align scrapes to every hour.
# See https://docs.victoriametrics.com/vmagent.html#scrape_config-enhancements
# scrape_align_interval: <duration>
# scrape_offset allows specifying the exact offset for scrapes.
# Example values:
# - "5m" - align scrapes to every 5 minutes.
# - "1h" - align scrapes to every hour.
# See https://docs.victoriametrics.com/vmagent.html#scrape_config-enhancements
# scrape_offset: <duration>
# series_limit is an optional limit on the number of unique time series
# a single target can expose during all the scrapes on the time window of 24h.
# By default, there is no limit on the number of exposed series.
# See https://docs.victoriametrics.com/vmagent.html#cardinality-limiter .
# The series_limit can be set on a per-target basis by specifying `__series_limit__`
# label during target relabeling phase.
# See https://docs.victoriametrics.com/vmagent.html#relabeling
# series_limit: ...
# no_stale_markers allows disabling staleness tracking.
# By default, staleness tracking is enabled for all the discovered scrape targets.
# See https://docs.victoriametrics.com/vmagent.html#prometheus-staleness-markers
# no_stale_markers: <boolean>
# Additional HTTP client options for target scraping can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
## HTTP API client options
The following additional options can be specified in the [scrape_configs](#scrape_configs)
and in the majority of [supported service discovery configs](#supported-service-discovery-configs):
```yaml
# authorization is an optional `Authorization` header configuration.
# authorization:
# type: "..." # default: Bearer
# credentials: "..."
# credentials_file: "..."
# basic_auth is an optional HTTP basic authentication configuration.
# basic_auth:
# username: "..."
# password: "..."
# password_file: "..."
# bearer_token is an optional Bearer token to send in every HTTP API request during service discovery.
# bearer_token: "..."
# bearer_token_file is an optional path to file with Bearer token to send
# in every HTTP API request during service discovery.
# The file is re-read every second, so its contents can be updated without the need to restart the service discovery.
# bearer_token_file: "..."
# oauth2 is an optional OAuth 2.0 configuration.
# See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#oauth2
# oauth2:
# ...
# tls_config is an optional TLS configuration.
# See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config
# tls_config:
# ...
# headers is an optional HTTP headers to pass with each request.
# headers:
# - "HeaderName1: HeaderValue"
# - "HeaderNameN: HeaderValueN"
# proxy_url is an optional URL for the proxy to use for HTTP API queries during service discovery.
# proxy_url: "..."
# proxy_authorization is an optional `Authorization` header config for the proxy_url.
# proxy_authorization:
# type: "..." # default: Bearer
# credentials: "..."
# credentials_file: "..."
# proxy_basic_auth is an optional HTTP basic authentication configuration for the proxy_url.
# proxy_basic_auth:
# username: "..."
# password: "..."
# password_file: "..."
# proxy_bearer_token is an optional Bearer token to send to proxy_url.
# proxy_bearer_token: "..."
# proxy_bearer_token_file is an optional path to file with Bearer token to send to proxy_url.
# The file is re-read every second, so its contents can be updated without the need to restart the service discovery.
# proxy_bearer_token_file: "..."
# proxy_oauth2 is an optional OAuth 2.0 configuration for the proxy_url.
# See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#oauth2
# proxy_oauth2:
# ...
# proxy_tls_config is an optional TLS configuration for the proxy_url.
# See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config
# proxy_tls_config:
# ...
# proxy_headers is an optional HTTP headers to pass to the proxy_url.
# proxy_headers:
# - "HeaderName1: HeaderValue"
# - "HeaderNameN: HeaderValueN"
```
Reference in a new issue View git blame Copy permalink