github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-11 14:53:49 +00:00
Code Issues Projects Releases Packages Wiki Activity

Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files

Browse source
This commit is contained in:
Aliaksandr Valialkin 2023-02-11 14:43:19 -08:00
commit c2b724d3ab
No known key found for this signature in database
GPG key ID: A72BEC6CD3D0DED1
2 changed files with 110 additions and 23 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

127
docs/sd_configs.md
View file

@ -33,7 +33,7 @@ Please file feature requests to [our issue tracker](https://github.com/VictoriaM
## azure_sd_configs
Azure SD configuration allows retrieving scrape targets from [Microsoft Azure](https://azure.microsoft.com/en-us/) VMs.
Azure SD configuration discovers scrape targets from [Microsoft Azure](https://azure.microsoft.com/en-us/) VMs.
Configuration example:
@ -66,12 +66,17 @@ scrape_configs:
# resource_group: "..."
# port is an optional port to scrape metrics from.
# Port 80 is used by default.
# port: ...
# Additional HTTP API client options can be specified here.
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<private_ip>:<port>`, where `<private_ip>` is the machine's private IP and the `<port>` is the `port`
option specified in the `azure_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_azure_machine_id`: the machine ID
@ -154,6 +159,10 @@ scrape_configs:
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<service_or_node_addr>:<service_port>`, where `<service_or_node_addr>` is the service address. If the service address is empty,
then the node address is used instead. The `<service_port>` is the service port.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_consul_address`: the address of the target
@ -195,6 +204,9 @@ scrape_configs:
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<public_ip>:<port>`, where `<public_ip>` is a public ipv4 address of the droplet, while `<port>` is the port specified in the `digitalocean_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_digitalocean_droplet_id`: the id of the droplet
@ -236,6 +248,10 @@ scrape_configs:
# port: ...
```
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to the `<addr>:<port>`, where `<addr>` is the discovered DNS address, while `<port>` is either the discovered port for SRV records or the port
specified in the `dns_sd_config`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_dns_name`: the record name that produced the discovered target.
@ -276,6 +292,11 @@ scrape_configs:
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip_address>:<port>`, where `<ip_address>` is the exposed ip address of the docker container, while the `<port>` is either the exposed port
of the docker container or the port specified in the `docker_sd_configs` if the docker container has no exposed ports.
If a container exposes multiple ip addresses, then multiple targets will be discovered - one per each exposed ip address.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_docker_container_id`: the id of the container
@ -333,9 +354,12 @@ One of the following roles can be configured to discover targets:
* `role: services`
The `services` role discovers all Swarm services and exposes their ports as targets.
For each published port of a service, a single target is generated. If a service has no published ports,
a target per service is created using the port parameter defined in the SD configuration.
The `services` role discovers all Swarm services.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip>:<port>`, where `<ip>` is the endpoint's virtual IP, while the `<port>` is the published port of the service.
If the service has multiple pulbished ports, then multiple targets are generated - one per each port.
If the service has no published ports, then the `<port>` is set to the `port` value obtained from `dockerswarm_sd_configs`.
Available meta labels for `role: services` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
@ -357,9 +381,12 @@ One of the following roles can be configured to discover targets:
* `role: tasks`
The `tasks` role discovers all Swarm tasks and exposes their ports as targets.
For each published port of a task, a single target is generated. If a task has no published ports,
a target per task is created using the port parameter defined in the SD configuration.
The `tasks` role discovers all Swarm tasks.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip>:<port>`, where the `<ip>` is the node IP, while the `<port>` is the published port of the task.
If the task has multiple published ports, then multiple targets are generated - one per each port.
If the task has no published ports, then the `<port>` is set to the `port` value obtained from `dockerswarm_sd_configs`.
Available meta labels for `role: tasks` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
@ -397,6 +424,9 @@ One of the following roles can be configured to discover targets:
The `nodes` role is used to discover Swarm nodes.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip>:<port>`, where `<ip>` is the node IP, while the `<port>` is the `port` value obtained from the `dockerswarm_sd_configs`.
Available meta labels for `role: nodes` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_dockerswarm_node_address`: the address of the node
@ -468,6 +498,10 @@ scrape_configs:
# values: ["...", "..."]
```
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<instance_ip>:<port>`, where `<instance_ip>` is the private IP of the instance, while the `<port>` is set to the `port` value
obtaine from `ec2_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_ec2_ami`: the EC2 Amazon Machine Image
@ -510,6 +544,10 @@ scrape_configs:
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<instance_host>:<instance_port>`, where `<instance_host>` is the discovered instance hostname, while the `<instance_port>`
is the discovered instance port. If the instance has no port, then port 80 is used.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_eureka_app_name`: the name of the app
@ -577,6 +615,9 @@ Files must contain a list of static configs in one of the following formats:
...
```
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to one of the `target` value specified in the target files.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_filepath`: the filepath from which the target was extracted
@ -623,6 +664,10 @@ Credentials are discovered by looking in the following places, preferring the fi
2. a JSON file in the well-known path `$HOME/.config/gcloud/application_default_credentials.json`
3. fetched from the GCE metadata server
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<iface_ip>:<port>`, where `<iface_ip>` is private IP of the discovered instance, while `<port>` is the `port` value
specified in the `gce_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_gce_instance_id`: the numeric id of the instance
@ -675,6 +720,9 @@ The service at `url` must return JSON response in the following format:
The `url` is queried periodically with the interval specified in `-promscrape.httpSDCheckInterval` command-line flag.
Discovery errors are tracked in `promscrape_discovery_http_errors_total` metric.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to one of the targets returned by the http service.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_url`: the URL from which the target was extracted
@ -737,9 +785,12 @@ One of the following `role` types can be configured to discover targets:
* `role: node`
The `role: node` discovers one target per cluster node with the address defaulting to the Kubelet's HTTP port.
The target address defaults to the first existing address of the Kubernetes node object in the address type order
of `NodeInternalIP`, `NodeExternalIP`, `NodeLegacyHostIP` and `NodeHostName`.
The `role: node` discovers one target per cluster node.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip>:<port>`, where `<ip>` is to the first existing address of the Kubernetes node object in the address type order
of `NodeInternalIP`, `NodeExternalIP`, `NodeLegacyHostIP` and `NodeHostName`,
while `<port>` is the kubelet's port on the given node.
Available meta labels for `role: node` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
@ -755,7 +806,13 @@ One of the following `role` types can be configured to discover targets:
* `role: service`
The `role: service` discovers a target for each service port for each service.
The `role: service` discovers Kubernetes services.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<service_name>.<namespace>:<port>`, where `<service_name>` is the service name, `<namespace>` is the service namespace
and `<port>` is the service port.
If the service has multiple ports, then multiple targets are discovered for the service - one per each port.
This is generally useful for blackbox monitoring of a service. The target address will be set to the Kubernetes DNS name
of the service and respective service port.
@ -776,9 +833,12 @@ One of the following `role` types can be configured to discover targets:
* `role: pod`
The `role: pod` discovers all pods and exposes their containers as targets. For each declared port of a container
a single target is generated. If a container has no specified ports, a port-free target per container is created
for manually adding a port via relabeling.
The `role: pod` discovers all pods and exposes their containers as targets.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<ip>:<port>`, where `<ip>` is pod IP, while `<port>` is the exposed container port.
If the pod has multiple container ports, then multiple targets are generated for the pod - one per each exposed container port.
If the pod has no exposed container ports, then the `__address__` for pod target is set to the pod IP.
Available meta labels for `role: pod` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
@ -806,7 +866,11 @@ One of the following `role` types can be configured to discover targets:
* `role: endpoints`
The `role: endpoints` discovers targets from listed endpoints of a service. For each endpoint address one target is discovered per port.
The `role: endpoints` discovers targets from listed endpoints of a service.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<addr>:<port>`, where `<addr>` is the endpoint address, while `<port>` is the endpoint port.
If the endpoint has multiple ports, then a single target per each port is generated.
If the endpoint is backed by a pod, all additional container ports of the pod, not bound to an endpoint port, are discovered as targets as well.
Available meta labels for `role: endpoints` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
@ -831,9 +895,12 @@ One of the following `role` types can be configured to discover targets:
* `role: endpointslice`
The `role: endpointslice` discovers targets from existing endpointslices. For each endpoint address referenced in the endpointslice
object one target is discovered. If the endpoint is backed by a pod, all additional container ports of the pod,
not bound to an endpoint port, are discovered as targets as well.
The `role: endpointslice` discovers targets from existing endpointslices.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<addr>:<port>`, where `<addr>` is the endpoint address, while `<port>` is the endpoint port.
If the endpoint has multiple ports, then a single target per each port is generated.
If the endpoint is backed by a pod, all additional container ports of the pod, not bound to an endpoint port, are discovered as targets as well.
Available meta labels for `role: endpointslice` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
@ -857,8 +924,13 @@ One of the following `role` types can be configured to discover targets:
* `role: ingress`
The `role: ingress` discovers a target for each path of each ingress. This is generally useful for blackbox monitoring of an ingress.
The target address will be set to the host specified in the ingress spec.
The `role: ingress` discovers a target for each path of each ingress.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to the host obtained from ingress spec.
If the ingress has multiple specs with multiple hosts, then a target per each host is created.
This is generally useful for blackbox monitoring of an ingress.
Available meta labels for `role: ingress` during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
@ -911,6 +983,9 @@ scrape_configs:
# See https://docs.victoriametrics.com/sd_configs.html#http-api-client-options
```
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<addr>:<port>`, where `<addr>` is the service address, while `<port>` is the service port.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_nomad_address`: the address of the target
@ -988,6 +1063,7 @@ scrape_configs:
# all_tenants: ...
# port is an optional port to scrape metrics from.
# Port 80 is used by default.
# port: ...
# availability is the availability of the endpoint to connect to.
@ -1006,7 +1082,9 @@ One of the following `role` types can be configured to discover targets:
* `role: hypervisor`
The `role: hypervisor` discovers one target per Nova hypervisor node.
The target address defaults to the `host_ip` attribute of the hypervisor.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<host>:<port>`, where `<host>` is the discovered node IP, while `<port>` is the port specified in the `openstack_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
@ -1020,7 +1098,9 @@ One of the following `role` types can be configured to discover targets:
* `role: instance`
The `role: instance` discovers one target per network interface of Nova instance.
The target address defaults to the private IP address of the network interface.
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to `<host>:<port>`, where `<host>` is the private IP address of the discovered instance, while `<port>` is the port specified in the `openstack_sd_configs`.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
@ -1119,6 +1199,9 @@ scrape_configs:
replacement: "$1:9100"
```
Each discovered target has an [__address__](https://docs.victoriametrics.com/relabeling.html#how-to-modify-scrape-urls-in-targets) label set
to the FQDN of the discovered instance.
The following meta labels are available on discovered targets during [relabeling](https://docs.victoriametrics.com/vmagent.html#relabeling):
* `__meta_yandexcloud_instance_name`: the name of instance

6
lib/promscrape/discovery/openstack/api.go
View file

@ -69,6 +69,10 @@ func getAPIConfig(sdc *SDConfig, baseDir string) (*apiConfig, error) {
}
func newAPIConfig(sdc *SDConfig, baseDir string) (*apiConfig, error) {
port := sdc.Port
if port == 0 {
port = 80
}
cfg := &apiConfig{
client: &http.Client{
Transport: &http.Transport{
@ -78,7 +82,7 @@ func newAPIConfig(sdc *SDConfig, baseDir string) (*apiConfig, error) {
availability: sdc.Availability,
region: sdc.Region,
allTenants: sdc.AllTenants,
port: sdc.Port,
port: port,
}
if sdc.TLSConfig != nil {
opts := &promauth.Options{