Added docs for telegraf proxmox and vector (#6619)

### Describe Your Changes

initial docs to implement #6618 more platforms can be added on this
branch or on future commits.

### Checklist

The following checks are **mandatory**:

- [x ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
This commit is contained in:
Mathias Palmersheim 2024-07-15 11:08:11 -05:00 committed by GitHub
parent 2fa03306c3
commit 27d03c1c98
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 332 additions and 0 deletions

View file

@ -0,0 +1,71 @@
---
title: Proxmox
weight: 1
sort: 1
menu:
docs:
identifier: "proxmox"
parent: "data-ingestion"
weight: 1
# sort: 1
aliases:
- /data-ingestion/proxmox.html
- /data-ingestion/Proxmox.html
---
# Proxmox Data Ingestion
Since Proxmox Virtual Environment(PVE) and Proxmox Backup Server(PBS) support sending data using the InfluxDB We can use the InfluxDB write support built into VictoriaMetrics
Currently PVE and PBS only support using an Authorization Token for authentication and does not support basic auth or a username and password.
## Proxmox Virtual Environment (PVE)
If want help Sending your data to Managed VictoriaMetrics check out [our blog](https://victoriametrics.com/blog/proxmox-monitoring-with-dbaas/).
1. Login to PVE as an administrator
2. Go to DataCenter > MetricServer > Add > InfluxDB
![PVE Metric Navigation](pve-nav.webp)
3. Set the parameters as follows:
- Name: VictoriaMetrics (can be changed to any string)
- Server: the hostname or IP of your VictoriaMetrics Instance
- Port: This will vary depending how you are sending data to VictoriaMetrics, but the defaults for all components are listed in the [data ingestion documentation](https://docs.victoriametrics.com/data-ingestion.html)
- Protocol: use HTTPS if you have TLS/SSL configured otherwise use HTTP
- Organization: leave empty since it doesn't get used
- Bucket: leave empty since it doesn't get used
- Token: your token from vmauth or leave blank if you don't have authentication enabled
- If you need to ignore TLS/SSL errors check the advanced box and uncheck the verify certificate box
4. Click the `Create` button
![PVE Metric Form](pve-form.webp)
5. Run `system_uptime{object="nodes"}` in vmui or in the explore view in Grafana to verify metrics from PVE are being sent to VictoriaMetrics.
You should see 1 time series per node in your PVE cluster.
## Proxmox Backup Server (PBS)
1. Login to PBS as an administrator
2. Go to Configuration > Metrics Server > Add > InfluxDB
![PBS Metric Navigation](pbs-nav.webp)
3. Set the parameters as follows:
- Name: VictoriaMetrics (can be set to any string)
- URL: http(s)://<ip_or_host>:<port>
- set the URL to https if you have TLS enabled and http if you do not
- Port: This will vary depending how you are sending data to VictoriaMetrics, but the defaults for all components are listed in the [data ingestion documentation](https://docs.victoriametrics.com/data-ingestion.html)
- Organization: leave empty since it doesn't get used
- Bucket: leave empty since it doesn't get used
- Token: your token from vmauth or leave blank if you don't have authentication enabled
4. Click the `Create` button
![PBS Metric Form](pbs-form.webp)
5. Run `cpustat_idle{object="host"}` in vmui or in the explore view in Grafana to verify metrics from PBS are being to VictoriaMetrics.
# References
- [Blog Post for configuring Managed VictoriaMetrics and Proxmox VE](https://victoriametrics.com/blog/proxmox-monitoring-with-dbaas/)

View file

@ -0,0 +1,35 @@
---
# sort: 14
title: Data Ingestion
weight: 0
menu:
docs:
parent: 'victoriametrics'
identifier: 'data-ingestion'
weight: 7
aliases:
- /data-ingestion.html
- /data-ingestion.html
- /dataingestion/
---
# Data Ingestion
In This Folder you will find instructions for sending data to VictoriaMetrics from a variety of platforms.
If your tool is not listed it is likely you can ingest your data into VictoriaMetrics using one of the protocols listed in our [Prominent features]({{< ref "/Single-server-VictoriaMetrics.md#prominent-features" >}}) section.
If you are unsure what port number to use when pushing data to VictoriaMetrics single node, vminsert, vmagent, and vmauth we have listed the default ports below.
- VictoriaMetrics Single: 8428
- vmagent: 8429
- vmauth: 8427
- vminsert: 8482
In the rest of the documentation we will assume you have configured your push endpoint to use TLS/SSL on port 443 so the urls in the rest of the documentation will look like `https://<victoriametrics_url>` instead of `http://<victoriametrics_url>:8428` for VictoriaMetrics single.
## Documented Collectors/Agents
* [Telegraf]({{< relref "Telegraf.md" >}})
* [Vector]({{< relref "Vector.md" >}})
## Supported Platforms
* [Proxmox Virtual Environment and Proxmox Backup Server]({{< relref "Proxmox.md" >}})

View file

@ -0,0 +1,105 @@
---
title: Telegraf
weight: 1
sort: 1
menu:
docs:
identifier: "telegraf"
parent: "data-ingestion"
weight: 1
# sort: 1
aliases:
- /data-ingestion/telegraf.html
- /data-ingestion/Telegraf.html
---
# Telegraf Setup
You will need to add the following output section to a Telegraf configuration file and reload Telegraf to enable shipping data from Telegraf to VictoriaMetrics.
All the options examples below can be combined to fit your use case
To avoid storing Passwords in configuration files you can store as a key value pair in `/etc/default/telegraf` on Linux as follows
```
victoriametrics_url="https://metrics.example.com"
victoriametrics_user="telegraf"
victoriametrics_password="password"
victoriametrics_token="my_token"
```
and they can be referenced in a Telegraf configuration file by prepending the variable name with `$` ex. `$victoriametrics_url` will be translated to `https://metrics.example.com` if it referenced in a Telegraf configuration using the values from `/etc/default/telegraf` in the values seen above.
Otherwise please replace the variables below to fit your setup.
If you want to mimic this behavior on windows please read [Influx Data's blog on storing variables in the registry](https://www.influxdata.com/blog/using-telegraf-on-windows/)
## Minimum Configuration with no Authentication
```toml
[[outputs.influxdb]]
urls = ["$victoriametrics_url"]
database = "victoriametrics"
skip_database_creation = true
exclude_retention_policy_tag = true
content_encoding = "gzip"
```
## HTTP Basic Authentication (Username and Password)
This is the same as the minimum configuration, but adds the `username` and `password` options
```toml
[[outputs.influxdb]]
urls = ["$victoriametrics_url"]
username = "$victoriametrics_user"
password = "$victoriametrics_password"
database = "victoriametrics"
skip_database_creation = true
exclude_retention_policy_tag = true
content_encoding = "gzip"
```
## Bearer Authentication (Token)
This is the same as the minimum configuration but adds the authorization header
```
[[outputs.influxdb]]
urls = ["$victoriametrics_url"]
http_headers = {"Authorization" = "Bearer $victoriametrics_token"}
database = "victoriametrics"
skip_database_creation = true
exclude_retention_policy_tag = true
content_encoding = "gzip"
```
## Route certain metrics
If you only want to route certain metrics to VictoriaMetrics use the `namepass` option with a comma separated listed of the measurements you wish to send to VictoriaMetrics.
```
[[outputs.influxdb]]
urls = ["$victoriametrics_url"]
username = "$victoriametrics_user"
password = "$victoriametrics_password"
database = "victoriametrics"
skip_database_creation = true
exclude_retention_policy_tag = true
content_encoding = "gzip"
namepass = ["cpu","disk","measurement1","measurement2"]
```
## Ignore TLS/SSL Certificate errors
This is the same as the minimum configuration but adds `insecure_skip_verify = true` to the configuration to ignore TLS certificate errors.
This is not recommended since it can allow sending metrics to a compromised site.
```
[[outputs.influxdb]]
urls = ["$victoriametrics_url"]
username = "$victoriametrics_user"
password = "$victoriametrics_password"
database = "victoriametrics"
skip_database_creation = true
exclude_retention_policy_tag = true
content_encoding = "gzip"
insecure_skip_verify = true
```
# References
- [Install Telegraf](https://docs.influxdata.com/telegraf/v1/install/)
- [InfluxDBv1 output for Telegraf](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/influxdb)
- [Storing Telegraf variables in the windows registry](https://www.influxdata.com/blog/using-telegraf-on-windows/)
- [Telegraf variables](https://docs.influxdata.com/telegraf/v1/configuration/#example-telegraf-environment-variables)

View file

@ -0,0 +1,121 @@
---
title: Vector
weight: 1
sort: 1
menu:
docs:
identifier: "Vector"
parent: "data-ingestion"
weight: 1
# sort: 1
aliases:
- /data-ingestion/Vector.html
- /data-ingestion/vector.html
---
# Vector
To Send data to Vector you need to configure with a Prometheus remote write sink and forward metrics to that sink from at least 1 source.
You will need to replace the values in `<>` with your to match your setup.
## Minimum Config
```yaml
sources:
host_metrics_source:
type: host_metrics
sinks:
victoriametrics_sink:
type: prometheus_remote_write
inputs:
- host_metrics_source
endpoint: "https://<victoriametrics_url>/api/v1/write"
healthcheck:
enabled: false
```
## Basic Authentication
This adds support for basic authentication by defining the auth strategy, user, and password fields:
```yaml
sources:
host_metrics_source:
type: host_metrics
sinks:
victoriametrics_sink:
type: prometheus_remote_write
inputs:
- host_metrics_source
endpoint: "https://<victoriametrics_url>/api/v1/write"
auth:
strategy: "basic"
user: "<victoriametrics_user"
password: "<victoriametrics_password>"
healthcheck:
enabled: false
```
## Bearer / Token Authentication
This adds support for bearer/token authentication by defining the auth strategy and token fields:
```yaml
sources:
host_metrics_source:
type: host_metrics
sinks:
victoriametrics_sink:
type: prometheus_remote_write
inputs:
- host_metrics_source
endpoint: "https://<victoriametrics_url>/api/v1/write"
auth:
strategy: "bearer"
token: "<victoriametrics_token>"
healthcheck:
enabled: false
```
## VictoriaMetrics and VictoriaLogs
This combines the Bearer Authentication section with the [VictoriaLogs docs for Vector](https://docs.victoriametrics.com/victorialogs/data-ingestion/vector/),
so you can send metrics and logs with 1 agent to multiple sources:
```yaml
sources:
host_metrics_source:
type: host_metrics
journald_source:
type: journald
sinks:
victoriametrics_sink:
type: prometheus_remote_write
inputs:
- host_metrics_source
endpoint: "https://<victoriametrics_url>/api/v1/write"
auth:
strategy: "bearer"
token: "<token>"
healthcheck:
enabled: false
victorialogs_sink:
inputs:
- journald_source
type: elasticsearch
endpoints:
- "https://<victorialogs_url>/insert/elasticsearch/"
mode: bulk
api_version: "v8"
healthcheck:
enabled: false
query:
_msg_field: "message"
_time_field: "timestamp"
_stream_fields: "host,container_name"
```
# References
- [Vector documentation](https://vector.dev/docs/)
- [VictoriaLogs documenation for using vector]({{< ref "/victorialogs/data-ingestion/vector" >}})

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB