VictoriaMetrics/docs/VictoriaLogs/data-ingestion/opentelemetry.md
Andrii Chubatiuk 01430a155c
vlinsert: added opentelemetry logs support
Commit adds the following changes:

* Adds support of OpenTelemetry logs for Victoria Logs with protobuf encoded messages

*  json encoding is not supported for the following reasons:
   - It brings a lot of fragile code, which works inefficiently.
   - json encoding is impossible to use with language SDK.

* splits metrics and logs structures at lib/protoparser/opentelemetry/pb package.

* adds docs with examples for opentelemetry logs.

---
Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4839

Co-authored-by: AndrewChubatiuk <andrew.chubatiuk@gmail.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-09-03 20:12:05 +02:00

3.5 KiB

weight title disableToc menu aliases
4 OpenTelemetry setup true
docs
parent weight
victorialogs-data-ingestion 4
/VictoriaLogs/data-ingestion/OpenTelemetry.html

VictoriaLogs supports both client open-telemetry SDK and collector.

Client SDK

Specify EndpointURL for http-exporter builder.

Consider the following example for golang SDK:

 // Create the OTLP log exporter that sends logs to configured destination
 logExporter, err := otlploghttp.New(ctx,
  otlploghttp.WithEndpointURL("http://victorialogs:9428/insert/opentelemetry/v1/logs"),
 )

Optionally, stream fields could be defined via headers:

 // Create the OTLP log exporter that sends logs to configured destination
 logExporter, err := otlploghttp.New(ctx,
  otlploghttp.WithEndpointURL("http://victorialogs:9428/insert/opentelemetry/v1/logs"),
   otlploghttp.WithHeaders(map[string]string{"VL-Stream-Fields": "telemetry.sdk.language,severity"}),
 )

Given config defines 2 stream fields - severity and telemetry.sdk.language.

See also HTTP headers

Collector configuration

VictoriaLogs supports given below OpenTelemetry collector exporters:

Elasticsearch

exporters:
  elasticsearch:
    endpoints:
      - http://victorialogs:9428/insert/elasticsearch
receivers:
  filelog:
    include: [/tmp/logs/*.log]
    resource:
      region: us-east-1
service:
  pipelines:
    logs:
      receivers: [filelog]
      exporters: [elasticsearch]

Loki

exporters:
  loki:
    endpoint: http://victorialogs:9428/insert/loki/api/v1/push
receivers:
  filelog:
    include: [/tmp/logs/*.log]
    resource:
      region: us-east-1
service:
  pipelines:
    logs:
      receivers: [filelog]
      exporters: [loki]

OpenTelemetry

Specify logs endpoint for OTLP/HTTP exporter in configuration file for sending the collected logs to VictoriaLogs:

exporters:
  otlphttp:
    logs_endpoint: http://localhost:9428/insert/opentelemetry/v1/logs

Optionally, stream fields could be defined via headers:

exporters:
  otlphttp:
    logs_endpoint: http://localhost:9428/insert/opentelemetry/v1/logs
    headers:
     VL-Stream-Fields: telemetry.sdk.language,severity

See also HTTP headers

Substitute localhost:9428 address inside exporters.oltphttp.logs_endpoint with the real address of VictoriaLogs.

The ingested log entries can be queried according to these docs.

See also: