How to export logs to the OTEL collector

This guide explains how to export Aidbox logs in OpenTelemetry format

Aidbox supports exporting logs using the Protobuf protocol in line with the OTEL specification. This guide configures Aidbox to export logs to the OpenTelemetry collector, but the setup can also be modified to export to other logs consumers that adhere to this specification.

Prerequisites

1. OTEL collector should be deployed and configured to receive logs.

2. Aidbox should be configured with Aidbox configuration project.

How to enable export logs to the OTEL collector

To enable exporting logs to the OTEL collector:

1. Import aidbox.log

2. Define open-telemetry-appender

3. Add otel-appender to the services

4. Restart Aidbox

{:ns     main
 :import #{aidbox
           aidbox.log ; import aidbox.log
           config}

 open-telemetry-appender
 {:zen/tags   #{aidbox/service}
  :engine     aidbox.log/open-telemetry-appender
  :config     {:url "http://url-to-otel-collector/v1/logs"}} ; logs consumer endpoint

 box
 {:zen/tags #{aidbox/system}
  :config   config/zen-config
  :services {:otel-appender open-telemetry-appender}}} ; add otel-appender

How to check the OTEL collector receives logs

Set up logging exporter and logs pipeline in the OTEL collector configuration:

receivers:
  otlp:
    protocols:
      http:

exporters:
  logging:
    loglevel: debug

service:
  pipelines:
    logs:
      receivers: [otlp]
      exporters: [logging] # OTEL prints logs to the stdout

See Aidbox logs in the OTEL collector stdout

Open OTEL collector stdout and see the logs.

Check logs sending status

The common endpoint for checking status of sending metrics process

GET /telemetry/:zen-namespace/:zen-symbol-name/$status

In this case

GET /telemetry/main/open-telemetry-metrics-exporter/$status

queue-size: 10
history:
- ts: 1700661071
  processed-count: 34