Aidbox
Search
K

Aidbox Metrics Server

Setup and environment variable

DefineBOX_METRICS_PORTenvironment variable with monitoring server port number.

Start metrics server

Aidbox starts monitoring server on startup automatically.
To check the monitoring server works make GET <AIDBOX_BASE_URL>:<BOX_METRICS_PORT> request. The output should be a string "aidbox metrics".

Metrics server endpoints

There are three types of metrics Aidbox collects and exposes. All endpoints are available on a separate port, e.g. GET <AIDBOX_BASE_URL>:<BOX_METRICS_PORT>/metrics.
Endpoint
Update frequency
GET /metrics
continuous
GET /metrics/minutes
every minute
GET /metrics/hours
every hour
The /metrics/hours response can take some time since it collects a lot of information from the database. Make sure your metrics scraper timeout is sufficient.

Prometheus example scrapers configuration

global:
# omitted global configuration values
external_labels:
monitor: 'aidbox'
scrape_configs:
# omitted default scrappers configuration
- job_name: aidbox
honor_labels: true
scrape_interval: 10s
metrics_path: /metrics
static_configs:
- targets: [ 'aidbox.example.com:9999' ] # should be <AIDBOX_BASE_URL>:<BOX_METRICS_PORT
- job_name: aidbox-minutes
honor_labels: true
scrape_interval: 1m
metrics_path: /metrics/minutes
static_configs:
- targets: [ 'aidbox.example.com:9999' ] # should be <AIDBOX_BASE_URL>:<BOX_METRICS_PORT
- job_name: aidbox-hours
honor_labels: true
scrape_interval: 10m
scrape_timeout: 30s # increased timeout
metrics_path: /metrics/hours
static_configs:
- targets: [ 'aidbox.example.com:9999' ] # should be <AIDBOX_BASE_URL>:<BOX_METRICS_PORT

Collected metrics

HTTP

Metric
Update frequency
Description
aidbox_http_request_duration_seconds_bucket
continuous
request duration as cumulative counters for buckets
aidbox_http_request_duration_seconds_count
continuous
request duration events count
aidbox_http_request_duration_seconds_sum
continuous
sum of request duration events value
aidbox_http_request_wait_seconds_bucket
continuous
queue waiting time as cumulative counters for buckets
aidbox_http_request_wait_seconds_count
continuous
queue waiting time events count
aidbox_http_request_wait_seconds_sum
continuous
sum of queue waiting time events value

Postgres

Metric
Update frequency
Description
pg_requests_total
continuous
number of executed selects requests
pg_inserts_total
continuous
number of executed insert statements
pg_updates_total
continuous
number of executed update statements
pg_deletes_total
continuous
number of executed delete statements
pg_blks_hit
continuous
number of shared block cache hits
pg_blks_read
continuous
number of shared blocks read
pg_tup_fetched
continuous
fetched tuples number
pg_tup_returned
continuous
returned tuples number
pg_errors_total
continuous
number of errors
pg_activity_count
continuous
number of PG workers
pg_idx_scan
every minute
number of index scans
pg_seq_scan
every minute
number of sequential scans
pg_stat_statements_total_calls
every minute
number of times executed
pg_stat_statements_stddev_execution_time
every minute
statement execution time
pg_stat_statements_mean_execution_time
every minute
mean statement execution time
pg_table_size
every hour
table size
pg_database_size
every hour
database size
pg_activity_max
every hour
maximum number of connections

Hikari (Postgres connection pool)

Metric
Update frequency
Description
hikari_active_count
continuous
number of active connections
hikari_idle_count
continuous
number of idle connections
hikari_acquire_created_seconds_bucket
continuous
time taken to create an actual physical connection
hikari_acquire_created_seconds_count
continuous
number of created physical connections
hikari_acquire_created_seconds_sum
continuous
total amount of time to create all physical connections
hikari_acquired_total
continuous
number of obtained connections
hikari_acquire_wait_seconds_bucket
continuous
time taken to obtain a connection
hikari_acquire_wait_seconds_sum
continuous
total amount of time to obtain all connections
hikari_acquire_used_seconds_bucket
continuous
time consumed by a connection
hikari_max_size
every hour
maximum number of connections

JVM

Metric
Update frequency
Description
jvm_gc_time
continuous
garbage collector execution time
jvm_gc_count
continuous
garbage collector count of launch
jvm_heap_memory
continuous
heap memory usage
jvm_non_heap_memory
continuous
non-heap memory usage
jvm_thread_count
continuous
number of live threads including both daemon and non-daemon thread
jvm_thread_peak_count
continuous
peak live thread count
jvm_thread_daemon_count
continuous
number of daemon thread
jvm_available_processors_size
every hour
number of processors available to the JVM
jvm_max_memory_size
every hour
maximum amount of memory that JVM will attempt to use
jvm_total_memory_size
every hour
total amount of memory in JVM

Disable PostgreSQL metrics

If you have a different pg exporter you can disable Aidbox PostgreSQL metrics for avoiding metrics duplication.
In this case, you should set BOX_METRICS_POSTGRES_ON to false value