Observability

AuraBoot is built on Spring Boot's Micrometer instrumentation and adds platform-specific gauges and counters for the meta engine, command pipeline, BPM, and crawler subsystems. The goal is that every production incident can be reproduced from the time series and structured logs alone.
Three pillars
| Pillar | Backend | Source of truth |
|---|---|---|
| Metrics | Prometheus | /actuator/prometheus on each node |
| Logs | Loki / ELK | stdout JSON lines |
| Traces | OpenTelemetry collector | OTLP exporter, propagated via traceparent |
Each request carries a traceparent header. Logs and metrics emitted while handling that request are tagged with the same trace id, so a single trace id pivots you across all three pillars.
When to use
- Setting up a new environment: wire Prometheus scraping and Grafana dashboards before exposing the URL to users.
- Diagnosing latency: start from the trace, drill into spans, and confirm with metrics.
- Capacity planning: long-horizon Prometheus queries against
http_server_requests_seconds_*. - Auditing: structured logs are the canonical record for security events.
Metric naming conventions
Platform metrics are prefixed with auraboot_ and use Prometheus snake_case; tags are lowercase snake_case too.
| Metric | Example | Meaning |
|---|---|---|
auraboot_command_execution_total | auraboot_command_execution_total{command_code="qualify_crm_lead", model_code="crm_lead", status="success"} | Command execution counter |
auraboot_command_execution_duration_seconds | auraboot_command_execution_duration_seconds{command_code="qualify_crm_lead", model_code="crm_lead"} | Command execution latency timer (with histogram) |
auraboot_api_requests_total | auraboot_api_requests_total{path="/api/...", method="POST", status="200"} | API request counter |
auraboot_active_sessions | gauge | Current active session count |
auraboot_llm_requests_total | auraboot_llm_requests_total{provider="...", model="...", status="success"} | LLM request counter |
auraboot_plugin_install_total | auraboot_plugin_install_total{plugin_code="...", status="success"} | Plugin install counter |
Avoid unbounded tags (raw user input, record ids). Cardinality explosions are the single most common Prometheus self-DoS.
How it works
# application.yml (excerpt)
management:
endpoints:
web:
exposure:
include: ${ACTUATOR_ENDPOINTS:health,info,metrics,prometheus}
metrics:
tags:
application: auraboot
tracing:
sampling:
probability: 1.0 # 100% in dev; lower to 0.1 in production
propagation:
type: w3c
otlp:
tracing:
endpoint: http://localhost:4318/v1/traces # dev default; point at the collector service in productionThe OTLP endpoint above is the dev default. In production it usually points at the collector's service address (for example
http://otel-collector:4318/v1/tracesinside Kubernetes).
Prometheus scrape config:
scrape_configs:
- job_name: auraboot
metrics_path: /actuator/prometheus
scrape_interval: 15s
static_configs:
- targets: ["app-1:6443", "app-2:6443", "app-3:6443"]
labels:
env: prodUseful queries in Grafana:
# p95 command latency by command, last 5 minutes
histogram_quantile(0.95,
sum by (le, command_code) (rate(auraboot_command_execution_duration_seconds_bucket[5m])))
# command execution error ratio
sum(rate(auraboot_command_execution_total{status="error"}[5m]))
/ sum(rate(auraboot_command_execution_total[5m]))
# API request rate by status code
sum by (status) (rate(auraboot_api_requests_total[1m]))Structured logs
Every log line is one JSON object with at least: @timestamp, level, logger, message, trace.id, tenant.id, user.id (when authenticated). A line that lacks trace.id cannot be correlated and is a defect.
{"@timestamp":"2026-05-28T10:11:12.345Z","level":"INFO","logger":"c.a.cmd.CommandPipeline","message":"command executed","trace.id":"4bf92f3577b34da6a3ce929d0e0e4736","tenant.id":"tenant_001","command_code":"qualify_crm_lead","status":"success","duration_ms":42}Verify
curl -sS http://app-1:6443/actuator/prometheus | grep auraboot_command_executionreturns non-empty series.- A Grafana dashboard renders p50/p95/p99 for at least: command execution latency and error ratio, API request rate, LLM request volume.
- Sampled traces in the OTLP backend include spans for HTTP -> command -> JDBC, and span attributes include
tenant.id. - Log volume is JSON, every line carries
trace.id, and a single trace id can be grepped end-to-end across nodes. - Alerting rules exist for: p99 command latency, command execution error ratio, API 5xx rate, JVM heap pressure, Postgres connection pool saturation.