Observability
Barbacane provides comprehensive observability features out of the box: structured logging, Prometheus metrics, and distributed tracing with OpenTelemetry support.
Logging
Structured logs are written to stdout. Two formats are available:
JSON Format (Default)
Production-ready structured JSON logs:
barbacane serve --artifact api.bca --log-format json
{"timestamp":"2024-01-15T10:30:00Z","level":"INFO","target":"barbacane","message":"request completed","trace_id":"abc123","request_id":"def456","method":"GET","path":"/users","status":200,"duration_ms":12}
Pretty Format (Development)
Human-readable format for local development:
barbacane serve --artifact api.bca --log-format pretty --log-level debug
2024-01-15T10:30:00Z INFO barbacane > request completed method=GET path=/users status=200 duration_ms=12
Log Levels
Control verbosity with --log-level:
| Level | Description |
|---|---|
error | Errors only |
warn | Warnings and errors |
info | Normal operation (default) |
debug | Detailed debugging |
trace | Very verbose tracing |
barbacane serve --artifact api.bca --log-level debug
Or use the RUST_LOG environment variable:
RUST_LOG=debug barbacane serve --artifact api.bca
Metrics
Prometheus metrics are exposed on the dedicated admin API port (default 127.0.0.1:8081):
curl http://localhost:8081/metrics
Available Metrics
| Metric | Type | Description |
|---|---|---|
barbacane_requests_total | counter | Total requests by method, path, status, api |
barbacane_request_duration_seconds | histogram | Request latency |
barbacane_request_size_bytes | histogram | Request body size |
barbacane_response_size_bytes | histogram | Response body size |
barbacane_active_connections | gauge | Current open connections |
barbacane_connections_total | counter | Total connections accepted |
barbacane_validation_failures_total | counter | Validation errors by reason |
barbacane_middleware_duration_seconds | histogram | Middleware execution time |
barbacane_dispatch_duration_seconds | histogram | Dispatcher execution time |
barbacane_wasm_execution_duration_seconds | histogram | WASM plugin execution time |
barbacane_upstream_endpoint_requests_total | counter | Attempts sent to an endpoints pool member, by endpoint and outcome (success, failure) |
barbacane_upstream_endpoint_retries_total | counter | Attempts sent to an endpoint after another one failed, by endpoint |
barbacane_upstream_endpoint_circuit_state | gauge | An endpoint’s circuit breaker: 0 closed, 1 open, 2 half-open |
An endpoint label is scheme://host:port, never the full URL, so it carries no user information or query; its values are the endpoints the artifact lists. The request span records the endpoint a dispatch used as upstream_endpoint, and breaker transitions are logged with it.
Labels stay bounded whatever clients send. path is the route template, or one label for requests that match no route. method is GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE, PATCH or QUERY, and OTHER for any other method, including one declared under OpenAPI 3.2 additionalOperations.
Plugin metrics
A plugin with the telemetry capability emits counters and histograms. Each metric is its own family, barbacane_plugin_<plugin>_<metric>, with the plugin name’s - written _: the observability middleware’s slo_violation counter is barbacane_plugin_observability_slo_violation_total. The labels the plugin sends are the series’ labels.
- A metric name is
[a-zA-Z_][a-zA-Z0-9_]*; a label name is the same and does not start with__; a label value is a string, number or boolean. A histogram’s buckets run from 1 ms to 10 s, for observations in seconds. - Each plugin has at most 1,000 series. A family belongs to the plugin metric that created it: another plugin’s metric with the same exported name, or the same metric with the other type, is refused.
- Every refused write is counted in
barbacane_plugin_metrics_dropped_total{plugin,reason},reasonbeingoversize,cardinality,non_finite,invalid_name,invalid_labels,name_conflictortype_conflict.
Prometheus Integration
Configure Prometheus to scrape metrics:
# prometheus.yml
scrape_configs:
- job_name: 'barbacane'
static_configs:
- targets: ['barbacane:8081']
metrics_path: '/metrics'
scrape_interval: 15s
Example Queries
# Request rate
rate(barbacane_requests_total[5m])
# P99 latency
histogram_quantile(0.99, rate(barbacane_request_duration_seconds_bucket[5m]))
# Error rate
sum(rate(barbacane_requests_total{status=~"5.."}[5m])) / sum(rate(barbacane_requests_total[5m]))
# Active connections
barbacane_active_connections
Distributed Tracing
Barbacane supports W3C Trace Context for distributed tracing and can export spans via OpenTelemetry Protocol (OTLP).
Enable OTLP Export
barbacane serve --artifact api.bca \
--otlp-endpoint http://otel-collector:4317
Or use the environment variable:
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317 barbacane serve --artifact api.bca
Trace Context Propagation
Barbacane automatically:
- Extracts
traceparentandtracestateheaders from incoming requests - Generates a new trace ID if none provided
- Injects trace context into upstream requests
This enables end-to-end tracing across your entire service mesh.
Span Structure
Each request creates a span tree:
barbacane.request (root span)
├── barbacane.routing
├── barbacane.validation
├── barbacane.middleware.jwt-auth.request
├── barbacane.middleware.rate-limit.request
├── barbacane.dispatch.http-upstream
├── barbacane.middleware.rate-limit.response
└── barbacane.middleware.jwt-auth.response
Span Attributes
Spans include attributes like:
http.method,http.route,http.status_codebarbacane.api,barbacane.operation_idbarbacane.middleware,barbacane.dispatcher
Integration with Collectors
Works with any OpenTelemetry-compatible backend:
Jaeger:
barbacane serve --artifact api.bca --otlp-endpoint http://jaeger:4317
Grafana Tempo:
barbacane serve --artifact api.bca --otlp-endpoint http://tempo:4317
Datadog (with collector):
# otel-collector-config.yaml
exporters:
datadog:
api:
key: ${DD_API_KEY}
Production Setup
A typical production observability stack:
barbacane serve --artifact api.bca \
--log-format json \
--log-level info \
--otlp-endpoint http://otel-collector:4317
Combined with:
- Prometheus scraping
/metricson the admin port (default 8081) - OpenTelemetry Collector receiving OTLP traces
- Grafana for dashboards and alerting
- Log aggregation (Loki, Elasticsearch) ingesting stdout
Example Alert Rules
# prometheus-rules.yaml
groups:
- name: barbacane
rules:
- alert: HighErrorRate
expr: |
sum(rate(barbacane_requests_total{status=~"5.."}[5m]))
/ sum(rate(barbacane_requests_total[5m])) > 0.01
for: 5m
labels:
severity: critical
annotations:
summary: "High error rate detected"
- alert: HighLatency
expr: |
histogram_quantile(0.99, rate(barbacane_request_duration_seconds_bucket[5m])) > 1
for: 5m
labels:
severity: warning
annotations:
summary: "P99 latency exceeds 1 second"
What’s Next?
- CLI Reference - All command-line options
- Reserved Endpoints - Full metrics endpoint documentation
- Spec Extensions - Barbacane spec extensions reference