Skip to content

Metrics

Prometheus metrics for monitoring and observability

Accessing Metrics

Auditty Edge exposes Prometheus-style metrics that provide complete visibility into log processing, suppression efficiency, and system health.

Metrics are exposed on port 2112 at the /metrics endpoint.

From within the pod:

wget -qO- http://localhost:2112/metrics

From Prometheus: Configure Prometheus to scrape the metrics endpoint. Enable prometheusScrape.enabled: true in your Helm values to add scraping annotations automatically.

The 5 Essential Metrics

These five counters tell you everything you need to know about Auditty's core value proposition:

1. Log Volume Ingested (Count)

auditty_events_processed_total

Total number of log events ingested from container runtime since startup.

Labels: namespace, workload

Use: Baseline traffic volume — how many log lines are your workloads producing?

2. Log Volume Ingested (Size)

auditty_event_size_bytes_total

Total bytes ingested from container runtime before processing.

Labels: namespace, workload

Use: Raw data volume — how much bandwidth/storage would you need without Auditty?

3. Log Volume Suppressed (Count)

auditty_events_suppressed_total

Total number of log events suppressed (removed from output).

Labels: namespace, workload, policy, reason

Use: Volume reduction — how many log lines did Auditty eliminate?

Suppression Rate: (events_suppressed / events_processed) * 100%

4. Log Volume Suppressed (Size)

auditty_suppressed_size_bytes_total

Total bytes removed via suppression.

Labels: namespace, workload, policy, reason

Use: Cost savings — how much storage/bandwidth did Auditty save?

Savings Rate: (suppressed_bytes / event_bytes) * 100%

Dollar Savings: Configure your log platform cost per GB in Settings to see dollar savings on the Dashboard and Metrics pages.

5. Synthetic Events Injected

auditty_synthetic_events_total

Total synthetic events generated (suppression summaries, diagnostics).

Labels: namespace, workload, event_type

Use: Overhead tracking — Auditty replaces N suppressed logs with 1 summary.

Example: Suppress 1,000 logs → emit 1 summary → synthetic_events_total increments by 1

Key Business Metrics

Data Volume Tracking

  • auditty_written_size_bytes_total — Actual bytes written to the log file (ground truth output)
  • auditty_synthetic_payload_size_bytes_total — Bytes emitted as synthetic events

Shadow Mode (Testing)

  • auditty_events_shadow_suppressed_total — Events that would be suppressed in shadow mode
  • auditty_shadow_suppressed_size_bytes_total — Bytes that would be suppressed in shadow mode

Use: Test suppression policies without actually suppressing logs. Compare shadow metrics to actual metrics to validate policy changes.

Operational Metrics

These metrics help monitor system health and troubleshoot issues:

Pipeline Health

auditty_event_channel_depth, auditty_event_queue_depth, auditty_active_tailers

Error Tracking

auditty_component_errors_total, auditty_interception_failures_total

Performance

auditty_processing_duration_seconds — Event processing duration histogram by component

Pattern Management

auditty_patterns_stored, auditty_pattern_evictions_total

Label Architecture

Static Labels (set at startup)

  • environment — Environment name (from config)
  • cluster — Cluster identifier (from config)
  • node_id — Kubernetes node name
  • version — Auditty Edge version
  • execution_id — Unique ID for this process instance

Dynamic Labels (vary by metric)

  • namespace, workload — Kubernetes context (most metrics)
  • component — Component name (forwarder, processor, etc.)
  • policy, reason — Suppression context
  • event_type — Type of synthetic event
  • destination — Output destination (file, s3, etc.)

Data Integrity Verification (Auditty)

Auditty continuously verifies Auditty's lossless guarantee behind the scenes using the conservation counters above: ingested == forwarded + suppressed, with suppressed events fully represented by both the emitted summaries and the Vault archive. This runs automatically on every metrics refresh and has no dashboard presence — data integrity is the default, not something to announce. Anything short of full reconciliation is escalated directly to the Auditty team for investigation.

Outlier Passthrough

auditty_outlier_detected_total counts events whose suppression was bypassed because a promoted numeric field’s value fell far outside its fingerprint’s learned normal range — see Suppression → Outlier Passthrough. auditty_suppression_skipped_total{reason="outlier_passthrough"} mirrors the same events from the suppression-decision side, alongside reason="vault_unhealthy" for events that were forwarded because the Vault was not accepting writes — suppression stops rather than summarising into an archive that cannot store the originals.

For operators, the underlying pattern-stability gauges (auditty_fingerprints_active_total / auditty_fingerprints_singleton_total) remain available at the /metrics Prometheus endpoint — a rising singleton share usually means a recent log format change made templating too granular for a workload. See Suppression → Promoting Numeric Fields for the related identity-ledger controls.

Forwarding Auditty Metrics to External Platforms

Auditty Edge emits its own Prometheus metrics (prefixed with auditty_) for internal monitoring and the Auditty dashboard. If you are using a Prometheus remote-write pipeline or an agent (e.g. Datadog Agent, Splunk OTEL Collector, Grafana Agent) that scrapes all Prometheus endpoints and forwards them to an external platform, Auditty metrics will be forwarded too.

This can result in unexpected ingestion costs on your external platform, especially with high-cardinality label combinations (namespace × workload × policy).

Recommended actions:

  • Filter by prefix: Exclude metrics matching auditty_* from your remote-write or agent scrape config if you do not need them externally.
  • Datadog: In your Datadog Agent config, add auditty_ to metrics_exclude or use autodiscovery annotations to skip the Auditty metrics endpoint.
  • Splunk OTEL: Use a filter processor in your pipeline to drop metrics with the auditty_ prefix.
  • Grafana Agent / Prometheus: Use metric_relabel_configs to drop metrics matching auditty_.*.
  • Keep selectively: If you want some Auditty metrics externally (e.g. for cost-savings dashboards), allowlist only the 5 essential metrics and drop the rest.

Summary

  • 5 essential metrics tell you about ingestion, suppression, and synthetic event overhead
  • Shadow mode metrics let you test policies without affecting actual log flow
  • Operational metrics help monitor pipeline health and troubleshoot issues
  • Labels provide Kubernetes context for filtering and aggregation
  • Data integrity verification runs continuously behind the scenes in Auditty, confirming the lossless guarantee from the conservation counters
  • Access metrics at http://localhost:2112/metrics from any Edge pod