aeat.adapters.outbound.llm._run_telemetry module

Local-only LLM run-timing telemetry recorder.

Persists one LLMRunRecord per completed (or failed) LLM classification/completion invocation to encrypted secure-object storage under LLM_RUN_TELEMETRY_NAMESPACE, mirroring UsageRecorder’s persistence shape. Every record is written at SensitivityClass DIAGNOSTIC and carries ONLY timing and outcome metadata (provider label, duration, success flag, optional error-kind string) – never prompt text, response text, or any transaction/financial content, honouring sensitive-financial-data-secure-storage-only. Nothing here ever leaves the host: there is no network transport, only the same encrypted local secure-object backend every other diagnostic store uses.

This is the durable capture half of the local-only run-diagnostics surface (aeat app diagnostics run-health): a slow or failing LLM-backed classification run is otherwise invisible until an operator notices a stuck CLI invocation.

prune() bounds this store’s growth with a retention window (aeat_llm_run_telemetry_retention_days) and a maximum record count (aeat_llm_run_telemetry_max_records), mirroring prune()’s list-then-delete-by-reconstructed-key shape. The object key each record was saved under embeds a random UUID4 suffix (so two runs starting in the same microsecond never collide); that suffix is persisted inside the record’s own payload alongside its natural fields so pruning can reconstruct the exact save-time key and issue a matching delete, without a parallel index.

See also

LLMRunTelemetryRecorder

Public recorder that appends and reads these local-only records.

LLMRunRecord

Timing/outcome-only payload stored for each completed LLM run.

build_run_health_report()

Application diagnostic that aggregates these records for operators.

diagnostics_telemetry

Remote-telemetry preview/flush layer that aggregates only the same non-sensitive accounting signal through a separate consent gate.

LLM_RUN_TELEMETRY_NAMESPACE

Secure-object namespace used for the encrypted local store.

class LLMRunRecord(**data)[source]

Bases: BaseModel

One local LLM run-timing record: duration, provider, and outcome only.

Carries no prompt or response text and no transaction content – only the accounting metadata needed to diagnose a slow or failing run.

Parameters:
run_id: str
caller: str
provider: str
model: str
duration_ms: int
succeeded: bool
error_kind: str
started_at: datetime
class LLMRunTelemetrySummary(**data)[source]

Bases: BaseModel

Aggregated LLMRunTelemetryRecorder statistics for one provider or overall.

Parameters:

  • entries (int)

  • succeeded (int)

  • failed (int)

  • min_duration_ms (int | None)

  • max_duration_ms (int | None)

  • mean_duration_ms (Decimal | None)

entries: int
succeeded: int
failed: int
min_duration_ms: int | None
max_duration_ms: int | None
mean_duration_ms: Decimal | None
class LLMRunTelemetryRecorder(root_dir=None)[source]

Bases: object

Append local LLM run-timing records to encrypted secure-object storage.

Mirrors UsageRecorder’s persistence shape: each record() call appends one redacted-free LLMRunRecord (there is no free text to redact – the model carries only accounting metadata) through secure_object_repository_for_active_bucket().

Variables:

root_dir – Logical partition used for run-telemetry records.

Parameters:

root_dir (Path | None)

record(record)[source]

Append record to encrypted secure-object storage.

Parameters:

record (LLMRunRecord) – Run-timing record to append.

Return type:

Path

Returns:

Logical daily run-telemetry path for operator display only.

Raises:
load_records(since=None, until=None)[source]

Load run-telemetry records, optionally filtered by an inclusive date range.

Parameters:

  • since (date | None) – Inclusive lower date bound, or None for no lower bound.

  • until (date | None) – Inclusive upper date bound, or None for no upper bound.

Return type:

tuple[LLMRunRecord, ...]

Returns:

Loaded LLMRunRecord entries in file-iteration order.

summarize(since=None, until=None, *, provider=None)[source]

Aggregate run records into a LLMRunTelemetrySummary.

Parameters:

  • since (date | None) – Inclusive lower date bound, or None for no lower bound.

  • until (date | None) – Inclusive upper date bound, or None for no upper bound.

  • provider (str | None) – Optional provider filter; None aggregates every provider.

Return type:

LLMRunTelemetrySummary

Returns:

Aggregate run-timing summary.

prune(*, retention_days=None, max_records=None)[source]

Delete records older than the retention window or beyond the count cap.

Applies a two-stage bound, mirroring prune()’s list-then-delete-by-reconstructed-key shape: first every record older than retention_days (measured against the current time) is removed, then – if more than max_records remain – the oldest excess records beyond the cap are removed too. Both bounds default to the centralized aeat_llm_run_telemetry_retention_days and aeat_llm_run_telemetry_max_records settings.

Parameters:

  • retention_days (int | None) – Age cutoff in days; records strictly older than this are removed. Defaults to the centralized setting.

  • max_records (int | None) – Maximum record count to retain after the age cutoff is applied; the oldest excess records beyond this count are removed. Defaults to the centralized setting.

Return type:

int

Returns:

Number of removed run-telemetry objects. A record whose key no longer resolves (e.g. removed by a concurrent prune) is silently skipped rather than counted or raised.