aeat.adapters.persistence.storage.sql.secure_objects module

Encrypted SQL byte-object repository for sensitive application payloads.

class SecureObjectRepository(*, engine=None, namespace_registry=None, active_session_bucket_id=None, require_secure_active_session=False)[source]

Bases: object

Repository over encrypted byte objects stored in the primary database.

Parameters:
property namespace_registry: StorageHierarchyRegistry | None

Return the StorageHierarchyRegistry bound here, if any.

property engine: Engine

Return the bound SQLAlchemy Engine.

Exposed so a sibling plaintext ORM table (e.g. a derived, non-sensitive routing index) can be written in the SAME database file and, where the driver supports it, the same transaction as this repository’s encrypted rows – without duplicating the bucket-to-engine routing this repository already resolved at construction.

exists(namespace, object_key)[source]

Return whether namespace / object_key is present.

Return type:

bool

Parameters:

  • namespace (str)

  • object_key (str)

exists_by_raw_key(namespace, hashed_object_key)[source]

Return whether namespace carries a row with this raw HMAC digest.

Used by the archive restore pipeline when the natural key was not present in the source bundle. Same master-key constraint as save_with_raw_key().

Return type:

bool

Parameters:

  • namespace (str)

  • hashed_object_key (bytes)

iter_all_records_raw(*, batch_size=256)[source]

Yield every stored row as a SecureObjectRawRow without decryption.

Walks every row in secure_objects ordered by (namespace, object_key) without attempting to decrypt the payload. The query bypasses the encrypted-column type decorators so rows sealed under a rotated master key still surface verbatim — this is what the outbound sync coordinator’s ciphertext-layer mirror consumes, mirroring on-wire ciphertext to a remote storage provider without ever decrypting domain data.

Parameters:

batch_size (int) – SQLAlchemy yield_per chunk size. The default keeps memory bounded for very large substrates while still amortising session overhead across multiple rows.

Yields:

One SecureObjectRawRow per persisted row. The order is (namespace ASC, object_key ASC) so consumers can checkpoint progress deterministically.

Return type:

Iterator[SecureObjectRawRow]

list_namespaces()[source]

Return the distinct namespaces present in secure_objects sorted.

Used by the integrity diagnostic so consumers do not have to hardcode the namespace list (which drifts as new domain repositories register their own namespaces).

Return type:

tuple[str, ...]

quarantine_unreadable_rows()[source]

Move every undecryptable row into secure_objects_quarantine.

Iterates every populated namespace, probes each row’s payload through decrypt_encrypted_bytes_column(), and for rows that fail tag verification copies the original (encrypted) payload plus all metadata into the quarantine table, then deletes the row from secure_objects. The quarantine table mirrors secure_objects with the addition of a quarantined_at timestamp so the archive is auditable.

Decryptable rows are NOT touched; the quarantine table is created on first use; nothing is auto-deleted from the user’s data even after quarantine. The operator can recover the quarantined rows manually from the table if a missing master key is later recovered (for example, restored from a recovery key backup).

Return type:

tuple[SecureObjectNamespaceIntegrity, ...]

Returns:

A tuple of SecureObjectNamespaceIntegrity records describing how many rows were quarantined per namespace.

probe_namespace_integrity(namespace)[source]

Count decryptable and undecryptable rows in namespace.

Returns a SecureObjectNamespaceIntegrity for the namespace.

This method answers a strictly crypto-layer question – can the payload ciphertext be unwrapped under the current master key – and intentionally bypasses the classification and schema-version contracts that consumer reads enforce. Used by aeat config repair to surface namespaces holding rows from a prior keychain master-key generation.

Return type:

SecureObjectNamespaceIntegrity

Parameters:

namespace (str)

iter_namespace_decryptability(namespace)[source]

Yield SecureObjectDecryptabilityRow metadata for one namespace.

This is the row-level companion to probe_namespace_integrity(). It decrypts only to validate the AEAD tag, never returns plaintext, and exposes the HMAC lookup digest plus storage metadata needed by repair diagnostics.

Return type:

Iterator[SecureObjectDecryptabilityRow]

Parameters:

namespace (str)

list_keys(namespace)[source]

Return stored lookup digests under namespace as hex strings.

Natural object keys are HMAC digested before storage and cannot be recovered from the index. Domain repositories that need natural IDs should iterate list_records() and read IDs from decrypted payloads.

Return type:

tuple[str, ...]

Parameters:

namespace (str)

list_records(namespace, *, expected_class, max_supported_version)[source]

Yield secure-object rows under namespace or fail on unreadable rows.

The default listing path is fail-closed: it walks the namespace through iter_records_with_failures() and raises SecureObjectUnreadableError before yielding a partial readable subset. Use iter_records_with_failures for explicit mixed readable/unreadable diagnostics.

Parameters:

  • namespace (str) – The storage namespace whose rows are listed.

  • expected_class (SensitivityClass) – The SensitivityClass all rows in this namespace must carry.

  • max_supported_version (int) – The consumer’s current schema_version ceiling; a row above it, or below it without a complete registered upgrade chain, is treated as unreadable.

Return type:

Iterator[SecureObjectRecord]

load_many(namespace, object_keys, *, expected_class, max_supported_version)[source]

Yield requested secure-object rows or fail closed on unreadable rows.

This is the targeted equivalent of list_records(): it performs a single WHERE namespace = ? AND object_key IN (...) read for the requested natural keys, decrypts matching rows, and raises SecureObjectUnreadableError before yielding a partial readable subset if any matching row is unreadable. Missing keys are omitted, mirroring repeated load() calls that return None for absent rows. expected_class is the SensitivityClass every returned row must be classified under; a mismatch fails closed.

Return type:

Iterator[SecureObjectRecord]

Parameters:
iter_many_with_failures(namespace, object_keys, *, expected_class, max_supported_version)[source]

Yield readable/unreadable outcomes for requested natural object keys.

Rows are selected by raw HMAC digests derived from object_keys and returned in stored digest order. Missing keys produce no item, matching load() returning None. Present rows use the same classification, schema-version, AEAD, and revision-lineage checks as namespace scans. expected_class is the SensitivityClass every yielded row must be classified under; a mismatch fails closed.

Return type:

Iterator[SecureObjectRecord | SecureObjectUnreadable]

Parameters:
iter_records_with_failures(namespace, *, expected_class, max_supported_version, batch_size=256)[source]

Yield a typed outcome per stored row under namespace.

Each row is represented by either a SecureObjectRecord (the row decrypts cleanly and matches the consumer’s classification and schema-version contract) or a SecureObjectUnreadable (the on-wire ciphertext exists but cannot be decrypted under the current master key, or its metadata fails the consumer’s contract).

The iterator is fault-isolated: a failure on row N does not prevent rows > N from being inspected. Consumers count the failures and decide how to report them; nothing is auto-deleted.

Parameters:

  • namespace (str) – The storage namespace whose rows are scanned.

  • expected_class (SensitivityClass) – The SensitivityClass all rows in this namespace must carry; rows with a differing classification are yielded as SecureObjectUnreadable.

  • max_supported_version (int) – The consumer’s current schema_version ceiling. Rows above it, or below it without a complete registered upgrade chain, are yielded as SecureObjectUnreadable.

  • batch_size (int) – SQLAlchemy yield_per chunk size for the raw row scan. The default keeps memory bounded for large namespaces while preserving deterministic (object_key ASC) order.

Yields:

One SecureObjectListItem per stored row — either a SecureObjectRecord or a SecureObjectUnreadable.

Raises:

StorageValidationError – When batch_size is less than 1.

Return type:

Iterator[SecureObjectRecord | SecureObjectUnreadable]

load(namespace, object_key, *, expected_class, max_supported_version)[source]

Load and decrypt one secure-object row, returning None when absent.

Returns a SecureObjectRecord when the row is present and decrypts under the expected class/version.

Parameters:

  • namespace (str) – The storage namespace to look in.

  • object_key (str) – The natural string key identifying the record.

  • expected_class (SensitivityClass) – The SensitivityClass the consumer expects.

  • max_supported_version (int) – Highest schema_version the consumer supports.

Return type:

SecureObjectRecord | None

save(*, namespace, object_key, classification, schema_version, written_at, payload, write_provenance='secure-object-repository', source_event_id=None, expected_revision_id=None)[source]

Encrypt and upsert one byte payload keyed by a natural string id.

The natural object_key is HMAC-digested at the column boundary. To upsert against a pre-computed digest (e.g. when restoring an archive bundle whose natural key was lost in the original HMAC), use save_with_raw_key() instead.

Parameters:

  • namespace (str) – The storage namespace to write into.

  • object_key (str) – Natural string identifier for this record. Digested via HMAC before being stored on disk.

  • classification (SensitivityClass) – The SensitivityClass for this record.

  • schema_version (int) – Envelope schema version to stamp on the row.

  • written_at (datetime) – Timezone-aware write timestamp.

  • payload (bytes) – Plaintext envelope bytes. Encrypted at the column boundary.

  • write_provenance (str) – Human-readable string identifying the write origin.

  • source_event_id (str | None) – Optional opaque domain-event identifier for audit trails.

  • expected_revision_id (str | None) – Optional optimistic-concurrency guard.

Return type:

None

save_many(writes)[source]

Encrypt and upsert several payloads in one SQL unit of work.

Return type:

None

Parameters:

writes (tuple[SecureObjectWrite, ...])

namespace_payload_hashes(namespace)[source]

Return {object_key_digest: payload_hash} for every row in namespace.

A decryption-free scan of the object_key (HMAC digest) and payload_hash columns, for diff-based writers that persist a namespace as one row per logical entry: an entry whose freshly-serialised payload_hash matches the stored value is unchanged and need not be rewritten. The digest is the same value secure_object_key_digest() produces for the entry’s natural key, so a caller compares secure_object_key_digest(key) against these keys without decrypting anything.

Return type:

dict[bytes, str | None]

Parameters:

namespace (str)

apply_batch(writes, deletions=())[source]

Atomically upsert writes and remove deletions in one unit of work.

The single session_scope transaction commits every upsert and every digest-addressed deletion together, so a diff-based per-row writer (e.g. the transaction catalogue) keeps the all-or-nothing guarantee the whole-blob save had — including when the same call must also commit sibling-catalogue writes (bucket-event history, invoices) passed in writes. A crash mid-batch rolls the whole unit back.

Deletions are addressed by raw HMAC digest (see SecureObjectDeletion); the digest passes straight through the HashedLookup column comparison without re-hashing.

Return type:

None

Parameters:
save_with_raw_key(*, namespace, hashed_object_key, classification, schema_version, written_at, payload, write_provenance='secure-object-repository', source_event_id=None, expected_revision_id=None)[source]

Encrypt and upsert one byte payload keyed by a pre-computed digest.

The 32-byte hashed_object_key is passed straight through the HashedLookup column without re-hashing. Used by the archive restore path to round-trip rows whose natural key is not present in the bundle (e.g. the path-keyed setup-profile and inventory namespaces).

Parameters:

  • namespace (str) – Storage namespace string.

  • hashed_object_key (bytes) – 32 raw HMAC-SHA256 bytes (the digest produced by HashedLookup.compute under the same master key the row was originally written with).

  • classification (SensitivityClass) – SensitivityClass to upsert at.

  • schema_version (int) – Envelope schema version captured on the row.

  • written_at (datetime) – Timezone-aware datetime captured on the row.

  • payload (bytes) – Plaintext envelope bytes (the column encrypts).

  • write_provenance (str) – Human-readable string identifying the write origin (e.g. caller module or operation name). Defaults to the repository’s default provenance marker.

  • source_event_id (str | None) – Optional opaque identifier of the domain event that triggered this write; stored verbatim for audit trails.

  • expected_revision_id (str | None) – Optional optimistic-concurrency guard; when supplied the upsert is rejected if the row’s current revision does not match.

Raises:
Return type:

None

peek_metadata(namespace, object_key)[source]

Return SecureObjectMetadata for one object without decrypting it.

Returns None when no row matches. Never decrypts the payload column; callers use this to fingerprint an envelope they intend to discard (e.g. the workflow-state reset recovery path).

Return type:

SecureObjectMetadata | None

Parameters:

  • namespace (str)

  • object_key (str)

delete(namespace, object_key)[source]

Delete one object if it exists.

Return type:

bool

Parameters:

  • namespace (str)

  • object_key (str)