cadrumo.adapters.persistence.storage.sql.secure_objects module

Encrypted SQL byte-object repository for sensitive application payloads.

class SecureObjectMigrationTarget(namespace, object_key, expected_class, current_version)[source]

Bases: NamedTuple

One explicitly governed row participating in an atomic schema cutover.

Parameters:
namespace: str

Alias for field number 0

object_key: str

Alias for field number 1

expected_class: SensitivityClass

Alias for field number 2

current_version: int

Alias for field number 3

class SecureObjectRepository(*, engine=None, namespace_registry=None, active_session_bucket_id=None, require_secure_active_session=False)[source]

Bases: SecureObjectWriteOperations

Repository over encrypted byte objects stored in the primary database.

Parameters:
static object_key_digest(object_key)[source]

Derive the stored lookup digest for one natural object key.

Return type:

bytes

Parameters:

object_key (str | bytes)

property namespace_registry: StorageHierarchyRegistry | None

Return the bound hierarchy registry, if any.

The registry is a StorageHierarchyRegistry.

property engine: Engine

Return the bound SQLAlchemy Engine.

Exposed so a sibling plaintext ORM table (e.g. a derived, non-sensitive routing index) can be written in the SAME database file and, where the driver supports it, the same transaction as this repository’s encrypted rows – without duplicating the bucket-to-engine routing this repository already resolved at construction.

guarded_session_scope()[source]

Yield a session over this repository’s engine, session-checked first.

The counterpart to engine for a sibling plaintext table. Taking the engine directly is the one route into this database that skips _check_session_freshness(), which every operation on this repository otherwise applies – so a caller reading a routing index through a raw scope reads after a session seal, an idle expiry, or a move to another bucket, with nothing to stop it.

The existing callers are safe only because each happens to perform a guarded load first. That is protection by call ORDER: it holds until someone adds an entry point that reaches the raw read first, and nothing states the ordering or would notice it changing. This method makes the check structural rather than incidental.

No namespace is passed: a sibling table is not a secure-object namespace, so only the session half of the check applies.

Return type:

Generator[Session]

exists(namespace, object_key)[source]

Return whether namespace / object_key is present.

Return type:

bool

Parameters:

  • namespace (str)

  • object_key (str)

exists_many(namespace, object_keys)[source]

Return the subset of object_keys present in namespace.

The set-based companion of exists(), and the implementation behind it: one indexed IN (...) read per key slice answers the whole membership question, where per-key exists() calls cost one session and one statement each — the dominant cost of a bulk content-addressed ingest that deduplicates against what is already stored. Natural keys are HMAC-digested exactly as the object_key column stores them, so membership is decided on the digest and reported back as the caller’s natural keys. Nothing is decrypted.

Return type:

frozenset[str]

Parameters:
exists_by_raw_key(namespace, hashed_object_key)[source]

Return whether namespace carries a row with this raw HMAC digest.

Used by the archive restore pipeline when the natural key was not present in the source bundle. Same master-key constraint as save_with_raw_key().

Return type:

bool

Parameters:

  • namespace (str)

  • hashed_object_key (bytes)

iter_all_records_raw(*, namespace=None, batch_size=256)[source]

Yield every stored row as a SecureObjectRawRow without decryption.

Walks every row in secure_objects ordered by (namespace, object_key) without attempting to decrypt the payload. The query bypasses the encrypted-column type decorators so rows sealed under a rotated master key still surface verbatim — this is what the outbound sync coordinator’s ciphertext-layer mirror consumes, mirroring on-wire ciphertext to a remote storage provider without ever decrypting domain data.

Parameters:

  • namespace (str | None) – Optional exact namespace filter applied by SQL before iteration. None preserves the archive-wide traversal.

  • batch_size (int) – SQLAlchemy yield_per chunk size. The default keeps memory bounded for very large substrates while still amortising session overhead across multiple rows.

Yields:

One SecureObjectRawRow per persisted row. The order is (namespace ASC, object_key ASC) so consumers can checkpoint progress deterministically.

Return type:

Iterator[SecureObjectRawRow]

list_namespaces()[source]

Return the distinct namespaces present in secure_objects sorted.

Used by the integrity diagnostic so consumers do not have to hardcode the namespace list (which drifts as new domain repositories register their own namespaces).

Return type:

tuple[str, ...]

quarantine_unreadable_rows()[source]

Move every undecryptable row into secure_objects_quarantine.

Iterates every populated namespace, probes each row’s payload through the row-identity-bound secure-object decryptor, and for rows that fail tag verification copies the original (encrypted) payload plus all metadata into the quarantine table, then deletes the row from secure_objects. The quarantine table mirrors secure_objects with the addition of a quarantined_at timestamp so the archive is auditable.

Decryptable rows are NOT touched; the quarantine table is created on first use; nothing is auto-deleted from the user’s data even after quarantine. The operator can recover the quarantined rows manually from the table if a missing master key is later recovered (for example, restored from a recovery key backup).

Return type:

tuple[SecureObjectNamespaceIntegrity, ...]

Returns:

A tuple of SecureObjectNamespaceIntegrity records describing how many rows were quarantined per namespace.

probe_namespace_integrity(namespace)[source]

Count decryptable and undecryptable rows in namespace.

Returns a SecureObjectNamespaceIntegrity for the namespace.

This method answers a strictly crypto-layer question – can the payload ciphertext be unwrapped under the current master key – and intentionally bypasses the classification, schema-version, and namespace-registration contracts that consumer reads enforce. It probes over whatever namespaces are physically present in the table (as surfaced by list_namespaces()), including orphan, legacy, and unregistered ones – those are precisely the rows a repair diagnostic exists to find – so it runs the session / route freshness check but not the namespace-registration check. Used by aeat config repair to surface namespaces holding rows from a prior keychain master-key generation.

Return type:

SecureObjectNamespaceIntegrity

Parameters:

namespace (str)

iter_namespace_decryptability(namespace)[source]

Yield SecureObjectDecryptabilityRow metadata for one namespace.

This is the row-level companion to probe_namespace_integrity(). It decrypts only to validate the AEAD tag, never returns plaintext, and exposes the HMAC lookup digest plus storage metadata needed by repair diagnostics. Like its namespace-level companion it is a crypto-layer probe over whatever namespaces are physically present, so it runs the session / route freshness check but not the namespace-registration check.

Return type:

Iterator[SecureObjectDecryptabilityRow]

Parameters:

namespace (str)

list_keys(namespace)[source]

Return stored lookup digests under namespace as hex strings.

Natural object keys are HMAC digested before storage and cannot be recovered from the index. Domain repositories that need natural IDs should iterate list_records() and read IDs from decrypted payloads.

Return type:

tuple[str, ...]

Parameters:

namespace (str)

list_records(namespace, *, expected_class, max_supported_version)[source]

Yield secure-object rows under namespace or fail on unreadable rows.

The default listing path is fail-closed: it walks the namespace through iter_records_with_failures() and raises SecureObjectUnreadableError before yielding a partial readable subset. Use iter_records_with_failures for explicit mixed readable/unreadable diagnostics.

Parameters:

  • namespace (str) – The storage namespace whose rows are listed.

  • expected_class (SensitivityClass) – The SensitivityClass all rows in this namespace must carry.

  • max_supported_version (int) – The consumer’s current schema_version ceiling; a row above it, or below it without a complete registered upgrade chain, is treated as unreadable.

Return type:

Iterator[SecureObjectRecord]

load_many(namespace, object_keys, *, expected_class, max_supported_version)[source]

Yield requested secure-object rows or fail closed on unreadable rows.

This is the targeted equivalent of list_records(): it performs a single WHERE namespace = ? AND object_key IN (...) read for the requested natural keys, decrypts matching rows, and raises SecureObjectUnreadableError before yielding a partial readable subset if any matching row is unreadable. Missing keys are omitted, mirroring repeated load() calls that return None for absent rows. expected_class is the SensitivityClass every returned row must be classified under; a mismatch fails closed.

Return type:

Iterator[SecureObjectRecord]

Parameters:
load_many_current(namespace, object_keys, *, expected_class, current_version, refuse_legacy)[source]

Load exact current-version rows in one addressed snapshot.

Every selected outer row is checked before any payload is decrypted or yielded. A legacy row therefore reaches the owning repository’s explicit-cutover refusal without an implicit upgrade or partial read.

Core types: SensitivityClass.

Return type:

Iterator[SecureObjectRecord]

Parameters:
migrate_many_atomically(namespace, object_keys, *, expected_class, current_version, validate_upgraded_payloads, write_provenance)[source]

Validate every upgraded payload, then persist all replacements atomically.

Older rows are decrypted and chain-upgraded through the normal read policy. The caller receives the complete natural-keyed payload set in validate_upgraded_payloads before any replacement is written. Only after that callback succeeds are all older rows replaced in one compare-and-swap batch, so a malformed sibling or concurrent write leaves every original row intact.

Core types: SensitivityClass.

Return type:

Mapping[str, SecureObjectRecord]

Parameters:
migrate_targets_atomically(targets, *, validate_upgraded_payloads, write_provenance)[source]

Validate and CAS-replace an explicit cross-namespace target set once.

Return type:

Mapping[tuple[str, str], SecureObjectRecord]

Parameters:
iter_many_with_failures(namespace, object_keys, *, expected_class, max_supported_version)[source]

Yield readable/unreadable outcomes for requested natural object keys.

Rows are selected by raw HMAC digests derived from object_keys and returned in stored digest order. Missing keys produce no item, matching load() returning None. Present rows use the same classification, schema-version, AEAD, and revision-lineage checks as namespace scans. expected_class is the SensitivityClass every yielded row must be classified under; a mismatch fails closed.

Return type:

Iterator[SecureObjectRecord | SecureObjectUnreadable]

Parameters:
iter_records_with_failures(namespace, *, expected_class, max_supported_version, batch_size=256)[source]

Yield a typed outcome per stored row under namespace.

Each row is represented by either a SecureObjectRecord (the row decrypts cleanly and matches the consumer’s classification and schema-version contract) or a SecureObjectUnreadable (the on-wire ciphertext exists but cannot be decrypted under the current master key, or its metadata fails the consumer’s contract).

The iterator is fault-isolated: a failure on row N does not prevent rows > N from being inspected. Consumers count the failures and decide how to report them; nothing is auto-deleted.

Parameters:

  • namespace (str) – The storage namespace whose rows are scanned.

  • expected_class (SensitivityClass) – The SensitivityClass all rows in this namespace must carry; rows with a differing classification are yielded as SecureObjectUnreadable.

  • max_supported_version (int) – The consumer’s current schema_version ceiling. Rows above it, or below it without a complete registered upgrade chain, are yielded as SecureObjectUnreadable.

  • batch_size (int) – SQLAlchemy yield_per chunk size for the raw row scan. The default keeps memory bounded for large namespaces while preserving deterministic (object_key ASC) order.

Yields:

One SecureObjectListItem per stored row — either a SecureObjectRecord or a SecureObjectUnreadable.

Raises:

StorageValidationError – When batch_size is less than 1.

Return type:

Iterator[SecureObjectRecord | SecureObjectUnreadable]

load(namespace, object_key, *, expected_class, max_supported_version)[source]

Load and decrypt one secure-object row, returning None when absent.

Returns a SecureObjectRecord when the row is present and decrypts under the expected class/version.

Parameters:

  • namespace (str) – The storage namespace to look in.

  • object_key (str) – The natural string key identifying the record.

  • expected_class (SensitivityClass) – The SensitivityClass the consumer expects.

  • max_supported_version (int) – Highest schema_version the consumer supports.

Return type:

SecureObjectRecord | None

peek_metadata(namespace, object_key)[source]

Return SecureObjectMetadata for one object without decrypting it.

Returns None when no row matches. Never decrypts the payload column; callers use this to fingerprint an envelope they intend to discard (e.g. the workflow-state reset recovery path).

Return type:

SecureObjectMetadata | None

Parameters:

  • namespace (str)

  • object_key (str)

peek_many_schema_versions(namespace, object_keys)[source]

Return schema versions for exact natural keys without loading ciphertext.

Return type:

Mapping[str, int]

Parameters:
peek_many_revision_ids(namespace, object_keys)[source]

Return the stored revision id of each present natural key without loading ciphertext.

A key with no row is absent from the result; a present row written before revisions were recorded maps to None.

Return type:

Mapping[str, str | None]

Parameters:
delete(namespace, object_key)[source]

Delete one object if it exists.

Return type:

bool

Parameters:

  • namespace (str)

  • object_key (str)