aeat.adapters.persistence.storage._rotation module

Master-key rotation for ciphertext-at-rest envelopes.

The substrate’s at-rest encryption derives a per-consumer key from the project master key via HKDF-SHA256 over a stable, per-consumer hkdf_context. Callers supply the old and new keys as MasterKeyProvider instances. Rotating the master key requires walking every consumer’s persisted ciphertext, decrypting under the old key, and re-writing under the new key. This module is the single sanctioned path for that operation.

Rotation operates at the bytes level - it never parses the inner adapters.persistence.storage.Envelope payload. That keeps the rotation contract content-preserving across every consumer’s payload type without requiring rotation code to know the typed payload schemas.

The rotation is per-file atomic - each cipher envelope is re-written via tempfile + os.replace so a crash mid-rotation leaves either the old or the new ciphertext on disk, never a torn state. The whole rotation is not transactional across files: both keys must remain available until rotation completes.

Detection of “already rotated” works by attempting to decrypt under the new key first; on AEAD-tag verify success the file is skipped. On failure, the helper falls back to the old key.

class RotationPlanEntry(**data)[source]

Bases: BaseModel

One consumer’s directory + HKDF context that the rotation must visit.

Variables:

  • store_dir – Directory that contains the consumer’s *.envelope.json files (or, when target_filename is set, the directory containing that specific file).

  • hkdf_context – The same hkdf_context the consumer’s repository uses at save / load time.

  • envelope_suffix – Filename suffix the consumer uses; defaults to .envelope.json (matches every repository). Ignored when target_filename is set.

  • target_filename – Optional exact filename inside store_dir. Use this for single-file consumers whose on-disk filename does not end in .envelope.json (e.g. usage-ratios.json written by the usage-ratios service). When set, the rotation visits exactly store_dir / target_filename and ignores every other file in the directory.

Parameters:

  • store_dir (Path)

  • hkdf_context (bytes)

  • envelope_suffix (str)

  • target_filename (str | None)

store_dir: Path
hkdf_context: bytes
envelope_suffix: str
target_filename: str | None
lock_path_for(envelope_path)[source]

Return the writer-canonical lock target for envelope_path.

Aligns the rotation’s exclusive_file_lock target with the sidecar lock the consumer’s writer acquires at save() time. Without this alignment, rotation and writer would contend on different .lock files and lose the OS-level serialisation the lock was meant to provide.

  • Multi-file envelopes (envelope_suffix set, default .envelope.json): the writer convention is <id>.lock ( lock_target_for helpers). Strip the configured envelope_suffix from the envelope name and append .lock.

  • Single-file envelopes (target_filename set, e.g. usage-ratios.json): the writer convention is <base>.lock (target.with_suffix('.lock')). Use Path.with_suffix() directly.

The lock file exclusive_file_lock actually opens is the returned path with an additional .lock suffix appended; the rotation and writer therefore land on the same lock-byte target.

Return type:

Path

Parameters:

envelope_path (Path)

class RotationSummary(**data)[source]

Bases: BaseModel

Frozen result of a rotate_master_key() call.

Variables:

  • rotated – Count of envelope files re-encrypted under the new key.

  • skipped – Count of envelope files already decryptable under the new key (resume idempotency).

  • errors – Count of envelope files that could not be parsed, decrypted under either key, or re-encrypted.

Parameters:
rotated: int
skipped: int
errors: int
rotate_master_key(plan, *, old_master_key_provider, new_master_key_provider)[source]

Re-encrypt every envelope listed in plan under the new master key.

Rotation contract:

  • For each envelope file, first attempt decryption under the new master-key provider. On success the file is already rotated; bump skipped and continue. (Resume-idempotency contract: a half-complete rotation can be re-run safely.)

  • On failure, try the old master-key provider. On success, re-encrypt the recovered plaintext bytes under the new provider, build a fresh CipherEnvelope, and replace the on-disk file atomically.

  • On failure under both providers, log the path and bump errors. Rotation continues so partial-rotation ground state is visible in the summary.

Parameters:
Return type:

RotationSummary

Returns:

A frozen RotationSummary.

default_rotation_plan(settings)[source]

Return the canonical rotation plan as a tuple of RotationPlanEntry records.

Enumerates every master-key-encrypted file-envelope consumer’s directory + HKDF context. Operators with custom directories / additional consumers pass an extended plan to rotate_master_key() directly.

Scope boundary: this plan covers only the *.envelope.json file consumers whose ciphertext is derived directly from the project master key (via the per-consumer HKDF context above). The SQL secure_objects store is NOT in this plan and intentionally so: its payloads are encrypted under the per-bucket DEK (the column layer resolves the active adapters.persistence.storage.master_key._bucket_session.BucketSession DEK, not the master key). A master-key / passphrase custody change rewraps that DEK without changing its value, so the secure_objects ciphertext stays valid and never requires re-encryption on master-key rotation.

Return type:

tuple[RotationPlanEntry, ...]

Parameters:

settings (_RotationPlanSettings)

rotate_blob_stores(blob_store_roots, *, old_master_key_provider, new_master_key_provider)[source]

Re-wrap every blob’s per-record DEK across the given blob-store roots.

The blob store wraps each blob’s DEK directly under the master key; on master-key rotation, every wrapped DEK must be re-wrapped or the blob is unrecoverable. This helper composes the per-store EncryptedBlobStore.rotate_master_key() results into a single summary.

Parameters:
Return type:

RotationSummary

Returns:

A frozen RotationSummary covering every visited blob.

default_blob_store_roots(settings)[source]

Return the canonical blob-store roots covered by master-key rotation.

The substrate persists wrapped DEKs in:

Each root is a directory whose blobs/<hex[:2]>/<hex>.manifest.json files carry the per-blob wrapped_dek field. Operators with custom blob stores extend this tuple before calling rotate_blob_stores().

Roots that resolve to the same absolute path (operator override / shared deployment) are deduplicated so the rotation does not walk the same blob twice.

Return type:

tuple[Path, ...]

Parameters:

settings (_BlobStoreSettings)