aeat.core.corpus_manifest package

Directory-level integrity manifest for CORPUS-class on-disk data.

The substrate’s SensitivityClass.CORPUS policy mandates SHA-256 integrity tracking for plaintext-at-rest reference material. This module is the canonical implementation: a single manifest model covers every file under a corpus root with a per-file SHA-256 + size record, plus a self-attesting manifest_sha256 so a manifest-only tamper is detectable.

Manifests are plaintext JSON on disk (corpus material is plaintext; the manifest is the integrity gate, not the secrecy gate). Per-record fields are validated against path-traversal at construction.

Operator workflow (via the aeat security verify-corpus CLI):

  • aeat security verify-corpus --corpus manuals — re-walk the manuals root and exit non-zero with a per-file diff on drift.

  • aeat security verify-corpus --corpus manuals --regenerate — re-walk and rewrite the manifest in place after intentional corpus updates.

This module also builds and verifies distributable corpus bundles: a single .zip archive carrying every corpus file plus an embedded CorpusManifest (see build_corpus_bundle() and verify_corpus_bundle()), for offline installation of the bundled corpus checksummed against its own manifest. The SHA-256 manifest alone is an integrity gate, not an authenticity gate; _bundle_signing (re-exported here) adds the Ed25519 authenticity layer on top – see sign_corpus_bundle() and verify_corpus_bundle_signature().

class CorpusEntry(**data)[source]

Bases: BaseModel

One file’s integrity record under a corpus root.

Variables:

  • relative_path – POSIX-style path from the corpus root. Validated against path-traversal at construction.

  • sha256 – Lower-case hex SHA-256 of the file’s bytes.

  • content_length – Size of the file in bytes (>= 0; the empty file is permitted).

Parameters:

  • relative_path (str)

  • sha256 (str)

  • content_length (int)

relative_path: str
sha256: str
content_length: int
class CorpusManifest(**data)[source]

Bases: BaseModel

Self-attesting manifest covering every file under a corpus root.

The manifest_sha256 field is a SHA-256 over the canonical JSON serialisation of (manifest_version, corpus_root_name, generated_at, entries) with sorted keys. On load, the same digest is re-derived and compared; a mismatch raises CorpusManifestTamperError.

Variables:

  • manifest_version – Wire-format version. Higher than the consumer supports raises CorpusManifestError at load time.

  • corpus_root_name – Stable identifier for the corpus ("manuals", "legal", etc.).

  • generated_at – UTC timestamp the manifest was built.

  • entries – Frozen tuple of CorpusEntry records, sorted by relative_path for deterministic manifest_sha256.

  • manifest_sha256 – Self-attesting digest. Re-computed on load.

Parameters:
manifest_version: int
corpus_root_name: str
generated_at: datetime
entries: tuple[CorpusEntry, ...]
manifest_sha256: str
class CorpusManifestDiff(**data)[source]

Bases: BaseModel

Result of comparing a manifest against the live corpus on disk.

A manifest is in drift iff any of added / removed / changed is non-empty.

Parameters:
corpus_root_name: str
added: tuple[str, ...]
removed: tuple[str, ...]
changed: tuple[str, ...]
property is_clean: bool

Return True iff every tracked file’s hash matches the manifest.

build_corpus_manifest(corpus_root, *, corpus_root_name, generated_at=None)[source]

Walk corpus_root and build a freshly self-signed manifest.

Parameters:

  • corpus_root (Path) – The corpus directory to walk.

  • corpus_root_name (str) – Stable identifier for this corpus (logged in the manifest body so an operator can grep for it).

  • generated_at (datetime | None) – Optional override for the timestamp. When None the canonical clock core.time.now() is consulted, so the deterministic-output seam (core.time.frozen_clock()) pins it under replay; an explicit value still overrides.

Return type:

CorpusManifest

Returns:

A CorpusManifest covering every regular file under corpus_root.

Raises:
verify_corpus_manifest(corpus_root, *, manifest)[source]

Compare manifest against the live state of corpus_root.

Returns a diff enumerating files added since the manifest was built, files removed, and files whose SHA-256 changed. Empty diff = clean.

Parameters:

  • corpus_root (Path) – The corpus directory to verify.

  • manifest (CorpusManifest) – The manifest to verify against.

Return type:

CorpusManifestDiff

Returns:

A CorpusManifestDiff enumerating added, removed, and changed files. An empty diff means the corpus is clean.

Raises:

FileNotFoundError – If corpus_root does not exist.

save_corpus_manifest(manifest, target)[source]

Atomically persist manifest as JSON to target.

Return type:

None

Parameters:
load_corpus_manifest(target)[source]

Load and verify a manifest’s self-attesting digest.

Parameters:

target (Path) – Source file. Must exist.

Return type:

CorpusManifest

Returns:

The validated CorpusManifest loaded from target.

Raises:

  • FileNotFoundError – If target does not exist.

  • CorpusManifestError – If the on-disk version exceeds the consumer’s supported version, or the JSON is structurally invalid.

  • CorpusManifestTamperError – If the manifest’s recorded manifest_sha256 does not match the digest re-derived from the rest of its body.

manifest_path_for(corpus_root)[source]

Return the canonical manifest sidecar path inside corpus_root.

Return type:

Path

Parameters:

corpus_root (Path)

assert_corpus_clean(corpus_root)[source]

Verify the on-disk manifest matches the corpus root.

Loads the manifest sidecar, walks the corpus, and raises CorpusManifestDriftError on any drift. This is the operator-facing assertion used by the CI gate.

Return type:

None

Parameters:

corpus_root (Path)

class CorpusBundleVerification(**data)[source]

Bases: BaseModel

Result of verifying a bundle’s archived files against its embedded manifest.

An empty missing / unexpected / mismatched triple means the bundle is clean and safe to extract. missing is a file the manifest declares but the archive does not carry; unexpected is an archive member the manifest does not declare; mismatched is an archived file whose SHA-256 or byte length disagrees with its manifest record.

Parameters:
manifest: CorpusManifest
missing: tuple[str, ...]
unexpected: tuple[str, ...]
mismatched: tuple[str, ...]
property is_clean: bool

Return True iff every archived file matches the embedded manifest.

build_corpus_bundle(corpus_root, *, corpus_root_name, output_path, generated_at=None)[source]

Pack corpus_root into a checksummed zip bundle at output_path.

Walks corpus_root with the same file-selection rules as build_corpus_manifest() (hidden files and the manifest sidecar itself are excluded), builds the manifest, then writes a new zip archive containing the manifest under _BUNDLE_MANIFEST_MEMBER plus every corpus file under its POSIX-relative path. The write is atomic: the archive is built at a temporary path in the same directory and renamed into place only on success, so a failure mid-write never leaves a partial bundle at output_path.

Parameters:

  • corpus_root (Path) – The corpus directory to bundle.

  • corpus_root_name (str) – Stable identifier recorded in the manifest (mirrors build_corpus_manifest()).

  • output_path (Path) – Destination .zip path. Parent directories are created as needed; an existing file at this path is overwritten.

  • generated_at (datetime | None) – Optional override for the manifest timestamp; see build_corpus_manifest().

Return type:

CorpusManifest

Returns:

The CorpusManifest embedded in the written bundle.

Raises:
verify_corpus_bundle(bundle_path)[source]

Verify a zip bundle’s archived files against its embedded manifest.

Loads the manifest member (_BUNDLE_MANIFEST_MEMBER), re-derives its self-attesting digest (a mismatch means the manifest member itself was tampered with, raising CorpusManifestTamperError), then re-hashes every other archive member and compares against the manifest’s per-file records.

Parameters:

bundle_path (Path) – Path to the .zip bundle to verify.

Return type:

CorpusBundleVerification

Returns:

A CorpusBundleVerification enumerating any missing, unexpected, or hash-mismatched files. An empty result means the bundle is clean.

Raises:

  • FileNotFoundError – If bundle_path does not exist.

  • CorpusBundleError – If bundle_path is not a valid zip archive, or the embedded manifest is absent, structurally invalid, or at an unsupported version.

  • CorpusManifestTamperError – If the manifest member’s recorded manifest_sha256 does not match its own body.

assert_corpus_bundle_verifies(bundle_path)[source]

Verify bundle_path and raise on any drift; return the embedded manifest on success.

The operator-facing assertion: call this before extracting or trusting a downloaded/copied bundle. Raises CorpusBundleVerificationError naming every missing, unexpected, or mismatched file when the bundle does not verify clean.

Return type:

CorpusManifest

Parameters:

bundle_path (Path)

Submodules