aeat.agent.eval._replay module

Determinism-replay for operator tool calls.

Captures golden tool-call records (the tool name, its arguments, and the resolved invocation) and replays them, asserting the resolution is byte-identical. A non-deterministic mapping is an assurance failure in regulated work: the same operator action must always project onto the same deterministic CLI invocation. Pure: the resolver is injected, so this module stays free of the entrypoints layer it replays.

ToolCallResolver

A resolver projects a (tool_name, args) call onto its deterministic invocation.

alias of Callable[[str, tuple[str, …]], tuple[str, …]]

class GoldenToolCall(**data)

Bases: BaseModel

A captured tool call and its resolved invocation.

Parameters:

• tool_name (str)

• args (tuple[str, ...])

• invocation (tuple[str, ...])

tool_name: str

args: tuple[str, ...]

invocation: tuple[str, ...]

record_tool_call(tool_name, args, *, resolve)

Capture a golden tool call by resolving it once.

Return type:

GoldenToolCall

Returns:

GoldenToolCall containing the original call and invocation.

Parameters:

• tool_name (str)

• args (tuple[str, ...])

• resolve (Callable[[str, tuple[str, ...]], tuple[str, ...]])

replay_tool_call(record, *, resolve)

Return True when re-resolving the record yields its captured invocation.

Return type:

bool

Parameters:

• record (GoldenToolCall)

• resolve (Callable[[str, tuple[str, ...]], tuple[str, ...]])

divergent_replays(records, *, resolve)

Return the records whose replay diverges from their captured invocation.

Return type:

tuple[GoldenToolCall, ...]

Returns:

Tuple of divergent GoldenToolCall records.

Parameters:

• records (Iterable[GoldenToolCall])

• resolve (Callable[[str, tuple[str, ...]], tuple[str, ...]])