aeat.application.command_search._index module

Hybrid command index: per-column BM25 lexical + model2vec semantic, RRF-fused.

Ranks a free-text query against the command corpus so the search meta-tool bridges the operator’s natural vocabulary to a command’s own tokens. Three cooperating parts, the same shape the corpus grounding search uses (ADR mcp-progressive-discovery P2):

  • a per-column FTS5 lexical index that weights the command KEY and TOOL NAME above curated OUTCOME ALIASES above the human DESCRIPTION above the per-verb HELP, so a homonym token in a low-value column no longer outranks the correct command whose key carries it (the import mis-rank the review found);

  • a model2vec semantic side (behind the capability-gated aeat-cli[search] extra, reused through the corpus_search public facade) that embeds the command docs and the query into one vector space and cosine-ranks them, so a concept query reaches the right verb across a Spanish/English or concept/verb vocabulary gap the stemmer cannot bridge;

  • Reciprocal Rank Fusion (RRF, k=60) over the two rankings.

The semantic side is a strict enhancement: when the search extra (or SQLite FTS5) is absent, CommandIndex.search() degrades cleanly — FTS5 BM25 alone, then a pure-Python token-overlap scorer — so a minimal install always ranks better than a bare substring match and never hard-fails.

The index is SDK-independent and pure (it takes plain command documents), so it is unit-tested directly without the MCP transport.

RRF_K

RRF damping constant. The canonical k=60 keeps a strong hit on one ranker from being drowned by the other ranker’s long tail.

class CommandDoc(**data)[source]

Bases: BaseModel

One command’s searchable document, split into weighted columns.

The four tiers rank a query hit by WHERE it lands: key_and_name (the command key tokens and tool name) is the most discriminative, aliases carries curated outcome vocabulary for composite verbs, description is the human summary, and help is the per-verb CLI help — the noisiest tier. Each tier becomes its own BM25-weighted FTS5 column; a query token hitting the key ranks above the same token hitting the help.

Parameters:

  • command_key (str)

  • tool_name (str)

  • key_and_name (str)

  • description (str)

  • aliases (str)

  • help (str)

command_key: str
tool_name: str
key_and_name: str
description: str
aliases: str
help: str
property combined_text: str

The union of every tier, for the semantic embedding and token fallback.

class CommandHit(**data)[source]

Bases: BaseModel

One ranked command match.

Parameters:
command_key: str
tool_name: str
rank: int
score: float
class CommandIndex(docs, *, query_embedder=None, enable_semantic=True)[source]

Bases: object

A hybrid searchable index over the command corpus.

Fuses a per-column FTS5 BM25 lexical ranking with a model2vec semantic cosine ranking via RRF; both degrade cleanly (FTS5-only, then token-overlap only) so a minimal install keeps a working search. search() returns ranked CommandHit records for a free-text query.

Parameters:
search(query, *, limit=20)[source]

Return up to limit ranked command hits for query.

A blank query or one with no searchable terms returns no hits. The lexical side ranks the matching commands (per-column BM25, or token-overlap when FTS5 is absent); the semantic side re-ranks that candidate set by cosine similarity when the search extra is present; the two rankings are RRF-fused. When the semantic side is unavailable the fused order is exactly the lexical order.

Return type:

tuple[CommandHit, ...]

Parameters:
build_command_index(docs, *, query_embedder=None, enable_semantic=True)[source]

Build a CommandIndex from the command documents.

Return type:

CommandIndex

Parameters: