cadrumo.application.provisioning_runtime module

Canonical local-runtime lifecycle operations for model provisioning.

class RuntimeResident(**data)[source]

Bases: BaseModel

One model the local runtime reports as currently loaded.

Sourced from the runtime’s own /api/ps report, which is what makes the figure attributable: it is memory held by models this runtime loaded, as distinct from the device-wide free shortfall NVML measures. The gap between the two is peer-process usage.

Parameters:

data (Any)

name: str
size_bytes: int | None
size_vram_bytes: int | None
ollama_endpoint(chat_url, path)[source]

Derive a sibling runtime endpoint (/api/<path>) from the configured chat URL.

Return type:

str

Parameters:
forget_runtime_unreachable(settings=None)[source]

Drop the remembered unreachable answer for the configured runtime.

Called by the one action that changes the answer on purpose – starting the runtime – so the reads that follow ask the endpoint instead of replaying a failure recorded seconds before the server came up.

Return type:

None

Parameters:

settings (Settings | None)

read_runtime_residents(settings=None)[source]

Read the local runtime’s resident model set, or None when it could not be read.

A short-timeout GET /api/ps. Loads nothing and runs no inference. The None return is a distinct state from an empty tuple and the two must not be collapsed: empty means “measured, nothing is resident”, which permits attributing a device shortfall to peer processes, while None means the attribution input is missing and assess_model_load_contention() refuses rather than guessing.

Return type:

tuple[RuntimeResident, ...] | None

Parameters:

settings (Settings | None)

cadrumo_selected_models(settings=None)[source]

Return the model identifiers Cadrumo itself selected, the only ones it may unload.

The boundary of the unload action. A model resident in the local runtime that Cadrumo did not select belongs to whoever loaded it, and evicting it would be the same overreach as signalling a peer process – it is reported in the contention snapshot and never acted on.

Return type:

frozenset[str]

Parameters:

settings (Settings | None)

runtime_model_names_match(left, right)[source]

Return whether two runtime model names denote the same tagged model.

An untagged name means :latest to the runtime, so qwen3 and qwen3:latest match while qwen3:1.7b and qwen3:8b do not – a different size is a different download and a different memory claim. Every “is this exact model present or loaded” question uses this comparison.

Return type:

bool

Parameters:
matches_selected_model(name, selected)[source]

Return whether a model name belongs to a family Cadrumo selected, ignoring the tag suffix.

Family-level on purpose: it bounds which models Cadrumo may unload or remove and which residents count as reclaimable, so a differently sized variant of a selected model stays within Cadrumo’s reach. It never answers whether one exact model is present; runtime_model_names_match() does.

Return type:

bool

Parameters:
class ContentionSnapshot(**data)[source]

Bases: ProvisioningOutcome

The measured verdict on whether one model load is safe to perform right now.

Read the fields as three separate claims. admitted is the decision. causes is ContentionCause and says why a refusal happened, keyed to the condition that applies – unloading a model Cadrumo selected, closing a peer application, or measuring what could not be measured. residents is the attribution evidence: the runtime’s own loaded set, which is what separates memory Cadrumo can reclaim from memory it may only observe.

Nothing on this model evicts, signals, or otherwise touches a process Cadrumo does not own. Pressure caused by a peer is reported and refused, never managed.

Parameters:
model: str
requirement_bytes: NonNegativeInt
safety_margin_bytes: NonNegativeInt
accelerator: AcceleratorKind
free_vram_bytes: int | None
free_system_memory_bytes: int | None
binding_free_bytes: int | None
residents: tuple[RuntimeResident, ...] | None
resident_attributed_bytes: int
unloadable_models: tuple[str, ...]
shortfall_bytes: int | None
admitted: bool
causes: tuple[ContentionCause, ...]
property required_bytes: int

Return the requirement plus the configured safety margin – the figure actually compared.

assess_model_load_contention(model, requirement_bytes, *, profile=None, residents=None, residents_measured=True, settings=None)[source]

Judge whether loading model is safe right now, failing closed on any unknown.

Acting fails closed where reporting fails open. The free figures, never the totals, are compared against the model’s declared requirement plus the configured safety margin. An unreadable free figure refuses the load: “could not tell” is not evidence of headroom, and is precisely the state that destroyed running work on the machine this was built for. The only escape is the explicit cadrumo_llm_contention_check_override setting, which admits an unmeasurable machine and never a measured shortfall.

On a measured shortfall the cause is attributed rather than assumed, because the attributions are not interchangeable. Memory held by the runtime’s own residents is reclaimable through unload_runtime_model() – and only for the models Cadrumo selected. Memory unexplained by those residents is a peer process’s, and the refusal says so instead of telling the operator to unload something they do not own.

This function never loads, pulls, evicts, or signals anything. It reads.

Parameters:

  • model (str) – The model identifier whose load is being judged.

  • requirement_bytes (int) – The model’s declared memory requirement.

  • profile (HardwareProfile | None) – Measured hardware; probed when omitted.

  • residents (tuple[RuntimeResident, ...] | None) – The runtime’s resident set; read when omitted. None together with residents_measured false means unreadable.

  • residents_measured (bool) – False when the resident set could not be read. Distinguishes an unreadable resident set from a measured-empty one.

  • settings (Settings | None) – Settings carrying the margin and override; loaded when omitted.

Return type:

ContentionSnapshot

Returns:

A ContentionSnapshot. This module never raises on absence; the dispatch boundary converts a non-admitted snapshot into its typed refusal.

class UnloadOutcome(**data)[source]

Bases: ProvisioningOutcome

The result of an explicit unload of one Cadrumo-selected resident model.

Never raises and never escalates: a model Cadrumo did not select, or one that is not resident, returns unloaded false with the reason. The action is self-scoped by construction – it asks the local runtime to release a model it loaded on Cadrumo’s behalf, and has no mechanism to affect any other process on the device.

Parameters:
model: str
unloaded: bool
was_resident: bool
unload_runtime_model(model, *, residents=None, residents_measured=True, settings=None)[source]

Release one Cadrumo-selected model from the local runtime, touching nothing else.

Two guards, in order, and both matter. The model must be one cadrumo_selected_models() names, so Cadrumo cannot evict a model another operator or application loaded into the shared runtime. It must then be resident, which is what keeps this action from becoming a load: the runtime’s release call would otherwise bring a non-resident model in before releasing it, and this module must never cause a model load.

The release is a zero-keep-alive request carrying no prompt, so the runtime drops the model without running inference.

Parameters:
Return type:

UnloadOutcome

Returns:

An UnloadOutcome describing what happened and, on refusal, why.

class LoadOutcome(**data)[source]

Bases: ProvisioningOutcome

The result of an explicit load of one pulled model into the runtime’s memory.

already_loaded marks a model that was resident before the request, so nothing was sent. contention is populated when the admission check refused, and is the reason nothing was loaded.

Parameters:
model: str
loaded: bool
already_loaded: bool
contention: ContentionSnapshot | None
elapsed_ms: int | None
load_runtime_model(model, requirement_bytes, *, profile=None, settings=None)[source]

Load a pulled model into the runtime’s memory so the first read does not pay the cold start.

Idempotent: a model that is already resident is reported loaded and nothing is sent. A model that is not on disk is refused rather than fetched, because a load must never become an implicit download. The admission check runs before the load, exactly as it does before a pull, and the result is confirmed against a re-read of the resident set.

The request carries no prompt, so the runtime loads the model without running inference, and a keep-alive long enough to outlast the gap between an operator’s load and their first read. Loading leaves the weights unchanged, so a remembered fitness verdict stays valid.

Return type:

LoadOutcome

Returns:

A LoadOutcome. Never raises.

Parameters:
class PullProgress(**data)[source]

Bases: BaseModel

One progress report from an in-flight model fetch.

total_bytes is absent early in a pull, before the runtime has resolved the manifest, so a renderer must tolerate a percentless report rather than computing a ratio against zero.

Parameters:

data (Any)

status: str
completed_bytes: int | None
total_bytes: int | None
property percent: int | None

Return whole-percent progress, or None when the total is not yet known.

class RuntimePullRecord(**data)[source]

Bases: BaseModel

The last model fetch this process attempted, for status projections.

Process-local on purpose: it answers “what did this session last try” for a status area, and a model name with a timestamp is not worth a new persisted record. A fresh process reports no pull rather than a stale one.

Parameters:

data (Any)

model: str
pulled: bool
attempted_at: datetime
bytes_fetched: int | None
failed_condition_id: str | None
last_runtime_pull()[source]

Return the most recent fetch this process attempted, or None.

Return type:

RuntimePullRecord | None

class PullOutcome(**data)[source]

Bases: ProvisioningOutcome

The result of an explicit model fetch, including a fetch that never started.

contention is populated when the pre-fetch admission check refused, and is the reason no bytes moved. Keeping the snapshot rather than flattening it to a string is what lets the caller name WHICH cause applied – unloading a resident Cadrumo selected and closing a peer application are different instructions and only one of them is ours to offer.

Parameters:
model: str
pulled: bool
contention: ContentionSnapshot | None
bytes_fetched: int | None
pull_runtime_model(model, requirement_bytes, *, profile=None, settings=None, on_progress=None)[source]

Fetch model into the local runtime, refusing BEFORE any bytes move.

The admission check runs first and a refusal returns without contacting the runtime at all. That ordering is the point of the action: a multi-gigabyte download that completes and then cannot be loaded has spent the operator’s bandwidth to arrive at the refusal it could have been given immediately.

Never raises, in keeping with every other action here. An unreachable runtime is a typed closed outcome – this function does not start a daemon, and nothing on this path pulls implicitly: reaching it at all requires an operator to have asked.

Parameters:

  • model (str) – Runtime identifier to fetch.

  • requirement_bytes (int) – The model’s declared memory requirement, checked against measured headroom before fetching.

  • profile (HardwareProfile | None) – A measured hardware profile; probed when omitted.

  • settings (Settings | None) – Resolved settings; loaded when omitted.

  • on_progress (Callable[[PullProgress], None] | None) – Called for each progress report the runtime emits. Failures in the callback are the caller’s to handle; nothing here swallows them.

Return type:

PullOutcome

Returns:

A PullOutcome. pulled false with contention populated means the fetch was refused before it began.

class ReadinessOutcome(**data)[source]

Bases: ProvisioningOutcome

Whether a model is loaded and actually answering, not merely present.

resident and answered are separate claims on purpose. A model can be present on disk and not loaded, or loaded and too slow to be useful, and an operator debugging a stalled read needs to know which of the two they have.

Parameters:
model: str
ready: bool
resident: bool
answered: bool
elapsed_ms: int | None
verify_model_ready(model, *, settings=None, timeout_s=None)[source]

Confirm model is resident and answers a trivial prompt within a bound.

The prompt is deliberately minimal and its CONTENT is irrelevant – this verifies the transport and the load, not the model’s quality, so nothing here inspects what came back beyond the fact that something did.

Never raises. An unreachable runtime and a model that is absent names the provision verb rather than pulling it, because an implicit multi-gigabyte fetch triggered by a verification command is exactly what the lifecycle design forbids.

Return type:

ReadinessOutcome

Parameters:
class InstalledModel(**data)[source]

Bases: BaseModel

One model the local runtime reports as PRESENT ON DISK, loaded or not.

Distinct from RuntimeResident, and the distinction is the whole reason both exist. A resident model is loaded into memory right now; an installed model occupies disk whether or not anything ever loads it. Removal reclaims disk, so it reads this inventory – reading the resident set instead would report a multi-gigabyte model as absent merely because it is cold.

Parameters:

data (Any)

name: str
size_bytes: int | None
digest: str | None

The runtime’s content digest for the pulled weights; changes when a pull replaces them.

read_installed_models(settings=None)[source]

Read the runtime’s on-disk model inventory, or None when it could not be read.

A short-timeout GET /api/tags. Downloads nothing and loads nothing. As with read_runtime_residents(), None and the empty tuple are different states and must not be collapsed: empty means “measured, no models are installed”, which is a fact callers may act on, while None means the inventory is unknown and every caller here refuses rather than guessing.

The reported size is the runtime’s own figure for the bytes that model occupies, which is what makes a freed-bytes report a measurement rather than an estimate.

Return type:

tuple[InstalledModel, ...] | None

Parameters:

settings (Settings | None)

class RemoveOutcome(**data)[source]

Bases: ProvisioningOutcome

The result of removing one Cadrumo-selected model from the local runtime’s store.

freed_bytes is a measurement, not an estimate, and is populated only when the removal was confirmed against a re-read of the inventory. The figure is the runtime’s own reported size for that model, so an operator can reconcile it against the disk the runtime’s store actually gives back. When the confirming read fails, the removal may well have succeeded, but no figure is reported – an unreconcilable number is worse than none.

Never raises. A model Cadrumo did not select, a model that is not installed, and an unreadable inventory each return removed false with the reason.

Parameters:
model: str
removed: bool
was_installed: bool
freed_bytes: int | None
remove_runtime_model(model, *, installed=None, installed_measured=True, settings=None)[source]

Delete model from the local runtime’s store and report the bytes reclaimed.

Deletion is delegated to the runtime that owns the store; nothing here unlinks a file, walks a directory, or reaches any path Cadrumo manages. That boundary is deliberate and load-bearing: the model store is a third-party cache, and no encrypted bucket, key material, or secure-storage object is reachable from this action by construction.

The selection guard mirrors unload_runtime_model() – Cadrumo removes only models cadrumo_selected_models() names, so a model another operator or application pulled into the shared runtime is reported and never deleted. Deleting a peer’s multi-gigabyte model would be a far more expensive overreach than evicting one from memory, since the bytes must be re-fetched.

The freed figure is measured across the action: the size is read from the inventory BEFORE the delete, and reported only after a re-read confirms the model is gone.

Parameters:
Return type:

RemoveOutcome

Returns:

A RemoveOutcome describing what happened and, on refusal, why.