The LLM Contract

What this is, in one line

This is the kernel's promise about how an agent talks to a language model — a single, provider-agnostic shape that every model adapter (OpenAI, Anthropic, Gemini, Ollama) must fit into.

Everything on this page lives in kernel/llm/llm.py. The kernel is frozen: it defines only the contract (Python Protocols and dataclasses), with zero I/O. The real network calls live one layer down, in adapters that implement this contract.

The big picture

You hand the model a list of messages plus a bag of options. You get back one response (content + token usage). That is the whole loop.

The adapter does the dirty work of translating Agent Substrate's neutral types into each vendor's wire format and back. The agent above it never sees that translation.

The LLMClient Protocol

A Protocol is a shape, not a base class. Any object that has these methods is an LLMClient — no inheritance required. That is how a brand-new provider plugs in: implement the shape, and you are done.

Here is the contract, trimmed to its essence:

class LLMClient(Protocol):
    """Contract every LLM provider adapter must satisfy."""

    model: str  # which model this client speaks to, e.g. "gpt-4o"

    async def generate(
        self,
        messages: list[ChatMessage],
        *,
        options: GenerationOptions = GenerationOptions(),
        ctx: RunMeta | None = None,
    ) -> LLMResponse: ...

    def generate_stream(
        self,
        messages: list[ChatMessage],
        *,
        options: GenerationOptions = GenerationOptions(),
        ctx: RunMeta | None = None,
    ) -> AsyncIterator[TextDelta | ReasoningDelta | CompletionEvent]: ...

    async def count_tokens(self, messages: list[ChatMessage]) -> int: ...

Three things to notice:

• model is a plain attribute — a string naming the model the client is wired to ("gpt-4o", "claude-sonnet-4", "llama3.2"). Reading it tells you who is on the other end of the remote.
• messages is always a list[ChatMessage]. A ChatMessage is a role-tagged turn (system / user / assistant / tool) carrying multimodal ContentBlocks — text, images, tool calls, and more. (See the core content types for the full block list — we just use them here.)
• ctx is optional run metadata (RunMeta) carrying the cancellation token and deadline. A well-behaved adapter calls ctx.check() before its first network call and respects ctx.is_expired() between streamed chunks, so a cancelled run stops promptly instead of burning tokens.

Swapping providers without touching agent code

Because the agent depends on the shape, not the brand, switching models is a one-line change at the wiring site — never inside the agent.

The agent points at the diamond, never at a box. Repoint the wiring at a different box and the agent is none the wiser.

GenerationOptions — the knobs

Everything that tunes a call — beyond the messages themselves — lives in one frozen dataclass so every provider reads the same field names.

A call with no options is valid — the default GenerationOptions() is a sensible empty bag, which is why the Protocol can use it as a default argument.

LLMResponse — what comes back

The return value of generate() is small and immutable:

@dataclass(frozen=True, slots=True)
class LLMResponse:
    content: list[ContentBlock]   # the model's reply, as multimodal blocks
    usage: Usage                  # tokens consumed by this call

• content is a list[ContentBlock] — the same universal block type used everywhere else in agent_substrate The reply may be plain text (TextBlock), a reasoning trace (ThinkingBlock), a tool request (ToolUseBlock), or a mix.
• usage is a Usage record: input_tokens, cached_tokens, output_tokens, and reasoning_tokens, plus a total_tokens property. Cached and reasoning counts are broken out so you can attribute cost accurately (cached prompt tokens are billed cheaper, reasoning tokens come from extended thinking).

From inputs to that one response

generate() vs generate_stream() — two ways to get the answer

Both take the same messages, options, and ctx. They differ only in how the answer arrives.

How streaming events relate to the final response

generate_stream() yields a sequence of small deltas, then one final assembling event:

• TextDelta — an incremental chunk of visible text.
• ReasoningDelta — an incremental chunk of the model's thinking trace.
• CompletionEvent — the last event, carrying the fully assembled content plus the final Usage.

In other words: the deltas are the live commentary, the CompletionEvent is the box score. The content + usage you would have gotten from a single generate() call is exactly what the trailing CompletionEvent hands you.

EmbeddingClient — turning text into vectors

Not every model call generates words. Sometimes you need a vector — a list of numbers that captures the meaning of a piece of text, so you can compare texts by closeness (the engine behind search and RAG).

EmbeddingClient is the contract for the thing that produces those coordinates:

class EmbeddingClient(Protocol):
    """Contract every embedding provider adapter must satisfy."""

    async def embed(self, texts: list[str]) -> EmbeddingResult: ...

    async def embed_single(self, text: str) -> list[float]: ...

• embed(texts) — batch: turn many strings into many vectors in one call. Returns an EmbeddingResult (embeddings: list[list[float]], the model used, and usage_tokens).
• embed_single(text) — convenience: one string in, one list[float] vector out.

Same Protocol pattern as LLMClient: any object with these two methods is an embedding client, so a new embedding provider plugs in without changing callers.

How the four pieces fit together

Where this lives

Next: Messaging & Streaming — the delta and event types that flow out of generate_stream(), and the progress channel every agent in the tree publishes to.