Memory & Context Compaction

Long-running agents quickly exceed model context limits or incur compounding prompt expenses if raw message histories are re-sent on every execution turn. Agent Substrate implements a decoupled, two-tier memory architecture: In-Session Hot Token Compaction and Cross-Session Vector Retrieval.

Smart Token Budget Compaction

When the active conversation exceeds a configured token threshold (e.g. 10,000 tokens), the built-in compactor compresses earlier turns into an immutable structured state digest while keeping recent interaction turns intact.

python

from substrate.agents import ReActAgent
from substrate.agents.context import TokenCompactor

agent = ReActAgent(
    model="claude-3-7-sonnet",
    compactor=TokenCompactor(
        max_active_tokens=8000,
        target_summary_tokens=1500,
        strategy="sliding_window_summary",
        preserve_system_prompts=True
    )
)

Compaction Lifecycle

Compaction Strategies & Benchmarks

Choose the strategy that best fits your latency and memory fidelity requirements:

Strategy	Token Reduction	Latency Overhead	Context Retention Guarantee
`sliding_window_summary`	~82% – 88%	~450ms (Background async)	Key entities, tools executed, and user decisions are distilled into typed JSON state.
`truncate_oldest`	~50% – 70%	0ms (Instant)	Strictly drops the oldest conversational turns, preserving only the most recent N turns.
`hybrid_entity_extract`	~75% – 85%	~600ms (Structured LLM pass)	Extracts semantic knowledge entities, tool outputs, and facts into a relational memory table.

Cross-Session Vector Memory

For recall across distinct user sessions and historical threads, attach a durable vector backend backed by PostgreSQL pgvector or Qdrant:

python

from substrate.capabilities.memory import VectorMemoryBackend

# Initialize vector memory collection
memory = VectorMemoryBackend(
    connection_string="postgresql+asyncpg://postgres:secret@localhost:5432/substrate",
    collection_name="customer_preferences",
    embedding_model="text-embedding-3-small"
)

# Store memory during agent execution
await memory.store(
    thread_id="thr_user_8923",
    text="User prefers AWS us-east-1 and Terraform for infrastructure deployments.",
    metadata={"category": "infra_pref", "confidence": 0.98}
)

# Recall relevant memories dynamically in subsequent sessions
relevant_notes = await memory.recall(
    query="What is the preferred cloud provider and region?",
    top_k=3
)

Vector Backend Parameters

Parameter	Type	Default	Description
`connection_string`	`str`	Required	Async connection URL for PostgreSQL pgvector or Qdrant cluster endpoint.
`collection_name`	`str`	`"default"`	Logical namespace isolating memory collections across workspaces.
`embedding_model`	`str`	`"text-embedding-3-small"`	Embedding model identifier used to vectorize conversational facts.
`top_k`	`int`	`5`	Maximum number of semantically relevant memories injected into the context window.

Next Steps

ReAct Reasoning Loops — Configure step bounds and tool selection policies.
Safety Guardrails — Enforce regex tripwires and spend ceilings.
PostgreSQL WAL & Replay — Learn how compacted memory is journaled.

Was this page helpful?

Suggest edits Raise issue