Summarizing conversation

Summarize older turns, keep the last two verbatim

from strands.agent.conversation_manager import SummarizingConversationManager


SUMMARY_PROMPT = """Summarize the conversation so far as short bullet points in the third person.
Keep every fact the user stated about themselves (name, role, preferences such as a favorite color).
Give each tool result one short line. Do not address the user."""
"""Strands' default summary lists topics and tool results and may drop personal facts; this one keeps them."""


def build_conversation_manager() -> SummarizingConversationManager:
    """Summarize most (80%) of the older messages once the context is 70% full; keep the 2 most recent verbatim."""
    return SummarizingConversationManager(
        summary_ratio=0.8,
        preserve_recent_messages=2,
        summarization_system_prompt=SUMMARY_PROMPT,
        proactive_compression={"compression_threshold": 0.7},
    )

Chat until it summarizes

uv run bootcamp.py phase1 t4 --starter

uv run bootcamp.py phase1 t4

Start from a fresh session: add a new session name, e.g. uv run bootcamp.py phase1 t4 --starter --session colors1 (next try colors2), then type, one at a time: My favorite color is blue, List every department with its budget and number of active employees, My favorite food is pizza, What is my favorite color?

The system prompt and the tool description alone take 1,000-1,300 tokens (it depends on the model's tokenizer), so the 1,890-token threshold is crossed only once the department table and its tool call are in the history. Then, before the next model call, the manager asks the model to summarize the oldest messages and prints [context] prompt ~1,900 of 2,700 tokens: older messages summarized (N so far). Each later summary includes the previous one.

The two numbers work together: a summary has to bring the prompt well under the threshold. With a low threshold and a small ratio, the prompt after a summary is still over the line, so the manager summarizes again before every model call.

Experiments

• Open phase1/sessions/session_summary_session/ (session_summary_session_starter/ for your starter file): the session on disk still has every message, only the model's context was summarized.
• Swap in SlidingWindowConversationManager(window_size=2) and repeat in a fresh session (a new --session name). Now the [context] line says dropped and the agent forgets blue: a window drops old turns, a summary keeps their gist. (A warning that it found no valid trim point during the database question is expected: a window never splits a tool call from its result.)
• Set summary_ratio=0.5 and compression_threshold to 0.5 and repeat in a fresh session: the [context] line now appears before nearly every model call, each one an extra summarization call you pay for.
• Remove summarization_system_prompt to get Strands' default summary and repeat. Read the summary in the session's agent.json: is your favorite color still in it?
• Raise CONTEXT_WINDOW_TOKENS to 200,000 (realistic). No [context] line in a short chat: summarization costs an extra model call, so it only runs when it pays off.
• Run uv run bootcamp.py llm before and after a long chat to see what the extra summary calls cost.