Skip to content

Canned Responses

Conversational agents routinely receive trivial inputs — a bare greeting, a "thanks", an "ok", a "bye". These need no retrieval, routing, tools, or LLM round-trip; they need a short, deterministic acknowledgement. On a hosted model that round-trip is several seconds and real cost spent on nothing.

ReActAgent supports an opt-in canned-response short-circuit: a declarative list of rules, evaluated at the top of every turn, that return a configured reply without calling the LLM. The framework owns the event contract, so a canned turn is a correctly-framed terminal turn — never an empty message. Internally, canned responders are data-fed lifecycle responders that return a Reply.

from latent.agents import ReActAgent, CannedResponder, patterns

agent = ReActAgent(
    name="support",
    model="anthropic/claude-sonnet-4-5",
    system_prompt=BIG_PROMPT,
    tools=[search, lookup],
    canned_responses=[
        CannedResponder("greet", patterns(r"hi|hello|hey", reply="Hi! How can I help?")),
        CannedResponder("thanks", patterns(r"thanks|thank you|thx", reply="You're welcome!")),
    ],
)

Default is an empty list — existing agents are byte-for-byte unchanged.

Lifecycle API

For code-first responders, use @responder. It is sugar for a before_turn lifecycle hook that returns reply text:

from latent.agents import ReActAgent, responder

class SupportAgent(ReActAgent):
    @responder
    def quick_reply(self, messages: list[Message]) -> str | None:
        return "Hi!" if messages[-1].content == "hi" else None

Use the lower-level @lifecycle("before_turn") hook with Prefix or SystemSuffix when a turn should still call the model:

from latent.agents import lifecycle
from latent.agents.actions import Prefix, SystemSuffix

class SupportAgent(ReActAgent):
    @lifecycle("before_turn")
    def opener(self, messages: list[Message]):
        return [Prefix("Hi. "), SystemSuffix("Do not greet again.")]

SystemSuffix is appended to SystemContent.dynamic, after the cache-control breakpoint, so the static prompt prefix remains stable for prompt caching.

How canned responses work

Each CannedResponder holds a name (used in trace/debug attribution) and a single reply callable:

Reply = Callable[[list[Message]], str | None | Awaitable[str | None]]

reply(messages) receives the full conversation and returns the reply text, or None to decline. Responders are evaluated first-match-wins, in declaration order on the same chain as @responder hooks. The first responder that returns a non-empty string fires; the agent emits the turn and returns without an LLM call.

reply may be sync or async, and may inspect the whole history (e.g. "only greet on the first turn").

patterns() — the common regex case

patterns(*regexes, reply, max_len=40) builds a reply callable that fullmatches the normalized last user message (case-insensitive, surrounding whitespace stripped) and returns reply on a hit, else None:

CannedResponder("greet", patterns(r"hi|hello|hey", reply="Hi!"))
  • Inputs longer than max_len (default 40) are rejected — a backstop against an over-broad pattern swallowing a real question.
  • reply is validated non-empty at build time (patterns(..., reply="") raises ValueError).

Custom matchers

Any (messages) -> str | None callable works directly. A matcher that already returns a reply (or None) needs no glue:

def my_matcher(messages: list[Message]) -> str | None:
    text = messages[-1].content if messages else ""
    return "Goodbye!" if text.strip().lower() in {"bye", "goodbye"} else None

CannedResponder("bye", reply=my_matcher)

Guarantees

  • Input guardrails always run first. GuardrailMiddleware wraps stream(), so a canned response is only produced for inputs that already cleared the safety gate. The short-circuit never bypasses scanning.
  • Correct event contract by construction. The framework emits StepBoundary → TextDelta via the shared emit_text_turn helper. invoke() segments the event stream by StepBoundary and drops anything before the first boundary; centralizing emission makes the empty-message failure mode impossible for callers.
  • Fail-open. A reply callable that raises is logged and falls through to the LLM — a trivial-input optimization never breaks a real conversation.
  • Empty reply declines. A reply returning None or "" falls through to the LLM; an empty string is never shipped.
  • No Usage event. A canned turn makes no LLM call, so it emits no Usage — cost dashboards correctly show zero spend. The absence of LLMCallStart/Usage plus the _debug["canned_response"] marker distinguishes a canned turn from a real one.

Observability

When a responder fires, the agent records self._debug["canned_response"] = {"responder": <name>}, making "which responder fired" queryable per turn. A sudden rise in canned-turn rate is the early-warning signal that a matcher has gone greedy and is swallowing real questions — the reason patterns() defaults to strict fullmatch plus the max_len cap.