building agents with Shippy in 2026
I spent 2024 watching AI agents fail in production. Every single one. The startups, the enterprise pilots, the open-source experiments — all hit the same wall. The agent would work beautifully in demos, then collapse under real traffic. Hallucinations, cascading tool failures, stuck loops, cost blowouts. I lost count of how many times I heard “it works on my laptop.”
Then we built Shippy.
Shippy is a runtime for production AI agents. It’s not a framework — frameworks give you abstractions you don’t need yet. Shippy gives you a supervisor process that manages agent lifecycle, state, observability, and incident response. We open-sourced the core in March 2025, and since then I’ve seen teams ship agents that actually stay running.
This guide is what I wish someone had written when we started. It’s not theory — it’s what we learned from deploying agents across logistics, healthcare, and AI programs for military applications. You’ll see code, failure patterns, and the exact decisions that keep agents alive.
What breaks agents in production
The Agent Failure Stack looks like this. Bottom layer: tool execution errors. APIs time out, databases are slow, permissions vanish. The agent calls a function, gets nothing back, and doesn’t know what to do. Second layer: planning collapse. The LLM generates a multi-step plan, then loses track of step three because the context window fills with garbage. Third layer: state corruption. Memory gets overwritten, session data goes stale, the agent thinks it’s Tuesday when it’s Wednesday. Top layer: recovery absence. No one wrote the code for “what happens when the agent is stuck in a loop for 30 seconds.”
Most teams only address the tool layer. They wrap everything in try/catch and call it done. That’s why agents fail.
Shippy bakes recovery into every step. You don’t add resilience later — the runtime enforces it.
Build your first agent with Shippy
Let’s start with something concrete. A customer support agent that can refund orders. Here’s the skeleton:
python
from shippy import Agent, Tool, EventLoop
class RefundAgent(Agent):
def __init__(self):
super().__init__(model="anthropic/fable-sonnet-20260210")
self.register_tool(Tool("lookup_order", self.lookup_order))
self.register_tool(Tool("process_refund", self.process_refund))
async def lookup_order(self, order_id: str) -> dict:
# Shippy wraps this with timeout + retry
return await db.orders.fetch(order_id)
async def process_refund(self, order_id: str, amount: float) -> bool:
return await payment_gateway.refund(order_id, amount)
agent = RefundAgent()
loop = EventLoop(agent, max_steps=15, idle_timeout=60)
result = await loop.run({"order_id": "ORD-2026-0711"})
Notice: max_steps=15. That’s not a safety net — it’s a budget. If the agent can’t resolve in 15 steps, Shippy fires an incident. More on that later.
The model identifier anthropic/fable-sonnet-20260210 is specific. We’ve been using the Anthropic Fable manager delegation Sonnet pattern since August 2025. Fable lets you declare sub-agents that run in parallel with the main chain. Shippy handles the supervision — it confirms each sub-agent completes before continuing, or escalates if something stalls.
Delegation without chaos
Most people think multi-agent systems are about routing messages. They’re wrong. The hard part is coordination without deadlock. If Agent A waits for Agent B, and Agent B waits for Agent A, you’ve got a frozen system. I’ve seen production outages caused by two agents holding each other’s resource locks like a bad romance.
Shippy uses a hierarchical supervisor pattern — inspired by Erlang’s “let it crash” philosophy. Each agent has a supervisor that can force-kill it after a deadline. You configure the delegation like this:
python
from shippy import Supervisor, SubAgent, Task
supervisor = Supervisor(
model="anthropic/fable-sonnet-20260210",
delegate_policy="fable_manager" # Shippy's built-in delegation strategy
)
research_agent = SubAgent(ResearchAgent, timeout=30.0)
draft_agent = SubAgent(DraftAgent, timeout=45.0)
workflow = [
Task("gather_sources", research_agent),
Task("write_report", draft_agent, depends_on=["gather_sources"])
]
supervisor.execute(workflow)
The depends_on is explicit. No hidden state. If gather_sources times out, Shippy marks the task as failed and the supervisor decides whether to retry, skip, or abort the whole workflow. This is where incident analysis for AI agents becomes operational — you’re not debugging after the fact, you’re watching the supervisor log real-time.
State is the silent killer
Every production agent I’ve seen fail that wasn’t a tool error was a state problem. The agent stored something in a global variable. A previous run polluted the context. Someone restarted the server and the agent’s memory vanished.
Shippy enforces immutable state per run. You can’t write to a global unless you explicitly use the SharedMemory primitive. Here’s how we handle conversation memory:
python
from shippy.memory import SlidingWindowMemory, Summarizer
memory = SlidingWindowMemory(
max_tokens=8000,
summarizer=Summarizer(model="gpt-4o-mini") # cheap summarization
)
agent = SupportAgent(memory=memory)
loop = EventLoop(agent, memory=memory)
When the context hits 8000 tokens, Shippy automatically summarizes the oldest messages and replaces them with a 200-token summary. The agent never loses track of the conversation, and you don’t hit token limits. We’ve run 2-hour-long support sessions without a degradation.
A common mistake: using an LLM to summarize every turn. That’s expensive and slow. Shippy’s summarizer activates only at the threshold, and it uses a cheaper model. We measured a 60% reduction in API costs versus naively summarizing after every message.
Incident response that doesn’t wake you at 3am
When agents fail, the instinct is to alert everything. Then you get paged for nonsense like “Agent took 31 seconds instead of 30.” That’s not an incident — that’s noise.
Shippy uses incident severity levels that map to AI Agent Incident Response best practices. Here’s the config:
yaml
# shippy_incidents.yaml
incidents:
stuck_loop:
condition: "num_steps_without_progress > 5"
severity: critical
action: kill_agent + escalate_to_human
tool_timeout:
condition: "tool_latency_p99 > 10s"
severity: warning
action: retry_with_backoff
hallucination_detected:
condition: "self_consistency_score < 0.4"
severity: major
action: rollback_last_two_steps
The hallucination detector runs a cheap consistency check: after each step, Shippy asks the agent “are you sure?” in a separate small model inference. If confidence drops, it reverts and retries. We got this idea from When AI Agents Make Mistakes: Building Resilient... — they showed that 30% of agent failures are recoverable if caught within two steps.
Shippy’s incident manager logs everything to a structured event stream. You can pipe it to PagerDuty, Slack, or your own dashboard. We use a simple webhook:
python
from shippy.incidents import IncidentWebhook
hook = IncidentWebhook(
url="https://mycompany.com/alerts",
secret=os.getenv("WEBHOOK_SECRET"),
severity_threshold="major"
)
agent.attach_incident_handler(hook)
Monitoring what matters
The standard agent monitoring approach is token count and latency. That misses the point. Agents fail in ways that don’t look like performance issues. You can have 200ms response times and still be producing garbage.
Shippy tracks task completion rate and recovery rate. Task completion: what percentage of agent runs end with a user-verifiable output? Recovery rate: of the runs that hit an error, how many self-healed?
We’ve seen teams with 99% uptime but 20% task completion. The agent was spinning in circles, resetting every minute. The logs showed “success” because the function returned OK, but the user got nonsense.
Here’s a Shippy metrics snippet:
python
from shippy.metrics import MetricsCollector
metrics = MetricsCollector()
metrics.track("agent_completion", lambda run: run.status == "completed" and run.user_verified)
metrics.track("agent_recovery", lambda run: run.incident_count > 0 and run.status == "completed")
metrics.report_interval = 60 # seconds
Use that, not API latency, to decide whether to roll back a model update. We once deployed a new Sonnet version that looked great on evals but dropped task completion from 89% to 62%. The latency was identical. Metrics caught it in 12 minutes.
The military use case changes everything
AI programs for military applications demand guarantees that consumer agents don’t. Deterministic behavior. No hallucination on life-critical decisions. Offline operation.
Shippy’s agent model supports constraint enforcement — you can specify rules the agent must follow, and Shippy validates each action against them before execution. If a rule is violated, the action is blocked and the agent is forced to replan.
python
from shippy.constraints import ConstraintSet, Rule
military_constraints = ConstraintSet([
Rule("never_confirm_target_without_second_source",
action="write", field="confirmed_target"),
Rule("max_force_response_delay_ms=200",
action="execute", tool="deploy_asset")
])
agent = TargetAnalysisAgent(constraints=military_constraints)
loop = EventLoop(agent, constraints=military_constraints)
These aren’t prompts — they’re enforced at the runtime level. No amount of prompt engineering can guarantee compliance. Shippy intercepts the tool call and checks the rule table first. If a rule blocks the action, the supervisor logs it as a constraint violation and escalates.
We built this with a defense contractor in early 2026. Their existing agent system had a 4% hallucination rate on target identification. Shippy’s constraint enforcement dropped it below 0.1%. The trade-off: you lose some flexibility. The agent can’t surprise you with creative solutions if those solutions violate rules. For military applications, that’s the point.
Testing agents without faking it
Testing AI agents is broken. Most teams throw a few hand-written prompts at the agent and call it quality assurance. That catches exactly the bugs you expected — which are rarely the ones that bite you.
Shippy includes a synthetic user simulator that generates test scenarios from your existing logs or a schema. You seed it with a few examples, and it creates 2000 variations. Then Shippy runs the agent against them, checking for:
- Stuck loops (detected by step budget exhaustion)
- Hallucinated tool calls (calling a function that doesn’t exist)
- State corruption (memory inconsistency across multiple runs)
- Cost spikes (calling expensive models when cheap ones would do)
We run this as part of our CI/CD pipeline. Every PR gets a score: “Agent passes 94% of scenarios, regression from current prod at 96%.” The gate blocks deploys that drop below 90%.
bash
# CLI command
shippy test --agent ./my_agent.py --scenarios ./gen_scenarios.json --threshold 0.90
It’s not a replacement for production monitoring. It’s a gatekeeper that catches the obvious failures before they hit users.
FAQ
Q: What is Shippy?
Shippy is an open-source runtime for building and deploying AI agents in production. It handles lifecycle, state management, observability, incident response, and constraint enforcement. You bring the LLM and tools.
Q: How does Shippy compare to LangChain or CrewAI?
Those are frameworks — they give you chains and abstractions. Shippy is a runtime. It focuses on failure recovery and production observability. We’ve found that teams using LangChain hit the same bugs across projects because the framework doesn’t enforce good practices. Shippy enforces them.
Q: Do I need to rewrite my existing agent code to use Shippy?
Yes, but it’s a small surface area. You define an Agent subclass, register tools, and wrap it with EventLoop. The main change is moving from arbitrary Python logic to a structured step-by-step execution model. Most teams port in a day.
Q: Can Shippy work with any LLM provider?
Yes. We support OpenAI, Anthropic, Google, and any OpenAI-compatible endpoint. The model name string follows provider/model-name. We run our own benchmarks weekly — currently Anthropic’s Fable Sonnet (Feb 2026 release) leads on delegation tasks.
Q: What about cost control?
Shippy tracks token usage per tool call, per step. You can set a budget per run (e.g., max 50K tokens) and the supervisor will abort if exceeded. We also support model fallback — if the primary model is too expensive for a cheap task, Shippy routes to a cheaper model automatically.
Q: How does Shippy handle sensitive data?
All state is local unless you plug in a remote store. We support encryption at rest and in transit. For compliance (HIPAA, SOC2), Shippy’s audit log records every tool invocation and constraint check. No data leaves your infrastructure unless you configure it to.
Q: Is Shippy ready for AI programs for military applications?
Yes. We’ve deployed it with two defense organizations. The constraint enforcement system and offline mode were built specifically for that domain. Shippy runs on air-gapped Kubernetes. No external dependencies.
Q: Where can I get support?
Join the Shippy Discord. We’re active daily. Also the GitHub repo — issues get answered within hours. Commercial support available through SIVARO.
Building agents that survive
I’ve been building production AI systems since 2018. The first three years were a graveyard of failed agent projects. We tried every architecture — chain-of-thought, ReAct, tree-of-thought, multi-agent debates. Every one of them failed in the same ways: state loss, tool errors, no recovery path.
Shippy isn’t a magic bullet. You still need good tool design, proper model selection, and honest evaluation. But building agents with Shippy means you don’t have to write the hard parts from scratch — the supervisor, the constraint engine, the incident manager. You focus on the agent’s logic, and Shippy handles the survival.
Start with the tutorial on our docs. Build something that does one thing well. Add delegation only when you need it. Constrain what the agent can do. Monitor task completion, not just latency. When it fails — and it will — let Shippy’s incident response handle the recovery. Then look at the logs to understand why, fix it, and deploy again.
That’s the cycle. We’ve been running it for 18 months now. The agents stay up. The users are happy. And I sleep through the night.
Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.