An AI agent framework is a library or platform that provides the building blocks for building agents: the execution loop, tool integration, memory, state handling, retries and tracing. Frameworks save teams from rewriting the same plumbing, though many production agents are still built directly against a model API because the plumbing is not that large.
Key takeaways
- Frameworks handle the loop, tool calling, state, retries and observability.
- Options range from low-level SDKs to opinionated multi-agent platforms.
- Abstraction has a cost: harder debugging and less control over prompts.
- Tracing and evaluation support matter more than the number of features.
- Starting without a framework is a legitimate choice for simple agents.
What a framework provides
| Capability | Why it matters |
|---|---|
| Agent loop | Runs the decide-act-observe cycle with limits |
| Tool registry | Defines tools and their schemas consistently |
| State and memory | Tracks conversation and task state |
| Structured output | Validates model responses |
| Error handling | Retries, timeouts and fallbacks |
| Multi-agent support | Orchestration and handoffs |
| Tracing and evaluation | Step-level visibility, essential for debugging |
| Human-in-the-loop | Pausing for approval before risky actions |
The main categories
| Category | Examples | Character |
|---|---|---|
| Provider SDKs | OpenAI Agents SDK, Claude Agent SDK | Close to the model, minimal abstraction |
| Graph and workflow libraries | LangGraph | Explicit control flow, good for complex branching |
| Data-centric frameworks | LlamaIndex | Strong RAG and data connectors |
| Multi-agent frameworks | CrewAI, AutoGen | Role-based teams of agents |
| Typed Python frameworks | Pydantic AI | Schema-first, strong validation |
| Enterprise platforms | Semantic Kernel and similar | Integration with existing stacks |
Names and capabilities change quickly in this space, so check current documentation before committing.
How to choose
- Start from the problem: a single agent with three tools needs far less than a multi-agent research system.
- Check tracing: can you see every prompt, tool call and result? This matters more than convenience features.
- Check prompt control: hidden prompts you cannot edit will eventually block you.
- Check the exit path: how hard is it to move off the framework later?
- Check ecosystem fit: language, deployment target and existing infrastructure.
- Check evaluation support: agents need test suites, not just demos.
A worked example
A team builds an internal assistant with three tools: document search, a ticketing API and a calendar.
- Without a framework: roughly a loop, a tool dispatch table, JSON validation, retries and logging. A few hundred lines, fully understood and controlled.
- With a framework: less code, built-in tracing and memory, but behaviour depends on the framework’s prompt templates and update cycle.
For this scope, either is reasonable. For a multi-agent system with branching, checkpoints and human approvals, a framework’s state management becomes far more valuable.
Common pitfalls
- Choosing a framework before understanding the problem.
- Hidden prompts causing unexplained behaviour.
- Version churn breaking working systems.
- Abstractions obscuring token usage and cost.
- Treating a demo as evidence of production readiness.
Limitations
- Frameworks move fast; documentation lags.
- Debugging through layers is harder than debugging plain code.
- Opinionated designs may not match your architecture.
- None removes the need for evaluation, guardrails and monitoring.