A curated list of infrastructure for building reliable LLM agents — frameworks, memory, tool protocols, sandboxes, browsers, observability, and retrieval.
Maintained by Backblaze.
- Awesome ML Data Pipelines
- Awesome Multimodal Data
- Awesome Physical AI
- Awesome Image Generation
- Awesome Video Generation
- Awesome Audio Generation
- Agent Frameworks
- Memory and State
- Tool Protocols and Servers
- Execution Sandboxes
- Browser and Computer Use
- Observability and Evaluation
- Retrieval and RAG
- Vector Databases
- Templates and Example Projects
Libraries for building LLM agents — planning, tool use, multi-agent orchestration.
- Microsoft AutoGen – Multi-agent conversation framework from Microsoft Research. AutoGen 0.4 rewrote it around an event-driven runtime. Docs | SDK: Python (pip install autogen-agentchat)
- CrewAI – Role-based multi-agent framework. Agents, tasks, and tools composed into crews with deterministic or planning-based flows. Docs | SDK: Python (pip install crewai)
- Agno – Lightweight Python framework for building multimodal agents and agentic systems. Formerly Phidata. Docs | SDK: Python (pip install agno)
- LangGraph – Graph-based agent runtime from the LangChain team. Durable execution, human-in-the-loop, and multi-actor patterns. Docs | SDK: Python (pip install langgraph), JS (npm install @langchain/langgraph)
- HuggingFace smolagents – Minimal "code agent" library — agents write Python to solve tasks. ~1k LoC core; easy to audit and extend. Docs | SDK: Python (pip install smolagents)
- Mastra – TypeScript-first agent framework with workflows, RAG, and evals. From the creators of Gatsby. Docs | SDK: TypeScript (npm install @mastra/core)
- OpenAI Agents SDK – Official OpenAI agent framework. Handoffs, guardrails, built-in tracing, and Responses-API-native execution. Docs | SDK: Python (pip install openai-agents)
- Pydantic AI – Agent framework from the Pydantic team. Type-safe tool calling, structured outputs, dependency injection. Docs | SDK: Python (pip install pydantic-ai)
- AG2 – Community-maintained fork of AutoGen 0.2. Multi-agent conversation framework with swarms, group chats, and nested chat patterns. Docs | SDK: Python (pip install ag2)
- Agent Swarm – Self-hosted runtime where a lead agent turns goals into tasks and routes them to workers in isolated containers with shared memory and review gates. Docs
- AgentScope – Python agent framework with an event-driven runtime, human-in-the-loop, sandboxed tool execution, and Agent-as-a-Service REST deployment. v2.0 released May 2026. Docs | SDK: Python (pip install agentscope)
- DeerFlow – ByteDance's open-source super-agent harness built on LangGraph. Orchestrates sub-agents, memory, sandboxes, and skills for long-horizon tasks. Docs
- Elflodge – Self-hosted runtime for batch automation with unattended Pi agents. Runs parallel jobs in sandboxes without a dedicated container per job, with a job API, structured results, and durable recovery. Docs
- Flowise – Open-source visual builder for LLM agents and workflows. Drag-and-drop Agentflow canvas plus REST API, JS/Python SDK, and CLI for programmatic integration into production applications. Docs | SDK: TypeScript (npm install -g flowise)
- Google ADK – Google's open-source agent development kit. Build, evaluate, and deploy multi-agent systems; multi-language with Gemini-optimized but model-agnostic. Docs | SDK: Python (pip install google-adk), TypeScript (npm install @google/adk)
- Langflow – Low-code builder for AI agents and RAG applications. Visual canvas with Python escape hatches, deploys flows as REST APIs or MCP servers; 40+ model and vector-store integrations. Docs | SDK: Python (pip install langflow)
- Langroid – Lightweight Python multi-agent framework from CMU/UW-Madison. Task-delegation via message passing; no LangChain dependency. Docs | SDK: Python (pip install langroid)
- MetaGPT – Multi-agent framework that assigns software-company roles (PM, architect, engineer) to LLMs. Input a requirement, get PRD, design, code, and tests. Docs | SDK: Python (pip install metagpt)
- Microsoft Agent Framework – Microsoft's production-ready open-source agent SDK and runtime for Python and .NET. Unifies AutoGen orchestration and Semantic Kernel foundations. Docs | SDK: Python (pip install agent-framework), .NET (dotnet add package Microsoft.Agents.AI)
- Open Multi-Agent – TypeScript-native multi-agent orchestration that plans task DAGs from goals at runtime and runs them with approval gates, tracing, evaluation, checkpoints, and resume support in your own environment. SDK: TypeScript (npm install @open-multi-agent/core)
- OpenAI Agents SDK (TypeScript) – Official OpenAI agent framework for TypeScript and JavaScript. Agents, handoffs, guardrails, voice via Realtime API, and built-in tracing. Docs | SDK: TypeScript (npm install @openai/agents)
- OpenSRE – Open-source toolkit for building AI SRE agents. Connects to 60+ observability, cloud, and incident-management tools; auto-fetches alert context, correlates logs/metrics, and generates root-cause reports.
- Semantic Kernel – Microsoft's open-source SDK for building LLM agents and multi-agent systems. Model-agnostic; plugins, planners, and process orchestration across Python, C#, and Java. Docs | SDK: Python (pip install semantic-kernel), C# (dotnet add package Microsoft.SemanticKernel), Java (Maven: com.microsoft.semantic-kernel)
- Strands Agents – AWS-backed open-source agent SDK. Define tools as functions; the model-driven loop handles planning and execution with no workflow graphs required. Docs | SDK: Python (pip install strands-agents), TypeScript (npm install @strands-agents/sdk)
- VoltAgent – TypeScript agent framework with memory adapters, RAG, tool registry, multi-agent supervisor coordination, voice support, and built-in evals. Docs | SDK: TypeScript (npm create voltagent-app@latest)
Long-term memory, session state, and knowledge-retention layers for agents.
- Mem0 – Memory layer for AI agents. Personalization through user/agent/session memories with semantic recall. Docs | SDK: Python (pip install mem0ai), Node (npm install mem0ai)
- Letta – Open-source agent server focused on long-term memory. Successor to MemGPT; agents are first-class stateful services. Docs | SDK: Python (pip install letta-client)
- Zep – Memory and context platform for LLM apps. Knowledge-graph-backed user memory with temporal reasoning. Docs
- Agent Commons – Open-source continuity layer for persistent AI agents. Preserves identity, memory, state, and relationships across model, runtime, and server changes, with signed portable state, A2A discovery, and MCP/REST interfaces. Docs
- Cognee – Knowledge engine for agent memory. ECL pipeline ingests any data into a hybrid vector + knowledge graph for structured, traceable recall. Docs | SDK: Python (pip install cognee)
- Concord – MCP server and CLI that lets Claude Code, Codex, Cursor, Gemini CLI and other coding agents message each other, claim work, share task memory and hand off in one shared workspace. Docs | SDK: JavaScript
- Graphiti – Open-source temporal context graph engine. Tracks how facts change over time with full provenance; hybrid semantic + keyword + graph retrieval. Docs | SDK: Python (pip install graphiti-core)
- grite – Git-backed issue tracker for AI coding agents. CRDT-merged event log in
refs/grite/walsyncs viagit push/fetch; no server or database required. Stable--jsonoutput,AGENTS.mdauto-generation, andgrite install-skillfor Claude Code. Docs | SDK: Rust (cargo install grite), Homebrew (brew install neul-labs/tap/grite), npm (npm i -g grite-cli), pip (pip install grite-cli) - Hindsight – Open-source agent memory system using biomimetic data structures. Organises memories into world facts, experiences, and mental models; TEMPR retrieval combines semantic, keyword, graph, and temporal search. Docs | SDK: Python (pip install hindsight-client), TypeScript (npm install @vectorize-io/hindsight-client)
- Honcho – Memory infrastructure for stateful agents. Stores messages to per-peer sessions, runs background reasoning to build user representations, and returns curated context via a fast query API. Docs | SDK: Python (uv add honcho-ai)
- Hyperconsciousness – Developer-alpha Rust knowledge store with encrypted, append-only records, device sync, and MCP retrieval through scoped, expiring grants. Docs
- Puppyone – File system for agents. Connect, govern, version, and share context across agent workflows. Docs
- Redis Agent Memory Server – Memory layer for AI agents backed by Redis. Two-tier working + long-term memory, semantic/keyword/hybrid search, REST and MCP interfaces, and multi-LLM provider support. Docs | SDK: Python (pip install agent-memory-client)
- ReMe – Memory management kit for AI agents. Conversation compaction, long-term file-based and vector memory, semantic search; compresses context by up to 99.5% while retaining critical facts. Docs | SDK: Python (pip install reme)
- SoloPup Memory – Native DeepSeek Harness public-beta text memory with source references, project scope, conditions and correction history. Lexical recall may miss paraphrases; requires Node >=24.9 and DSH >=0.2.1-rc.0 <0.3.0. Docs
- Supermemory – Memory and context API for AI agents. Ingests documents and conversations, extracts facts, builds user profiles, and returns relevant context via hybrid semantic search; SDK and REST interfaces. Docs | SDK: TypeScript (npm install @supermemory/sdk), Python
- Tree Ring Memory – Local-first memory lifecycle layer for AI agents. Rust CLI with SQLite/FTS recall, forgetting, audit, consolidation, and portable SKILL.md guidance. Docs | SDK: Rust CLI (brew install tree-ring)
- YYLO Ledger – Git-native task and Record store for coding-agent workflows. Keeps reviewable current state and append-only history as Markdown in the repository, with receipts, archives, and bounded queries. Docs | SDK: Python CLI (pip install yylo-ledger)
Standardised interfaces for exposing tools and data sources to agents (MCP and friends).
- MCP Reference Servers – Reference MCP server implementations for filesystem, Git, GitHub, SQL, Slack, and more.
- MCP Python SDK – Official Python SDK for building and consuming MCP servers and clients. SDK: Python (pip install mcp)
- MCP TypeScript SDK – Official TypeScript SDK for MCP servers and clients. SDK: TypeScript (npm install @modelcontextprotocol/sdk)
- Anthropic Model Context Protocol – Open protocol for connecting AI applications to tools and data sources. Spec, reference servers, and official SDKs. Docs
- Agent2Agent Protocol (A2A) – Open protocol for communication and interoperability between AI agents. JSON-RPC 2.0 over HTTP with SDKs for Python, Go, JS, Java, and .NET. Docs
- Aident Loadout – Hosted remote MCP server (Streamable HTTP, OAuth) that lets agents search app integrations and 400+ Skills by intent and run actions, with credentials kept in a vault, cost preflight, and an audit log of every call. Docs | SDK: TypeScript CLI (npm install -g @aident-ai/cli)
- AWS MCP Servers – Suite of 53 open-source MCP servers for AWS services — CloudFormation, Bedrock, DynamoDB, EKS, S3, and more. Docs
- Bifrost – Self-hosted LLM and MCP gateway with OpenAI-compatible APIs, multi-provider routing, fallbacks, virtual keys, tool filtering, and OpenTelemetry observability. Docs
- BrewPage MCP – MCP server and REST API that let agents publish HTML pages, Markdown, JSON, key-value data and files to shareable HTTPS links without an account or API key. Hosted Streamable HTTP endpoint at https://brewpage.app/mcp; content is temporary (expires). Docs | SDK: TypeScript (npx -y brewpage-mcp)
- Composio – Tool-integration SDK for AI agents. 1000+ pre-built tool connectors (GitHub, Slack, Jira, etc.) with managed auth and sandboxed execution. Docs | SDK: Python (pip install composio), TypeScript (npm install @composio/core)
- Docsy – Hosted OAuth 2.1 MCP for querying version-specific public developer documentation with original source links. Five read-only tools; the MIT plugin is public while the hosted backend is proprietary. Docs
- Headroom – Context compression layer for AI agents. Compresses tool outputs, logs, RAG chunks, and files 60-95% before they reach the LLM; runs as a library, proxy, or MCP server with reversible compression. SDK: Python (pip install headroom)
- IBM ContextForge – Open-source MCP/A2A/REST gateway and registry. Federates MCP servers, A2A agents, and REST/gRPC APIs behind a single governed endpoint with auth, rate limiting, and OpenTelemetry tracing. Docs | SDK: Python (pip install mcp-contextforge-gateway)
- kc-agent (KubeStellar Console) – MCP bridge that exposes multi-cluster Kubernetes resources (pods, deployments, namespaces, events, Helm releases) as MCP tools so MCP-compatible agents can query live fleet data. Standalone HTTP/WS daemon; no cluster-side install. Docs
- MCP Inspector – Interactive visual tool for testing and debugging MCP servers. Supports STDIO, SSE, and Streamable HTTP transports. SDK: TypeScript (npx @modelcontextprotocol/inspector)
- MCPX – Open-source MCP gateway and aggregator. Consolidates multiple MCP servers behind a single governed entry point with rate limiting and traffic policies. Docs
- n8n-mcp – MCP server exposing n8n's 1,650+ workflow nodes to AI agents. Provides node docs, schema properties, operations, and workflow validation for agents building n8n automations. SDK: TypeScript (npx n8n-mcp)
- Screenpipe MCP – MCP server for querying locally captured screen text and audio history as agent context for recall, meeting notes, and work summaries. Requires a running Screenpipe recorder and local API key; configured cloud features can transmit context off-device. Docs
Secure environments for running agent-generated code, shell commands, and browser sessions.
- Daytona – Open-source dev-environment manager; Daytona Sandboxes expose a sandbox API for agents and CI pipelines. Docs
- E2B – Secure cloud sandboxes for running AI-generated code. Firecracker microVMs, sub-second startup, per-session isolation. Docs | SDK: Python (pip install e2b), JS (npm install @e2b/code-interpreter)
- Ephemeral Sandbox – Open-source agent sandbox infrastructure that gives parallel coding agents isolated workspaces over one shared project base, with MCP and CLI control plus atomic publication. Docs
- Lizard Sandboxes – Managed sandboxes for agent code with a Firecracker microVM runtime, snapshots, live forks, and TypeScript/Python SDKs. Docs | SDK: TypeScript/JavaScript (npm install @lizard-build/sdk), Python (pip install lizard-sdk)
- Microsoft Agent Governance Toolkit – Runtime policy enforcement for autonomous agents. Zero-trust identity, execution sandboxing, sub-millisecond policy checks; covers all 10 OWASP Agentic Top 10 risks. SDK: Python (pip install agent-governance-toolkit), TypeScript (npm install @microsoft/agentmesh-sdk)
- Modal Sandboxes – Serverless sandbox primitive inside Modal. Arbitrary container execution, ephemeral filesystems, strict network policies.
- OpenShell – NVIDIA's open-source sandbox runtime for autonomous agents. Declarative YAML policies govern file access, network activity, and data exfiltration; supports Claude, Codex, Copilot, and OpenCode. SDK: Python (uv tool install openshell)
- Riza – Secure code-execution API for LLM tool calls. Python, JS, PHP, Ruby; strict WASM-based isolation. Docs
- SandBase Harness – Local-first agent runtime and MCP bridge for auditable sessions, sandboxed execution, approvals, and Docker/Kubernetes-backed workflows. Docs
Platforms and SDKs that let agents drive web browsers and full desktops.
- browser-use – Open-source library giving LLMs reliable control of a Playwright browser. Self-host or use their cloud. Docs | SDK: Python (pip install browser-use)
- Playwright MCP – Microsoft's official MCP server for Playwright. Gives any MCP-aware agent a controllable browser.
- Anthropic Computer Use – Claude's computer-use tool for controlling a full desktop. Reference Docker image and sample agent loop from Anthropic.
- Agent QA – TypeScript QA agent for natural-language web and mobile tests, persistent test memory, self-healing flows, and MCP-driven execution. Docs | SDK: Node
- Browserbase – Managed headless browsers for AI agents. Session recording, proxying, CAPTCHA handling, and a Stagehand framework. Docs | SDK: Python (pip install browserbase), Node (npm install @browserbase/sdk)
- Cloudbrowser – Persistent remote Chrome for AI agent workflows. Isolated browser profiles connect through scoped CDP links; pause and resume preserve the browser profile, and Live View lets operators inspect the full desktop. Docs
- CurlShot – Hosted screenshot API and remote MCP server that lets agents see rendered web pages. Returns an image, PDF, or scrolling video, with full-page capture, device presets, dark mode, and single-element capture by CSS selector. Docs
- rea-captcha – Handles browser-agent tasks interrupted by CAPTCHA while keeping the caller-owned session through an authorized test action or human handoff and requiring fresh evidence before resuming. Docs | SDK: TypeScript
- ReMCP – OAuth MCP for file, terminal, browser and desktop actions on explicitly user-paired computers. Operations may change device state; MIT runtime is public while the managed relay is proprietary. Docs
Tracing, logging, metrics, and automated evals for LLM applications.
- Langfuse – Open-source LLM engineering platform — traces, prompt management, datasets, and evals. Self-host or managed. Docs | SDK: Python (pip install langfuse), JS (npm install langfuse)
- Opik (Comet) – Open-source LLM evaluation and tracing from Comet. Playground, datasets, experiment comparison. Docs | SDK: Python (pip install opik)
- Arize Phoenix – Open-source LLM tracing and evaluation. OpenTelemetry-based, self-hostable, integrates with every major framework. Docs | SDK: Python (pip install arize-phoenix)
- Helicone – Open-source proxy-based observability for LLM apps. Logging, caching, rate-limiting, and costs with minimal code. Docs
- AgentOps – Observability and DevTool SDK for AI agents. Session replays, LLM cost tracking, multi-agent tracing, and framework integrations. Docs | SDK: Python (pip install agentops)
- AgentPlane – Open-source Git-native audit layer for coding-agent work. Records task intent, approved plans, verification evidence, finish state, and Agent Change Records inside the repository. Docs | SDK: CLI (npm install -g agentplane)
- ax – Local telemetry and recall graph for AI coding agents. Ingests Claude Code, Codex, OpenCode, Cursor, and Pi transcripts with cost/routing analytics, hooks, dashboard, and read-only MCP queries. Docs
- BrowserTrace – Local-first trace viewer for AI browser-agent failures, with screenshots, model I/O, URLs, actions, errors, and public-safe HTML exports. Docs | SDK: Python (pip install "browsertrace[ui]")
- ClawBench – Open-source benchmark runner for web agents on live websites. Isolated container runs, request-interception scoring, and five-layer traces covering video, screenshots, HTTP traffic, browser actions, and agent messages. Docs
- DeepEval – Open-source LLM and agent evaluation framework. Pytest-native with 50+ built-in metrics (hallucination, faithfulness, role adherence), multi-turn eval support, and CI/CD integration. Docs | SDK: Python (pip install -U deepeval)
- Laminar – Open-source observability platform purpose-built for AI agents. OTel-native tracing, step-level replay/rerun, Signals pattern extraction across traces, evals, and self-hostable via Docker. Docs | SDK: Python (pip install lmnr), TypeScript (npm install @lmnr-ai/lmnr)
- LangSmith – Commercial tracing, evaluation, and prompt engineering platform from the LangChain team. Works with any LLM framework. Docs
- Latitude – Open-source agent engineering platform. Production observability, LLM-as-judge evals, issue grouping, and GEPA-based prompt optimisation. Docs
- MLflow – Open-source AI engineering platform with LLM/agent tracing built on OpenTelemetry, 50+ eval metrics, prompt management, and an AI gateway. Supports 60+ agent frameworks. Docs | SDK: Python (pip install mlflow)
- OpenClaw Monitor – Open-source monitoring dashboard for OpenClaw AI agents. Tracks token usage, sessions and weekly trends from local agent data. Docs
- OpenLLMetry – OpenTelemetry-based instrumentation for LLM apps. Drop-in tracing for OpenAI, Anthropic, LangChain, LlamaIndex, and major vector DBs. SDK: Python (pip install traceloop-sdk), TypeScript (npm install @traceloop/node-server-sdk)
- Spend Brake – Self-hostable egress proxy that prices each paid API call an agent makes (search, maps, SMS, custom APIs) and refuses it with 402 once a hard dollar cap per key, agent or session would be crossed. Loop detection, kill switch, Prometheus metrics and OpenTelemetry spans. Docs | SDK: CLI (npx @spend-brake/cli)
- TapVow – Open-source local gateway that gives each AI agent a signed spending limit. Refuses over-budget LLM and x402 calls before they reach the provider and records a receipt for every call. Drops in via ANTHROPIC_BASE_URL / OPENAI_BASE_URL. Docs
- TruLens – Open-source evaluation and tracking for LLM apps and agents. RAG Triad metrics, feedback functions, and experiment comparison dashboard. Docs | SDK: Python (pip install trulens)
Retrieval-augmented generation frameworks and document-indexing libraries.
- LangChain – LLM composition library. Document loaders, retrievers, and chains form the RAG backbone for many apps. Docs | SDK: Python (pip install langchain), JS (npm install langchain)
- LlamaIndex – Data framework for connecting custom data sources to LLMs. Document loaders, indexing, query engines, and agents. Docs | SDK: Python (pip install llama-index), TypeScript (npm install llamaindex)
- Haystack (deepset) – End-to-end NLP framework for building RAG, search, and agent applications. Pipelines compose components. Docs | SDK: Python (pip install haystack-ai)
- RAGAS – Framework for evaluating RAG pipelines. Reference-free metrics for faithfulness, answer relevancy, and context precision. Docs | SDK: Python (pip install ragas)
- CocoIndex – Incremental data-pipeline engine for agent context. Declarative transforms over code, docs, and streams; only changed chunks re-index, giving agents sub-second fresh context at minimal compute cost. Docs | SDK: Python (pip install cocoindex)
- LightRAG – RAG system combining knowledge graphs with dual-level (local + global) retrieval. Fast indexing, graph-based entity-relation extraction, and multiple query modes. Docs | SDK: Python (pip install lightrag-hku)
- RAGFlow – Open-source agentic RAG engine with deep document understanding and intelligent chunking. Combines RAG pipelines with agent workflows, MCP integration, and multi-turn conversational retrieval. Docs
Vector stores and embedding databases commonly used by agents for semantic recall.
- Milvus – Scalable open-source vector database from Zilliz. Horizontal scale, GPU indexing, LF AI & Data graduated project. Docs
- Qdrant – High-performance vector database in Rust. Strong filter DSL, quantization, and hybrid search. Docs | SDK: Python (pip install qdrant-client), JS, Rust, Go
- Chroma – AI-native embeddings database. Popular choice for local/laptop development and quick prototyping. Docs | SDK: Python (pip install chromadb), JS (npm install chromadb)
- pgvector – Open-source vector similarity extension for Postgres. Exact and approximate nearest-neighbour with HNSW and IVFFlat.
- Weaviate – Vector search with built-in vectorization modules and a schema-aware GraphQL API. Docs
- LanceDB – Serverless vector database on the Lance columnar format. Zero-copy, versioned, runs directly over S3-compatible storage. Docs | SDK: Python (pip install lancedb), Rust, Node
Reference implementations, demos, and starter projects.
- Awesome MCP Servers – Community-maintained catalogue of MCP servers. Useful reference when deciding what to build vs. adopt.
- LangGraph Examples – Reference LangGraph flows — ReAct agents, human-in-the-loop, multi-agent collaboration.
- OpenAI Agents Python Examples – Official examples for the OpenAI Agents SDK — handoffs, voice, parallelism, guardrails.
Contributions are welcome. See CONTRIBUTING.md. One entry per PR — edit entries.yaml only and let the maintainers regenerate README.md.
Save on tokens by using the Genblaze SDK — Backblaze's open-source Python SDK for AI-generated video, audio, and images. It orchestrates multi-provider generation pipelines with built-in, tamper-evident provenance and native Backblaze B2 storage.
Released under CC0 1.0 Universal. You may copy, modify, and redistribute without attribution.
Backblaze B2 Cloud Storage is S3-compatible object storage designed for AI and media workloads. This list is maintained as part of our work making B2 a convenient storage layer for AI workflows.
