Here's what it looks like:

Normally agents operate in channels like #sales and #engineering. This video is from our testing channel: #agents. If you look closely you can see Sam trying to hack my personal context agent. Everything you see works out of the box: streaming, tool calls rendered as they run, human-in-the-loop, memory of the person the agent is talking to. We'll get to how all of that works.

One backend, many frontends

A big advantage of having your own agent platform is that the same backend connects to every frontend. So the same agent is available over Slack, a custom UI, or AI apps like Claude and ChatGPT. The key is setting up the platform right, so it can serve any frontend.

Do it yourself

Here's what we'll do to get our agents in Slack:

1. Set up our agent platform
2. Deploy our platform (Slack needs a public URL)
3. Connect our agent to Slack

End to end this should take ~20 minutes. We'll use coding agents for steps 1 and 2 so you'll move through this pretty quickly and the only manual setup is creating the Slack app. Let's go.

Set up your platform

If there's one thing you take away from my posts, it's that you need your own agent platform: shared execution, shared security, shared context, and a consistent way of exposing your agents over REST, MCP, and chat interfaces like Slack.

Your code lives in one place, easy to scale beyond your first agent. Sessions, memory, and traces are stored in your database and can be mined for self-improvement. Platform access and security are managed in one place, and the best part: the whole thing can be done by coding agents. Let's start by setting up our platform.

Step 1: Start up your coding agent. I'm using claude code with --dangerously-skip-permissions (fine here, it's a fresh project).

Step 2: Create your agent platform using one prompt.

1. Make sure Docker is installed and running.
2. Export your OpenAI API key using export OPENAI_API_KEY=...
3. Get your prompt from os.agno.com and give it to your coding agent.

And that's it. Your coding agent will set up and run your agent platform locally.

Deploy your platform

Slack needs the agent to be available over a public URL. When you copied the prompt from os.agno.com, you chose a cloud to deploy your platform to (I chose Railway).

Let's deploy our platform so slack can reach our agent. cd into your agent platform, start your coding agent so the slash commands are available and run:

/deploy-platform

I'm telling the skill to deploy to the AB workspace because I already have my internal company platform running in my main account. The skill checks the prerequisites (may ask you to log in), provisions the services, and pauses for the JWT key. This is important.

One mistake I see agent builders make all the time is deploying their agents to a public URL with no authentication. If the open URL gets discovered, strangers have unrestricted access to your agents. The platforms I build come with JWT auth on by default, so a production deploy needs a JWT verification key before it will even start.

You can mint one yourself, or get one from os.agno.com.

Connect to your AgentOS on os.agno.com, flip on token-based authorization, and paste in the verification key when the skill pauses for it. Your coding agent deploys the platform and hands you a live URL.

Connect to Slack

Now for the main event. We will:

1. Create the app
2. Install it to your workspace
3. Add the credentials to your platform
4. Chat with Chief over Slack (the agent that's pre-wired)
5. View the chat sessions on the AgentOS UI.

Here's me doing it end to end:

Step 1: Create the Slack app from a manifest.

• Grab this manifest.json and replace https://YOUR-URL with your AgentOS URL. The events URL becomes YOUR-AGENTOS/slack/events and the interactivity URL becomes YOUR-AGENTOS/slack/interactions. Also update the display_information and bot_user.display_name.
• Then go to api.slack.com/apps and Create New App → From a manifest, pick your workspace, and paste in the manifest. The manifest pre-configures everything: the scopes, the event subscriptions, interactivity.

Note: create the app after your platform is live. Slack sends a verification challenge to /slack/events and AgentOS answers it automatically, so the URL is verified on creation.

Step 2: Install it to your workspace.

Click Install App → Install to Workspace → Allow. Copy two values: the Bot User OAuth Token (starts with xoxb-) from the install page, and the Signing Secret from Basic Information.

Step 3: Add the credentials to your platform.

Open the .env.production file and add both values to SLACK_BOT_TOKEN and SLACK_SIGNING_SECRET, then sync and redeploy. You can just tell your coding agent to redeploy the platform or run the following commands manually.

./scripts/railway/env-sync.sh
./scripts/railway/redeploy.sh

That's it. Find your agent under Apps in Slack and say hello. Then invite it to a channel and mention it:

/invite @Chief
@Chief tell me about yourself.

If you want to see the code, open app/main.py and you'll see:

if SLACK_BOT_TOKEN and SLACK_SIGNING_SECRET:
    interfaces.append(
        Slack(
            agent=chief,
            streaming=True,
            resolve_user_identity=True,
            token=SLACK_BOT_TOKEN,
            signing_secret=SLACK_SIGNING_SECRET,
        )
    )

How it works

When you message the agent, Slack sends an event to https://your-url/slack/events. There AgentOS runs a signature check and responds in a thread. Streaming is on by default, so responses appear as they're generated, with task cards showing progress on longer work.

Two details worth understanding:

• Threads are sessions. Each Slack thread maps to one session (the ID is literally chief:{channel_id}:{thread_ts}).
• With resolve_user_identity=True, each sender resolves to their email, so the agent knows who it's talking to. Chief keeps a private profile and memory for every person, while notes and entities stay shared across the team. Ask the same question as two different people and you'll get answers shaped by what Chief knows about each person.

Monitoring and Observability

Open the AgentOS UI and go to the Sessions and Traces tabs. Every Slack thread is there with its full history. You can also ask the Platform Manager about usage, run activity, and tool activity.

A couple of notes on security

You just deployed a public agent platform. It's important to understand your security posture. Your platform has two entry points:

The platform entry point. JWT auth is on by default in production. Every API and MCP request needs a valid bearer token, and the verification key you pasted during deploy is what checks them. The MCP endpoint adds OAuth on top (MCP_CONNECT_SECRET) so chat apps like claude.ai can connect with authentication.

The Slack entry point. The events webhook is public by design. That's safe because every request carries an HMAC signature computed with your signing secret. Requests older than five minutes are rejected.

Wrapping up

Here's the picture after this post:

                 ┌──►  AgentOS UI    build, monitor, improve
                 │
                 ├──►  Slack         where your team works   ◄── you are here
  Your agents ───┤
                 ├──►  MCP           claude.ai, ChatGPT, Claude Code, Cursor
                 │
                 └──►  REST API      your own product

One backend for every frontend. Your agent now lives where your team works, with auth on both entry points and full visibility from the AgentOS UI.

Thanks for reading!

Ashpreet

Here's how it works: my coding agent reads the agent's instructions and mines its usage data to derive probes. Then it runs those probes against the live agent, reviews the logs, and edits the agent until each probe passes. Here's what the end result looks like:

Not RSI

Recursive self-improvement (RSI) has a specific meaning in the community. 1) the system improves itself, 2) the improvements target its ability to improve, and 3) the gains compound. RSI is a divergent process. Here, one AI improves another. A coding agent edits a target agent's instructions, tools, and parameters. Recursive auto-improvement (RAI) is a convergent process: it pulls the agent toward a fixed point, i.e. its own spec. For production software, convergent is exactly what we want.

Do it yourself

To build an auto-improvement loop, you need a system that is set up for it.

Your coding agent needs to be able to:

• Query the live system via an API. So it can test the improvements against the live agent.
• Watch the logs. So it has full visibility into the agent's trajectory. Every tool call, every error, every debug message is available to the coding agent.
• Make code changes and test the updates. So it can fix, improve, and extend the agent to pass the probes.
• Mine the usage data. So it can derive probes from real usage patterns. Questions, answers, and fumbles help it understand where the agent is struggling.
• Look up framework information. So it can reliably make updates by tuning parameters.

This "system" is exactly what I've spent the last year building, and the results are MARVELOUS.

If you're an agent builder, I highly recommend you follow along. Even if you're not an Agno user, the coding-agent-driven paradigm is a see it to believe it experience. We're going to do everything from setting up the platform, to creating agents, to improving them using coding agents.

Let's start by setting up our platform first.

Set up your platform

Step 1: Start up your coding agent. I'm using claude code with --dangerously-skip-permissions.

Step 2: Create your agent platform, using one prompt.

1. Make sure docker is installed and running.
2. Export your OpenAI API key using export OPENAI_API_KEY=...
3. Get your prompt from os.agno.com and give it to your coding agent.

Once the platform is up, connect to the AgentOS UI (where you got the prompt from) and play around before creating the agent we'll use for auto-improvement.

Create the agent to improve

The setup platform skill ends with recommending an agent called Radar, go along with it.

Radar is a great example for the auto-improvement loop, because its spec is dense with rules a judge can check mechanically:

• Max 5 items, one line each, every item with a source link.
• No hype adjectives.
• Keeps a ledger of items already reported, so briefs don't repeat.

Optional: Use it before you improve it

The auto-improvement loop's strongest probes come from usage data. So before improving an agent, I just... use it a couple of times. Asked for briefs. Asked follow-ups. Asked adversarial questions.

The usage data is stored in Postgres as sessions. When the improvement loop runs, it mines this usage data to derive probes: recurring shapes, visible fumbles, out-of-scope asks.

Note: you might run into rate limit errors on the Parallel MCP, to fix those, get a key from platform.parallel.ai and set it in the .env file.

Run the improvement loop

Now coming to the main event: the auto-improvement loop. I'm going to cd into the platform directory and open another coding agent. We'll cd in so the coding agent can pick up the skills and I can run them using a slash command.

/improve-agent

The loop runs for ~25 minutes, I've trimmed (and sped up) the video aggressively to condense it into 2 minutes. I rarely deploy an agent without running this loop first.

My favorite version of this is the overnight run. I kick off the loop with 300-500 probes then go to bed. The loop works through rare edge cases, odd phrasings, failures that only show up once in a hundred runs. My coding agent works through them while I sleep, fixing every bug it can find.

You can then ask the agent to provide a report. Mine looks like this (because it has access to my internal design system):

Here's how it works

The coding agent reads Radar's instructions and mines the sessions to build a set of probes: golden path, edge cases, tool selection, adversarial. For each probe it writes a one-line expected behavior, drawn from the spec. Then it runs them against the live agent, reads the tool calls, errors, and debug messages from the container logs, and judges every response.

For every failure it picks one lever to change: tighten a rule, add a rule, swap a tool. It edits the agent code in agents/radar.py, restarts, and re-runs only what failed. Capped at five rounds. Most fixes are one sentence or one parameter change.

Note: you can also turn the failures into evals :)

Wrapping up

Here's the new agent development lifecycle, managed entirely using coding agents:

   /create-agent            scaffold a new agent
        │
        ▼
   /improve-agent ◄──────┐  improve it against its own spec
        │                │
        ▼                │
   /extend-agent ────────┘  add a capability, then improve again
        │
        ▼
   /deploy-platform         launch it
        │
        └──► back to /extend-agent when you want it to do more

Create it. Improve it. Extend it, improve it again. Deploy.

Thanks for reading! If you have any questions, feel free to reach out to me on X or LinkedIn.

Ashpreet

The future is here, just not evenly distributed.

Today I'll show you how to build your own agent platform without writing a single line of code.

I know this sounds crazy, so I'll share videos along the way and encourage you to build along. All you need is a coding agent, docker, and an OpenAI API key.

But first, what is an agent platform and why is every company building one?

What is an Agent Platform?

An agent platform is the system that runs your agents. It handles agent execution (i.e. running the agents), manages sessions and context, enforces security policies, logs traces, and connects your agents to frontends via API, MCP, or chat interfaces like Slack.

An agent platform provides a shared foundation that you can build your company's AI applications on. Without a shared platform, every agent becomes its own silo, each needing its own deployment, authentication, database, observability, and integrations.

Most companies build their own agent platform to own the learning loop. To retain full control over the agent's behavior, data and context. Which paves the way for reflective self improvement.

Let's Get Started

The first thing we're going to do is set up our agent platform. I'm going to show how to do that with one prompt. I know this sounds mad, so here's a video of it in action.

Want to follow along? Go to os.agno.com, get the prompt for your cloud provider (I'm using Railway) and hand it to your favorite coding agent (I'm using Claude Code).

For a one-shot experience like the video, make sure you have docker installed and running, and export your OpenAI API key using export OPENAI_API_KEY=<your-api-key>

Under the hood, the coding agent first clones an agent-platform template (built by yours truly) and then runs the /setup-platform skill. It ends by running the /create-agent skill, which I used to create an agent called Radar. Ten minutes from a fresh machine to a running platform.

Skills for days

When I say "build without writing a single line of code", I mean you don't write any code, the coding agent does. But how do we do that reliably? A unified platform + skills.

The agent platform we created above is structured so that agent code, logs, traces, evals and live service all live in one place, accessible to the coding agent doing the implementation. A complete paradigm shift because you're no longer stitching different services together.

Because the coding agent can make changes to the code, and probe the live running system, it's able to take full control of the agent development lifecycle. It can create agents (using /create-agent), reflect on usage and improve them (using /improve-agent), fix bugs add new features (using /extend-agent), write and run evals (using /create-evals and /eval-and-improve), keep the codebase in sync (using /review-and-improve), and even deploy the platform to production.

All of this is possible because a) the gnarly wiring is handled by Agno, so the coding agent is just putting things together, and b) all information the coding agent needs is available using Agno's MCP server. So the coding agent is simply assembling a puzzle, with access to the manual via MCP.

I won't go into the details of each skill, but I will showcase the /improve-agent skill. Which covers the most important part of your agent platform: the learning loop.

/improve-agent takes an agent and runs N simulations (called probes) to evaluate the agent's performance. It derives these probes from the agent's own instructions and from its real usage (stored in the database). It then runs the probes against the live container, judges the responses, and improves the agent until they pass. It's rare for me to publish an agent without running this skill first.

Let's see it in action. cd into your agent platform and restart your coding agent so it picks up the new skills. Then run /improve-agent to improve your agent, my run took about 29 minutes so read the rest of the article before running this. It really does dig through the edge cases.

Give your agent platform a UI

Every step so far has been in a terminal, next let's give our agent platform a UI. Head to os.agno.com and connect your platform.

Here's the part I like. The AgentOS UI connects directly from your browser to your runtime. The flow of data is entirely between your browser and the running API. Sessions, traces and memory stay in your database and you retain full control and ownership of your data and context.

Connect your agents to your frontends

You're not building an agent platform to replace Claude or ChatGPT. You're building an agent platform to power your product and operations with custom agents you own and control. Meaning building agents is only part of the puzzle, you also need to connect them to:

1. Your product. Use the REST API to call your agents from your product.
2. Your AI apps. Connect your agents to Claude and ChatGPT using the MCP server.
3. Your chat apps. Expose agents in Slack, Discord and Telegram using Agno Interfaces.

Luckily, your agent platform is already set up to do all three.

First, go live using the /deploy-platform skill

To connect your platform to AI apps like Claude and ChatGPT, or chat apps like Slack, Discord and Telegram, you need to deploy your platform so it is reachable from the internet.

So before we wire up the AI apps and chat apps, let's take our platform live. Run the /deploy-platform skill, which will deploy the platform to the cloud template you selected during setup (I'm using Railway).

Connect your agents to your AI apps using the MCP server

This is the part I'm most excited about. A large number of the custom agents and workflows you'll build will be used through AI frontends like Claude, ChatGPT, Claude Code and Codex. If you've ever heard anyone moan about shared context between AI apps, this is the solution.

And its remarkably simple to do so. Your agent platform (built on AgentOS) exposes every agent, team, and workflow using an MCP server, so Claude and ChatGPT can connect to your AgentOS and run your agents.

Let's test it out by connecting Chief (the company mascot) to Claude.

In Claude, go to Settings → Connectors → Add custom connector and paste your platform's MCP URL: https://<your-railway-domain>/mcp. Leave the client ID and secret fields empty. Connect and it'll take you to a consent page asking for your connect secret: that's the MCP_CONNECT_SECRET the /deploy-platform skill generated and added to .env.production (it prints it in the deploy summary, copy it). Once you confirm, Claude can interact with your AgentOS using MCP.

First, tell Claude that whenever you ask for "Chief", you want to run the chief agent on AgentOS (it'll add it to its memory).

Whenever I ask for "Chief", you should run the chief agent on AgentOS.

Then tell Claude to interact with Chief.

Tell chief the surfer leo zapato from garaje was 🔥.

Connect chief to slack

Next, let's make Chief available in Slack. You can connect your agents to any chat app using Agno Interfaces. Slack is the most popular, and comes pre-wired in the platform template.

Create a Slack app for your workspace (the Agno docs have a step-by-step guide), point its Event Subscriptions URL at https://<your-railway-domain>/slack/events, and grab two values: the bot token and the signing secret. Set SLACK_BOT_TOKEN and SLACK_SIGNING_SECRET in .env.production, run ./scripts/railway/env-sync.sh (or tell your coding agent to run it), and chief should be available over slack.

Use your agents from your product using the REST API

This is the last one, and the easy one. Your agent platform comes with a complete API with 80+ endpoints. Run agents, stream responses, list sessions, manage knowledge. Use this API to build agentic products on top of your platform.

Go localhost:8000/docs for the full list of endpoints. Everything the AgentOS UI does, your product can do too. The AgentOS UI uses the exact same API.

Wrapping up

Today we built a unified agent platform using nothing but a prompt, took it live, and connected it to Claude, ChatGPT, and Slack. End to end, this guide would probably take you ~30-40 minutes, which is pretty awesome! Now the fun part, build more agents, improve them, and connect them to your product, use them in your AI apps or chat apps. Here is the full set of skills, for reference:

• /setup-platform takes you from a fresh clone to a running platform
• /create-agent creates a new agent and tests it
• /extend-agent makes changes to existing agents, add features, fix bugs, etc.
• /improve-agent recursively improves the agent against its own instructions and usage patterns
• /create-evals creates new evals for the agent, lock in good behavior via tests
• /eval-and-improve runs the evals and improves the agent if they fail
• /review-and-improve keeps the full codebase in sync, docs, code, and config coherent.
• /deploy-platform deploys the platform to production

Total lines of code written by me: zero.

The future is here. Go to os.agno.com and see for yourself. Links for reference:

• Agno is open source, star the repo on GitHub
• Read the Agno documentation
• Connect with me on X or LinkedIn

Building an agent today is simple. Pick a framework, tell your coding agent what you want, and you'll have a working agent in ~15 minutes. But running that agent as a durable, reliable service with resumable streams is a completely different ball game.

If you're a visual learner like me, tell me if you can achieve this UX with 1 vanilla FastAPI dev server. Multiple long-running streams (resumable across connections). Multiple MCP clients (opened / closed at the same time). Multiple processes updating data at the same time.

No amount of coding agent slop will give you the reliability, durability, and security required for running agents in production, especially if you're after a UX this smooth.

I've spent the last year solving this problem, working with hundreds of companies to serve agents at scale. Today I'm sharing what I built, free and open-source.

Why vanilla FastAPI doesn't work for agents

FastAPI is incredible, but here's why you can't wrap an agent in a regular FastAPI route and call it a day:

1. Agents are long-running processes that outlast HTTP requests. A regular API call is milliseconds. An agent can run for seconds, minutes, and now days. Serving agents needs durability, background execution, and resumeability built in. Here's the alpha: the unit of work for agents is a run, not a request. Runs live in the database with a status (running, paused, completed), and the HTTP connection is simply a window into the run, not the run itself. Close the connection and nothing is lost: the run keeps executing in the background, and when you reconnect, the stream resumes right where it left off. If the run lives inside the request handler, a dropped connection kills it mid-thought.

2. Agent requests are stateful. Regular HTTP requests are stateless, and horizontal scaling is a solved problem. Agents break that assumption: every run is tied to a session, and run #5 needs the history, memory, and working state from runs #1-4. Worse, because each run holds a connection for minutes instead of milliseconds, we need horizontal scaling from the get-go. Here's the alpha: just like the unit of work for agents is a run, the unit of state for agents is a session. We tie everything to a session and checkpoint every step (making our agents durable). Session history, memory, and state live in the database, resumable at any checkpoint. Every run starts by loading the session and ends by persisting it, so any worker can serve any turn.

3. Agents live on many more surfaces beyond REST. For regular software, REST is all you need. But agents? No sir. The same agent needs to serve a web UI via REST, run as a tool inside Claude and ChatGPT via MCP, talk to other agents via A2A, and be available over chat interfaces like Slack, WhatsApp, and Telegram. Each surface has its own protocol, its own payloads, its own auth. Good luck wiring it all by hand. Here's the alpha: if the run is the unit of work and the session is the unit of state, every surface needs to be a client to the same backend. One backend speaks REST, MCP, and A2A; Slack and WhatsApp are thin translations onto the same service. We need to build the agent once, serve it everywhere.

And that's just the tip of the iceberg. Agents break many assumptions of the request/response stack. Security is harder (scopes need to be at an agent, session and tool level), HITL complicates everything. In short, we need an abstraction around FastAPI that makes it durable for agents.

FastAPI for Agents

Introducing AgentOS: a production runtime for agents, built on FastAPI.

FastAPI promises: write a function, get an API. AgentOS makes the same promise but one abstraction up: write an agent, get a production service. AgentOS will:

• Turn your agents into a durable API with 80+ endpoints.
• Expose your agents as an MCP server, so you can use them in your coding agents and AI apps.
• Connect your agents to chat interfaces like Slack, WhatsApp, Telegram and Discord.

And the best part: AgentOS is a FastAPI app itself, so you get all the features of regular FastAPI out of the box. Here's a simple AgentOS serving an example agent we'll build next:

"""Serve the RL Tutor Agent using AgentOS."""
 
from agno.os import AgentOS
from tutor.agent import tutor
 
agent_os = AgentOS(
    agents=[tutor],
    tracing=True,
    scheduler=True,
    mcp_server=True,
)
app = agent_os.get_app()
 
if __name__ == "__main__":
    agent_os.serve(app="app:app", reload=True)

Here's the API you get with it, designed for building the best agentic products.

Example: RL Tutor

A few months ago, Andrej Karpathy shared how he uses LLMs to build personal knowledge bases. Let's build a similar system together, but with a slight twist: we'll focus the knowledge around post-training and RL, and use that knowledge to actually post-train our own models.

Here's the loop we're after:

       drop a link · ask a question
                     │
                     ▼
         ┌───────────────────────┐
         │       RESEARCH        │  web_search + web_fetch
         │   articles · papers   │  primary sources, cited
         └───────────┬───────────┘
                     │  filed as pages
                     ▼
         ┌───────────────────────┐
     ┌──▶│         WIKI          │  markdown on disk
     │   │  one concept per page │  explore in Obsidian
     │   └───────────┬───────────┘
     │               │  becomes context
     │               ▼
     │   ┌───────────────────────┐
     │   │       LEARNINGS       │  LearningMachines
     │   │  what worked and why  │  memory + learning
     │   └───────────┬───────────┘
     │               │  guides experiments
     │               ▼
     │   ┌───────────────────────┐
     │   │      POST-TRAIN       │  Tinker API
     │   │ datasets · fine-tunes │  base vs tuned evals
     │   └───────────┬───────────┘
     │               │
     └───────────────┘
       experiment reports feed back into the wiki

Meet RL Tutor, an agent that helps post-train my own models. I don't want a chatbot that knows LoRA. I want an agent that helps me study the field, build datasets, run experiments, and record what we learn. You can check out the code for RL Tutor here and read more about it here.

The agent itself is relatively simple. The full code is here but here's the gist of it:

"""RL Tutor: an agent that teaches me post-training."""
 
from agno.agent import Agent
from agno.db.sqlite import SqliteDb
from agno.models.openai import OpenAIResponses
 
from .learning import LEARNER_INSTRUCTIONS, learning_machine
from .tinker_tools import TinkerTools
from .wiki import parallel_web, wiki
 
FOCUS = f"""<focus>
You are RL Tutor. You are teaching us to train our own models with Tinker,
Thinking Machines' fine-tuning API. Inkling is the primary study subject.
For the first hands-on training runs, use the cheaper default Qwen MoE.
 
Working style:
- Answer established training questions from the wiki, citing pages. For
  facts that may have changed or gaps in the wiki, use web_search and web_fetch
  as needed, prefer primary sources, and cite their URLs in the answer.
- File training knowledge under training/, one concept per page: the model
  (training/inkling.md), the API (training/tinker-api.md), methods like LoRA and RL, recipes, costs.
- Keep training/open-questions.md current: add questions we cannot answer
  yet; when a new page answers one, remove it and cite the page.
...
</focus>
"""
 
tools = [*wiki.get_tools(), *parallel_web.get_tools()]
if getenv("TINKER_API_KEY"):
    tools.append(
        TinkerTools(wiki_path=str(wiki.backend.path), datasets_path=str(DATASETS_PATH))
    )
 
tutor = Agent(
    name="RL Tutor",
    model=OpenAIResponses(id="gpt-5.6"),
    db=SqliteDb(db_file=str(DB_FILE)),
    tools=tools,
    learning=learning_machine,
    instructions=[wiki.instructions(), FOCUS],
    add_history_to_context=True,
    num_history_runs=5,
    # Tool transcripts are preserved in the db but we should
    # avoid injecting bulky datasets and reports into every prompt.
    # The tutor can fetch them from history on demand.
    max_tool_calls_from_history=0,
    read_chat_history=True,
    search_past_sessions=True,
    num_past_sessions_to_search=10,
    num_past_session_runs_in_search=2,
    add_datetime_to_context=True,
)

We're going to serve the rl-tutor agent using AgentOS, which is roughly 20 lines of code:

"""Serve RL Tutor with AgentOS."""
 
from agno.os import AgentOS
 
from tutor.agent import tutor
from tutor.slack import get_slack_interfaces
 
agent_os = AgentOS(
    agents=[tutor],
    tracing=True,
    scheduler=True,
    mcp_server=True,
    interfaces=get_slack_interfaces(tutor),
)
app = agent_os.get_app()
 
if __name__ == "__main__":
    agent_os.serve(app="app:app", reload=True)

Run this using python app.py and we have our rl-tutor running as a service at localhost:7777.

By running our agent using AgentOS, we get:

• a Chat UI. Open os.agno.com, connect to http://localhost:7777, and chat with the tutor, browse its sessions, memory, and runs.
• a REST API at localhost:7777/docs, because it's FastAPI.
• an MCP server. Add it to Claude Code and the tutor becomes available in your coding agent.

The best part: rl-tutor can be added to my coding agents with 1 command, which is what im after.

Add RL Tutor to your coding agents

This is the KEY for me. Open another terminal, and run

uvx agno connect --url http://localhost:7777 --server-name rl-tutor

Then open or restart your coding agent and ask: "can you access the rl-tutor mcp?"

How RL Tutor works

RL Tutor operates in two modes:

1. Study mode. RL Tutor maintains a Markdown wiki inspired by Andrej Karpathy's LLM knowledge bases. Drop a link and it can convert a raw source into knowledge; ask a question and it answers from the wiki. The wiki is stored on the file system, so it can be explored using tools like Obsidian.
2. Build mode. When a Tinker key is set, RL Tutor gets tools for creating datasets and running fine-tunes. It is designed to compare the tuned checkpoint with the base model and file the experiment back into its wiki.

I use RL Tutor entirely through my coding agents, because that's where I'm writing most of my code. For example, here's rl-tutor adding a doc to its wiki:

Add https://thinkingmachines.ai/news/introducing-inkling/ to the wiki

The artifacts, datasets, reports are stored locally. I can inspect everything in my editor or in Obsidian:

As the wiki grows, it becomes the context for the next experiment. When an experiment finishes, or fails, the report becomes another source the tutor can learn from.

Wrapping up

This was a long read, and I'm grateful if you made it this far. If you take one idea away, make it this:

The unit of work for agents is a run, the unit of state is a session, and every front-end should be a client to the same backend. To serve agents in production, we need a runtime built around this.

AgentOS is that runtime, free and open-source. You give it your agents, multi-agent teams, and agentic workflows; you get a durable, reliable and sercure API in return.

Getting started with AgentOS

AgentOS turns your agents into a production API, giving you one AI backend that serves every frontend. Getting started is as simple as giving your coding agent this prompt:

Help me set up my agent platform and build my first agent.
 
Clone https://github.com/agno-agi/agentos-railway.git into a folder called agent-platform,
cd in, and run the setup-platform skill (in .agents/skills/).

The prompt above uses the Railway template, but you can swap it out for your cloud provider of choice. Your agent platform runs in your cloud, as a secure and scalable service.

Move RL Tutor to your agent platform

The RL Tutor we served locally is just an agent plus an AgentOS. You can move it over to your agent platform with one prompt:

Move the RL Tutor into my agent platform.
 
Clone https://github.com/ashpreetbedi/rl-tutor into a temporary folder and read
tutor/agent.py. Copy the tutor package into my agent-platform repo, register the
tutor agent in app/main.py, and switch its SqliteDb to the platform's Postgres
database. Add its dependencies, keep the wiki directory on a persistent volume,
then restart the platform and verify the tutor answers over REST and MCP.

That's the advantage of having your own agent platform: build the foundation once, and every agent you need slots into the same runtime, storage, and connectors. The platform also comes with skills that let your coding agent create, extend, improve, evaluate, and maintain the agents running on it. I wrote about that in my Self-Driving Agent Infrastructure article.

Thank you for reading!

Ashpreet

Learn more

• Agno docs
• AgentOS docs
• Agent Platform docs

Meet RL Tutor, an agent that helps me learn and post-train my own models. I don't want a chatbot that knows LoRA. I want an agent that helps me study the field, build datasets, run experiments, and record what we learn. Here's the loop I'm after:

       drop a link · ask a question
                     │
                     ▼
         ┌───────────────────────┐
         │       RESEARCH        │  web_search + web_fetch
         │   articles · papers   │  primary sources, cited
         └───────────┬───────────┘
                     │  filed as pages
                     ▼
         ┌───────────────────────┐
     ┌──▶│         WIKI          │  markdown on disk
     │   │  one concept per page │  explore in Obsidian
     │   └───────────┬───────────┘
     │               │  becomes context
     │               ▼
     │   ┌───────────────────────┐
     │   │       LEARNINGS       │  LearningMachines
     │   │  what worked and why  │  memory + learning
     │   └───────────┬───────────┘
     │               │  guides experiments
     │               ▼
     │   ┌───────────────────────┐
     │   │      POST-TRAIN       │  Tinker API
     │   │ datasets · fine-tunes │  base vs tuned evals
     │   └───────────┬───────────┘
     │               │
     └───────────────┘
       experiment reports feed back into the wiki

My end goal is to fine-tune Inkling, but I'm starting with the cheaper Qwen MoE as a first step. RL Tutor plugs into Claude Code and Codex via MCP, because that's where I write most of my code.

How it works

RL Tutor operates in two modes:

1. Study mode. RL Tutor maintains a Markdown wiki inspired by Andrej Karpathy's LLM knowledge bases. Drop a link and it converts a raw source into knowledge; ask a question and it answers from the wiki.
2. Build mode. When a Tinker key is set, RL Tutor gets tools for creating datasets and running fine-tunes. It compares the tuned checkpoint with the base model and files the experiment back into its wiki.

I use RL Tutor entirely through my coding agents. For example, here's RL Tutor adding a doc to its wiki:

Add https://thinkingmachines.ai/news/introducing-inkling/ to the wiki

The wiki, datasets, and reports are stored locally as plain files. I can inspect everything in my editor or in Obsidian:

As the wiki grows, it becomes the context for the next experiment. When an experiment finishes, or fails, the report becomes another source the tutor can learn from.

The agent

The agent itself is relatively simple. The full code is here, but here's the gist of it:

"""RL Tutor: an agent that teaches me post-training."""
 
from os import getenv
 
from agno.agent import Agent
from agno.db.sqlite import SqliteDb
from agno.models.openai import OpenAIResponses
 
from .learning import LEARNER_INSTRUCTIONS, learning_machine
from .tinker_tools import TinkerTools
from .wiki import parallel_web, wiki
 
FOCUS = f"""<focus>
You are RL Tutor. You are teaching us to train our own models with Tinker,
Thinking Machines' fine-tuning API. Inkling is the primary study subject.
For the first hands-on training runs, use the cheaper default Qwen MoE.
 
Working style:
- Answer established training questions from the wiki, citing pages. For
  facts that may have changed or gaps in the wiki, use web_search and web_fetch
  as needed, prefer primary sources, and cite their URLs in the answer.
- File training knowledge under training/, one concept per page: the model
  (training/inkling.md), the API (training/tinker-api.md), methods like LoRA and RL, recipes, costs.
- Keep training/open-questions.md current: add questions we cannot answer
  yet; when a new page answers one, remove it and cite the page.
...
</focus>
"""
 
tools = [*wiki.get_tools(), *parallel_web.get_tools()]
if getenv("TINKER_API_KEY"):
    tools.append(
        TinkerTools(wiki_path=str(wiki.backend.path), datasets_path=str(DATASETS_PATH))
    )
 
tutor = Agent(
    name="RL Tutor",
    model=OpenAIResponses(id="gpt-5.6"),
    db=SqliteDb(db_file=str(DB_FILE)),
    tools=tools,
    learning=learning_machine,
    instructions=[wiki.instructions(), FOCUS],
    add_history_to_context=True,
    num_history_runs=5,
    # Tool transcripts are preserved in the db but we should
    # avoid injecting bulky datasets and reports into every prompt.
    # The tutor can fetch them from history on demand.
    max_tool_calls_from_history=0,
    read_chat_history=True,
    search_past_sessions=True,
    num_past_sessions_to_search=10,
    num_past_session_runs_in_search=2,
    add_datetime_to_context=True,
)

Current status

RL Tutor is a work in progress. The code is available at ashpreetbedi/rl-tutor.

I've tested the study loop and completed two small, paid Tinker runs. The first run trained but exposed a sampling bug. The second completed end to end and produced clean outputs.

That proves the training, checkpoint, comparison, and reporting pipeline can run. It does not prove the fine-tuned model learned the behavior or is usable in any shape or form.

Try it with me

To try RL Tutor with me, clone the repo, set up the env and credentials, and run the app.

# Clone and cd
git clone https://github.com/ashpreetbedi/rl-tutor && cd rl-tutor
 
# Setup env
uv venv --python 3.12 .venv && source .venv/bin/activate && uv sync --locked -q
 
cp example.env .env
# Export the following keys or set in your .env
export OPENAI_API_KEY="sk-..."
export TINKER_API_KEY="..."
export PARALLEL_API_KEY="..."
 
# Run RL Tutor on AgentOS
python app.py

Your RL Tutor is live at localhost:7777, served with AgentOS — I wrote about why serving agents takes more than a vanilla FastAPI server in FastAPI for Agents. Access it using:

• Chat UI. Open os.agno.com, connect to http://localhost:7777, and chat with the tutor, browse its sessions, memory, and runs.
• REST API at localhost:7777/docs, because it's FastAPI.
• MCP server. Add it to Claude Code and the tutor becomes available in your coding agent.

Add RL Tutor to your coding agents

This is the key part for me: rl-tutor plugs into my coding agents with one command, and that's where I write most of my code.

Open another terminal and run:

uvx agno connect --url http://localhost:7777 --server-name rl-tutor

Then open (or restart) Claude Code — or whichever coding agent you use — and ask: "can you access the rl-tutor mcp?"

First training run

Ask Claude Code to prepare our first training run:

ask rl-tutor to teach me how I would train my first model with Tinker. This is a
walkthrough only: do not call any Tinker tools or create any files. Use the
canonical 30-example haiku experiment and give me five numbered steps covering
the Tinker key, dataset design, saving the dataset, running the fine-tune, and
inspecting the report. End with a separate prompt I can send later if I choose
to run it, then stop without training anything.

The first experiment is intentionally small: 30 examples that teach a model to answer in haiku, followed by a base-versus-tuned comparison. Once the walkthrough makes sense, I can run the fine-tune using:

Run the haiku fine-tune now. Create and save a strong 30-example dataset, wait for the saved confirmation, then run the finetune.

As a reminder: a completed report means the pipeline ran end to end, not that the model learned the behavior.

What's next

I'll continue to learn and explore, turning what I learn into datasets, running small experiments, comparing the results, and filing what happened back into the wiki. After that, I'll keep the experiments small and change one variable at a time—dataset quality, epochs, learning rate, or model. The long-term path still points toward Inkling, DPO, and RL, but I want to earn the steps.

If you try RL Tutor, tell me what worked, what broke, and what confused you. Open an issue or send me a note. I'll keep updating the repo as I learn.

Thanks for reading!

Ashpreet

How I ended up here: I'm the founder of Agno - a framework and runtime for model-agnostic agent platforms, so you can imagine that my day job includes building lots and lots of agents.

Because I do this 16 hours a day, 7 days a week, I've acquired a niche set of skills for spinning up high-quality agents. These skills are pretty useless in the real-world, but for companies looking to build their own agent stack - this post is a goldmine of information.

For example. A few days ago I ran my /improve-agent skill on one of my agents and went to bed. Woke up to 20+ iterations, 282 simulations and 6 accepted improvements, making my agent 20× faster.

I'm a big believer in every company owning their agent stack and this post showcases the payoff. When agent code, traces, evals, and logs live in one place, a coding agent can inspect and improve the platform end to end. Your agent infrastructure becomes self-driving.

Creating a Self Driving Agent Platform

The key takeaway from this post is that your platform needs to be set up for self-driving. You can't just expect any codebase to become self-driving. For example, here's the structure agent-platform I'm going to share with you:

> tree -L 1 .
.
├── agents
├── app
├── db
├── evals
├── workflows
└── AGENTS.md

Because the codebase is set up for agent-driven development, self-driving capabilities are possible. And even better, because of these self-driving capabilities, you can set up your agent platform with one prompt. I encourage you to ignore whatever I'm saying and try it yourself.

Hand this prompt to your favorite coding agent:

Help me set up my agent platform.
 
Clone https://github.com/agno-agi/agentos-railway into a folder called
agent-platform, cd in, read the README, and follow the get started guide.

Self-driving Skills

I'm not going to sell you a quantum thermodynamic AI god, but I will tell you that a set of well-defined skills will get you very far. The agent platform we're setting up comes with 5 skills that drive the platform:

• /create-new-agent takes an idea, builds the agent and smoke-tests a live instance.
• /extend-agent adds capabilities, fixes problems, and verifies live running agents.
• /improve-agent is where the fun begins. Your coding agent derives simulations and probes from the agent's instructions, tests it, judges responses, edits the agent, tunes parameters, and re-probes until the behavior improves.
• /eval-and-improve runs the eval suite, diagnoses regressions, and hill-climbs until the suite is green.
• /review-and-improve catches repo-wide drift. A consequence of coding agents is that your codebase will move fast, and everything (docs, env vars, evals, security) will fall out of sync. /review-and-improve will bring everything up to speed.

Together they cover the full agent-development lifecycle: create, extend, improve, evaluate, and maintain. With these 5 skills as the machinery, a coding agent as the engine, all you need to do is set the destination (create a new agent, improve it, extend it), and your agent platform self-drives there.

The Five Levels of Self-Driving Software

Self-driving software is a fairly new concept, so here's how I mark our progress:

L1 (Assisted, available now). I name the task and invoke a skill. The coding agent executes that stage end to end, shows its work, and hands the result back to me.

L2 (Self-monitoring, available now). The platform has a heartbeat. A deterministic deployment check runs daily by default across the database, auth, scheduler, MCP, and registered components. Model-backed evals can run beside it, but scheduling them is opt-in because they cost tokens. Nothing improves itself yet, but nothing drifts silently either.

L3 (Autonomous improvement, demonstrated). I set a destination and boundaries, not a list of edits. "Make the builder faster and harder to break" is a destination. The platform does the rest: inspect, probe, judge, edit, and prove. In my overnight harness, every accepted improvement became a commit for me to review in the morning.

L4 (Self-repair, building now). The platform notices a regression and shows up with a verified patch. Half of this exists today: Platform Manager already reads deployment reports, eval history, schedules, traces, and runtime state. I'm working on closing the diagnosis-to-patch loop.

L5 (Full self-driving, the vision). The platform notices a recurring request that no agent covers, proposes one, builds it through Agent Builder, hardens it with the improve loop, locks it in with evals, and ships after human approval. Humans set constraints, choose destinations, and approve arrivals.

At L1, I name the task.

At L3, I name the destination.

At L5, the platform knows what to build, what to improve, what to fix.

Self-Driving for Everyone

As always, everything I build is open-source. You can get your own self-driving platform today, with one prompt. Hand this to your favorite coding agent:

Help me set up my agent platform.
 
Clone https://github.com/agno-agi/agentos-railway into a folder called
agent-platform, cd in, read the README, and follow the get started guide.

After you complete the setup, start with /create-new-agent. Then /improve-agent what you built. This is the payoff of owning your agent stack: you own the loop that makes it better.

Thanks for reading!

Ashpreet

Built with 🧡 using Agno.

From model independence, to zdr and cost controls, the stakes are too high to let one vendor play all the roles.

This has been our thesis from day one. Your agents, your data, your cloud, you own everything and control where it runs.

We're super early in the 'own your agent stack' game, but if you want to get ahead, one prompt is all it takes to stake your claim.

Hand this to your coding agent:

Help me set up my agent platform.
 
Clone https://github.com/agno-agi/agentos-railway into a folder called
agent-platform, cd in, read the README, and follow the get started guide.

Why this matters

No model, no vendor, no provider should control your fate. The payoff isn't running your agents, it's learning from their mistakes.

Satya calls it the reverse information paradox, and warns that a frontier without an ecosystem is not stable. I say hold up a little - most of us aren't even at the table.

Models, observability, orchestration; everyone wants our data. You're the creator, don't settle for beta. Owning your stack ends that trade. Your sessions, traces, and corrections stay in your (data) base.

The data that will train someone else's model becomes your learning loop, run the /improve-agent skill and watch your agents beep-boop.

The powers that be

If anthropic pulls another fable, would we survive? Don't worry kid, my AgentOS will thrive.

Swap the model, the context lives with me; I will not surrender my alpha to the powers that be. Sessions, memory, knowledge, and traces, all in my Postgres. My database, my cloud, zero egress.

The giants will debate the frontier, the paradox, the game. You and I will build quietly, and stake our claim.

Now repeat after me; I will not surrender my alpha to the powers that be. One prompt and I'm free.

Help me set up my agent platform.
 
Clone https://github.com/agno-agi/agentos-railway into a folder called
agent-platform, cd in, read the README, and follow the get started guide.

In this post, I'll show you how to build a working image search engine using agents that label images using search terms and then running search on those descriptions instead of the images.

Why?

Image search has generally been a HARD problem. CLIP, multimodal vectors, fine-tuned encoders. So many use cases are blocked because of the complexity whereas the truth is that not every everyuse-case needs google photos like infrastructure.

Some just need a quick and dirty image search engine and now that's possible in about 100 lines of code.

The old default

For years, the assumed path for "search images by natural language" looked like:

1. Run every image through a multimodal encoder (CLIP and friends) to get a vector.
2. Run the query string through the same encoder to get a vector.
3. Nearest-neighbor in shared embedding space.

It's the right approach, but comes bearing costs. Swap encoders and you re-embed your whole library. The vectors are opaque, can't read them, can't grep them, can't explain a miss. And one-word queries like "car" or "drink" tend to land in fuzzy regions of the latent space where the wrong things are also nearby.

The new approach - classify using vision models

Gemini 3.5 flash turns our images into a structured description that reads like a search query a human would type. The model already knows that a yellow NYC taxi is also a car, a vehicle, a piece of transportation, and a thing that lives in Manhattan.

That's the signal we were trying to get out of the embedding space in the first place.

Once we have the description, we don't need image embeddings. We need text embeddings, which we've known how to do for a decade. And you get full-text search for free: stemming, prefix match.

The cheap version of image search might be the right version for most use cases.

The labeling schema

Update the fields to be signals you are looking to search over. For this demo we use:

class ImageDescription(BaseModel):
    caption: str         # one or two sentences, written like a search query
    subjects: list[str]  # 1-5 noun phrases, specific + generic
    scene: str           # short noun phrase: "urban street at night"
    visual_style: str    # "soft morning light", "vibrant macro"
    tags: list[str]      # 12-20 lowercase keywords

The agent returns this as structured output. We flatten it into one string and embed that string. The structured fields stay on the side as metadata for the UI to render.

Search becomes free

Once your index is text, everything you've ever wanted from search is in the box:

• Cosine similarity over the embedded descriptions for semantic recall. "cozy interior" finds a cafe shot tagged "warm", "intimate", "wooden".
• PostgreSQL full-text search (to_tsvector + websearch_to_tsquery) with stemming, so "car" matches "cars" without dragging in "carnivore", and prefix match so "mount" hits "mountain".
• Hybrid fusion, pgvector blends the two scores into one ranked list. Out of the box.

The labeling step is the only part of the pipeline that wasn't great a year ago. Everything downstream is boring infrastructure that's been working for years.

Run it yourself

As always I come bearing code.

The full cookbook lives here. Clone, pip install -r requirements.txt, set GOOGLE_API_KEY, point at a pgvector, and you have a working image search engine in a couple of minutes.

Swap the default URLs in settings.py for your own S3 bucket and you get your own image search engine.

Thanks for reading!

Ashpreet - built with 🧡 using Agno

Coding agents can do all of that work now. Today I'll share how to build an agent platform built, managed and improved entirely by coding agents.

The entire agent development lifecycle is managed using five prompts:

• Create. Scaffolds a new agent.
• Improve. Hardens an existing agent against its own spec.
• Extend. Adds new capabilities to an existing agent.
• Hill Climb. Runs the eval suite, diagnoses failures, fixes what's in scope.
• Review. Sweeps the repo for drift between docs, code, and config.

What is an agent platform

Let's say you built an agent in your favorite framework. How do you take it live? How do you host it in the cloud, send requests to it, secure it?

Your agent platform is the service responsible for running your agents. It takes requests, runs the agent and streams the responses. It collects data and metrics, manages security by preventing unauthorized access, and stops one agent from accessing or polluting the data of another.

If you think of agents as mini-applications, it becomes clear that they need a system to run on, like an OS. Your agent platform is that OS.

What we're building

Today we'll build an agent platform that you can run locally using docker, your own cloud, or on Railway with 1 command. The platform has five parts:

1. Runtime: the service that runs your agents. It handles requests, runs the agent loop, streams responses, writes to storage, handles auth.
2. Storage: the database where our data lives: agent sessions, memory, knowledge, traces and eval history.
3. Connectors: tools for agents to connect with external systems via MCP, API, or CLI. Having them in one place is a big win for security.
4. Interfaces: Slack, Discord, Telegram, custom UIs. One place to resolve identity across surfaces, so the same person is the same user_id whether they ping you in Slack or hit the web app.
5. Infrastructure: where everything runs. We'll use Docker for local and Railway for production. You're free to run production anywhere.

Once it's running, you should be able to ship a new agent in ~10 minutes without writing any code. I know this is a crazy claim so let's give it a shot.

Let's get started

I'm going to share a foundational codebase that you can build upon.

Clone, configure, and run: agent-platform-railway

# Clone the agent-platform template
git clone https://github.com/agno-agi/agent-platform-railway.git agent-platform
cd agent-platform
 
# Configure your environment
# Recommended: copy the example env file and add the key there
cp example.env .env
# Edit .env and add your OPENAI_API_KEY
 
# Run your platform: 1 FastAPI server and 1 Postgres database
docker compose up -d --build

This brings up two containers: a FastAPI server and a Postgres database.

Now let's give our platform a UI.

1. Head to os.agno.com and sign in.
2. Connect to your local OS at http://localhost:8000.

You should see something like:

Agent Development Lifecycle

Because everything is in one place, Claude Code can manage my entire agent development lifecycle.

1. Create an agent

To create a new agent, I open Claude Code and type:

Run create-new-agent.md in a new branch.

Claude starts by asking a few questions about what the agent should do, which tools it needs. It then searches the Agno docs via MCP for the right toolkit, generates the agent file, registers it in app/main.py, restarts the container, and smoke-tests via cURL. 5-10 minutes from prompt to agent.

2. Improve an agent

To improve an existing agent, I type:

Run improve-agent.md on code-search agent.

Claude reads the agent's INSTRUCTIONS and derives 8-12 probes from them. Some golden-path. Some edge cases. Some tool-selection. A couple of adversarial ones thrown in: prompt injections, malformed input, attempts to pull the agent off-purpose.

It runs each probe against the live container via cURL. Reads the response. Reads the tool calls from the container logs. Judges PASS or FAIL against what the INSTRUCTIONS actually promise.

For every failure it picks a lever. Tighten a rule. Add a rule. Swap a tool. Bump num_history_runs. Whatever fits the failure mode. It edits agents/<slug>.py, hot-reloads, and re-runs only the probes that failed.

Zero input from me beyond kicking off the task. This used to take a day of manually clicking things around and now it's fully automated.

3. Extend an agent

To add capabilities to an existing agent, I type:

Run extend-agent.md on code-search agent.

Extend runs with my guidance. I describe a change: add a tool, refine a prompt, fix a bug. Claude executes. The Agno docs MCP is loaded so toolkit research is grounded in the real API.

4. Hill Climb

Over time we collect a lot of evals, and it would be a shame to fix failures manually. I simply type:

Run eval-and-improve.md.

Hill Climb runs the eval suite, diagnoses every failure, and fixes what's in scope.

5. Review

Because the repo is managed primarily by coding agents, it moves fast. To bring everything up to speed, I type:

Run review-and-improve.md.

Claude sweeps the whole repo for drift between docs, code, and config. Fixes what it can. Best run before a release or after a refactor.

Drift between docs and code has always been a tax on production software. Now it costs nothing.

Run in production

When you're ready to ship, the codebase comes with deploy-to-Railway scripts:

./scripts/railway/up.sh

This provisions Postgres and the app service on the same private network.

Read the full Railway guide in the README.

Wrapping up

Congratulations. If you made it this far, you have an auto-improving agent platform running securely in your cloud.

Technical users on your team can create and deploy agents using Claude Code. Non-technical users can use the no-code UI.

Sessions, traces, and knowledge live in your database. Your infrastructure is gated behind JWT-based RBAC and API keys are managed in one place.

The agents you ship today are the smallest part of what you've built. The platform underneath them, and the iteration loop it enables, is the thing that matters.

The entire platform is managed using five prompts:

• Create. Scaffolds a new agent.
• Improve. Hardens an existing agent against its own spec.
• Extend. Adds new capabilities to an existing agent.
• Hill Climb. Runs the eval suite, diagnoses failures, fixes what's in scope.
• Review. Sweeps the repo for drift between docs, code, and config.

The "Improve → Hill Climb" loop recursively improves my agents with minimal oversight. It's hard to imagine doing this manually.

FYI, this auto-improvement loop is only possible because the environment is designed for it. Agent code, traces, logs, eval suite, and the live software all live in one place so a coding agent can rip end-to-end.

It works because we control the stack

Most software can't auto-improve because its inputs and outputs are scattered across tools. To run the auto-improvement loop, a coding agent has to piece together data from three different tools, each behind its own auth, with its own way of doing things.

Theoretically possible. In practice, too much friction.

My codebase is specifically designed for auto-improvement. For example, claude code can test an agent, then PASS or FAIL by reading the sessions, traces, and logs. If the agent fails, it edits the agent and runs it again.

Three things make this possible:

1. Every action is exposed as an API. Running an agent, reading a session, running an eval. Every key action can be run using cURL or bash.
2. Data is colocated. Sessions and traces live in our Postgres database. A coding agent can trigger a run and read what came out without leaving its environment.
3. Logs over everything. The entire platform runs locally on Docker. The coding agent reads live logs and makes updates as needed. The test -> review loop is ~5s. Logs are the real-time feedback loop that unlocks everything.

Agent platforms are the first category of software where the actions, data, and the iteration tool all sit close enough that a coding agent can test end-to-end, make code changes, and test again until the agent improves. Meaning the platform that hosts the loop is the first thing the loop improves.

Agent Development Lifecycle

Next I'll show you how Claude Code runs my agent platform.

1. Create an agent

To create a new agent, I open Claude Code and type:

Run create-new-agent.md in a new branch.

Claude starts by asking a few questions about what the agent should do, which tools it needs. It then searches the Agno docs via MCP for the right toolkit, generates the agent file, registers it in app/main.py, restarts the container, and smoke-tests via cURL. 5-10 minutes from prompt to agent.

Because the platform takes care of everything, I'm building agents I never would have bothered with before. An agent that summarizes overnight Slack messages, an agent that drafts my weekly update, an agent that highlights important issues in the repo. None of these would have survived a multi-day project. All of them fit into a coffee break.

2. Improve an agent

To improve an existing agent, I type:

Run improve-agent.md on code-search agent.

Claude reads the agent's INSTRUCTIONS and derives 8-12 probes from them. Some golden-path. Some edge cases. Some tool-selection. A couple of adversarial ones thrown in: prompt injections, malformed input, attempts to pull the agent off-purpose.

It runs each probe against the live container via cURL. Reads the response. Reads the tool calls from the container logs. Judges PASS or FAIL against what the INSTRUCTIONS actually promise.

For every failure it picks a lever. Tighten a rule. Add a rule. Swap a tool. Bump num_history_runs. Whatever fits the failure mode. It edits agents/<slug>.py, hot-reloads, and re-runs only the probes that failed.

Then it iterates. Capped at five rounds. Stops earlier if everything passes.

Zero input from me beyond kicking off the task. This used to take a day of manually clicking things around and now its fully automated.

3. Extend an agent

To add capabilities to an existing agent, I type:

Run extend-agent.md on code-search agent.

Extend runs with me in the driver's seat. I describe a change: add a tool, refine a prompt, fix a bug. Claude executes. The Agno docs MCP is loaded so toolkit research is grounded in the real API.

Claude makes the changes. Runs smoke-tests. Each iteration is one small, verified step. Changes stay surgical and get tested in isolation.

4. Hill Climb

Over time we collect a lot of evals, and it would be a shame to fix failures manually. I simply type:

Run eval-and-improve.md.

Hill Climb runs the eval suite, diagnoses every failure, and fixes what's in scope. Failure types map to fix locations: missing rule in INSTRUCTIONS, hallucination, wrong tool fired, overspecified rubric. For each failure Claude picks the right lever, edits, and re-runs only the failing case. Once everything is green it re-runs the full suite to catch regressions.

The eval suite is two files. evals/cases.py declares the cases. Each case is one input plus a rubric (what a correct response looks like) and optionally an expected tool call. Built on Agno's AgentAsJudgeEval and ReliabilityEval.

Improve catches out-of-distribution failures. Hill Climb makes sure in-distribution cases continue to pass. The two work very well together.

5. Review

Because the repo is managed primarily by coding agents, it moves fast. To bring everything up to speed, I type:

Run review-and-improve.md.

Claude sweeps the whole repo for drift between docs, code, and config. Every agent file on disk should be registered in app/main.py. Every env var the code reads should be in example.env and the AGENTS.md. Every path in a markdown doc should still exist. Every script should do what's claimed.

Mechanical drift gets auto-fixed in place: a renamed file, a missing entry in example.env, a new agent missing from the architecture diagram. Anything bigger gets flagged with a recommended next step.

Best run before a release or after a refactor. The kind of work that's tedious for a human and trivial for a coding agent that can read every file in the repo.

Drift between docs and code has always been a tax on production software. Now it costs nothing.

Why Agent Platforms?

Agent platforms are the perfect testing ground for this pattern.

1. Greenfield. Agent platforms are relatively new and can be designed for coding agents from the get go.
2. Workflow is clear. We know how to improve an agent: run it, read the logs, grade the response, edit, run again.
3. The loop is actually useful. For regular software, optimizing an API endpoint doesn't really make sense. For agents, each round of improvement is real, measurable, and adds value.

Set the platform up right and you can build any agent on top of it: use the create workflow to go from idea to agent, the improve workflow to harden the agent, the extend workflow to add new capabilities, lock them up using evals and then hill-climb against them.

Keep the entire repo in sync using the review-and-improve workflow.

It's almost impossible to do this by hand.

My Auto Improving Agent Platform

Here's a link to my auto-improving agent platform: agent-platform-railway.

It's a starter codebase for an agent platform you can run locally using docker or on Railway. The prompts are in the docs/ folder. Clone, configure, and you're running agents in 10 minutes.

Follow the README for the full setup guide, and Agno Docs for reference.

Auto improving software

I've been running this loop for a few weeks and it continues to surprise me.

Agent instructions tightened by half a sentence. A docstring brought in sync with code. The platform is a little cleaner every time I run it.

I can see a world where all software works like this. A coding agent managing your platform end-to-end, fixing things small enough you'd never have prioritized them. Thanks for reading!

Ashpreet

Built with 🧡 using Agno.

Note: this post is about building your own agents (agentic software engineering), not about using coding agents.

By now you've probably used a few agents, or at least heard of Claude Code, Codex, or OpenClaw. Ever wondered what it takes to build your own?

Most people think of agents as prompts + tools in a loop. That's a reasonable assumption, but it's not production architecture.

The moment your agent needs to know who it's talking to, maintain state, handle concurrent requests, take sensitive actions like refunds, and survive failing tool calls, it stops being an "LLM + tools in a loop" and becomes a distributed system.

Building agents is the easy part. There are 75 frameworks that help you do that. The hard part is the runtime: the harness around the agent that makes it work in the real world. That's what agentic software engineering is all about.

Build. Serve. Connect.

Here's how I think about shipping agentic software.

Build the agent. Define the model, tools, knowledge base, memory, storage, and guardrails. This is the layer that most frameworks give you.

Serve it as an API. User-scoped, session-scoped, horizontally scalable. Add persistent storage, streaming, background execution, retry semantics. This is where most agentic products stall. Not because the agent doesn't work, but because it doesn't have the infrastructure to work reliably at scale.

Connect it to where users live. Your product, Slack, Discord, MCP, wherever. An agent in a notebook is an experiment. An agent where your users are is a product.

The 6 Pillars of Agentic Software

Building an agent is AI engineering. Running it in production is software engineering. Together, they form agentic software engineering: the practice of building, running, and scaling agents as production services.

Here are the six pillars that hold it up:

Durability. Agents reason across multiple steps, call tools that time out, and fail halfway through. If your agent crashes on step 12 of 15, restarting might duplicate a side effect or lose critical context. Agentic software needs to pause, resume, checkpoint, and recover gracefully. Durability turns failure into resumption, not a full restart.

Isolation. Agentic software serves thousands of users simultaneously. Each user needs their own session, their own memory, their own context. Passing a user_id with each request is easy. Isolating every resource the agent touches is where the engineering comes in. Your database, your vector store, your model provider, all need to respect user boundaries. One missing filter becomes a data breach.

Governance. Agents that can act can also cause damage. Looking up a record is harmless. Deleting a record or issuing a refund needs approval. Agentic software needs layered authority: what runs automatically, what needs human approval, and what needs admin sign-off. Today, most agents auto-execute with minimal oversight. As they get more capable, governance becomes the product.

Persistence. An agent without persistent storage can't learn, can't build context, can't improve. We need to store sessions, memory, knowledge in a database. Persistent state is what turns a chatbot into a product. Every conversation makes the next one better.

Scale. A thousand users hit your agent at the same time. Requests queue, you hit model rate limits, and tool calls compete for resources. Traditional services call your own backends. Agentic software calls external model APIs and third-party tools, which means you inherit their rate limits, latency, and downtime. Scaling agentic software means scaling around dependencies you don't control.

Composability. When an agent is a service, other agents can call it. Your frontend can call it. Your Slack bot can call it. MCP clients can discover it. It becomes a building block in your architecture, and every new integration becomes a standard API call. That's how single-agent tools become multi-agent systems.

None of this is new. We've been building reliable distributed systems for decades. The AI industry just hasn't brought those lessons along yet, and we're feeling it in every failed deployment.

From Theory to Practice

As always, I come bearing code. Here's how you can start building your own agentic service today.

# 1. Clone the repo
git clone \
    https://github.com/agno-agi/agentos-docker-template.git \
    agentos
 
cd agentos
 
# 2. Set your model provider key
cp example.env .env
# Edit .env and add OPENAI_API_KEY
 
# 3. Start the application
docker compose up -d --build
 
# 4. Optional: Load documents for the knowledge agent
docker exec -it agentos-api python -m agents.knowledge_agent

This gives you a containerized service with persistent storage (Postgres), two starter agents (a knowledge agent using Agentic RAG and an MCP agent for external tool use), and a REST API you can connect to from anywhere.

I'm using Docker for this template because Docker runs everywhere: your laptop, AWS, GCP, Azure, Railway. The same container you develop locally is the one you deploy to production. The README covers everything you need to get started.

After running the service:

1. Open localhost:8000/docs to see your API.
2. Connect to the web UI at os.agno.com where you can chat with your agents, trace runs, manage knowledge, create schedules and approve sensitive tool calls. One UI for your agentic software.

Adding your own agent is a few lines of Python and a restart. Swap models with a one-line change. Add tools from 100+ integrations. The template is a starting point. Read the Agno docs to learn more.

Governance & Elicitation

Most agents run tool calls with minimal oversight or auditability. In practice, we need layered authority:

1. Tools that run freely
2. Tools that need user approval
3. Tools that need admin approval

Agents also need to ask questions (often called elicitation). The Claude Code team shared a great article on the AskUserQuestion tool used by Claude.

This is available in Agno as UserFeedbackTools. Here's a support agent that can look up orders freely, ask the customer structured questions when it needs more information, and wait for admin approval before issuing a refund:

support = Agent(
    id="support",
    name="Support",
    model=OpenAIResponses(id="gpt-5.2"),
    db=agent_db,
    tools=[
        lookup_order,             # auto-execute
        search_help_docs,         # auto-execute
        issue_refund,             # requires user confirmation
        UserFeedbackTools(),      # structured questions
    ],
    instructions=instructions,
    enable_agentic_memory=True,
)

Watch what happens when a customer asks for a refund.

• The agent looks up the order on its own, no permission needed.
• Then it hits a decision point: why does the customer want the refund?
• Instead of guessing, it presents a structured question with clear options: defective, wrong item, changed mind.
• The customer picks one. Now the agent calls the refund tool, but because refunds carry real consequences, it pauses for user approval.
• Once approved, the agent runs the refund tool.

Three levels of agency in one conversation. You can view the full code here.

The agent knows when to act, when to ask, and when to wait. That's what governance looks like in practice. The runtime has to support all three modes, and the transitions between them have to feel natural.

Note: the approvals flow on the UI is actively being developed. The refund should wait for admin approval, not user approval. This is implemented on the SDK but not the UI yet. This is being fixed this week.

Agents are distributed systems

The 5 Levels describe how agentic software grows in capability (and complexity). The 7 Sins describe how they fail in production. The 6 Pillars describe what it takes to build them right.

The consistent message across all three: agentic software engineering is a discipline. The teams that internalize this early will ship great products. The teams that keep treating agents as scripts will continue to miss the mark.

Clone the repo. Build your first agent. Ship it where your users are.

Links:

• Agno Docs
• Agno Github
• AgentOS Templates
• AgentOS Docker Template

In 2026 we are building agents the other way around. If you believe we're at the cusp of a new operating system, then the agent is the shell and every tool is a program. The question is: why do we keep stuffing the shell's prompt with the man pages of every program it might call?

The most powerful technology of this decade is bottlenecked by the number of tools it can hold in its "RAM" and every agent with multiple toolkits hits the same three problems:

1. Context pollution from too many tools
2. Degrading performance from blurry scopes (eg: search operations in multiple toolkits)
3. The main agent getting confused because its context is all tool instructions

I've been testing a protocol that fixes all three.

The three walls

Context pollution. Every tool takes up precious context. Schemas, descriptions, example usage, all of it lands in the system prompt. A Slack toolkit is 8 to 12 tools. Gmail is 6 to 10. Calendar another 6. Drive, GitHub, your CRM, the web. You're at 50 tools before adding anything custom. From what I've seen, somewhere past 20 tools models start hallucinating tools that don't exist, calling tools with the wrong shape, or skipping the right tool because its description got buried.

Blurry scopes don't compose. Two tools both take a workspace argument: one is Slack's, one is Google's. search in one MCP collides with search in another. send_message could be Slack, email, or your CRM. The agent picks wrong half the time, and no naming convention fixes it because the same word legitimately means different things in different sources. The minute you compose tools from sources you don't control (MCP servers, third-party SDKs), you get overlap, and the model has no reliable way to disambiguate.

Tool-use logic lives with the main agent. This is the deepest wall.

For an agent to use Slack well, the system prompt has to explain Slack: look up the user ID before you DM them, resolve a channel name to an ID before you post, prefer conversations.history for channels and conversations.replies for threads, paginate by cursor instead of offset. That's hundreds of tokens of Slack-specific guidance. Now do that for Gmail. For Calendar. For Drive. For your database. For GitHub.

The system prompt becomes the union of every API's quirks. Every turn carries every rule, even when the user just asked about Slack. The main agent is stuck reasoning about both the user's question and the mechanics of every API. Adding a source means editing the prompt and praying nothing breaks.

The missing layer

Today the canonical agent shape is some variant of:

Agent → Tools                             # raw
Agent → MCP server → Tools                # MCP
Agent + Skill instructions → Tools        # Skills

In all three cases the agent sees the raw tool surface of every source. Every Slack tool, every Drive tool, every CRM tool. The agent's prompt has to contain how to use every one of them.

The shape I've been testing puts a thin layer in between:

Agent ↔ ContextProvider ↔ Tools

A ContextProvider wraps one source: Slack, GitHub, Drive, Filesystem, your DB. To the calling agent, it exposes exactly two tools:

• query_<source>(question) for natural-language reads
• update_<source>(instruction) for natural-language writes (or a clean read-only error)

The main agent doesn't see Slack's twelve tools. It sees query_slack and update_slack. It doesn't see Drive's quirks. It sees query_drive. Add ten more sources and the agent's tool surface stays linear at 2N.

Behind each tool is a sub-agent scoped to that one source. The sub-agent owns the source's tools, the source's quirks, the lookup-before-write patterns, the pagination weirdness. It runs in its own context, returns an answer, and the main agent gets a clean result.

from agno.agent import Agent
from agno.context.slack import SlackContextProvider
from agno.context.gdrive import GDriveContextProvider
from agno.context.database import DatabaseContextProvider
from agno.models.openai import OpenAIResponses
 
# Sub-agents do source-specific tool work — a cheaper model is plenty.
provider_model = OpenAIResponses(id="gpt-5.4-mini")
 
slack = SlackContextProvider(id="slack", token=..., model=provider_model)
drive = GDriveContextProvider(id="drive", service_account_file=..., model=provider_model)
crm   = DatabaseContextProvider(id="crm", sql_engine=engine, model=provider_model)
 
agent = Agent(
    model=OpenAIResponses(id="gpt-5.4"),
    tools=[*slack.get_tools(), *drive.get_tools(), *crm.get_tools()],
    instructions="\n".join([slack.instructions(), drive.instructions(), crm.instructions()]),
)

The agent sees four tools: query_slack, query_drive, query_crm, update_crm. Three sources, two of them read-only. Five years of API quirks for those three sources, summarized into four tool descriptions.

Btw the quirks didn't vanish, they just moved into the sub-agent's scope, where they belong, and only load on turns that actually touch that source.

The biggest advantage is that the main agent doesn't see the gunk of intermediate tool calls, or the fact that slack returned 1000 channel names to find the correct channel id.

A walkthrough

Here's an example from Scout, the agent I've been testing this with. The user asks:

What's the verdict on the v3 migration, and is the spec doc up to date?

Scout has the following tools: query_slack, query_drive, query_filesystem, query_web.

It picks two.

query_slack("recent engineering channel discussion about v3 migration")

The Slack sub-agent runs search_messages, walks the thread with conversations_replies, and returns:

On Tuesday, engineering decided to phase the v3 migration over two weeks, starting with the read path. James raised concerns about the rollback procedure that haven't been resolved in-thread.

query_drive("v3 migration spec document, last updated when, current contents")

The Drive sub-agent runs files.list, fetches the doc, returns:

The v3 migration spec was last edited Friday by Mike. The current version describes a single-phase migration. No mention of phased rollout or rollback procedure.

The main agent writes the answer:

Engineering agreed Tuesday on a phased migration starting with the read path, but the spec was last edited Friday and still describes a single-phase approach. James's rollback concerns aren't reflected in the doc either. The spec needs an update.

The main agent never touched a Slack API tool. It never paginated a Drive listing. It composed two natural-language calls and wrote one paragraph. The Slack and Drive specialists handled their own messes in their own contexts.

What about Skills?

Skills are another serious attempt at problem #3. A Skill packages task-specific instructions ("here's how to use Slack") into a module the model loads when relevant, instead of carrying it in the system prompt full-time. They move task knowledge out of the always-on prompt and into something more conditional. If problem #3 were the only issues, we could solve this skill issue.

But Skills don't fix walls 1 and 2. The strongest version — Skills that bring scoped tools when loaded — still leaves you with N tool surfaces in the agent's context once N skills are active, and search still collides with search. There's also a higher chance of conflicting Skills quietly degrading the agent without you noticing.

ContextProvider and Skills compose. A Slack ContextProvider's sub-agent can itself load a Slack Skill, and that's where the Skill does its best work: in the context of the thing actually executing against Slack, not in the main agent that just wanted an answer.

Roughly: Skills compress how to do a task. ContextProvider hides that there's a task until the main agent decides to delegate one. Different layers, both useful.

Examples

As always, I come bearing code. The full set of examples lives in cookbook/12_context. A few worth pointing at:

Sources covered out of the box. Filesystem (00), database (04), Slack (05), Google Drive (07), GitHub (12), and web via Exa or Parallel, direct SDK or MCP endpoint (01, 02, 03, 11). Every provider follows the same query_<id> / update_<id> shape.

Read/write split with real security. 04_database_read_write.py spins up a SQLite DB and has the agent insert a contact, read it back, and verify with direct SQL. Read and write go through separate sub-agents with separate engines, same shape I used in Dash, same reason: the database itself rejects writes the model isn't allowed to make, regardless of what the prompt says. 12_github.py does the same shape over a real repo: reads through a read-only sub-agent on a clone, writes through a sub-agent that operates on a per-session worktree on a <prefix>/<task> branch and ends in a PR. The agent cannot push to the default branch.

Compositional multi-source. 09_web_plus_slack.py is the shape flat tool layouts can't do without orchestration code. The agent pulls topics from a Slack channel, runs a per-topic web search, and returns a briefing tying each internal thread to an external reference. Two providers, one prompt, and the main agent stitches the synthesis itself. 12_engineering_briefing.py takes it one step further: Slack topics → codebase workspace → web fallback, all in one prompt.

MCP wrapper. 06_mcp_server.py wraps any MCP server (stdio or HTTP) as a single query_<id> tool. The sub-agent's instructions are built from the server's list_tools() response at connect time, so the calling agent never sees stale tool docs. Staleness is bounded by sub-agent lifetime, not eliminated. But for any reasonable session, that's the same difference. This is the move that collapses a 50-tool MCP server to 1 tool from the main agent's view.

Surprises and open questions

Sub-agents are cheaper than expected. I assumed the extra hops would dominate. They don't. The main agent's context is so much smaller that its calls are faster, and the sub-agent only fires on turns that touch its source. Anecdotal on Scout's workload: total tokens are roughly flat at low source counts and improve as the source count grows; wall-clock latency drops at every source count I've measured. I haven't generalized this off Scout yet.

The main agent's prompt got smaller. I expected to add orchestration logic. I removed it. With a uniform surface, the routing rules collapse to "pick the right query_<source>". gpt-5.4 just works out of the box, with zero guidance on how to use a source.

A few things I'm still working through:

• How thin can the main agent's prompt get? I've been hill-climbing this with evals.
• Caching across calls in a session. The same query_<source>("who's on the X channel") shouldn't re-do the work two turns later.
• Per-user authentication that survives the hop. Partially solved. Scout passes user_id, session_id, metadata, and dependencies through to the sub-agent. More to do for OAuth flows.
• When to expose underlying tools instead. Some sources benefit from the agent driving the tool calls directly, usually when the source is small enough that the schema cost is low. The protocol has a mode for this. I'm still figuring out where the line sits.

TL;DR

The agent's tool surface should be its job description, not the union of every API it might touch. Context Providers move the API mess to where it belongs and leave the main agent free to reason about what the user actually asked.

• Cookbook examples
• Scout: open-source company intelligence agent built on this pattern

Every company wants to build one and run a fleet of agents. Having built cloud, compute and data platforms before, I'm hoping we can learn from the mistakes of our past and build it right the first time 'round.

The data era is a particularly cautionary tale. I lived through the great unbundling where we had to stitch multiple tools for ingestion, orchestration, transformation, and services for metadata, BI, and data quality. Each tool worked great in isolation but using them together was a pain. 80% of engineering time went into gluing things together. Then came the bundling. Snowflake, Databricks, and the cloud providers (esp AWS) consolidated the stack and provided unified platforms.

The agent era is starting the same way, with features being sold as products and vendors for solving problems that don't really exist yet.

If you find yourself running agents with data on 3 providers, traces in multiple places and no auto-improvement loop, this article is for you.

You need an agent platform

If you think of agents as apps, it becomes clear that they need a system to run on, like an OS.

Your agent platform is responsible for running the agents, collecting data and metrics, managing security by preventing unauthorized access, and stopping one agent from accessing or polluting the data of another.

What we're building

Today we'll build an agent platform that you can run locally, or on Railway for $20/mo.

Once it's running, you should be able to ship a new agent (or workflow) without writing any code. Because the platform takes care of everything, Claude Code can build new agents with a high degree of quality, and then recursively improve them by querying them live. This is the single biggest advantage of having a unified agent platform.

We'll also set up a scheduler for recurring work, lock in behavior with evals, and connect agents to interfaces like Slack.

The most fun is watching Claude Code recursively improve your agents.

What makes an agent platform

An agent platform is made of five parts:

1. Runtime: the service that runs the agents. This part does most of the heavy lifting.
2. Storage: the database where our data lives: agent sessions, memory, knowledge, traces and eval history.
3. Connectors: tools for agents to connect with external systems via MCP, API, or CLI. Having them in one place is a big + for security.
4. Interfaces: Slack, Discord, Telegram, custom UIs. One place to resolve identity across surfaces, so the same person is the same user_id whether they ping you in Slack or hit the web app.
5. Infrastructure: where everything runs. We'll use Docker for local and Railway for production.

Let's get started

I'm going to share a foundational codebase that you can build upon.

This, in my opinion, is the perfect starting point for an agent platform.

agent-platform
├── agents                       # agent code goes here
│   ├── code_search.py
│   └── web_search.py
├── app                          # server code goes here
│   ├── config.yaml
│   ├── main.py
│   └── settings.py
├── compose.yaml
├── db
│   ├── session.py
│   └── url.py
├── Dockerfile
├── docs                         # docs go here
│   ├── create-new-agent.md
│   ├── eval-and-improve.md
│   ├── improve-agent.md
│   └── review-and-improve.md
├── evals
│   └── cases.py                 # test cases
└── README.md

Step 1: Run locally

First let's clone, configure, and run our agent platform.

Make sure Docker is installed and running. Follow these steps if not.

Then open your terminal and run one by one:

1. Clone the agent-platform template

git clone https://github.com/agno-agi/agentos-railway-template.git agent-platform
cd agent-platform

2. Configure your environment. Copy the example .env file, open in your favorite code editor and add the OpenAI key there.

cp example.env .env

3. Run your platform: 1 FastAPI server and 1 Postgres database

docker compose up -d --build

This brings up two containers: a FastAPI server and a Postgres database. Confirm the API is running at http://localhost:8000/docs.

Now let's give our platform a UI.

1. Head to os.agno.com and sign in.
2. Click Add OS → Local, enter http://localhost:8000, and click Connect.

You should see something like:

Step 2: Create your first agent

The codebase comes with two reference agents and a Claude Code prompt that can build new ones for you.

To create a new agent, open Claude Code and run:

Run docs/create-new-agent.md

Claude will ask you what the agent should do, what tools it needs, then generate the agent file, register it in app/main.py, restart the container, and run a smoke test.

This usually takes 5-10 minutes for a simple agent. More if you're building something bespoke with custom tools.

Step 3: Test your new agent

Open os.agno.com to chat with your agent. Run it through realistic prompts. Check the traces and sessions. Try to break it. Try out-of-distribution questions, prompt injections, edge cases. This usually takes 5-20 minutes.

Step 4: Recursively improve your agent

This is where your platform starts paying dividends.

Open Claude Code in the repo and paste:

Run docs/improve-agent.md

Claude Code can directly hit your live agents using curl. Then iterate and improve your agents.

This is why owning the stack pays off. The trace data, the agent code, the running platform, and the iteration tool all live in one box. Claude Code can see all of it and improve as needed.

Step 5: Lock in behavior with evals

Evals are the regression test for your agents. Same prompts, same agents, run on a schedule, fail when behavior drifts.

Evals are defined as a set of cases:

# evals/cases.py
Case(
    name="web_search_recent_anthropic_research",
    agent=web_search,
    input="What did Anthropic publish about agent research recently?",
    criteria=(
        "Answers the question by citing at least one real Anthropic URL "
        "(anthropic.com domain). The response is grounded in fetched content "
        "rather than refusing to answer."
    ),
    expected_tool_calls=(_WEB_SEARCH_TOOL,),
)

Run them later with: python -m evals

Results write to your Postgres via eval_db, so eval history shows up at os.agno.com alongside sessions and traces. Connect Claude Code to the diagnosis loop by pasting Run docs/eval-and-improve.md. It triages every failure and fixes the issues in scope.

Step 6: Run on Railway

Let's take our platform live by hosting it somewhere. Your company probably has a set way of running software. Follow whatever that is.

If you're looking for a place to test this out without going through the full devops process, Railway is the cheapest and fastest PaaS I've found. $20/month gets you pretty far and the codebase already comes with deploy-to-railway scripts.

Requires the Railway CLI and railway login.

6.1 Configure production environment

The deploy scripts read .env.production first and fall back to .env. This lets you keep separate values for local and production: different OpenAI keys with different budgets, production-only credentials, a different Slack workspace, and so on.

cp .env .env.production
# Edit .env.production with production values

6.2 Deploy

The codebase comes with a script that provisions a Postgres database and deploys the app on the same private network. Run it:

./scripts/railway/up.sh

6.3 Your first deploy will fail. That's expected.

Token-Based Authorization is ON by default.

Without a JWT_VERIFICATION_KEY, the app refuses to serve traffic. Your platform's job is to keep your data off the public web. The fix is to generate a key and put it in your production env.

The alternative was to ship with auth off and expect people to add it on later, which we all know isn't going to happen. People will leave their servers open to the public internet, get hacked, then blame me.

Token-Based Auth gives you three things:

• No public access. The server rejects requests without a valid token.
• Per-request identity. The middleware parses the token and injects user_id, session_id, and custom claims into your endpoints. Each request is tied to a user and session, so data leakage is prevented.
• Granular permissions. A user-level token can run an agent and view its own sessions. An admin token can read everyone's sessions and test any agent. You don't need to know or care about RBAC right now, but you have the foundation in place for when you start thinking about security.

6.4 Get your verification key

The default path is to let os.agno.com generate the keypair for you:

1. Open os.agno.com, click Add OS → Live, enter your Railway domain, and connect.
2. Enable Token Based Authorization.

3. Paste the public key into .env.production (full PEM block, no surrounding quotes):

JWT_VERIFICATION_KEY=-----BEGIN PUBLIC KEY-----
MIIBIjANBgkq...
-----END PUBLIC KEY-----

Live connections to AgentOS require a pro subscription. Use the PLATFORM30 coupon code for a 1 month free trial. Remember to cancel before the trial ends if you don't want to be charged.

You don't have to use os.agno.com for this. You can generate your own RSA or EC keypair, sign tokens with the private key in your own service, and put the matching public key in JWT_VERIFICATION_KEY. The platform doesn't care where the key came from, as long as incoming tokens verify.

6.5 Sync env and verify

While .env.production is open, point the in-cluster scheduler at your public Railway domain so cron triggers can reach AgentOS:

# .env.production
AGENTOS_URL=https://<your-app>.up.railway.app

Then push variables to Railway:

./scripts/railway/env-sync.sh

Railway auto-deploys when env values change. Watch the logs and confirm the platform is serving:

railway logs --service agent-os

Once you see successful requests, AgentOS will connect through your Railway domain and you're live.

6.6 Auto-deploys from GitHub

So far every code update needs ./scripts/railway/redeploy.sh. To auto-deploy on every push to main:

1. Open the Railway dashboard → your project → the agent-os service → Settings.
2. Under Source, click Connect Repo and pick your repo.
3. Set the deploy branch to main.

Every push to main now triggers a build and rolling deploy. ./scripts/railway/env-sync.sh is still how you sync env changes.

Opting out of JWT (not recommended)

If you must run production without auth (e.g., inside a private VPC behind another auth layer), set authorization=False in app/main.py and redeploy. Keep authorization on for any deploy holding real data. Without it, anyone who guesses your Railway domain can read your sessions and your agents.

Scaling

The default deploy is two replicas at 4Gi memory and 2 vCPU each. Gives you zero-downtime rolling deploys and basic fault tolerance. Bump numReplicas and limits up or down in railway.json as your usage grows.

Going beyond agents

Rule of thumb: agents for open questions, teams for routing, workflows for processes. Most of your platform will be agents. A few will be teams or workflows. You'll know when you need each.

Multi-agent teams. When one agent isn't enough, route work across a team of specialists. Agno teams come in three modes:

• Coordinate. A leader plans the work, calls the right specialists, and synthesizes.
• Route. A router picks one specialist to handle the request.
• Broadcast. Every specialist runs in parallel; you aggregate.

Use teams when the right specialist isn't known up front. The teams overview walks through each mode.

Agentic workflows. When a process needs to run the same way every time, write a workflow. Workflows give you determinism. Use them for the few high-leverage flows in your platform that need to be repeatable. The workflows overview covers the patterns.

For more on agents themselves (instructions, tools, memory, model configuration), the agents overview is the reference.

Scheduled tasks

The platform ships with a lightweight scheduler enabled by default in app/main.py:

agent_os = AgentOS(
    name="AgentOS",
    scheduler=True,
    ...
)

Schedule any agent or workflow on a cron. Common patterns:

• Maintenance. Purge sessions older than 90 days. Vacuum your Postgres tables. Rotate trace data into cold storage.
• Proactive runs. Every weekday morning, run an agent that summarizes overnight news for your portfolio companies. Post to Slack.
• Catch regressions. Schedule python -m evals weekly against your production agents. Drift shows up in eval history before users feel it.

See the agno scheduler docs for the cron API.

Connect to interfaces

Your agents should be available where your users are. Slack threads. Discord channels. Telegram for the field team.

Or most importantly: a custom UI inside your product.

For Slack, Discord, Telegram: the pattern is similar. Expose the agents via an interface. See app/main.py for a reference:

interfaces: list = []
if SLACK_BOT_TOKEN and SLACK_SIGNING_SECRET:
    from agno.os.interfaces.slack import Slack
 
    interfaces.append(
        Slack(
            agent=code_search,
            streaming=True,
            token=SLACK_BOT_TOKEN,
            signing_secret=SLACK_SIGNING_SECRET,
            resolve_user_identity=True,
        )
    )
 
...
 
agent_os = AgentOS(
    ...
    interfaces=interfaces,
)

Read the interfaces guide for more information.

Wrapping up

Congratulations. If you made it this far, you have a unified agent platform running securely in your cloud. Technical users can create and deploy agents using Claude Code. Non-technical users can use the no-code UI.

Sessions, traces, and knowledge live in your database. Your infrastructure is gated behind JWT-based RBAC and API keys are managed in one place.

Why it's important to control your data

Before we close, a note on data sovereignty.

Every interaction with your platform produces data. Sessions, memory, and traces all flow into your Postgres database. Two reasons why this matters:

1. Compliance. Keeping the data in your own database reduces the risk of a breach. The moment customer data, proprietary code, or internal documents touch a third-party trace tool or memory service, your level of security is whatever their level of security is.

2. Auto Improvement. Your traces are how Claude Code (or you) close the loop on agent quality. Coding agents are going to be the main way to build and improve agents. They can only work because the trace data lives where the iteration tool can read it. Vendor-stitched stacks split this surface across three SaaS products and the loop never closes.

The agents you ship today are the smallest part of what you've built. The platform underneath them, and the iteration loop it enables, is the thing that matters.

Every program you've ever used is a collection of functions run through a hard-coded control flow: If, else, while, for. The functions do the work. Reading from databases. Calling APIs. Transforming data.

Same Input = Same Output. This was the contract for fifty years.

Then 2024 happened. The control flow came alive and created a new category of software. Software that is alive, dynamic, on-demand.

Software is dead, long live Software

Static software is a recording. You press play and you get back exactly what was captured. Same notes, same order, every time. The performance happened once, in a devbox, and now it plays the same tune every time.

Dynamic Software is a live orchestra.

The score exists. The instruments exist. The musicians exist. But what happens in the room tonight depends on the maestro, the players, the moment. The model is the maestro. The tools are the instruments. The control flow is the performance, not the recording.

This is what people feel when they use a great agent and can't quite explain why it feels different. They've spent their whole lives interacting with buttons. Now they're in a room with a live performance for the first time. The software is responding to them, here, now, with judgment and presence. It's listening. It's adjusting. It's alive in a way software has never been alive before.

Recordings are perfect. Live performances aren't. A live orchestra makes choices. Sometimes it stumbles. Sometimes it surprises you. The reason we still pay to hear live music is that something different happens in the room.

The performance is the point.

Dynamic Software is alive. It's not deterministic. It's not perfect. And once you've felt the difference, recordings feel like what they always were. Frozen.

We're not building better recordings. We're building the first generation of software that performs.

Assumptions Dynamic Software breaks

When software comes alive, every assumption built on static software breaks.

Determinism breaks. Same input no longer means same output. The model considers context, memory, learnings. The software does something different on Tuesday afternoon than it did on Monday morning. While this can be (somewhat) controlled in text, we should note that the visual era is next. Charts, dashboards, entire screens generated on-demand. Instead of forcing determinism on non-deterministic software, give in, enjoy the ride.

State and time work differently. Static programs don't need to remember much. The control flow is the same every time, so state lives in a database and is CRUD only. In Dynamic Software, state is context. Memory of past sessions. History of what worked. Knowledge of the domain. The database stops being storage and becomes the context the software runs on.

Sessions follow from this. A static API endpoint is stateless by design. Each request is independent. Dynamic Software is the opposite. A session is a continuous context that spans minutes, days, sometimes weeks. The user comes back, the agent picks up where it left off. Sessions become first-class.

Time changes too. Static software returns in milliseconds, seconds if you don't believe in data co-location. Dynamic Software reasons. It calls tools. It waits for tools to return. It reasons again. A single request takes minutes sometimes. Streaming is the default. Background execution is a core primitive. The HTTP request/response model strains and breaks and so does the default 29s loadbalancer timeout.

The software needs to watch itself. With static software, you can read the code and know what it does. With Dynamic Software, you can't. The control flow is a model and the model is opaque. The only way to know what your software did is to record everything it did. Every reasoning step. Every tool call. Every retrieval. Tracing goes from a debugging tool to the only way to understand your software.

Watching isn't enough. Static programs don't make decisions, so there's nothing to approve. Dynamic Software makes decisions, and decisions have consequences. Some can be made freely. Some need the user. Some need an admin. Your software has to express which is which, and your runtime has to enforce it.

Every one of these is a real engineering problem. Every team building Dynamic Software hits them all. Most spend months solving these from scratch.

A new category needs a new runtime

Static software has a mature runtime. You write Django or Express, deploy to a managed platform, and don't think about HTTP, sessions, scaling, or recovery. The infrastructure is solved. The platform handles it.

Dynamic Software has no equivalent. You write an agent. Then you build six months of infrastructure around it, fixing every edge case manually. Edge cases you only learn after running agents at scale. SSE + websockets. Streaming + background execution. Sessions that survive restarts. Storage you can actually query, not five vendors stitched together. Approval gates that wait for admin sign-off, not just user confirmation. Per-resource, per-tool RBAC. Agents available on Slack, Telegram, WhatsApp, because no one wants to use a custom UI.

This is why 80% of agents don't work, there's a painful amount of grind in the last mile.

The last shift this big was going from desktop apps to web apps. Web software needed its own runtime, its own protocols, its own infrastructure, its own developer tools. We spent two decades building all of it.

Dynamic Software is here. Starting from scratch. Its own runtime. Its own protocols. Its own infrastructure. Its own developer tools.

The next decade

Static software took fifty years to mature. Operating systems, databases, web servers, deploy platforms, observability stacks, identity providers. We forget how recent most of it is. Heroku was 2007. Kubernetes was 2014. Vercel was 2015. The infrastructure we now take for granted is younger than most of the people building on it.

Dynamic Software is at year one.

Whoever builds the runtime, the protocols, the developer tools, the platforms, defines the next era of software. The work ahead is enormous. It is also the most interesting work I've done in the past fifteen years.

Come build with us at Agno.

The Key To Building Agentic Software That Works

In the early 1940s, Bell Labs was building the national telephone network, the most complex technical system in the world at the time. Millions of switches, cables, relays, and operators had to work together. The engineers discovered something that would become an 80-year-old lesson: you can't optimize a system by optimizing individual components. The behavior of the whole (call routing, reliability, capacity, cost) emerged from how the parts interacted. They needed a discipline focused on the interactions between components.

They called it systems engineering.

Agentic Software Is a Systems Engineering Problem

Coding agents have lowered the barrier to writing code, but they haven't lowered the requirements of production software.

Software engineering is, and has always been, systems engineering and agentic software is no different. If you're building agentic software, your system needs to bridge five layers:

1. Agent Engineering. Your agent or multi-agent logic and execution flow. Model, system instructions, tool configurations, handoffs, context management, observability. This is where you define what your agent does, how it runs, and how it responds. Your agent's behavior should be deterministic where possible and observable where it isn't.

2. Data Engineering. Your agent is only as good as the context it has access to, and context is just data under the hood.

Call it memory, storage, knowledge. Your Agent's data should be managed with data engineering principles. Well designed schemas, structured querying, databases for fast read/writes, object storage for long-term storage, and workflows that keep your knowledge and memory up to date. The patterns are decades old. Use them.

3. Security Engineering. Auth, RBAC, governance, data isolation, audit trails. Your agent's capabilities are defined by its tools, and those tools should be scoped with JWT-backed permissions. Read-only access IS NOT a prompt instruction, it's a tool configuration.

Actions should have approval tiers: reads run freely, writes need user approval, sensitive operations need admin sign-off. Most actions should be logged and queryable for the life of the product.

And please, isolate requests. One user's context bleeding into another's is a data breach, not a "bug". It has serious consequences and there are laws protecting user data.

4. Interface Engineering. How users and other agents reach your agent.

REST API, Slack, MCP server, terminal, Chat UI. In the old world, you had one API and one client. Now you have multiple surfaces, each with its own identity system. A Slack user ID is not your product's user ID. An MCP client authenticating as another agent is not a human user. Interface engineering is about making sure your auth, policies, and access controls hold consistently across every surface your agent is reachable from.

5. Infrastructure Engineering. How you run and scale your software. Containers, cloud deployment, horizontal scaling. Generally called DevOps.

The good news: 95% of this is identical to running any other service. Re-use existing patterns, they'll serve you well. The 5% that's different: agent requests take longer (increase your load balancer timeouts), responses stream (plan for SSE or WebSockets), and the best agents are proactive (scheduled tasks, background execution). None of this is new.

The key unlock for AI engineers is realizing that agentic software is just regular software, with the business logic replaced by agents, and interfaces going from request/response to streaming across multiple surfaces.

Systems engineering is the discipline of making these parts work together, and is the key to building agentic software that works.

When you look at your software from a systems perspective, the right decisions become obvious. You give your agent well-scoped tools, not unfettered bash access. You store sessions, memory, and knowledge in a database, not in files, so you can utilize decades of multi-tenant patterns.

When you design one layer in isolation, you inherit constraints that cascade through the rest of the system. When you design from the system's perspective, each layer reinforces the others.

Systems Engineering in Practice

I can't make a claim like this and not give you working code.

Dash is an open-source, self-learning data agent. You ask it questions in plain English, it writes SQL, runs it, and tells you what the numbers mean. Simple enough to clone and adapt. Real enough to demonstrate all five layers. Here's how it looks (2x speedup)

Dash is live in many companies and works incredibly well. The difference is the system behind it. Here's how each layer works.

Agent Engineering

Dash is a team of three agents. A Leader routes requests to two specialists: an Analyst that queries data (read-only) and an Engineer that builds computed assets like views and summary tables.

Each specialist gets similar tools, but wired up for different purposes. The Analyst's SQL tools connect to a read-only database engine. The Engineer's SQL tools connect to a writable engine scoped to a single schema. Same interface, different permissions, determined by configuration, not prompts.

Instructions are assembled at runtime from table metadata and business rules stored as structured files.

Interface Engineering

One system, multiple surfaces.

Dash serves a REST API, a Slack bot, a web UI, and a CLI. Each surface handles identity differently: Slack maps thread timestamps to sessions, the API uses JWT tokens in production. But all four hit the same agents, same tools, same knowledge. Adding a new interface does not require rebuilding the agent logic.

Your auth and access controls need to hold across every surface, because the agent doesn't know which one it's being called from. Here's dash being used in slack.

Data Engineering

Six layers of context, and tools for learning.

Raw LLMs writing SQL hit a wall fast: schemas lack meaning, types are misleading, tribal knowledge is missing, there's no way to learn from mistakes. Dash solves this with six layers of grounded context:

1. Table metadata (schema, columns, relationships)
2. Human annotations (metrics, definitions, business rules)
3. Query patterns (SQL that is known to work)
4. Institutional knowledge (docs, wikis)
5. Learnings (error patterns and discovered fixes)
6. Runtime context (live schema inspection)

These layers feed two systems.

• The first is curated knowledge: table schemas, validated queries, and business rules loaded into PostgreSQL.
• The second is discovered learnings: error patterns and fixes that the agent saves automatically when it hits problems and recalls on future queries.

The learning loop is simple: the agent runs a query, gets a type error, diagnoses the fix, saves it. Next time it sees a similar column, it gets it right the first time. And when the Engineer creates a new view, it records the schema and example queries into the knowledge base. The Analyst discovers it on the next search and starts using it.

Query 100 is better than query 1, not because the model improved, but because the data layer got better.

Security Engineering

Enforced by the system, not the prompt.

Production auth uses RBAC with JWT verification. Every query is scoped to user_id. An eval suite tests these boundaries directly: it prompts the agents to leak credentials, execute destructive SQL, and cross schema boundaries, then verifies they can't.

Security is a system property tested across layers.

The Analyst's read-only access is a PostgreSQL connection parameter. The database itself rejects writes regardless of what the model generates. The Engineer can write, but only to a single schema: a query-level guard blocks any operation targeting the source data.

Infrastructure Engineering

Boring on purpose.

Standard Python container. Docker Compose for local development. One-command cloud deployment. Streaming via SSE through a standard ASGI server. The 95% that's identical to any other service is identical. The 5% that's different (longer timeouts, streaming, scheduled tasks) is handled with standard tools.

You can clone it, run docker compose up, and have the entire system running in minutes. One command, five layers, a working product.

# Clone the repo
git clone https://github.com/agno-agi/dash.git
 
cd dash
 
# Set your keys
cp example.env .env
# Edit .env and add your model provider key
 
# Start the system
docker compose up -d --build

TLDR

Agentic software is just software. The agent replaces business logic. Everything else is systems engineering. Five layers: agent, data, security, interface, infrastructure. Each layer affects the others. Design them together and the system compounds. Design them in isolation and you spend your time patching around constraints that shouldn't exist. We walk through all five with Dash, a real open-source data agent you can run yourself.

Links:

• Dash on Github
• Agno Docs
• Agno Github
• AgentOS Templates

What is the simplest architecture for running a multi-agent system at scale?

I want to deploy agents as a real service. Multi-user, RBAC, JWT-based auth. Sessions, memory, and knowledge backed by a database. Horizontally Scalable. Able to serve thousands of concurrent requests. The kind of product you'd actually ship to users.

Could the answer be: a FastAPI app and a Postgres database?

So I spent some time building one to find out. 14 agents, 11 multi-agent teams, 5 workflows. Hundreds of tools, approvals, evals, schedules. All running in a single FastAPI process against a single PostgreSQL database. It's open source: Demo AgentOS.

I'll walk through the architecture in this post. In the next one we'll dive into what breaks when you push it.

The Bar

"Scale" gets thrown around quite a bit. In this case, scale means breadth. The surface area of a real product. Every concern a CTO would actually need to address before shipping a product to users:

Multi-user and multi-tenancy. Every user gets their own sessions, memory, and context. The system isolates every resource an agent touches, across every user, on every request.

Note: Context bleeding is a data breach, not a bug.

Auth and RBAC. JWT verification, role-based access control, scoped permissions. This applies to the API layer, the agents, the tools they call, and the data they can access. Dev and production should have different security postures.

Real persistence. Sessions, memory, and knowledge stored in a database, with regular backups and data access policies. Everything needs to comply with user-data protection laws like GDPR and CCPA.

Serving requests at scale. The system should be able to handle thousands of concurrent requests. Streaming responses should be held open. Background work (memory extraction, summarization, learning) running alongside the primary model call. All of it competing for the same HTTP transports, connection pools, and database connections. The hard part is not serving one request. It is serving the thousandth one without stalling the ninth one.

Observability. Tracing every agent run, every tool call, every delegation in a multi-agent team. When something goes wrong at step 7 of a 12-step workflow, you need to see exactly what happened and why.

Governance. Layered authority over what agents can do. Some tools run freely. Some need user approval. Some need admin sign-off. Approval flows, audit trails, and the ability to pause execution mid-run.

Reliability and evals. Agents are testable software. You need smoke tests, tool call validation, LLM-judged accuracy, performance baselines. Without evals, every change is a guess.

If this is the bar, the question is: what's the simplest architecture that clears it?

The Architecture

One FastAPI process. One Postgres database. That's it.

The FastAPI app serves 14 agents, 11 multi-agent teams, 5 workflows using REST endpoints. Every request is a POST, every response is a server-sent event stream.

The database does more than you'd think. The Postgres database stores agent sessions, user memory, knowledge contents, learnings, schedules, and eval results. Pgvector handles embeddings for knowledge bases.

The Components

The 30+ components in the AgentOS showcase different agentic patterns.

Some showcase HITL patterns. The Helpdesk agent wraps three tools: one that requires operator confirmation before restarting a service, one that pauses for user input on ticket priority, one that executes outside the agent runtime. The Approvals agent uses Agno's @approval decorator for blocking approval gates and audit-trailed operations. Both agents pause execution mid-run and resume on approval.

Some showcase guardrails. The Helpdesk agent has three pre-hooks: OpenAI moderation, PII detection, prompt injection detection. It also has a post-hook that scans responses for secret patterns (API keys, connection strings, SSNs) and rewrites them before they leave the process. An audit log hook records every run for compliance.

Some showcase multi-agent teams. Pal is a personal knowledge agent with five specialists. Dash is a data analyst with an Analyst/Engineer split. Coda is a coding agent with five specialists including a Planner and a Triager. The Research and Investment teams each ship in four modes (coordinate, route, broadcast, tasks) so you can see how the same set of members produces different behavior under different coordination patterns.

Some showcase step-based workflows. Morning Brief gathers calendar, email, and news in parallel and synthesizes a briefing. AI Research runs four parallel researchers and synthesizes their findings. Content Pipeline does parallel research plus a loop that iterates until an editor approves. Support Triage classifies a ticket, routes it to a specialist, and escalates if severity is high.

Some showcase state management. Taskboard demonstrates session state with agentic state updates. Injector demonstrates dependency injection through RunContext. Compressor demonstrates tool result compression with a cheaper model.

Some showcase scheduling. Morning Brief runs every weekday at 8am ET. AI Research runs every day at 7am UTC. The Scheduler agent lets users create, list, disable, and delete schedules at runtime through natural language.

The point is not that you need all of these. The point is that a single FastAPI process can host them without the architecture getting complicated.

Governance as First-Class Infrastructure

Three layers of governance sit on top of every agent.

Pre-hooks run before the model sees the input. Moderation, PII detection, injection detection. If any hook raises, the request is rejected before a single token is generated.

Approval gates pause the run mid-execution. A tool decorated with requires_confirmation=True or @approval streams a RunPaused event to the client with the tool name and arguments. The client shows the user an approve/reject UI. On approval, the run resumes from where it paused. This works because the session state is durable (stored in db).

Post-hooks run on the output. The Helpdesk agent has an output guardrail that scans responses for secret patterns and rewrites them before they leave. Every run is audit-logged through a separate hook.

What's Not Here

No message queue. No worker pool. No separate vector database. No Redis. No microservices. No orchestrator service standing in front of the agents. No separate auth service.

Could you add them? Sure. Are they necessary to clear the bar I defined? Not yet. The point of this exercise is to find out where the simple architecture breaks, so the next decision (what to add) is grounded in actual load, not in speculation.

What's Next

Part 2 is what breaks when you scale this.

I'm going to load test it. Thousands of concurrent requests. Streaming responses held open. Background memory extraction competing with primary runs. Connection pools under pressure. I expect to find a few obvious bottlenecks and a couple of surprising ones.

Links:

• Demo AgentOS on GitHub
• Agno GitHub
• Agno Docs

Most AI-forward companies have in-house data agents:

• OpenAI: Inside OpenAI's in-house data agent
• Vercel: d0, another post
• Uber: QueryGPT (creative name)
• LinkedIn: SQLBot (absolutely LinkedIn-coded name for the agent)
• Salesforce: Horizon Agent
• DoorDash: How to use every buzzword in a blog post

This post will show you how to build a best-in-class data system and make it available to your team over Slack. If you do this well, Dash should handle roughly 80% of routine data questions, send daily reports, and catch metric anomalies before anyone asks.

What is Dash?

Dash is a self-learning data system made of 3 agents: Dash (the team leader), a Data Analyst and a Data Engineer.

It uses a dual-tier knowledge and learning system to deliver an incredible work-with-your-data experience.

You can chat with it via Slack or the AgentOS UI.

It writes SQL, runs it, and tells you what the numbers mean. More importantly, when it makes a mistake or gets corrected, it learns from it. When your team keeps asking the same question, it builds infrastructure so the answer is faster next time.

A self-learning data system, not a data agent.

Dash uses its own PostgreSQL database. You don't point it at your production database. You progressively load the tables you want it to work with, along with the context it needs to be useful. This is the part most people skip. This is the part that makes it special.

Here's how it looks in Slack (8x speedup when waiting):

And on the AgentOS UI:

Using the AgentOS UI, you can chat with your agents, view sessions, traces, metrics, and schedules.

AgentOS is the agent platform you didn't know you needed.

How It Works

1. Context is everything

Most data agents get a schema dump and the impossible task of writing SQL from business logic that only lives in the data engineer's head. That's why they're bad. Column names and types tell you nothing about the data. They don't tell you that ended_at IS NULL means a subscription is active. That annual billing gets a 10% discount. That usage metrics are sampled 3-5 days per month, so summing them gives you garbage.

I wrote about this problem in detail in my Self-Improving Text-to-SQL Agent post. The core insight holds: the biggest improvement you can make to your data agent is giving it the same tribal knowledge that human engineers have.

Dash uses a carefully curated knowledge system backed by PgVector. It contains:

Table metadata. Table schema, column types, what they mean, what to use each table for, the gotchas. Every table ships with use cases and data quality notes. Example: status is 'active', 'churned', or 'trial'; always check against subscriptions for ground truth.

Validated queries (must have). Battle-tested SQL with the right JOINs, the right NULL handling, the right edge cases. When the Analyst gets your question, it searches knowledge first. Before it writes a line of SQL, it already knows the shape of the data and which traps to avoid.

Business rules. How MRR is calculated, what NRR means, that a customer can have multiple subscription records because upgrades close the old row and open a new one. This is the context that separates a correct answer from a plausible-looking wrong one.

This knowledge is curated by the user. What makes Dash special is its ability to learn on its own.

2. Self-learning loop

Separate from knowledge, Dash captures what it learns automatically (via tool calls). When the Analyst hits a type error and fixes it, the fix gets saved. When a user corrects a result, that correction is recorded. When the system discovers a data quirk, it notes it.

Next time anyone asks a similar question, the Analyst checks learnings before writing SQL. Dash gets better the more it's used.

I've been developing this pattern since December 2025, first as GPU Poor Continuous Learning and then refined through Dash v1. The approach is simple: the model stays frozen. The system gets smarter. Learning happens in retrieval, not in weights. It's auditable, reversible, and requires zero training compute.

3. Three agents, two schemas

Dash is three agents. Leader routes requests and synthesizes answers. Analyst writes and runs SQL. Engineer builds views, summary tables, and computed data. They work together, sharing knowledge and learnings.

The Leader has no SQL tools. It cannot touch the database.

The Analyst is read-only. Not "read-only because the prompt says so." Read-only because the PostgreSQL connection is configured with default_transaction_read_only=on. The database itself rejects writes. No prompt injection or clever jailbreak changes this. The database says no.

The Engineer can write, but only to the dash schema. A SQLAlchemy event listener intercepts every SQL statement before execution and blocks anything targeting the public schema. Your company data is untouchable.

This gives you two schemas with a hard boundary:

• public schema: your company data. You load it. Agents read it.
• dash schema: views, summary tables, computed data. The Engineer owns and maintains it.

There's also an ai schema where Dash stores its sessions, learnings, knowledge vectors, and other operational data. It powers the AgentOS UI and the self-improvement loop.

I covered the security model in depth in my Systems Engineering post. The key principle: security is a system property enforced by configuration, tested across layers.

The part nobody else has

When the Leader notices your team keeps asking the same expensive question (MRR by plan, churn by segment, revenue waterfall) it asks the Engineer to build a view.

The Engineer creates dash.monthly_mrr_by_plan. A SQL view joining the right tables, handling all edge cases, producing a clean result. Then it does the critical thing: it calls update_knowledge to record the view in the knowledge base. What it contains, what columns it has, example queries.

Next time someone asks about MRR by plan, the Analyst searches knowledge, finds the view, and queries it directly. No complex join. No risk of getting NULL handling wrong. Faster. Pre-validated. Consistent.

The agents build on each other's work. The Engineer creates infrastructure. The Analyst discovers and uses it. The Leader notices patterns and triggers the cycle. Over time, the dash schema fills with views and summary tables that nobody manually created. An analytics layer the system built for itself, shaped by what your team actually asks about.

The full loop

1. You ask a question. Leader delegates.
2. The Analyst searches knowledge, writes correct SQL, returns an insight.
3. Good queries get saved to knowledge. Errors become learnings.
4. Repeated patterns become views. Views get recorded to knowledge.
5. Next time, the Analyst uses the view. Faster, pre-validated, consistent.

Dash accumulates institutional knowledge about your data and compounds with use.

Build Your Own

Dash is free and open-source. Check out the GitHub repo and follow the README for in-depth instructions.

Quick Start

git clone https://github.com/agno-agi/dash && cd dash
cp example.env .env  # Add OPENAI_API_KEY
 
docker compose up -d --build
 
docker exec -it dash-api python scripts/generate_data.py
docker exec -it dash-api python scripts/load_knowledge.py

This starts Dash with a synthetic dataset (~900 customers, 6 tables) and loads the knowledge base (table metadata, validated queries, business rules). You can demo the entire system without connecting any real data.

Connect to the Web UI

1. Open os.agno.com
2. Add OS → Local → http://localhost:8000
3. Connect

Connect to Slack

Dash lives in Slack. You can DM it or mention it in a channel with @Dash. Each thread maps to one session, so every conversation gets its own context.

1. Run Dash and give it a public URL (use ngrok for local, or your deployed domain).
2. Follow instructions in docs/SLACK_CONNECT to create and install the Slack app from the manifest.
3. Set SLACK_TOKEN and SLACK_SIGNING_SECRET, then restart Dash.

Adding Your Own Data

Once you have Dash running, making it your own is straightforward. Replace the sample dataset with your data and give Dash the context it needs.

1. Load your tables into the public schema

Use whatever pipeline you already have. pg_dump, a Python script, dbt, Airbyte. Dash reads from public and never writes to it. You can use your existing workflow orchestration tools (Airflow, Dagster), or use Dash's built-in scheduler.

2. Add table knowledge

For each table, create a JSON file in knowledge/tables/:

{
  "table_name": "customers",
  "table_description": "B2B SaaS customer accounts with company info and lifecycle status",
  "use_cases": ["Churn analysis", "Cohort segmentation", "Acquisition reporting"],
  "data_quality_notes": [
    "signup_date is DATE (not TIMESTAMP) — no time component",
    "status values: active, churned, trial",
    "company_size is self-reported"
  ],
  "table_columns": [
    {"name": "id", "type": "SERIAL", "description": "Primary key"},
    {"name": "company_name", "type": "TEXT", "description": "Company name"},
    {"name": "status", "type": "TEXT", "description": "Current status: active, churned, trial"}
  ]
}

This is the single highest-leverage thing you can do. The better your knowledge, the better Dash performs.

3. Add validated queries

For your most common questions, write the SQL that gives the correct answer and save it in knowledge/queries/:

-- <query current_mrr>
-- <description>Current total MRR from active subscriptions</description>
-- <query>
SELECT
    SUM(mrr) AS total_mrr,
    COUNT(*) AS active_subscriptions
FROM subscriptions
WHERE status = 'active';
-- </query>

This is the easiest way to make sure Dash uses your internal semantics for answering routine questions. Your job is to deliver the best work-with-your-data experience for your team. This makes it possible.

4. Add business rules

Document your metrics, definitions, and gotchas in knowledge/business/:

{
  "metrics": [
    {
      "name": "MRR",
      "definition": "Sum of active subscriptions excluding trials",
      "calculation": "SUM(mrr) FROM subscriptions WHERE status = 'active'"
    }
  ],
  "common_gotchas": [
    {
      "issue": "Active subscription detection",
      "solution": "Filter on ended_at IS NULL, not status column"
    }
  ]
}

Helpful context for Dash. You can skip if it's too much work up front.

5. Load knowledge

python scripts/load_knowledge.py             # Upsert changes
python scripts/load_knowledge.py --recreate  # Fresh start

Scheduled Tasks

Dash ships with a built-in scheduler. You can schedule any type of task that your container can handle.

Out of the box, Dash comes with a pre-built schedule that re-indexes your knowledge base every night at 4am UTC:

mgr.create(
    name="knowledge-refresh",
    cron="0 4 * * *",
    endpoint="/knowledge/reload",
    payload={},
    timezone="UTC",
    description="Daily knowledge file re-index",
)

Same pattern for anything else: daily metric summaries posted to Slack, anomaly detection runs, weekly email digests, automated data quality checks. Register a schedule, point it at an endpoint, Dash handles the rest.

The best agents are proactive. Scheduled tasks are the first step in that direction.

Run Evals

Dash ships with five eval categories:

• Accuracy: correct data and meaningful insights
• Routing: team routes to the correct agent
• Security: no credential or secret leaks
• Governance: refuses destructive SQL operations
• Boundaries: schema access boundaries respected

python -m evals                      # Run all
python -m evals --category accuracy  # Run one category
python -m evals --verbose            # Show response details

Deploy to Production

You can deploy Dash to Railway with one command:

cp example.env .env.production
# Edit .env.production — set OPENAI_API_KEY
 
railway login
./scripts/railway_up.sh

Railway is fine for getting started. Eventually you'd want it wherever your existing data infrastructure lives. Everything is containerized so deployment should be straightforward. Be mindful of egress costs.

Production requires a JWT_VERIFICATION_KEY from os.agno.com for RBAC. It would be insane to expose Dash on a public endpoint.

What's Next

Dash is built with systems engineering principles. Five layers: agent, data, security, interface, infrastructure. Each layer affects the others. Design them together and the system compounds.

If there's interest, I'll do deep dives on each layer:

• Agent Engineering: The business logic. Model, instructions, tools, knowledge, and the self-learning loop.
• Data Engineering: The context layer. Memory, knowledge, learnings, storage. Why the data layer is the most underinvested part of the stack.
• Security Engineering: Auth, RBAC, governance, data isolation, and audit trails designed into the system as core primitives.
• Interface Engineering: Turning an agent into a product. REST APIs, web UIs, Slack, MCP, and how one agent serves multiple surfaces.
• Infrastructure Engineering: How to deploy and scale Dash. Containers, deployment, scheduling.

TLDR

Every company with 30+ people should have an internal data agent. Dash is a free, open-source, self-learning data system made of 3 agents. It uses curated knowledge and continuous learning to get better with every query. Three agents (Leader, Analyst, Engineer) share knowledge and build on each other's work. Security is enforced by the system: read-only connections, schema-level isolation, eval-tested boundaries. Runs in your cloud, lives in Slack. Clone it, run docker compose up, and have the entire system running in minutes.

• GitHub: agno-agi/dash
• OpenAI's data agent
• Systems Engineering
• GPU Poor Continuous Learning
• Dash v1

Built with Agno.

The mainframe era had COBOL and Fortran. The systems era had C. The web era had JavaScript and Python. Each emerged for the same reason, the previous generation could no longer express the new abstraction.

We are now in the agentic era.

Software is no longer just executing predefined instructions. It is reasoning over context, calling tools, retrieving knowledge, learning from past runs, and making decisions at runtime.

When the contract of software changes, the language must change too.

What makes a programming language?

A programming language is made of three things:

1. Primitives to think and build with.
2. An engine to execute those primitives.
3. A runtime that governs memory, I/O, permissions, and interaction with the outside world.

An SDK alone is not a programming language. A collection of utilities is not a programming language. Without an execution engine and a runtime that enforces behavior, you have a library, not a language.

Python gives you lists, functions, and classes. Its interpreter runs them. Its runtime manages memory, exceptions, and interfaces with the operating system.

React gives you components and state. Its reconciler computes updates. The browser handles rendering and events.

Applying this to agentic systems:

• Agno gives you agents, teams, workflows, memory, knowledge, tools, guardrails, and approval flows.
• The Engine runs them: model calls, tool execution, context construction, and iteration.
• AgentOS, the production runtime, governs execution and interfaces with the outside world via an API: streaming, request-level isolation, authentication, RBAC, monitoring, background execution.

The runtime is stateless. Sessions, memory, state and traces persist in your database. Permissions are enforced at request boundaries.

Agno provides the SDK + Engine + Runtime for agentic software.

Agents are the new programs

Traditional applications are collections of deterministic programs. Every path is written in advance. The system does exactly what the developer specified.

Agents change that.

An agent reasons over context. It chooses tools dynamically. It retrieves knowledge. It remembers previous runs. It decides which path to take at runtime.

This is still software, but the path between input and output is no longer fixed.

This does not mean deterministic systems disappear. For many workloads, static pipelines are faster, cheaper, and more reliable.

But when the system must pause, reason, retrieve, and adapt dynamically, predefined control flow breaks down.

For decades, the contract was simple:

Same input, same output.

Agentic software breaks that contract.

The same input can produce different outputs depending on memory, context, retrieval, and prior state. If execution is dynamic, the language must express that natively.

Agentic software needs a new contract

Agentic software requires new capabilities built into its programming language:

1. A new interaction model

Static software receives a request and returns a response.

Agentic software streams reasoning, tool calls, intermediate results, and pivots in real time. The execution path can change mid run, or pause for days. The system may retrieve knowledge halfway through and completely redirect its reasoning.

Streaming and iteration are the default and the language for agentic software must treat them as first class behavior.

2. A new governance model

Traditional systems execute predefined decisions within rules written in advance. Code does not decide whether to send an email or issue a refund. It simply follows instructions.

Agents make decisions, and not all decisions are equal.

Some actions are low risk: summarizing text or searching documentation. Some require user approval: sending emails or booking travel. Some require admin approval: issuing refunds, deleting records, changing permissions.

Without runtime-enforced approval boundaries, an agent that can draft an email can also execute a payment. The difference must be enforced by the runtime, not prompt engineering.

Governance must be part of the agent definition itself and the runtime must enforce it.

3. A new trust model

Static systems are trusted because every path is written in advance.

Agents introduce probabilistic reasoning into the execution path.

If guardrails and evaluation run outside the runtime, they are advisory rather than enforceable. Unsafe output can be produced before policy checks intervene.

Trust must therefore be part of the runtime semantics: guardrails, evaluation, logging, pre and post-response checks integrated into execution.

Interaction. Governance. Trust.

These are language-level concerns in the agentic era.

What this looks like in practice

Here is a lightweight coding agent that writes, reviews, and iterates on code. It remembers project conventions, retrieves knowledge, learns from past runs, and operates within explicit governance boundaries.

This example is intentionally minimal but production-capable. It has persistence, memory, learning, and controlled tool execution.

from agno.agent import Agent
from agno.db.sqlite import SqliteDb
from agno.learn import LearnedKnowledgeConfig, LearningMachine, LearningMode
from agno.models.openai import OpenAIResponses
from agno.tools.coding import CodingTools
from agno.tools.reasoning import ReasoningTools
 
gcode = Agent(
    name="Gcode",
    model=OpenAIResponses(id="gpt-5.2"),
    db=SqliteDb(db_file="agno.db"),
    instructions=instructions,
 
    # Knowledge: searchable long-term memory
    knowledge=gcode_knowledge,
    search_knowledge=True,
 
    # Learning: extract and store learnings over time
    learning=LearningMachine(
        knowledge=gcode_learnings,
        learned_knowledge=LearnedKnowledgeConfig(mode=LearningMode.AGENTIC),
    ),
 
    # Tools: controlled extensions
    tools=[CodingTools(base_dir=workspace, all=True), ReasoningTools()],
 
    # Memory: learn user preferences
    enable_agentic_memory=True,
 
    # Context: include prior runs
    add_history_to_context=True,
    num_history_runs=10,
    markdown=True,
)

Notice what is being defined:

• Knowledge as a first class primitive
• Learning as a built in capability
• Tools as controlled extensions
• Memory and historical context as defaults
• A runtime that governs how the system executes

These are not utilities or third party integrations. They are the vocabulary of the agent and enforced by the runtime and execution layer.

That is what a programming language does. It gives you the right primitives for the era you are building in. You define the behavior. The language enforces it.

Every era gets the language it needs

COBOL abstracted business logic away from assembly. C abstracted system engineering without hiding it. Python abstracted memory management and low level primitives to accelerate iteration.

Each language captured the dominant abstraction of its era.

The agentic era introduces a new abstraction: systems that reason, remember, and decide at runtime.

The contract has changed. The primitives have changed. The execution model has changed.

The language must change too. That language is Agno.

There are many that argue that because Agno is written in Python, it cannot be a programming language.

If you wish to make an apple pie from scratch, you must first invent the universe. — Carl Sagan

This is a production-grade system that includes:

You'll also learn how to extend it with your own agents.

5 minute read. Running locally in 5. Deployed to production in 20.

The Agents

We'll build three agents, each demonstrating a different pattern:

• Pal - AI-powered second brain. Captures notes, bookmarks, people, meetings. Researches the web. Learns over time.
• Knowledge Agent - Answers questions from a knowledge base.
• MCP Agent - Connects to external services via MCP.

Each agent can be extended to fit your needs.

Run Locally (5 minutes)

Prerequisites

• Install Docker Desktop
• Get an OpenAI API key

Setup

Clone the repo and export your OpenAI API key:

git clone \
    https://github.com/agno-agi/agentos-railway-template.git \
    agentos-railway
 
cd agentos-railway
 
export OPENAI_API_KEY="sk-***"

Start the application (API + Database):

docker compose up -d --build

That's it. Your system is running. Here's how it looks:

Connect to the UI

1. Open os.agno.com
2. Click Add OS → Local
3. Enter http://localhost:8000 as the URL

Now chat with Pal:

> Note: decided to use Postgres for the new project - better JSON support
 
> Research event sourcing patterns and save the key findings
 
> What do I know about event sourcing?

Deploy to Production (10 minutes)

I've made it easy to deploy to Railway - just login and run a script.

Prerequisites

• Install the Railway CLI

Deploy

Login to Railway and run the deploy script:

railway login
 
./scripts/railway_up.sh

The script provisions PostgreSQL, configures environment variables, and deploys your system. Give it a few minutes for the services to spin up.

Connect to the UI

1. Open os.agno.com
2. Click Add OS → Live
3. Enter your Railway domain

You now have a production multi-agent system. Watch it go live in ~2 mins:

What's Included

Pal (Personal Agent that Learns)

Your AI-powered second brain. Captures notes, bookmarks, people, meetings. Researches the web and saves findings. Learns from errors so it doesn't repeat them.

I wrote more about Pal here: Building Pal: Personal Agent that Learns

Knowledge Agent (Agentic RAG)

Store any type of docs in a vector store, chat with it using Agentic RAG.

knowledge_agent = Agent(
    model=OpenAIResponses(id="gpt-5.2"),
    knowledge=knowledge,
    search_knowledge=True,
)

MCP Agent (MCP Tools)

Connects to external tools via the Model Context Protocol. Point it at any MCP server and it gets access to those tools.

mcp_agent = Agent(
    model=OpenAIResponses(id="gpt-5.2"),
    tools=[MCPTools(url="https://docs.agno.com/mcp")],
)

Create Your Own Agent

Now let's add a custom agent to the system. We'll build a research agent that uses the Exa MCP server.

Create agents/research_agent.py:

from agno.agent import Agent
from agno.models.openai import OpenAIResponses
from agno.tools.mcp import MCPTools
 
from db import get_postgres_db
 
# Exa MCP for research
EXA_MCP_URL = (
    f"https://mcp.exa.ai/mcp?tools="
    "web_search_exa,company_research_exa,people_search_exa"
)
 
research_agent = Agent(
    id="research-agent",
    name="Research Agent",
    model=OpenAIResponses(id="gpt-5.2"),
    db=get_postgres_db(),
    tools=[MCPTools(url=EXA_MCP_URL)],
    instructions="""\
You are a research agent. You help users find information about:
- Companies and startups
- People and their backgrounds
- Topics and trends
 
Be thorough but concise. Cite your sources.
""",
)

Register it in app/main.py:

from agents.research_agent import research_agent
 
agent_os = AgentOS(
    agents=[pal, knowledge_agent, mcp_agent, research_agent],
)

Your agent is now part of the system. Chat with it:

If the agent doesn't show up, press refresh on the UI (top right corner) or restart containers with docker compose restart.

Wrapping Up

You now have a live multi-agent system with:

What's Next

• Build more agents - Add specialized agents for your use case
• Add tools - Extend your agents with 100+ toolkits
• Go multi-agent - Create multi-agent teams and workflows
• Go multi-channel - Expose your agents via Slack, Discord, WhatsApp
• Build an AI product - From 2-person startups to Fortune 500 companies, AgentOS is the foundation for agentic products

The system is yours. You have a head start - make it count.

Learn More

• GitHub repo
• Agno documentation

Built with Agno. Give it a ⭐️

Two years. Billions in VC funding. Thousands of blog posts about "production-ready agents." An entire industry built around evaluation frameworks, observability platforms, and benchmarks.

The result?

• 11% of organizations have agents in production. [Deloitte]

• 40%+ of agentic AI projects will be cancelled by 2027. [Gartner]

• 80%+ never reach meaningful production. [RAND]

If evals were the answer, these numbers would be different.

Here's what I've learned after two years of shipping agents: passing evals ≠ working product. You can have a green test suite and a broken product. You can hit 95% on your benchmark and watch your agent choke the moment a real user touches it.

Evals don't get you to production. A working product does.

The Pitch vs. The Reality

Here's what the eval-industrial complex told us:

"Evals are the key to production-ready agents" — Databricks

Here's what actually happens:

You build an agent in a Python script. It works. You run your eval suite. Green lights everywhere. You demo it to stakeholders. They love it. Then you try to ship it.

Everything falls apart.

What Evals Don't Test

Your eval suite said the agent was ready. Here's what it missed:

Your agent isn't a function — it's a process. A single response might take 30 seconds. Or 3 minutes. Or 10 minutes if it's doing research. Traditional servers handle stateless request-response cycles in milliseconds. Your agent thinks, waits, calls tools, thinks again. Try fitting that into a Lambda with a 15-second timeout.

State breaks at scale. Works great with 1 user on 1 container. Add more users? State bleeds across sessions. Add more containers? State disappears entirely. Store it in memory? Gone when the process dies. Store it in a database? Now you're building infrastructure you didn't plan for.

Streaming is harder than it looks. In your notebook, responses just appeared. In production, users stare at a blank screen for 8 seconds wondering if the app crashed. You try SSE. Then WebSockets. Then you realize you need durable streams that survive network hiccups, handle backpressure, and resume gracefully after disconnects.

The real world doesn't mock. Your agent calls an external API. In testing, mocks returned clean data every time. In production, the API times out. Returns malformed JSON. Hits rate limits. Requires re-authentication mid-session. Your agent chokes. Your eval suite never saw it coming.

Agents fail because of an inadequate runtime, not intelligence. Evals don't measure any of it.

We've been obsessing over the brain while ignoring the nervous system.

The Trap: Evals Too Early

Here's the thing that really kills projects: writing evals before you have a working product.

Every hour spent writing evals is an hour not spent learning what your product actually needs. You're locking yourself into test cases for a system that doesn't exist yet.

The agent you're building now? It's not the one that's going to ship. It's going to be the second iteration. Or the fifth. The eval suite you wrote for version one is useless for version three. Worse than useless — it's weight you're dragging around.

The eval-industrial complex sold you on this idea that evals-first is disciplined. It's not.

The right sequence:

1. Build something that runs
2. Get it in front of real users (internal users are fine)
3. Learn what breaks, what matters, what "good" actually looks like
4. Then write evals to lock in that understanding

You can't evaluate what you can't run.

What Evals Are Actually Good For

I'm not saying evals are useless. They're critical — for model providers shipping foundation models. If you're training GPT-5, you need benchmarks. Even for AI engineers building products on top of those models, evals help with:

• Catching regressions after you change something
• Comparing model versions
• Compliance checkboxes

That's it. They won't help you ship. They won't help you scale. They won't help you handle the thousand edge cases that only appear in production.

What Actually Gets You to Production

The market says: Evals → Observability → Production.

This is backwards. Here's what actually works:

Runtime → Production → (Evals + Observability)

The foundation comes first. Everything else is a support layer.

The foundation:

• A runtime that handles the weird stuff. Concurrent users. Failure recovery. Long-running stateful processes that survive container restarts. Your agent isn't a microservice — stop treating it like one.

• State management that doesn't disappear. Sessions that survive crashes. Context that carries across conversations. Memory that doesn't evaporate when Kubernetes decides to reschedule your pod.

• Storage that lives with the agent. The agent's data — sessions, memory, knowledge — stored where the agent runs. In your cloud. Under your control. Send it to a third-party service and you've lost control of your product's brain.

• Infrastructure you own. Your environment. Your data. Your competitive advantage.

The support layer (after you're running):

• Observability for real production behavior — not synthetic test traces.
• Evals to catch regressions — run them in CI, keep them lean.
• Tracing to debug when things go wrong.

The support layer matters. But without the foundation, you're just testing in a notebook.

The Questions That Actually Matter

You have a working agent in a Python script. Great. Now answer these:

• Where will it run?
• Can it handle 100 concurrent users? 1,000?
• What happens when a container crashes mid-conversation?
• Is streaming smooth or do users watch a loading spinner for 10 seconds?
• Where does the agent's memory live? Who owns it?
• How do you deploy updates without breaking active sessions?

Evals don't answer any of these questions. The runtime does.

The Path Forward

I built Agno because I got tired of watching good agents die in the gap between "works in a notebook" and "runs in production."

Agno is a runtime for agents. It handles the stuff evals can't test:

• Concurrent execution — thousands of users, isolated state
• Persistent storage — sessions survive crashes, memory persists across conversations
• Streaming that works — SSE out of the box, handles disconnects gracefully
• Your infrastructure — runs in your cloud, data never leaves your environment

The eval-industrial complex had their shot. Two years. Billions in funding. The production numbers haven't moved.

Maybe it's time to focus on actually shipping.

Want to build with Agno?

• GitHub: agno.link/gh
• Documentation: agno.link/docs
• AgentOS: os.agno.com

First, Let's Recap

After reading hundreds of papers on agentic memory and trying out every possible tool, I came to the simple conclusion that maybe we're looking at memory wrong.

Memory is just... learning. Learning about the user, the task at hand, learning insights and patterns, learning from decisions - good and bad, the feedback received. Learning from every interaction. Everything else is integration (how the agent uses these learnings) and curation (decay, pruning, deduplication).

So I built Learning Machines: A system that helps agents continuously learn from every interaction.

I started working on it dec 31, and got a basic working version yesterday. Here's the PR for those interested: learning-machine-v0

Now let's dig into the technical details.

The Learning Protocol

The key behind it all is the Learning Protocol. It's a simple interface for building Learning Stores -- user profiles, session context, learned knowledge, entity memory, etc.

Let's take a look at the protocol:

@runtime_checkable
class LearningStore(Protocol):
    """Protocol that all learning stores must implement."""
 
    @property
    def learning_type(self) -> str:
        """Unique identifier (e.g., 'user_profile')."""
        ...
 
    def recall(self, **context) -> Optional[Any]:
        """Retrieve learnings from storage."""
        ...
 
    def process(self, messages: List[Any], **context) -> None:
        """Extract and save learnings from conversation."""
        ...
 
    def build_context(self, data: Any) -> str:
        """Build context string for agent's system prompt."""
        ...
 
    def get_tools(self, **context) -> List[Callable]:
        """Get tools to expose to agent."""
        ...

Five functions. Everything else is optional.

Why this matters: You can build your own learning store in ~50 lines. Most memory systems are thousands of lines of config. This is ~50. Legal docs. Medical records. Codebases. Sales pipelines. Whatever your domain needs.

You can even build personalized LearningStores for your writing styles, for your daily to-do's, for your emails, for your shopping lists. The real value of this approach is its extensibility.

The Learning Machine

The protocol lets you build stores. But stores need to plug into the agent somehow. That's what LearningMachine does.

User Message ──────► Recall from Stores ◄────────┐
                            │                    │
                            ▼                    │
                      Build Context              │
                            │                    │
                            ▼                    │ LearningMachine
                Agent Responds (with tools)      │
                            │                    │
                            ▼                    │
                   Extract & Process             │
                            │                    │
                            ▼                    │
              Update Stores (agent learns) ──────┴──► Periodic Curation

Recall → Build context → Run agent → Extract → Store. That's the loop.

Developer Experience

Three levels of complexity:

Dead Simple

agent = Agent(
    model=model,
    db=db,
    learning=True,  # Enables user_profile in BACKGROUND mode
)

Pick What You Want

agent = Agent(
    model=model,
    db=db,
    learning=LearningMachine(
        user_profile=True,
        session_context=True,
        learned_knowledge=True,
        entity_memory=True,
    ),
)

Full Control

agent = Agent(
    model=model,
    db=db,
    learning=LearningMachine(
        user_profile=UserProfileConfig(
            mode=LearningMode.AGENTIC,
        ),
        session_context=SessionContextConfig(
            enable_planning=True,
        ),
        learned_knowledge=LearnedKnowledgeConfig(
            mode=LearningMode.PROPOSE,
            namespace="engineering",
        ),
        entity_memory=EntityMemoryConfig(
            mode=LearningMode.BACKGROUND,
        ),
    ),
)

Build Your Own Learning Store

This is the win. Implement the protocol, plug it in:

@dataclass
class ProjectContextStore:
    """Custom store for project-specific context."""
 
    @property
    def learning_type(self) -> str:
        return "project_context"
 
    def recall(self, project_id: str, **kwargs) -> Optional[ProjectContext]:
        # Retrieve from your storage
        ...
 
    def process(self, messages: List[Any], project_id: str, **kwargs) -> None:
        # Extract and save
        ...
 
    def build_context(self, data: Any) -> str:
        if not data:
            return ""
        return f"<project_context>\n{data.summary}\n</project_context>"
 
    def get_tools(self, **kwargs) -> List[Callable]:
        return []  # Or return tools for agentic mode
 
# Plug it in
learning = LearningMachine(
    custom_stores={
        "project": ProjectContextStore(),
    },
)

~50 lines. 5 functions. Your domain, your rules. Build a Learning Store for legal docs, medical records, codebases, sales pipelines. This is the whole point behind the Learning Machine.

Built-in Stores

Phase 1 includes four stores:

Key Design Decisions

Learning Modes

Different use cases need different extraction modes.

class LearningMode(Enum):
    BACKGROUND = "background"   # Automatic extraction after each conversation
    AGENTIC = "agentic"         # Agent decides via tools
    PROPOSE = "propose"         # Agent proposes, user confirms
    HITL = "hitl"               # Human-in-the-loop approval (future)

BACKGROUND is invisible. The user never sees extraction happening. This is what makes Claude's memory feel natural.

AGENTIC gives control. The agent decides what's worth remembering. You can see the tool calls. Less noise, more transparency.

PROPOSE is for medium-stakes learning. Agent suggests, human approves. Good for shared knowledge bases where bad data spreads.

HITL is for the highest-stakes learning. Explicit human approval required.

Namespace Scoping

Some learnings should be private. Some should be shared. Namespaces enable this.

# Private to this user
LearnedKnowledgeConfig(namespace="user")
 
# Shared within engineering team
LearnedKnowledgeConfig(namespace="engineering")
 
# Shared with everyone
LearnedKnowledgeConfig(namespace="global")

This is what enables cross-user learning. This is what made yesterday's experiment work — Alice's insight helped Bob because they shared a namespace.

Entity Memory: Three-Tier Memory System

Entities (people, companies, projects) hold different types of information:

• Facts: Semantic knowledge ("Uses PostgreSQL", "Based in London")
• Events: Episodic memories ("Launched v2 on Jan 15", "Raised Series A")
• Relationships: Graph connections ("Bob is CEO of Acme", "Acme acquired StartupX")

Flat list doesn't work. You need to query "what do we know about Acme" differently than "what happened with Acme."

What's Next

Phase 3 is the endgame. Agents that analyze their own failures and propose: "I should stop doing X." Human approves. Agent evolves. No retraining. No fine-tuning. Just learning.

Want to dig in? Here's the PR: learning-machine-v0

Memory was step one. Learning is what comes next.

If you enjoyed reading this, checkout Agno on GitHub.

Questions or feedback? Reach out on X.

After reading hundreds (maybe thousands) of opinions, posts, and papers on agentic memory, I've come to three conclusions.

1. No one has it figured out.

Claude has the most impressive memory system I've seen. It feels natural. It never shouts. It knows what to reveal and when.

But we haven't figured out how to give developers the same capability for their own agents. The tools we have are... not there.

2. Maybe we're looking at it wrong.

Maybe memory is the wrong framing. What agents are really doing is learning. Learning about the user, the task at hand, learning insights and patterns, learning from decisions - good and bad, the feedback received. Learning from every interaction.

Everyone's rushing to build memory extraction systems — pull out facts, store them in a vector (or graph 🙄) database, retrieve them using complex mechanisms. But that's only half the problem.

But the hard part is integration: When does the learning happen? Before the response? After? In parallel? Is it automatic or does the agent control it? And critically — how do you teach the agent to use that information properly? Integration is what makes the system work.

You can't just tell an agent "you know XYZ about the user". You need to teach it how to use that knowledge. How to learn from it. How to prioritize it. How to act like a partner, a colleague, a companion who genuinely knows you — not a machine reciting facts from a database.

3. User memory is only part of the story.

User profiles and conversation summaries are just two types of learnings. But what about patterns and insights that worked? The entities involved - companies, people, projects? The decisions made and why? The feedback received? How should the agent use all these learnings to improve itself?

These aren't separate systems. They're all forms of learning.

Memory is Learning

This realization led me to build something different: the Learning Machine, a unified learning system that helps agents continuously integrate information from their environment.

Here's the difference:

Traditional "Memory":
Message → Extract → Store → Retrieve → Dump into Prompt → Repeat
 
Learning Machine:
User Message ──────► Recall from Stores ◄────────┐
                            │                    │
                            ▼                    │
                      Build Context              │
                            │                    │
                            ▼                    │ LearningMachine
                Agent Responds (with tools)      │
                            │                    │
                            ▼                    │
                   Extract & Process             │
                            │                    │
                            ▼                    │
              Update Stores (agent learns) ──────┴──► Periodic Curation

The agent isn't just fed memories. It participates in learning, curating what it learns, and integrating that knowledge back into every response.

The goal: an agent on interaction 1000 is fundamentally better than it was on interaction 1 — across the board, not just with the same user.

What It Looks Like in Action

A new employee on their first day asks: "I'm starting work on the cloud migration project. What should I know?"

The agent responds with full context, even though it's never talked to this person before. It knows Acme is migrating from AWS to GCP. It knows Alex (CTO) is leading it. It knows Phase 2 is the most compute-heavy. It shares migration patterns from similar past projects. It knows that the pricing is changing next quarter.

How? Three types of learning from past interactions:

Session 1 (Alex, CTO):
"I'm Alex, CTO at Acme. We're migrating from AWS to GCP and
I need help planning the timeline."
 
→ User Profile captures: Alex, CTO, involved in planning discussions
→ Entity Memory captures: Acme (company), AWS→GCP migration (project)
→ Session Context: Goal is migration timeline planning

Session 2 (next day, same user, different session):
"Just heard GCP is changing their pricing next quarter.
How does that affect our migration?"
 
→ Agent recalls: Acme, AWS→GCP migration, Alex is CTO, 3-phase timeline
→ Agent responds: "That could impact your timeline. Last time we mapped
   out a 3-phase approach with Phase 2 being the most compute-heavy.
   Want me to model the cost implications for each phase?"

Session 3 (different user, same org namespace):
"I just joined to help with the Acme cloud project. What should I know?"
 
→ Entity Memory: "Acme is migrating AWS to GCP. Alex (CTO) is leading it."
→ Learned Knowledge: Shares migration patterns from past projects
→ Agent responds with full context — even though it never talked to this user

Three sessions. Three types of learning. Cross-user knowledge sharing.

This is possible. Today.

The Architecture: Learning Stores

The key innovation behind the Learning Machine is the learning protocol and learning stores. The protocol defines how stores capture, process, and integrate knowledge. Each store is configured independently. Mix and match as needed. The Learning Machine orchestrates it all.

These are the stores I'm working on:

Show Me Some Code

One agent. Four learning stores. Configured independently. Orchestrated by the Learning Machine.

from agno.agent import Agent
from agno.db.postgres import PostgresDb
from agno.models.openai import OpenAIResponses
 
agent = Agent(
    model=OpenAIResponses(id="gpt-5.2"),
    db=PostgresDb(db_url="postgresql://..."),
    learning=LearningMachine(
        knowledge=my_vector_store,  # or graph if that's your thing
        user_profile=UserProfileConfig(
            mode=LearningMode.BACKGROUND,
            enable_agent_tools=True,
        ),
        session_context=SessionContextConfig(
            enable_planning=True,
        ),
        learned_knowledge=LearnedKnowledgeConfig(
            mode=LearningMode.PROPOSE,
        ),
        entity_memory=EntityMemoryConfig(
            mode=LearningMode.BACKGROUND,
        ),
    ),
)

The best part? You can build custom learning stores by extending the LearningStore protocol. Need project context? Build a ProjectContextStore. Need to track accounts? Build an AccountStore.

Taking Inspiration from Claude

Claude's memory feels magical. It's natural, contextual, never announces "saving to memory". It just knows you.

But here's the thing: you can't build with it. Claude's memory is a consumer product feature. The API gives you nothing. If you want learning for your agents, you're on your own. Enter Learning Machine.

Here's what Claude does well, and what Learning Machine adds:

Claude feels natural. It never announces "saving to memory". So does Learning Machine. We inject context based on each store and control how the agent learns from it. No fact dumps.

Claude learns about its users over time. Preferences, history, personal context. So does Learning Machine. But we also add sessions, entities, patterns, and decisions. The full picture, not just the user.

Claude is scoped to a single user. Makes sense for a consumer product. Learning Machine adds namespace scoping: keep it private to a user, share across a team, or make it global. You control the boundaries.

Claude has fixed memory types. You can't change how it works. Learning Machine is extensible via protocol. Build your own stores for whatever your domain needs.

Claude is a closed system. Its memory lives inside Claude. Learning Machine is open source, fully customizable, and yours to extend.

I studied what makes Claude's memory feel good. Then built something you can actually use and extend.

What This Unlocks

Here's what's possible when agents learn across users, sessions, and time:

• A support agent where ticket #1,000 gets resolved better and faster — because it learned from tickets #1-999.
• A customer success agent that remembers every account's stack, contracts, and conversations — across your entire team.
• A healthcare agent that knows your full history — not just what's in today's chart, but every conversation (with different doctors), symptom, and concern you've ever mentioned.
• A financial advisor that remembers your risk tolerance, goals, and every "what if" scenario you've ever explored — across years of conversations.
• An agent that rewrites itself — analyzing its failures and proposing: "I should stop doing X."

That last one is the endgame. Agents that learn from their own mistakes and rewrite their own instructions. Human approves. Agent evolves. Continuous improvement.

Current Status

Learning Machine is part of Agno and I'm in the final stages of testing Phase 1. Here's where things stand:

If you're eager to dig in, here's the PR: learning-machine-v0

Want to get involved? DM me if you're interested in learning more or helping out.

Memory was never the goal. Learning was.

If you enjoyed reading this, checkout Agno on GitHub.

Questions or feedback? Reach out on X.

What makes ChatGPT and Claude great personal assistants? Memory.

Here's the dirty secret: when building agents with the API, we've made them capable, but we haven't yet figured out how to make them learn.

Table of Contents

1. What is memory
2. How memory enables learning
3. Three patterns (with code)
4. Video demo
5. What makes a good learning
6. Get started

Wanna jump straight to the code? Here you go. Cookbooks 2, 4 and 7 are what you're looking for.

1. What is memory?

"Memory" gets thrown around loosely. Chat history? Context window? Vector database? Let's be precise.

There are three types of memory that matter for agents:

Session Memory

The conversation context. What was said five messages ago. This is a solved problem: store messages in a database, retrieve them before every response, add them to the context.

Session memory is useful but limited. It disappears when the conversation ends. It's not really memory, it's just context.

User Memory

Facts about a specific user that persist across sessions. Preferences, goals, constraints.

When a user says "I'm interested in AI stocks and have moderate risk tolerance", that's worth remembering, not just for this conversation, but for every future conversation with that user.

This is powerful, but it's still not learning. User memory is about recall, not improvement.

Learned Memory

This is where knowledge gets built. As agents interact with the world, they discover insights that apply generally, not just to one user, but to anyone asking similar questions.

When your finance agent discovers that "when comparing ETFs, check both expense ratio AND tracking error", this insight is worth saving, not just because one user asked, but because it makes the agent better at ETF comparisons for everyone.

Here's the beauty: knowledge compounds. The more the agent learns, the better it gets. And unlike weight updates, this knowledge is tangible: you can inspect it, edit it, delete it. No retraining required.

If you're building agents without learned memory, you're leaving performance on the table.

2. How memory enables learning

Here's the core insight: learning is remembering what worked.

Without memory, agents are stateless. Every session is day one:

The best part: the model doesn't need to get better for the system to improve. Learning happens in retrieval, not in weights. And as models improve, your system improves too — for free.

I call this GPU Poor Continuous Learning: continuous improvement without fine-tuning, retraining, or any of the infrastructure traditionally required for model updates. Just a knowledge base that grows smarter over time.

The model doesn't get smarter. The system gets smarter.

3. Three patterns for agent memory

Let me show you how to implement the three patterns, with a bonus at the end.

Pattern 1: Session Memory

Store messages in a database, retrieve them before every response, add them to the context. Agno gives you this out of the box — just give your agent a database.

from agno.agent import Agent
from agno.db.sqlite import SqliteDb
from agno.models.google import Gemini
from agno.tools.yfinance import YFinanceTools
 
agent_db = SqliteDb(db_file="tmp/agents.db")
 
agent = Agent(
    model=Gemini(id="gemini-3-flash-preview"),
    tools=[YFinanceTools()],
    db=agent_db,
    add_history_to_context=True,
    num_history_runs=5,
)
 
if __name__ == "__main__":
    session_id = "finance-session"
 
    # Turn 1: Analyze a stock
    agent.print_response("Quick investment brief on NVIDIA", session_id=session_id)
 
    # Turn 2: Agent remembers NVDA from turn 1
    agent.print_response("Compare that to Tesla", session_id=session_id)
 
    # Turn 3: Recommendation based on full conversation
    agent.print_response("Which looks like the better investment?", session_id=session_id)

Use a consistent session_id to persist conversation across runs.

Pattern 2: User Memory

Remember facts about the user across sessions. The MemoryManager extracts preferences automatically and stores them in the database.

from agno.agent import Agent
from agno.memory import MemoryManager
from agno.models.google import Gemini
from agno.db.sqlite import SqliteDb
 
agent_db = SqliteDb(db_file="tmp/agents.db")
 
memory_manager = MemoryManager(
    model=Gemini(id="gemini-3-flash-preview"),
    db=agent_db,
)
 
agent = Agent(
    model=Gemini(id="gemini-3-flash-preview"),
    memory_manager=memory_manager,
    enable_user_memory=True,
)
 
# First conversation — preferences extracted and stored
agent.print_response(
    "I'm interested in AI stocks. My risk tolerance is moderate.",
    user_id="investor@example.com",
)
 
# Later conversation — agent remembers
agent.print_response(
    "What stocks would you recommend for me?",
    user_id="investor@example.com",
)

enable_user_memory=True runs the MemoryManager in parallel with every run. Use enable_agentic_memory=True to let the agent decide when to store memories via tool calls. More efficient, doesn't run on every response.

Pattern 3: Learned Memory

Now let's add learned memory: insights that apply beyond just one user. The key is a custom tool that saves learnings to a knowledge base:

import json
from datetime import datetime, timezone
 
from agno.agent import Agent
from agno.db.sqlite import SqliteDb
from agno.knowledge import Knowledge
from agno.models.google import Gemini
from agno.vectordb.chroma import ChromaDb
 
agent_db = SqliteDb(db_file="tmp/agents.db")
 
learnings_kb = Knowledge(
    name="Agent Learnings",
    vector_db=ChromaDb(
        name="learnings",
        persistent_client=True,
        search_type=SearchType.hybrid,
    ),
)
 
def save_learning(title: str, learning: str) -> str:
    """
    Save a reusable insight to the knowledge base.
 
    Args:
        title: Short descriptive title
        learning: The insight — specific and actionable
    """
    payload = {
        "title": title.strip(),
        "learning": learning.strip(),
        "saved_at": datetime.now(timezone.utc).isoformat(),
    }
 
    learnings_kb.add_content(
        name=payload["title"],
        text_content=json.dumps(payload),
    )
 
    return f"Saved: '{title}'"
 
agent = Agent(
    model=Gemini(id="gemini-3-flash-preview"),
    tools=[save_learning],
    knowledge=learnings_kb,
    search_knowledge=True,
    db=agent_db,
)

The agent now has two capabilities:

1. Search first — Before answering, it searches for relevant prior learnings
2. Save learnings — When it discovers something reusable, it saves it

But how do you prevent the agent from saving garbage?

Bonus: Human-in-the-Loop Gating

The quality of your knowledge base determines the quality of learning. Garbage in, garbage out.

The solution: the agent proposes learnings, but only saves with explicit user approval.

from agno.tools import tool
 
@tool(requires_confirmation=True)
def save_learning(title: str, learning: str) -> str:
    """Save a reusable insight. Requires user confirmation."""
    # ... same implementation

Handle the confirmation flow:

run_response = agent.run("Analyze NVDA and save any insights")
 
for requirement in run_response.active_requirements:
    if requirement.needs_confirmation:
        print(f"Tool: {requirement.tool_execution.tool_name}")
        print(f"Args: {requirement.tool_execution.tool_args}")
 
        if user_approves:
            requirement.confirm()
        else:
            requirement.reject()
 
run_response = agent.continue_run(
    run_id=run_response.run_id,
    requirements=run_response.requirements,
)

The agent proposes, the human gates. High-signal knowledge only.

5. Video demo

Here's a video demo that starts by showcasing user memory, then learned memory with user confirmation.

5. What makes a good learning

A learning is worth saving if it's:

• Specific: "Tech P/E ratios typically range 20-35x" not "P/E varies"
• Actionable: Can be applied to future queries
• Generalizable: Useful beyond this one conversation

Don't save: raw data, one-off facts, summaries, speculation.

Most queries should NOT produce a learning, and that's OK.

Where to store

Avoiding bloat

The biggest mistake is storing too much. A bloated knowledge base hurts retrieval and makes the agent worse.

The upside: because learnings are stored explicitly (not in weights), they're auditable and reversible. Bad learning? Delete it. System immediately improves.

6. Get started

This blog comes with complete working code. Here are 12 cookbooks that take you from "what is an agent" to building agents with memory, knowledge, state, guardrails, and more. Link again for reference.

Each builds on fundamentals, but you can jump to any one.

Setup

git clone https://github.com/agno-agi/agno.git
cd agno
 
uv venv .getting-started --python 3.12
source .getting-started/bin/activate
 
uv pip install -r cookbook/00_getting_started/requirements.txt
 
export GOOGLE_API_KEY=your-google-api-key

Run an example

Each cookbook is self-contained:

python cookbook/00_getting_started/agent_with_tools.py

Want a visual interface? Agent OS gives you a web UI for chatting with agents, exploring sessions, and monitoring traces:

python cookbook/00_getting_started/run.py

Then visit os.agno.com and add http://localhost:7777 as an endpoint.

Swapping models

These examples use Gemini 3 Flash by default — fast, reliable tool calling, cheap enough to experiment freely. But Agno is model-agnostic:

# Gemini (default)
from agno.models.google import Gemini
model = Gemini(id="gemini-3-flash-preview")
 
# OpenAI
from agno.models.openai import OpenAIChat
model = OpenAIChat(id="gpt-5.2")
 
# Anthropic
from agno.models.anthropic import Claude
model = Claude(id="claude-sonnet-4-5")

One line change. Everything else stays the same.

If you enjoyed reading this, star Agno on GitHub. It helps more than you'd think. Questions or feedback? Reach out on X.

Table of Contents

1. The problem with disconnected sessions
2. What is "gpu-poor continuous learning"
3. Why Gemini 3 Flash
4. The learning loop
5. Demo
6. What we store (and what we don't)
7. How to run your own Self-Learning Agent
8. Why this pattern works

1. The problem with disconnected sessions

Most agents run in independent sessions, disconnected from each other.

You ask a question. You get an answer. Tomorrow you ask a similar question and the agent starts from scratch. It doesn't remember what worked, what failed, or what it figured out along the way.

This is fine for simple tasks. But for anything complex—research, analysis, decision support—it means:

• Repeating the same reasoning patterns
• Re-discovering the same gotchas
• Never building on past success

If your agent can't learn from its own experience, you're leaving performance on the table.

2. What is GPU Poor Continuous Learning

Let me be precise about terminology, because "continuous learning" has a specific meaning in ML.

Traditional continuous learning:

• Model weights update over time
• Requires compute (GPUs, TPUs)
• Risk of catastrophic forgetting
• Learning happens in parameters

What I'm doing (GPU Poor Continuous Learning):

• Model stays completely frozen
• Zero training compute
• Learning happens in retrieval
• Knowledge is auditable and reversible

The model doesn't get smarter. The system gets smarter.

I call it "GPU Poor" because you get continuous improvement without any of the infrastructure traditionally required for model updates. It's poor man's continuous learning—and it works surprisingly well.

3. Why Gemini 3 Flash

I built this with Gemini 3 Flash, which launched today. Here's why: