Series · Part 1 of 2
Every Team Built an AI Agent. Nobody Knew Who Used Which Model, or What It Cost.
One gateway in front of every agent: role-based access, human approvals, and every LLM call logged per user, agent and model.
Part 1 of 3: the design. Part 2 runs it on a laptop, and Part 3 (coming soon) runs it in production on Kubernetes. Code: ovsec/agent-platform.
Every team building agents picks a different tool: Langflow, LangGraph, Pydantic AI. Soon every agent has its own chat window, its own login and its own model key, and nobody knows who asked what.
Building the agent is the easy part. The hard part is what surrounds it: finding agents, controlling access, getting a human’s approval before an agent acts, and keeping an audit log. I built that once, for any runtime.
A front door, not a framework
Don’t force one framework on everyone. Standardize how agents are reached instead. Every agent sits behind one gateway, agentgateway, and speaks one protocol, AG-UI. Because of that, the portal needs only one way to talk to all of them.
When Bob asks the invoice agent a question, the gateway:
- checks his roles and stops the request if he isn’t allowed
- routes it to the agent’s runtime, and tells the agent who’s asking
- handles the agent’s model calls, so no agent holds a provider key and every call is logged
Because every model call goes through one door, you get usage per person, per agent and per model in one place, and cost too once you use hosted models.
Routes aren’t written by hand. Each agent ships a small manifest (owner, runtime, allowed roles), and the gateway’s routes are generated from those manifests at deploy time.
Asking before acting
Agents that act should ask first. AG-UI has a standard “pause and ask”, and the portal shows it as an Approve / Reject card for every agent. Langflow pauses in its own way, so a small adapter translates:
Teams own their agents
Agents live in each team’s own repository, and a shared workflow releases them. The owning team decides when to go live.
Where it stands
Langflow agents and a LangGraph-based code agent run today (Part 2). Pydantic AI and CrewAI speak AG-UI already, so they need no new platform code. The runtime becomes each team’s choice, and every agent gets the same portal, access rules, approvals and audit trail.
Coming next: production on Kubernetes
Part 3 takes this platform from a laptop to a production Kubernetes cluster: the gateway, runtimes and portal as deployments, agents released from agent-catalog, and your company’s identity provider in place of Keycloak.
Built on open source: agentgateway, AG-UI, Langflow, assistant-ui and LangGraph.
Building an Agent Platform · 2 of 2 published
- 1 Every Team Built an AI Agent. Nobody Knew Who Used Which Model, or What It Cost.
- 2 An Enterprise Agent Platform on My Laptop, in 8 Containers
Related posts
Part 15 — AI Demystified
How Do You Know AI Is Actually Working?
Demos always look good. Production AI degrades silently. Here's the evaluation framework — from exact match to human review — and how to catch hallucinations.
ReadPart 0 — AI Demystified
5 Mental Models You Need Before Diving Into AI
Before you learn how AI works, learn how to think about it. These five mental models will save you hours of confusion.
ReadGet new posts in your inbox
No spam. No digest. Just a note when I publish something new.
Discussion
On this page