OpenAI is retiring Agent Builder and Evals. Here is the migration.

Bles Software · updated 5 August 2026

The dates are published and they are not far away. If your agents or your evaluation suite live in Agent Builder, this is a forced migration, and the only real question is whether you use it to fix the architecture or just move the mess somewhere else.

The dates

The recommended path is the Agents SDK for the orchestration and Promptfoo for evaluations. Both are reasonable. Neither is a drag-and-drop replacement for a visual builder, which is the part that catches teams out: the migration is not an export and an import, it is a rewrite of the glue.

What is actually at risk

People inventory the agents and forget the rest. In practice four things live in there and all four have to land somewhere.

• The agent graphs. The steps, the branches, the tool calls. This is the visible part and usually the easiest to rebuild, because the logic is understood.
• The reusable prompts. Often edited by non-engineers, often not in version control, often the thing nobody realises is load-bearing until it disappears.
• The evaluation suite. This is the expensive one. It encodes what "working correctly" means for your business, it was built case by case over months, and once Evals goes read-only on 31 October you cannot run it to prove the new system matches the old one.
• The operational knowledge. Which cases were known to fail, which prompt version fixed what, why a particular guardrail exists. This lives in the tool's history, not in anyone's head.

Move the evaluations first, not last. After 31 October you lose the ability to run the old suite, which means you lose your only objective proof that the migrated system behaves the same way.

The order that works

1. Inventory before you touch anything. Every agent, prompt, eval and integration, with an owner and a business process attached to each. Anything with no owner and no process is a candidate for deletion, and deleting is cheaper than migrating.
2. Export the evaluation cases now. Get the test cases, the expected outputs and the scoring logic out into your own repository while the tooling still runs. This is the single most time-critical step.
3. Rebuild the evaluation harness on Promptfoo. Run it against the current production system and confirm it reproduces the scores you already trust. Now you have a baseline the new build has to beat, or at least match.
4. Port the agents to the Agents SDK, highest value first. One at a time, each proved against the harness before the next one starts. Resist the temptation to port everything in parallel.
5. Put prompts in version control. If they were reusable in the platform, they become files with review and history. This is where a forced migration quietly pays for itself.
6. Add what the platform was hiding. Retry semantics, idempotency on anything that touches money, structured logs, and the transparency record EU AI Act Article 50 duties have required since 2 August 2026.
7. Cut over with the old system still available. Shadow run, compare against the harness, then switch. Keep the old path warm until the numbers hold for a full cycle.

The mistake worth avoiding

The tempting move is a like-for-like port: same graph, same prompts, new runtime, done. It is faster and it feels lower risk. It also carries forward every workaround that existed because the visual builder could not express something properly, and it wastes the one moment when a rewrite is already budgeted and approved.

The opposite mistake is to treat the deadline as permission to redesign everything. That is how a two-month migration becomes a six-month programme that misses 30 November. The line we hold is simple: rebuild the glue properly, keep the business logic identical, and prove it with the evaluation suite you rescued in week one.

How long it takes

For a handful of agents with a small evaluation suite, a competent team does this in four to six weeks. For a portfolio spread across several functions, with integrations into systems of record and a compliance story to keep intact, it is a quarter of work and it needs a fixed date, because the shutdown has one.

That is exactly the shape our Command Sprint covers: 90 days, fixed fee, migration off the retiring stack with the evaluation suite moved first, and a performance commitment against the numbers the old system produced. If your portfolio is smaller, a Focus or Scale Sprint fits and costs less. If you have engineers who can do it, the plan above is yours to use without paying anybody.