Workflow or AI agent? A practical decision framework

Why this decision matters

"Let's build an AI agent" has become the default answer to every AI project. In practice, most problems that teams call agents are better solved with a workflow: a fixed sequence of steps where a language model does one well-defined job at each step. Agents are powerful, but they are also slower, more expensive, harder to test and harder to explain to the people who have to trust them.

Choosing correctly at the start saves weeks. This guide gives you a simple framework you can reuse on any use case.

The three designs, in plain terms

Design	Who decides the next step?	Typical example
Workflow	Your code. The steps are fixed.	Classify a support ticket, extract the order ID, draft a reply for a human to approve.
Single agent	The model, in a loop, choosing from a set of tools.	A support assistant that decides whether to look up an order, check the refund policy or escalate.
Multi-agent	Several specialised agents that hand work to each other.	A research pipeline with a planner, several researchers and a reviewer.

A useful way to remember it: in a workflow, you own the control flow; in an agent, the model owns it.

Five questions that decide it

Ask these in order. The first "no" usually points to a workflow.

1. Can you write the steps down in advance? If a competent person would follow the same steps every time, encode them. That is a workflow.
2. Does the path really change from case to case? If the next step depends on what was found in the previous one (look up the order, then decide whether the policy applies), an agent may earn its place.
3. How bad is a wrong action? Agents take actions. If an action is irreversible or expensive (refunds, emails to customers, database writes), keep a human approval step, or keep it a workflow.
4. Do you need predictable cost and latency? Workflows make a known number of model calls. Agents can loop. If you have a strict budget or a 2-second response target, start with a workflow.
5. Can you test it? A workflow can be tested step by step with fixed inputs. An agent needs scenario-based evaluation. If you cannot invest in evaluation yet, you are not ready for an agent.

A worked example: support tickets

An e-commerce company receives thousands of tickets a day about orders, refunds and delivery.

Version 1, a workflow:

1. Classify the ticket (order status, refund, delivery, other) with structured output.
2. Extract fields: order ID, product, urgency.
3. Call the order-status API with the extracted ID.
4. Draft a reply using the ticket, the order data and the policy text.
5. Send to a human for approval.

Every step is testable. Cost per ticket is predictable. This alone removes most of the reading and typing work.

Version 2, an agent, only where it helps:

For the "refund" category, cases vary: partial refunds, damaged items, missing deliveries, exceptions. Here a single agent with three tools (get_order, check_refund_policy, escalate_to_human) can decide what to look up and when to stop. It runs inside the workflow, only for one category, with a step limit and an escalation rule.

This "workflow with an agent inside it" pattern is how most production systems actually look.

What an agent needs to be safe

If you do build an agent, give it:

• A small set of well-described tools. Five clear tools beat twenty vague ones.
• A step limit (for example, at most 6 tool calls) and a fallback when it is reached.
• Structured outputs for decisions, so your code can check them.
• A human in the loop for actions that matter.
• Logs of every step, so you can see why it did what it did.
• An evaluation set: 20–50 realistic cases with the expected outcome, run after every change.

When multi-agent is justified

Multi-agent systems make sense when the work naturally splits into roles that need different instructions, tools or context windows (for example, a planner that breaks down a task and specialists that each work on one part). They are rarely the right first step. Start with one agent, measure where it struggles, and split only when a single prompt can no longer hold the job.

A quick checklist

• Steps can be written down → workflow
• Path depends on intermediate results → consider one agent for that part only
• Irreversible actions → human approval
• Strict cost or latency → workflow first
• No evaluation set yet → not ready for an agent
• Clearly separate roles with different tools → maybe multi-agent, later

The best AI engineers are not the ones who build the most autonomous system. They are the ones who choose the simplest design that solves the problem, and can explain why.