Before you build an agent

A useful starting point for an AI project is a description of the work. Who needs something done? What counts as a satisfactory result? What happens when the result is wrong?

Those questions sound ordinary. They are also where many of the architectural decisions hide. A system that drafts a reply and a system that sends it may use the same model. They carry different responsibilities.

Describe the job before choosing the mechanism

Consider a team receiving partnership enquiries. The request might be to read incoming messages, find the relevant company information, and prepare a short brief for the person taking the meeting. That is a bounded job. “Run our partnerships with AI” leaves too many decisions unresolved.

For the bounded version, I would write down the input, expected output, permitted information sources and named reviewer. I would also record what the system is allowed to do without asking. Reading a public company page and making a contractual commitment should never sit behind the same permission.

Anthropic’s Building effective agents distinguishes predefined workflows from agents that choose their own next steps. That distinction is useful when deciding how much freedom a task actually requires. A fixed sequence can be easier to inspect when the steps are already known.

Make the first version deliberately small

For our hypothetical partnership brief, a first version could collect the message, retrieve a small set of approved reference material, draft the brief and leave it for review. It does not need to negotiate, send email or update every system in the company.

The smaller scope makes evaluation concrete. Did it identify the right company? Did it confuse an aspiration with an existing partnership? Can a reviewer trace each important claim back to a source? How much editing remains?

Keep examples that are awkward: an ambiguous company name, a stale website, a request that crosses two ventures. The easy examples demonstrate the idea. The awkward ones help define its limits.

Give uncertainty somewhere to go

A brief should distinguish a verified fact from an inference and an unanswered question. Otherwise the reviewer has to reverse-engineer the uncertainty from fluent prose.

The system also needs an exit. It should be able to say that it cannot identify the company, cannot access the required information, or has reached its allowed time or cost. A partial brief with a clear gap can be useful. A confident invention makes additional work for everyone downstream.

Decide how to recover

Before adding write access, ask how a mistake will be corrected. If the agent creates duplicate records after a timeout, can the system recognise that it already performed the action? If it prepares the wrong brief, can the reviewer replace it without losing the evidence trail?

These are ordinary software questions. They remain necessary when part of the software speaks in sentences.

For an initial release, my preference is a narrow permission set, a visible record of actions and a named person responsible for exceptions. Expand autonomy when the observed results justify it, one capability at a time.

Measure usefulness, including the work left behind

A convincing demonstration is a beginning. To judge the working system, count the accepted outputs, the material errors, the time spent reviewing and the cases that had to be handled manually. Keep latency and running cost alongside those measures.

The target is a job done well at an acceptable total cost. Sometimes that calls for an agent. Sometimes it calls for a simpler workflow. Establishing which one you need is part of the architecture.