The word "agent" has been applied to everything from a scripted FAQ widget to a system that files expense reports. That vagueness is expensive, because the two ends of that range differ by an order of magnitude in build cost, risk, and value. It is worth being precise about which one a project actually needs.
What a chatbot is
A chatbot answers questions inside a conversation. In its most common form it is a language model plus a system prompt, sometimes with a document set attached. It has no access to your records, cannot take an action, and does not know who it is speaking to beyond what they type.
That is not a criticism. For deflecting repetitive questions with public information — hours, policies, product details — it is often exactly the right tool, and it can be deployed in days. The failure mode is expecting it to do more.
What separates a business agent
Five things, and each one is engineering work rather than prompt work.
Grounding
An agent answers from a controlled source of truth rather than from general model knowledge. That requires a retrieval layer: documents indexed, chunked sensibly, kept in sync when the source changes, and returned with a citation so an answer can be checked. Most of the accuracy difference between a good and a bad deployment comes from this layer, not from the model.
Permissions
An agent knows who is asking and returns only what that person is entitled to see. Critically, this has to be enforced at retrieval — filtering the documents before they reach the model — rather than by instructing the model not to reveal things. An instruction is a request. A permission filter is a guarantee.
Tools
An agent can do something: create a record, schedule a meeting, look up an order, start a workflow. Each capability is an explicit, permissioned function rather than an open door. The design question for every tool is what happens if it fires when it should not have, and that answer determines whether it needs an approval step.
Escalation
A production agent needs a defined boundary and a clean exit. It should recognize when a request is outside its scope, when its confidence is low, or when a person has asked for a human — and hand over with the context attached rather than looping. Systems without a good exit path do more reputational damage than systems that simply answer less.
Auditability
Every question, retrieved source, and action should be logged. This matters for three separate reasons: debugging when it gets something wrong, accountability when a customer disputes what it said, and improvement — because the log of failed conversations is the highest-value input you have for making it better.
What this changes about the project
A chatbot is a configuration exercise. An agent is a software project with a data dependency, and it should be scoped like one — with a knowledge source that has an owner, an accuracy target measured against real questions, and a review cadence after launch.
The most common scoping mistake is treating the knowledge base as a one-time task. Documents change. A policy updated in March and an agent still quoting February is a worse outcome than no agent, because it is confidently wrong rather than unhelpful. Whoever owns the documents has to own the refresh.
How to tell which one you need
Three questions settle it in most cases:
- Does the answer depend on who is asking? If yes, you need permissions, which means an agent.
- Does the answer depend on data in your systems? If yes, you need retrieval and integration, which means an agent.
- Should it be able to change something? If yes, you need tools, approvals, and logging, which firmly means an agent.
Three noes means a chatbot will probably serve, and you should build that instead of a larger system that solves problems you do not have.
Measuring it honestly
Before launch, assemble fifty to a hundred real questions with known correct answers, and measure against them. Not curated questions — real ones, pulled from support tickets or internal messages, including the badly worded ones. Report the failure rate rather than hiding it, decide what rate is acceptable for the use case, and re-run the set whenever the knowledge source or model changes.
An agent measured this way can be deployed with confidence. One that was only ever tested by the people who built it will be tested by your customers instead.
We build custom AI agents and assistants with this full stack — grounding, permissions, tools, escalation, and audit logging — and we are equally willing to tell you when a much simpler system would do. Start with a conversation.