What is conversational ai?

Why it matters

The breadth of the term is a buying hazard. A menu bot and a grounded language-model agent are both sold as conversational AI at overlapping prices, and they differ by an order of magnitude in what they can resolve. The label tells you nothing, so the evaluation has to.

What to know

• Ask what happens with an unanticipated phrasing. That single question separates scripted from generative systems.
• Ask where the answer comes from. Scripts, intents, and retrieved documents are three different risk profiles.
• Ask what it can do beyond answering, since action-taking is where the remaining cost sits.
• Ask how it escalates, because handoff quality is what customers judge the whole system on.
• Ask how it is billed, because the unit reveals what the vendor is optimising for.

An example

Example: two vendors demo conversational AI. One resolves the five scripted questions perfectly and fails on the sixth. The other answers all six from documentation and admits when it cannot. The demos looked similar and the products are not comparable.

Common mistakes

• Evaluating conversational AI on demo questions the vendor chose
• Assuming the category label implies language-model capability
• Comparing prices without comparing what each system can resolve
• Ignoring the escalation path during evaluation

Questions