· 7 min read
Language models in support: where you cannot let them near a customer
We built a bot for a bank and refused to hand the model every scenario. Here is how to split questions into ones where a model belongs and ones where it is dangerous.
In 2025 we built a support bot for a bank. The client wanted a language model answering everything. We refused, and ended up giving the model roughly a third of the scenarios. The reasoning generalises to any support operation, not just finance.
The question that decides it: what does an error cost
A language model is a probabilistic tool. It will make mistakes, and the question is not how to eliminate that but what happens when it does.
If the model gets a branch's opening hours wrong, someone turns up at the wrong time, gets annoyed and writes a review. Unpleasant but recoverable. If the model states the wrong interest rate on a loan, that is a written commitment made in the bank's name. What follows is a complaint, a dispute, and reputational damage you cannot pay off.
We divide scenarios by the cost of a wrong answer, not by the difficulty of the question.
What we gave to deterministic logic
Everything with exactly one correct answer that comes from a system:
- Account balances and transaction history
- Loan application status
- Addresses, opening hours, ATM availability
- Product and tariff terms — quoted from the document, never paraphrased
- Card blocking and other state-changing actions
These are buttons and pre-written answers. Zero chance of invention, instant response, no inference cost. Such questions are about 60 % of all enquiries.
What we gave to the model
Free-form questions where the customer does not know what their problem is called: "I got charged twice, what now", "I'm moving house, do I need to change anything", "it won't let me log in, some error".
Here the model's job is not to answer but to understand and route. It identifies the topic and either answers strictly from the knowledge base or hands over to an agent.
Three constraints we would not launch without
Answer only from documents
The model receives the question plus excerpts from the bank's knowledge base, and is explicitly instructed not to answer beyond them. If the documents do not contain the answer, the correct behaviour is to say so and hand over to a human — not to assemble something from general knowledge about banks.
A source reference on every answer
Each answer cites the document section it came from. Two reasons. The customer can verify it. And we can verify it: when a case is disputed, we can see exactly what the model read.
No actions
The model does not block cards, change limits or submit applications. It can offer to, but confirmation goes through an ordinary button. A model that performs irreversible actions based on its reading of free text is an unjustified risk.
Where it landed
70 % of enquiries are resolved without an agent. Average response is 9 seconds against seven minutes on hold in the call centre. In a year of operation, not one instance of the bot stating incorrect product terms.
The share of questions escalated to a human is higher than off-the-shelf vendors promise. We consider that the right outcome: better to pass an extra 10 % to a person than to answer confidently and wrongly once.