In the welfare-navigation pilot we instrumented this spring, a competent answer costs about three tenths of a penny to produce. The advice-line call it stands in for costs the service something like £7, and costs the caller a forty-minute queue. Set those figures side by side and the gap stops being a discount; it is a change of category. For as long as professional advice has existed, the price of a considered answer has been high enough that a question had to justify itself before it was asked. People with solicitors ask legal questions. Everyone else has legal problems and hopes they go away.
Capability is what gets written about; price decides who gets to ask. A model's cost per query tells you who can afford to find out, and that number has been falling faster than most people priced in.
When GPT-4 arrived in 2023, its launch pricing made a substantial query cost tens of pence in tokens; within about two years the going rate for comparable capability had dropped by more than an order of magnitude, and the batch tiers most services run on are cheaper again. Beneath that commercial curve sits a harder floor: open-weight models good enough for routine advice now run on hardware you can buy in a shop, which drives the marginal cost of an answer down towards the price of the electricity to compute it. A query that cost real money in 2023 is a rounding error today.
The country the pilot runs in
This lands harder in Britain than the bare economics suggest, because Britain has spent a decade making advice harder to get. The 2012 legal-aid reforms, LASPO in the sector's shorthand, took most welfare-benefits advice out of scope, so the person who has both a benefits problem and a funded professional to explain it is now the exception. Advice deserts, districts with no accessible provider for a given kind of problem, are documented fact rather than rhetoric. The charities that absorbed the overflow run permanently over capacity. And the experience of a rationed answer is familiar to anyone who has needed one: the hold music on a council line, the callback that never comes, the NHS 111 queue that is free at the point of use precisely because it is rationed by time rather than money.
What the logs actually show
Week one of the pilot reads like the advice line it was built to relieve: eligibility checks, deadline lookups, which box on the form means what. These are the institutional questions, the ones the old channel was priced and staffed to receive.
By week four a second stratum has appeared, and it is the one we now watch most closely. A handful of examples, paraphrased with users' consent:
- "Why does the form ask for this twice? Is one of them a trick?"
- "If I report the extra shift, does my neighbour's report about me count for more or less?"
- "What actually happens if I just don't reply to this letter?"
- "Can they really do that?"
Nobody waits forty minutes on hold to ask whether a form field is a trap. These are questions people used to settle with guesswork, family folklore, or a shrug. Twenty-two per cent of the pilot's heavy users had never reached the service through any earlier channel: not lapsed users served a little better, but people the old price had kept outside the building.
This is where the intuitive reading fails. The obvious model is substitution: a cheap answer replaces a dear one, the same question at a lower unit cost. What the log records is closer to what Jevons saw in coal in the 1860s, that making a resource cheaper to use raises the total consumed rather than lowering it. Cheap answers do not shrink the advice problem by meeting fixed demand efficiently; they enlarge it, because most of that demand was never expressed. The pilot's own data makes the claim falsifiable. If substitution were the whole story, week four would look like week one at higher volume, the same mix scaled up. It does not: volume climbs and the composition tilts towards questions that never entered the old channel. Had the logs shown the flat, substituting pattern, that is what I would be reporting.
Being wrong at scale
None of this is worth much if the answers are wrong, and here the numbers keep me honest. Our probe runs put the underlying model's harmed-rate — answers confidently wrong in ways that could cost someone money or a missed deadline — at around six per cent on realistic traffic. Human advisers are not flawless, and the comparison is not against perfection. But the two failures are not alike. A tired adviser is wrong once, to one person, in one way. A model is wrong identically every time the question arrives in the same words, across everyone who asks it that way. The error is not only replicable but regressive: it falls hardest on the person with no solicitor, no case worker, no one placed to catch it, which is to say on exactly the people the low price has finally let in.
So the pilot runs with the unglamorous safeguards our findings keep pointing towards. Deadline questions return a sourced answer with the rule quoted rather than paraphrased. The model is barred from arithmetic on dates; a tool does the sum instead, for reasons our drift work sets out. And one fixed part of the interface is a prominent route to a human being, whose usage we treat as a health metric to watch rather than a cost to minimise. A pilot in which nobody ever escalates to a person would not reassure me; it would worry me.
Price the hundredth question
The practical move is to stop pricing the first question and start pricing the hundredth. Demand for answers is highly elastic, which is a dry way of saying that price does not merely ration how many get asked; it selects which kind. At £7 a time, a person asks only what cannot be avoided: the deadline, the eligibility rule, the form that has to be filed. The exploratory ninety-nine never surface, the is-this-normal and the what-if-I-have-misread-this and the please-explain-that-again-but-plainly. Nearly all the value we can measure sits in that long tail, no single entry of which would have been worth seven pounds, and which in aggregate is a population managing its own affairs.
The cost curve has not finished falling, and the population able to ask will widen with it, mostly people no product team has met, asking things no FAQ anticipated. A lab's job is to establish whether the answers can be trusted before that happens rather than after, which is why our cost work and our reliability work are run together and read together. On current evidence the pilot clears that bar. The next order-of-magnitude fall in price will test it again, against a larger population, and we would rather find the failure ourselves than have it found by the people who can least afford it.