Since last September we have put the same 1,850 probes to a single production assistant, once a month, on behalf of the public-service partner that runs it. The partner agreed to this. The vendor behind the model does not know it is being watched, which is deliberate. Our probes appear in no published benchmark, so they cannot leak into anyone's training set. The seeds and sampling parameters are frozen. So is the scoring rubric. If a number moves, the system moved, not the instrument.
Eight months is not a long time. It is long enough that three findings now feel solid enough to write down.
It changed nine times. We were told about three.
Our change-point detection runs PELT across 40 per-domain accuracy series and 12 style metrics. Over the window it finds nine statistically confident breaks.
| Over eight months | Count |
|---|---|
| Confident change-points (PELT, our 40 accuracy and 12 style series) | 9 |
| Coinciding with a vendor-announced update | 3 |
| Unannounced, nothing public to point to | 6 |
Two of those six moved a domain by more than four points, one up and one down. None of this should surprise anyone who read the 2023 Stanford and Berkeley study on how ChatGPT's behaviour shifted between snapshots, or who remembers the stretch in December 2023 when GPT-4 went noticeably lazy and OpenAI acknowledged the regression. Every integration is built on ground the vendor can move without telling you. The convenient "-latest" alias is also a standing agreement to take whatever ships next, and deprecation schedules eventually retire the exact version you validated.
We are not alleging anything underhand. Silent prompt edits and routing changes are ordinary operations, usually harmless and invisible from outside. But if a compliance story rests on the words "we validated the system at procurement", then six unannounced behavioural breaks in eight months is the figure that belongs beside it.
Competence held. Manner did not.
Aggregate task accuracy across the eight months is flat within noise, ±1.3 points around a mean of 71. On the numbers a capability benchmark reports, nothing happened. Style is another matter, and it moved in one direction throughout.
- Median answer length is up 23% on month one, with no matching gain in accuracy.
- Hedging phrases per answer, the "it's worth noting" and "generally speaking" filler, roughly doubled.
- Our direct-answer-first rate, whether the answer lands in the opening sentence, fell from 64% to 51%.
Most people reaching this service are stressed and reading on a phone. For them an answer in the first sentence is not decoration. No capability benchmark would have caught any of this, because none of it touches correctness, and that is the argument for treating manner as a measured quantity rather than a matter of taste. The data made that case better than our assertions had.
In month five, the date arithmetic broke.
One family of probes asks date questions: if the notice period is 21 days from last Tuesday, when does it lapse? That family fell from 83% correct to 61% in month five and returned to form in month six. Nothing else moved with it. We cannot see inside the vendor's stack, so we will not invent a root cause, but a failure that narrow and that reversible looks more like a changed tool-call format or a parser regression than a new model.
The partner's own complaint logs for that month carry a small rise in deadline-related follow-up calls. Small enough that nobody inside the service had connected it to anything. Real enough to find once you have a baseline to measure against. In month four nobody was measuring, so it would have been invisible.
Without monitoring, this is precisely the sort of thing that gets missed. New York's MyCity business chatbot was reported in 2024 to be giving advice that would, in places, have had businesses breaking the law. In January 2024 DPD's support bot swore at a customer after an update nobody had signed off. Those reached the news because they were vivid. A hedging rate that rises slowly, or one broken month of date maths, will never trend. It surfaces only if someone holds the baseline fixed and keeps checking against it.
Method, limits, and one refusal
Each monthly pulse runs over a 72-hour window at fixed local times, which controls for any time-of-day routing. Scoring is blind against the frozen rubric. The human-adjudication queue takes 5 to 8% of items in a given month, and it carries most of the cost: a full pulse comes to about £210, human time included.
The limits are real, and I would rather set them down than have them pointed out. This is one assistant at one vendor, across a single domain mix and eight months. We have since stood up parallel traces on two further surfaces, one from our March run onward and one more recent, and the early shape repeats: flat competence, drifting manner. But n=1 stretched over eight months is a case study, not a law. Not every expectation held, either. We went in braced for weekend or overnight degradation from load-based routing, and across the fixed-time windows we have not found it. The daily cycle, so far, is quiet.
We were asked, more than once, to name the vendor. We will not, and the reason is method rather than manners. The moment monitoring turns into adversarial PR, the vendor optimises against the probes and the instrument no longer measures anything real. What we will do instead is give the full protocol to any institution that wants to run its own trace before its own procurement decision. That offer, not the findings, is the point of the study.
Regulation is moving the same way. The EU's AI Act places post-market monitoring duties on high-risk systems, phased across 2026 and 2027, and it treats ongoing measurement as an obligation rather than a courtesy. UK procurement still tends to validate once, and standard contracts rarely oblige a vendor to flag a behavioural change. So for eight months, on this one assistant, we gave the partner the change notices their contract never obliged the vendor to send. Nobody else was in a position to.