In short
- The model you tested isn't necessarily the one you're serving a few months later.
- Tone and helpfulness often change before accuracy does.
- A frozen monthly test on the exact version you serve catches most changes early.
Testing a model before launch tells you how that version behaved on that day. After launch, the system around it keeps moving, and often the model does too.
What changes
- Vendor updates. Hosted models are updated, aliases such as "latest" move to new versions automatically, and older snapshots are eventually retired.
- Prompt edits. Teams adjust the system prompt to fix one problem and change behaviour somewhere else.
- Retrieval and tools. A new document source, search index or tool changes what the model sees, and therefore what it says.
- Routing and cost changes. Moving traffic to a cheaper model, or serving a compressed version, can change behaviour without anyone touching the prompt.
- Fine-tuning. Training on new data changes more than the behaviour you were aiming at. Research has shown that fine-tuning can weaken a model's safety behaviour even when the data looks harmless (Qi and colleagues, 2023).
Each change is usually reviewed by someone against their own goal. Few teams review the combined effect against the standard the system was approved for.
What drifts first
Accuracy is often the last thing to move. Tone, length, hedging and directness tend to shift earlier: answers get longer, add more caveats, or put the actual answer further down. For someone reading on a phone in a hurry, that can matter as much as a factual error, so measure manner as well as correctness.
How to catch it
- Keep a frozen test set. A few hundred representative questions, kept private so they can't leak into training data.
- Run it on a schedule. Monthly is a sensible default, plus after any change you make. Use fixed settings and the exact model version you serve.
- Compare over time. Track accuracy by task, harmful answers, refusals, answer length, and how often the answer comes first. Look for sudden steps as well as slow trends.
- Record versions. Log the model snapshot, prompt version and settings on every call, so a change in results can be traced to a change in the system.
- Watch the fallback. A sudden change in how often people ask for a human is often the first sign of trouble.
Put it in the contract
If you buy a model, or a service built on one, ask for advance notice of changes to the model, the system prompt and the routing, and for the ability to pin a version. The EU AI Act already treats post-market monitoring as an ongoing duty for high-risk systems. Budget for monitoring as a recurring cost, not a one-off test at procurement.