This is the practical companion to R-09. It assumes you have staged access to the weights and want the model doing useful work by the end of the day.
1. What you are actually deploying
Kaer-R1 is two things served together: a 7B dense reasoner, and a small classifier — the halting head — that reads the model's hidden state at nine checkpoints through a reasoning trace and predicts whether continuing will change the final answer. Inference stops when that probability drops below a threshold you set.
If you deploy only the base model and ignore the head, you have a mediocre 7B reasoner. The head is the product.
2. The three decisions that matter
| Decision | What it controls | Sensible starting point |
|---|---|---|
| Halting threshold | When the model stops thinking | Start at the released default, then tune on your traffic |
| Abstention cut-off | What escalates to a human or larger model | Bottom decile of halting score |
| Maximum budget | Worst-case latency per request | 4,096 thinking tokens |
The first two are separate numbers that both come from the halting score, and conflating them is the most common setup mistake we see. Stopping early is a cost decision. Escalating is a risk decision. They should not share a threshold because they do not share a consequence.
3. Wire up the fallback path first
Before you tune anything, build the escalation route: a human queue, a larger model, or both. A deployment with no fallback cannot use the abstention signal, which means it cannot use the main reason to choose this model.
Then treat fallback usage as a health metric to watch, not a cost to minimise. A pilot in which nobody ever escalates to a person would not reassure us; it would worry us.
4. Instrument these four things from day one
- Cost per solved task — what you paid divided by items answered correctly. Not accuracy. This is the number that will justify or kill the deployment.
- Halting score distribution — watch for a second hump. A bimodal histogram where there should be one mode means two things are being counted as one, which is how we lost six weeks once (post-mortem).
- Escalation rate — if it drifts down over time without a change on your side, something moved.
- Response length — cheap to log, and the earliest warning that a serving-side change altered behaviour.
5. Mistakes we see in the first week
- Quantising and not re-measuring. Our own 8-bit build scored within noise on capability and measurably worse on hedging and refusal manner. The cheap version of the model is a different model. Re-run your evaluation on the build you actually serve.
- Testing with tidy questions. Benchmark-shaped inputs are not what your users send. Test with real, badly-worded, compound, underspecified traffic or you have measured nothing relevant.
- Setting the threshold once. Traffic mix moves. Re-tune quarterly, and re-tune after any change to your prompt, your retrieval layer or your model build.
- Treating abstention as failure. An abstention that routes correctly is the system working. The metric to minimise is confident wrong answers, not declines.
- Serving it to the public. It has no safety hardening beyond the base model. Behind an evaluation harness on bounded tasks, yes. In front of strangers, no.
6. Before you go live
Run a behavioural read on your own traffic and look at the harmed cases specifically — answers confidently wrong in ways that could cost someone money or a missed deadline. Across models with near-identical headline accuracy we have measured harmed-rates spread from 4% to 19%. Your number is not our number, and it is the one that matters.
Questions, or a request for the evaluation harness: [email protected].