Short answer: If you need weights on your own hardware, Gemini is out — it is API-only, and Google's open-weight line is Gemma, not Gemini. That leaves Llama-class releases, Gemma, and specialised small models like Kaer-R1. Llama wins on breadth and ecosystem. Kaer-R1 wins on cost per solved task for bounded reasoning and on exposing a usable abstention score.
First, the licence question, because it decides everything else
"Open" is doing a lot of work in most comparisons. Three distinct things get called it:
| Category | What you get | Examples |
|---|---|---|
| API-only | Inference through a vendor endpoint. No weights. | Gemini, GPT-5, Claude |
| Open weights | Downloadable parameters under a bespoke licence. Usually no training data. | Llama, Gemma, Kaer-R1 |
| Open source | Weights plus data plus training code under an OSI licence. | Rare at scale |
If your constraint is that data cannot leave your infrastructure — the usual constraint in health, welfare and public-sector work — only the second and third rows exist for you, and the choice is made before any benchmark is consulted.
Compared, for a bounded reasoning workload
| Kaer-R1 7B | Llama-class open weights | Gemini | |
|---|---|---|---|
| Weights available | Yes, staged access | Yes, community licence | No — API only |
| Single-GPU serving | Yes | Small variants yes; large no | N/A |
| Breadth of capability | Narrow by design | Broad, strong ecosystem | Broad, frontier class |
| Adaptive compute | Learned per question | Operator-set budget | Operator-set budget |
| Exposed abstention score | Yes — halting head | No | No |
| Fine-tuning ecosystem | Minimal — research release | Extensive, mature tooling | Vendor tuning only |
| Evaluation card | Published, incl. failed variants | Vendor model card | Vendor model card |
| Cost / 1k hard-reasoning items (ours) | £3.10 measured | Depends on size and host | List pricing |
How to read the numbers. Kaer figures are our own, from R-09 and R-08. Third-party figures are published specifications and licences, or our own measurements under the stated protocol with the run date attached. No vendor supplied or reviewed them.
What the 7B actually buys you
The number worth carrying from R-09 is not the suite score. It is that adaptive halting beat the same model at a fixed 4,096-token budget — 61.4% against 58.1% — while using a fifth of the tokens. More thinking is not monotonically better. In our training corpus 6% of problems were non-monotone: right at 192 tokens, wrong at 1,536, because a longer chain talked the model out of a correct first instinct.
That effect is available to any model with a halting mechanism and unavailable to one running a fixed budget, regardless of parameter count. It is the clearest argument we have that architecture choices in the small-model range are not simply a poor imitation of scale.
Where Llama-class models are the better answer
Most of the time, honestly. If you need general assistance, multilingual coverage, an established fine-tuning toolchain, community-maintained quantisations, or anything approaching agentic tool use, a mature open-weight family is the correct default and Kaer-R1 is not competitive. Our own data-ablation work is built on an existing open-weight base; we did not pre-train from scratch and we are not going to pretend otherwise.
The measurement point nobody makes
Whichever you choose, do not choose it on a public leaderboard. Our paraphrase study (R-07) found a 19-point median accuracy spread across twelve phrasings of the same question — against an 11-point gap between adjacent-tier models. Wording moves the score more than changing model does. A leaderboard position measured on one fixed phrasing is a poll of one person.
Run the candidates against your own traffic, with your own phrasings, and score what happens when they get it wrong. That is a £40 exercise and it will contradict the leaderboard often enough to be worth the afternoon.