Nokia Open-Sources AnyJev: A Training-Free Layer That Turns Any Open LLM Into a Calibrated Decision Model
Nokia open-sources AnyJev, a training-free library that converts any open LLM into a calibrated decision model for classification tasks.
Nokia's applied research team has open-sourced AnyJev, a training-free Python library that converts open LLMs into calibrated decision models. The library uses cyclic shifts and batch prior correction to eliminate position and prior bias in next-token probabilities. Benchmarks on Qwen3-8B with the BANKING77 task show the order-flip rate drops from 0.230 to 0.073, and auto-decidable traffic at 5% error rises from 7.7% to 52.0% with the L1 calibration level.
- Nokia open-sourced AnyJev, a Python library turning any open LLM into a decision model.
- It uses cyclic shifts and batch prior correction to remove position and label bias.
- On Qwen3-8B BANKING77, it reduced order-flip rate from 0.230 to 0.073.
- Auto-decidable traffic at 5% error rose from 7.7% to 52.0% with L1 calibration.
- It supports Hugging Face and vLLM backends with shared-prefix scoring.
Full article808 words · extracted from marktechpost.com · click to collapse
Nokia’s applied research team has open-sourced AnyJev, a Python library that turns an open LLM into a decision model. It needs no training. It targets a common production job: picking one answer from a fixed set instead of writing a sentence.
Is it deployable? Yes, it installs from PyPI, ships under Apache-2.0, and has transformers and vLLM backends with shared-prefix scoring.
What is AnyJev?
AnyJev borrows its interface from Jev. Jev is the System One decision model that TypeSafe AI launched in September 2026 (our coverage). You give AnyJev a typed question and get back a decision with a probability you can threshold. That probability is read from the model’s next-token distribution. Nothing is generated, parsed, or trained.
The library supports 3 question types:
- A
choicequestion picks one of K options. - A
noulquestion is yes or no. - A
scorequestion places the answer in one of several ordered bins.
The Problem With Reading Logits Directly
Many open projects already restrict the next token to the option labels and read the scores. The Nokia research team flags 2 flaws in that shortcut. First, the answer can change when the options are reordered. Second, the probabilities are not calibrated.
The levels doc names 2 causes:
- The first is prior bias: the model favors some labels, such as “Yes” over “No”, whatever the input.
- The second is position bias: the model favors certain slots in the option list.
How AnyJev Works: L0 and L1
Every decision carries a level field.
L0 (zero labels, on by default) applies 2 fixes:
- Cyclic shifts. For a question with K options, the list is shown in K rotations, so every option appears in every position once. The results are combined in log space as a geometric mean. If the position bias is additive in logit space, this removes it exactly.
- Prior correction. By default, AnyJev uses batch calibration. It keeps a running mean of the predicted distributions on real inputs and divides it out at strength 0.75. The correction starts after 8 items.
L0 costs K prefills per decision, batched over a shared prefix. That is about 0.25 s per decision at batch 32 on one H100, with K = 20.
L1 (100 to 500 labels per question) adds temperature scaling on top of L0. The fitted values are saved as a small JSON artifact. L1 reshapes confidence but does not change the ranking of answers.
Benchmark Results
On Qwen3-8B with BANKING77 (20-way, 300 test items), the numbers look like this:
| Metric | Raw logits | AnyJev L0 | AnyJev L1 |
|---|---|---|---|
| Labels required | 0 | 0 | 100 to 500 |
| Flip rate when options reversed | 0.230 | 0.073 | 0.077 |
| Accuracy | 0.747 | 0.803 | 0.807 |
| Calibration error (ECE) | 0.240 | 0.184 | 0.095 |
| Auto-decidable at 5% error | 7.7% | 46.3% | 52.0% |
A few other results from the repo:
- L0 reduced order flips on all 9 model and task rows tested.
- On a typed-decisions set, Qwen3-32B with L1 reached an ECE of 0.036, compared with 0.144 published for Jev. On accuracy, the fine-tuned Laya still leads.
- The full ablation table covers Qwen, OLMo, Granite, Phi and Mistral models.
- Wu says the team tried AnyJev on an internal Nokia routing problem and saw promising results.
How to Use AnyJev
# pip install "anyjev[hf]"
from anyjev import Decider, Question
from anyjev.backends.hf import HFBackend
d = Decider(HFBackend("Qwen/Qwen3-8B"))
route = Question.choice("Which team should handle this?",
["billing", "technical", "sales", "other"], name="route")
r = d.decide({"conversation": [...]}, [route])
r["route"].distribution # probabilities per optionFor serving, you start vLLM with prefix caching and point a VLLMBackend at it.
Key Takeaways
- AnyJev turns open LLMs into Jev-style typed decision models with no training.
- L0 uses cyclic shifts and a batch prior to remove position and label bias.
- On Qwen3-8B BANKING77, the order-flip rate falls from 0.230 to 0.073.
- Auto-decidable traffic at 5% error rises from 7.7% to 52.0% with L1.
- It is Apache-2.0 on PyPI, with Hugging Face and vLLM backends.
Check out the GitHub Repo. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
Asif Razzaq
Asif Razzaq is the CEO of Marktechpost AI Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.