A single function Jev-like wrapper for LLMs, including vision models
A developer blog shows a Jev-style logprob wrapper that scores webcam frames with text and vision LLMs.
A developer blog describes a small Python wrapper that turns LLM token log probabilities into fast structured answers, including for vision models. The prompt asks for a single option letter while requesting top logprobs, then maps those letters to choices, booleans, or scores. The author adds image attachments to a Jev-like JSON format, sends webcam JPEGs, and asks whether a person or plant is visible, whether the scene is indoors, and how bright it is. Locally, Gemma 4 12B on an RTX 3090 processed about one frame per second across three questions, while OpenAI gpt-6-luna reached about 0.2 FPS in the same setup.
- Single-token answers plus logprobs score choice, boolean, and ordinal questions.
- Image attachments extend a text-only Jev request format for vision models.
- Gemma 4 12B reached about 1 FPS locally; gpt-6-luna about 0.2 FPS.
- The example targets llama.cpp chat completions and the OpenAI API.
- Vendors
- OpenAI
- Organizations
- OpenAI
- AI models
- Gemma 4 12Bgpt-6-luna
From the original article
6 min read at allanrbo.blogspot.com
I was intrigued by Jev and the self-hostable projects appearing around it, such as OpenJev and SemIf . Reading about them introduced me to a neat trick: reading an LLM's token probabilities. Apparently this is an old trick for some people.
Hacker News · AIBy allanrbo · allanrbo.blogspot.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.