One 1B decoder gates and writes memory-grounded conversation hints
Original title: When to Assist and What to Suggest: Joint Gating and Memory-Grounded Hint Generation in One Compact Decoder
End-to-end exact match is 0.2619 versus an oracle-gate ceiling of 0.2971 across 28,058 decision points.
A paper studies when a proactive assistant should speak and what hint to give, evaluating gating and generation together on 28,058 decision points. A single 1B decoder gates from an intermediate layer and generates in the same pass, matching or beating two-model pipelines on automatic end-to-end measures with 1.31-1.76x less wall-clock time than a matched 1B pipeline. Removing the memory profile costs 24.7 exact-match points, while retrieval and a larger generator add nothing on exact match.
- End-to-end precision behind a real gate is 0.2387.
- Compute savings from a gate fall from 4.39x with an 8B generator to 1.38x with a 1B one.
- Human raters gave 1,144 ratings over 32 dialogues.
- The single model rates level with the matched 1B pipeline and below the 8B model.
From the original article
1 min read at arxiv.orgProactive assistants offer short, memory-grounded hints during a conversation, deciding when to assist and what to suggest. Current systems split this between a gate and a larger generator; we evaluate both together on 28,058 decision points. Stage-wise generation scores are oracle-gate ceilings: behind a real gate, end-to-end exact match is 0.
By A. J. A. Jubayer Talukder · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.