Annotrace: A Workbench for Step-Level Annotation of Agent Traces
Annotrace labels agent traces at step level and reports 91% model agreement on 235 units.
Stories per day · 30 days
1 RSS
8 stories, newest first · one card per story
Stories tagged #annotation
Studying Comics at Scale with Artificial Intelligence: The PAGE Framework and Three Studies of American Comics and Japanese Manga
Researchers propose PAGE, an AI framework that parses and annotates thousands of American comics and Japanese manga.
It's Not What the Image Shows: Irrelevant Context Destabilises VLM Judges Without Informing Them
MIST shows irrelevant images change VLM judge labels about as often as aligned images, without improving agreement.
Optimal Sequential Annotations for Off-Policy Evaluation
Paper optimizes sequential annotation budgets for doubly robust off-policy evaluation of complex outcomes.
onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction
onPanda cuts median alignment annotation time 52% via token-level correction that keeps data on-policy for SFT and preference training.
onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction
onPanda corrects the first bad token and regenerates, cutting median alignment-annotation time by 52%.
Structured Claim-Level Discourse Representations for Dense Health Narratives
Researchers propose a claim-level discourse framework for health videos, finding 13.22 atomic claims per minute and that LLMs struggle with pragmatic profiling.
Zoom Annotation Flaws Could Let a Meeting Participant Hijack Another Attendee's Client
A Security researchers disclosed three Zoom annotation flaws enabling zero-click client hijacking; Zoom shipped fixes in June and July with no exploitation reported.
1