Semantic Action Graph: A Shared Representation for Agent Grounding and Human Interpretation of Sports Highlights
SportSAGE uses a semantic action graph schema to ground agent-generated sports highlights and let viewers query, navigate, and verify narratives.
The semantic action graph represents a sports match as performer, action, recipient, moment, and state nodes connected by role, temporal, and outcome edges, with a closed vocabulary and frame-addressable moments. Instantiated in SportSAGE, a design probe pairing a four-module agentic highlight pipeline with a graph interface, it was evaluated with 12 soccer fans. Participants were satisfied with generated highlight quality and used the interface to search, navigate, and interpret match highlights, suggesting one small human-readable schema can ground both agent generation and human interpretation.
- Closed-vocabulary graph with frame-addressable moments supports verification
- Pairs four-module agentic highlight pipeline with queryable interface
- Evaluated via feedback from 12 soccer fans
- Same schema serves agent grounding and human interpretation
Full article184 words · extracted from arxiv.org · click to collapse
Generative agents are increasingly used to select and narrate video highlights, but they typically operate over unstructured or frame-level representations. Their output is consequently difficult for a viewer to verify and steer toward individual preferences. We present the semantic action graph, a lightweight domain schema that represents a sports match as performer, action, recipient, moment, and state nodes connected by role, temporal, and outcome edges. The schema demonstrates three key properties: 1) connected event sequences, 2) a shared, closed vocabulary, and 3) frame-addressable moments, making it suitable to serve two consumers at once: an agentic pipeline that composes narrated highlights, and a visual interface through which viewers query and inspect the same structure. We instantiate it in SportSAGE, a design probe pairing a four-module highlight pipeline with a graph interface, and report feedback from 12 soccer fans. Participants were satisfied with the quality of the generated highlights and narratives, and used the graph interface to search, navigate, and interpret the match highlights. These results provide early evidence that one small, human-readable schema can ground agent generation and support human interpretation at the same time.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.20768