Gricea: An Open Science Platform for Conversational AI Research
Gricea packages conversational-AI studies as reusable artifacts and shows most papers omit replication details.
Gricea is an open-science platform that represents conversational AI studies as configurable, deployable artifacts combining study procedures, participant-facing systems, and task behavior. In a replication effort, configurations from 93 percent of eligible CUI 2026 papers were reproduced, while 96 percent of papers lacked information needed for faithful replication. A user study found researchers and practitioners from varied backgrounds could build runnable studies for open-ended questions.
- Gricea packages procedures, interfaces, and task behavior as runnable artifacts.
- Configurations from 93 percent of eligible CUI 2026 papers were replicated.
- Missing information in 96 percent of papers blocks faithful replication.
- Researchers built runnable studies for open-ended conversational-AI questions.
Full article148 words · extracted from arxiv.org · click to collapse
We need studies on conversational AI (CAI) at scale to understand human behavior and shape CAI design. However, fragmented reporting of systems and study configurations hinders replication, extension, and knowledge accumulation. We present Gricea, an open-science platform representing studies as configurable, deployable research artifacts that researchers can run, inspect, share, and reuse. Informed by a formative analysis of prior CAI research, Gricea couples study procedures, participant-facing systems, and conversational task behavior in. In a replication study using Gricea, we replicated configurations 93% of eligible CUI 2026 papers; while also flagging missing information in 96% of papers that hinder faithful replication --- further motivating Gricea's need. In a user study, researchers and practitioners from diverse backgrounds successfully constructed runnable studies addressing various open-ended research questions. Together, these findings demonstrate Gricea's support for constructing, reproducing, and extending CAI studies through shared research artifacts, enabling cumulative knowledge building through open science.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.22039