Evaluating Verified Autonomy in Quantum Engineering
Quantum-Harbor lab and QIQCBench (49 tasks) expose wide performance gaps across 17 frontier agentic systems in verified quantum engineering.
Researchers built Quantum-Harbor, a virtual laboratory providing a controlled execution environment where scientific AI agents interacting with quantum systems can have both actions and conclusions directly verified. QIQCBench contributes 49 expert-authored tasks spanning calibration and control, error correction and compilation, and sensing and networking. Across 17 frontier agentic systems, verified performance varied widely, exposing a substantial gap between demonstrated capability and reliable autonomous operation.
- Quantum-Harbor enables direct verification of agent actions and conclusions on quantum systems.
- QIQCBench offers 49 expert tasks covering calibration, error correction, compilation, sensing, networking.
- 17 frontier agentic systems show wide variation in verified performance.
- Highlights gap between demonstrating capability and achieving reliable autonomous operation.
Full article170 words · extracted from arxiv.org · click to collapse
Reliable quantum engineering is essential for turning quantum phenomena into practical technologies. As quantum platforms grow in scale and complexity, their characterization and operation require increasing human effort and coordination. Scientific artificial intelligence agents, which can plan experiments, operate instruments, and analyze observations, offer a promising route towards autonomous quantum engineering. Yet whether current agents can perform reliably in this setting has not been systematically established. To fill this gap, we developed Quantum-Harbor, a virtual laboratory that provides a controlled execution environment for agents to interact with quantum systems. This design enables direct verification of both the actions taken and the conclusions drawn. Building on this framework, we introduce QIQCBench, a benchmark of $49$ expert-authored tasks spanning multiple layers including calibration and control, error correction and compilation, sensing and networking. Across $17$ frontier agentic systems, QIQCBench reveals wide variation in verified performance. These results expose a substantial gap between demonstrating capability and achieving reliable operation, and establish Quantum-Harbor as a foundation for measuring progress towards verified autonomy in quantum engineering.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.17439