Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Design
Designer-RSI grows design-agent procedural memory from user traffic, raising Claude Sonnet 4 GenEval2 success to 99.3%.
Designer-RSI keeps a frozen frontier model operating professional design software through more than 230 tools while an external procedural memory of natural-language skills accumulates from experience. Five rounds over 1,406 real user briefs and 1,869 graded trajectories, with no weight updates or human labels, grow the skill bank from 76 to 139. GenEval2 execution success on Claude Sonnet 4 rises from 72.7% to 99.3%, with win rates of 61.8% and 67.6% against a no-skill agent on Claude Sonnet 4 and Claude Opus 4.6. Combining skill widening and deepening reaches a 58.5% win rate on 200 held-out briefs.
- Frozen model uses over 230 design tools; memory stores natural-language skills.
- Five rounds grow the skill bank from 76 to 139 without weight updates.
- Claude Sonnet 4 GenEval2 success rises from 72.7% to 99.3%.
- Widening plus deepening beats either mechanism alone on 200 held-out briefs.
Full article219 words · extracted from arxiv.org · click to collapse
Professional graphic design is a long-horizon agentic task in which structured, editable artifacts emerge from many interdependent actions, yet outcomes admit no reliable programmatic oracle. We introduce a continual adaptation framework in which a frozen frontier model operates professional design software through more than 230 tools, while an external procedural memory of natural-language skills accumulates and refines reusable design procedures from experience. The memory widens by acquiring procedures for recurring uncovered subtasks and deepens by revising existing procedures against their own successful and failed executions, while a matched replay gate admits only changes that repair failures without regressing observed successes. Five rounds over 1,406 real user briefs and 1,869 automatically graded trajectories, with no weight updates and no human labels, grow the bank from 76 documentation-derived skills to 139 and raise GenEval2 execution success on Claude-Sonnet-4 from 72.7% to 99.3% (+11.99 points in generation quality), with 61.8% and 67.6% win rates against the no-skill agent across four specialized design benchmarks on Claude-Sonnet-4 and Claude-Opus-4.6. We further show the two mechanisms are effective in combination: on 200 held-out briefs from user-traffic benchmark, widening or deepening alone reaches a 49.4% / 48.6% win rate over the no-skill agent, while their combination reaches 58.5% (p = 0.025). Procedural memory offers a practical route to continual adaptation of agents under noisy, unverifiable feedback.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.22086