WeAct selects LoRA adapters as actions for long-horizon agents
Original title: WeAct: Dynamic Weight Switching as Agent Actions
On Qwen3-32B, WeAct beats full fine-tuning by 6.2% and static LoRA merging by 27.5% on eight deep-research benchmarks.
WeAct trains independent capability-specific LoRA adapters on a frozen backbone and adds adapter selection to the agent's action space via native tool calling. The method is meant to avoid catastrophic forgetting when long-horizon agents need different skills at different stages. On eight deep-research benchmarks, Qwen3-32B with WeAct improves average accuracy by 6.2% relative to full fine-tuning while using less compute, and by 27.5% relative to the strongest static LoRA-merging method. New code and writing adapters can be added or updated without changing the backbone or other adapters.
- Adapters stay independent over a frozen backbone rather than a fixed merged configuration.
- Selection uses the model's native tool-calling interface in a shared context.
- A new code-computation adapter and an updated writing adapter leave other adapters unchanged.
- Gains are reported relative to full fine-tuning and static LoRA merging.
From the original article
1 min read at arxiv.orgLong-horizon agents must learn heterogeneous capabilities from interleaved trajectories. Updating shared parameters to learn new capabilities can degrade previously acquired ones, resulting in catastrophic forgetting. As a trajectory unfolds, different stages may require different capabilities, yet conventional joint training and static adapter merging maintain a fixed parameter configuration throughout execution.
By Rui Shi · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.