CARE-VI: Conservative Adaptive Reliability Estimation for Value Improvement in Off-Policy Actor-Critic Learning
CARE-VI introduces CARS, SEVA, and DARE components for reliability-aware value targets, achieving highest mean returns across twelve SAC, TD3, and TD7 settings.
CARE-VI is a framework for reliable temporal-difference target construction in off-policy actor-critic learning, combining Conservative Adaptive Ranking and Screening (CARS), Selector-Evaluator Value Assessment (SEVA), and Dynamic Adaptive Risk-aware Enhancement (DARE). The paper bounds boundary error and selected-value overestimation and proves fixed-policy recovery after finite-stage perturbations. Across SAC, TD3, and TD7 on four MuJoCo tasks, it achieves the highest mean return in all twelve experimental settings.
- CARS retains an ordered candidate action prefix within a preset budget, narrowing only on large boundary gaps
- SEVA orders candidates with selector critics and caps reviewed values at the selector reference
- DARE scales residual corrections using candidate reliability, selector-evaluator gaps, and a finite stage factor
- Highest mean return in all twelve SAC, TD3, and TD7 MuJoCo experiment settings
- Products
- CARE-VI
From the original article
1 min read at arxiv.orgReliable temporal-difference targets are central to off-policy actor-critic learning. Direct value improvement refines the next-state target with alternative actions, but the reliability of this refinement depends on how candidate actions are ranked, reviewed, and weighted.
By Xiang Zou · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.