DFQI transfers knowledge for offline Q-learning
Original title: Direct Knowledge Transfer for Offline Q-Learning
The method integrates source data via L1 penalty in target Q-function updates to improve policy learning with limited target data in marketing and healthcare.
Data-driven sequential decision making benefits from related environment observations when target data is limited. Existing transfer methods struggle with reward and transition differences. The paper introduces Direct Transfer Fitted Q-Iteration (DFQI) that updates target Q-function while borrowing source information through L1 penalty linking coefficients. Statistical guarantees are established for stationary discounted environments using sieve approximation.
- Directly integrates source information into target value-function learning
- Uses L1 penalty to link target and source coefficients
- Determines both jointly within each Bellman update
- Applies to common or heterogeneous transition kernels
From the original article
1 min read at arxiv.orgData-driven sequential decision making in marketing and healthcare can benefit from observations collected in related environments, particularly when data in the environment of interest are limited. Transferring such information requires accounting for differences in rewards and transition dynamics across environments.
By Zhanye Luo · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.