Stable Dynamic Programming and Reinforcement Learning with Dual Representations

dc.contributor.author	Wang, Tao
dc.contributor.author	Schuurmans, Dale
dc.contributor.author	Bowling, Michael
dc.contributor.author	Lizotte, Daniel
dc.date.accessioned	2025-05-01T21:12:55Z
dc.date.available	2025-05-01T21:12:55Z
dc.date.issued	2007
dc.description	Technical report TR07-05. We investigate novel, dual algorithms for dynamic programming and reinforcement learning, based on maintaining explicit representations of stationary distributions instead of value functions. In particular, we investigate the convergence properties of standard dynamic programming and reinforcement learning algorithms when they are converted to their natural dual form. Here we uncover advantages for the dual approach: dual update algorithms, since they are based on estimating normalized probability distributions rather than unbounded value functions, avoid divergence even in the presence of function approximation and off-policy updates. Moreover, dual update algorithms remain stable in situations where standard value function estimation diverges. \| TRID-ID TR07-05
dc.identifier.doi	https://doi.org/10.7939/R33G0M
dc.language.iso	en
dc.rights.uri	http://creativecommons.org/licenses/by/3.0/
dc.subject	Reinforcement learning
dc.subject	Dual representations
dc.subject	Reinforcement Learning
dc.title	Stable Dynamic Programming and Reinforcement Learning with Dual Representations
dc.type	http://purl.org/coar/resource_type/c_93fc
ual.jupiterAccess	http://terms.library.ualberta.ca/public

Files

Now showing 1 - 1 of 1