Technical Note—Cyclic Variables and Markov Decision Processes
Operations Research · 2020 · Volume 68 · Issue 4
Abstract
In this paper I develop a cyclic value function iteration, which is an adjustment to the standard value function iteration. When using this algorithm, the inclusion of cyclic variables of any size into the state space of an infinite horizon Markov decision process does not increase the computational complexity of solving for the value function. This result is proven theoretically and shown to closely hold in practice using Monte Carlo simulations.
Keywords
dynamic programming/optimal control, infinite state, Markov, economics, econometrics, marketing, estimation/statistical techniques, Optimization, Markov decision process, structural dynamic models, infinite-horizon dynamic programming, econometrics, cyclic variables, seasonality
Enable JavaScript for citation tools and additional paper interactions.