Technical Note—Successive Approximations in Value Determination for a Markov Decision Process
Operations Research · 1987 · Volume 35 · Issue 5
Abstract
We present a new procedure for solving the value-determination equations for an undiscounted Markov decision process by the successive approximations of a Gauss-Seidel or Jacobi iteration method. The key step is to use the Markov chain partitioning algorithm to compute the steady-state probabilities corresponding to a given policy. The gain, computed in terms of the steady-state probabilities, is substituted into the value-determination equations. After deleting one relative value and one value-determination equation, we show that the remaining system of equations satisfies the sufficient conditions for the convergence of a Gauss-Seidel or Jacobi iteration.
Keywords
119 successive approximations in value determination
Enable JavaScript for citation tools and additional paper interactions.