Sankhya: The Indian Journal of Statistics

2005, Volume 67, Pt. 1, 1--18

Existence of Optimal Markov Solutions for Ergodic Control of Markov Processes


Abhay G. Bhatt, Indian  Statistical Institute, Delhi, India
Vivek S. Borkar, Tata Institute of Fundamental Research, Mumbai, India

SUMMARY. For the ergodic control problem for a large class of controlled Markov processes in continuous time, existence of an optimal ergodic solution and an optimal, possibly time-inhomogeneous Markov solution, in both cases corresponding to a stationary Markov relaxed control policy, are known separately under suitable conditions (Bhatt and Borkar, 1996). The aim of this article is to refine this result to establish the existence of an optimal \textit{time-homogeneous} Markov solution. The proof is based upon Krylov's Markov selection procedure.

AMS (1991) subject classification. Primary 93E20.

Key words and phrases. Controlled Markov processes, ergodic control, Markov selection, optimal control.

Full paper (PDF)