Skip to main content

Strong consistency of a modified maximum likelihood estimator for controlled Markov chains

  • Bharat Doshi (a1) and Steven E. Shreve (a2)

A controlled Markov chain with finite state space has transition probabilities which depend on an unknown parameter α lying in a known finite set A. For each α, a stationary control law ϕ α is given. This paper develops a control scheme whereby at each stage t a parameter α t is chosen at random from among those parameters which nearly maximize the log likelihood function, and the control ut is chosen according to the control law ϕ αt. It is proved that this algorithm leads to identification of the true α under conditions weaker than any previously considered.

Corresponding author
Postal address: HP1B323, Holmdel, NJ 07733, U.S.A. Research carried out when the author was at Rutgers University.
∗∗ Postal address: Department of Mathematics, Carnegie-Mellon University, Pittsburgh, PA 15213, U.S.A.
Hide All

Research sponsored in part by the Air Force Office of Scientific Research (AFSC), United States Air Force, under Contract F-49620–79-C-0165.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Applied Probability
  • ISSN: 0021-9002
  • EISSN: 1475-6072
  • URL: /core/journals/journal-of-applied-probability
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Full text views

Total number of HTML views: 0
Total number of PDF views: 6 *
Loading metrics...

Abstract views

Total abstract views: 69 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 17th August 2018. This data will be updated every 24 hours.