Skip to main content

Multiscale Stochastic Approximation for Parametric Optimization of Hidden Markov Models

  • Shalabh Bhatnagar (a1) and Vivek S. Borkar (a2)

A two–time scale stochastic approximation algorithm is proposed for simulation-based parametric optimization of hidden Markov models, as an alternative to the traditional approaches to “infinitesimal perturbation analysis.” Its convergence is analyzed, and a queueing example is presented.

Hide All
1.Borkar V.S. (1990). The Kumar-Becker-Lin scheme revisited. Journal of Optimization Theory and Application 66: 287309.
2.Borkar V.S. (1993). On white noise representations in stochastic realization theory. SIAM Journal on Control and Optimization 31: 10931102.
3.Borkar V.S. (1995). Probability theory: An advanced course. New York: Springer-Verlag.
4.Borkar V.S. & Phansalkar V.V. (1994). Managing inter processor delays in distributed recursive algorithms. Sadhana 19(6): 9951003.
5.Chong E.K.P. & Ramadge P.J. (1993). Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times. SIAM Journal on Control and Optimization 31(3): 698732.
6.Chong E.K.P. & Ramadge P.J. (1994). Stochastic optimization of regenerative systems using infinitesimal perturbation analysis. IEEE Transactions on Automated Control 39(7): 14001410.
7.Dai L. & Ho Y.-C. (1995). Structural infinitesimal perturbation analysis (SIPA) for derivative estimation of discrete-event dynamic systems. IEEE Transactions on Automatic Control 40(7): 11541166.
8.Gelfand S.B. & Mitter S.K. (1992). Recursive stochastic algorithms for global optimization in Rd. SIAM Journal on Control and Optimization 29: 9991018.
9.Hirsch M.W. (1987). Convergent activation dynamics in continuous time networks. Neural Networks 2: 331349.
10.Ho Y.-C. & Cao X.-R. (1991). Perturbation analysis of discrete event dynamical systems. Boston: Kluwer.
11.Kushner H.J. & Clark D.S. (1978). Stochastic approximation methods for constrained and unconstrained systems. New York: Springer-Verlag.
12.Kushner H.J. & Vazquez-Abad F.J. (1994). Stochastic approximation methods for systems of interest over an infinite horizon. LCDS 94–4: 156.
13.Neveu J. (1975). Discrete parameter martingales. Amsterdam: North-Holland.
14.Parthasarathy K.R. (1967). Probability on metric spaces. New York: Academic Press.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Probability in the Engineering and Informational Sciences
  • ISSN: 0269-9648
  • EISSN: 1469-8951
  • URL: /core/journals/probability-in-the-engineering-and-informational-sciences
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 0
Total number of PDF views: 12 *
Loading metrics...

Abstract views

Total abstract views: 107 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 22nd February 2018. This data will be updated every 24 hours.