Skip to main content

Multiscale Stochastic Approximation for Parametric Optimization of Hidden Markov Models

  • Shalabh Bhatnagar (a1) and Vivek S. Borkar (a2)

A two–time scale stochastic approximation algorithm is proposed for simulation-based parametric optimization of hidden Markov models, as an alternative to the traditional approaches to “infinitesimal perturbation analysis.” Its convergence is analyzed, and a queueing example is presented.

Hide All
1.Borkar, V.S. (1990). The Kumar-Becker-Lin scheme revisited. Journal of Optimization Theory and Application 66: 287309.
2.Borkar, V.S. (1993). On white noise representations in stochastic realization theory. SIAM Journal on Control and Optimization 31: 10931102.
3.Borkar, V.S. (1995). Probability theory: An advanced course. New York: Springer-Verlag.
4.Borkar, V.S. & Phansalkar, V.V. (1994). Managing inter processor delays in distributed recursive algorithms. Sadhana 19(6): 9951003.
5.Chong, E.K.P. & Ramadge, P.J. (1993). Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times. SIAM Journal on Control and Optimization 31(3): 698732.
6.Chong, E.K.P. & Ramadge, P.J. (1994). Stochastic optimization of regenerative systems using infinitesimal perturbation analysis. IEEE Transactions on Automated Control 39(7): 14001410.
7.Dai, L. & Ho, Y.-C. (1995). Structural infinitesimal perturbation analysis (SIPA) for derivative estimation of discrete-event dynamic systems. IEEE Transactions on Automatic Control 40(7): 11541166.
8.Gelfand, S.B. & Mitter, S.K. (1992). Recursive stochastic algorithms for global optimization in Rd. SIAM Journal on Control and Optimization 29: 9991018.
9.Hirsch, M.W. (1987). Convergent activation dynamics in continuous time networks. Neural Networks 2: 331349.
10.Ho, Y.-C. & Cao, X.-R. (1991). Perturbation analysis of discrete event dynamical systems. Boston: Kluwer.
11.Kushner, H.J. & Clark, D.S. (1978). Stochastic approximation methods for constrained and unconstrained systems. New York: Springer-Verlag.
12.Kushner, H.J. & Vazquez-Abad, F.J. (1994). Stochastic approximation methods for systems of interest over an infinite horizon. LCDS 94–4: 156.
13.Neveu, J. (1975). Discrete parameter martingales. Amsterdam: North-Holland.
14.Parthasarathy, K.R. (1967). Probability on metric spaces. New York: Academic Press.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Probability in the Engineering and Informational Sciences
  • ISSN: 0269-9648
  • EISSN: 1469-8951
  • URL: /core/journals/probability-in-the-engineering-and-informational-sciences
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 0
Total number of PDF views: 15 *
Loading metrics...

Abstract views

Total abstract views: 130 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 19th July 2018. This data will be updated every 24 hours.