Skip to main content

A two Timescale Stochastic Approximation Scheme for Simulation-Based Parametric Optimization

  • Shalabh Bhatnagar (a1) and Vivek S. Borkar (a2)

A two timescale stochastic approximation scheme which uses coupled iterations is used for simulation-based parametric optimization as an alternative to traditional “infinitesimal perturbation analysis” schemes. It avoids the aggregation of data present in many other schemes. Its convergence is analyzed, and a queueing example is presented.

Hide All
1.Bhatnagar S. & Borkar V.S. (1997). Multiscale stochastic approximation for parametric optimization of hidden Markov models. Probability in the Engineering and Informational Sciences 11: 509522.
2.Borkar V.S. (1995). Probability theory: An advanced course. New York: Springer Verlag.
3.Borkar V.S. (1997). Stochastic approximation with two time scales. Systems and Control Letters 29: 291294.
4.Chong E.K.P. & Ramadge R.J. (1994). Stochastic optimization of regenerative systems using infinitesimal perturbation analysis. IEEE Transactions on Automatic Control 39(7): 14001410.
5.Chong E.K.P. & Ramadge P.J. (1993). Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times. SIAM Journal on Control and Optimization 31(3): 698732.
6.Hirsch M.W. (1987). Convergent activation dynamics in continuous time networks. Neural Networks 2: 331349.
7.Kushner H.J. & Clark D.S. (1978). Stochastic approximation methods for constrained and unconstrained systems. New York: Springer Verlag.
8.L'Ecuyer P. & Glynn P.W. (1994). Stochastic optimization by simulation: Convergence proofs for the GI/G/I queue in steady state. Management Science 40(11): 15621578.
9.Loeve M. (1977). Probability theory 2, 4th ed.New York; Springer.
10.Rubinstein R. (1981). Simulation and the Monte Carlo Method. New York: John Wiley.
11.Ruszczynski A. & Syski W. (1983). Stochastic approximation method with gradient averaging for unconstrained problems. IEEE Transactions on Automatic Control AC-28(12): 10971105.
12.Spall J.C. (1992). Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control 37(3): 332341.
13.Suri R. & Zazanis M.A. (1988). Perturbation analysis gives strongly consistent estimates for the M/G/1 queue. Management Science 34(1): 3964.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Probability in the Engineering and Informational Sciences
  • ISSN: 0269-9648
  • EISSN: 1469-8951
  • URL: /core/journals/probability-in-the-engineering-and-informational-sciences
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 0
Total number of PDF views: 12 *
Loading metrics...

Abstract views

Total abstract views: 125 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 22nd November 2017. This data will be updated every 24 hours.