Skip to main content
×
×
Home

Forward sensitivity analysis for contracting stochastic systems

  • Thomas Flynn (a1)
Abstract

In this paper we investigate gradient estimation for a class of contracting stochastic systems on a continuous state space. We find conditions on the one-step transitions, namely differentiability and contraction in a Wasserstein distance, that guarantee differentiability of stationary costs. Then we show how to estimate the derivatives, deriving an estimator that can be seen as a generalization of the forward sensitivity analysis method used in deterministic systems. We apply the results to examples, including a neural network model.

    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Forward sensitivity analysis for contracting stochastic systems
      Available formats
      ×
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Forward sensitivity analysis for contracting stochastic systems
      Available formats
      ×
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Forward sensitivity analysis for contracting stochastic systems
      Available formats
      ×
Copyright
Corresponding author
* Current address: Computational Science Initiative, Brookhaven National Laboratory, P.O. Box 5000, Upton, NY 11973, USA. Email address: tflynn@bnl.gov
References
Hide All
[1] Borovkov, A. A. and Hordijk, A. (2004). Characterization and sufficient conditions for normed ergodicity of Markov chains. Adv. Appl. Prob. 36, 227242.
[2] Burago, D., Burago, Y. and Ivanov, S. (2001). A Course in Metric Geometry (Graduate Stud. Math. 33). American Mathematical Society, Providence, RI.
[3] Flynn, T. (2015). Timescale separation in recurrent neural networks. Neural Comput. 27, 13211344.
[4] Flynn, T. (2016). Convergence of one-step adjoint methods. In Proceedings of the 22nd International Symposium on Mathematical Theory of Networks and Systems.
[5] Forni, F. and Sepulchre, R. (2014). A differential Lyapunov framework for contraction analysis. IEEE Trans. Automatic Control 59, 614628.
[6] Griewank, A. and Walther, A. (2008). Evaluating Derivatives, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia, PA.
[7] Hairer, M. (2006). Ergodic properties of Markov processes. Lecture given at the University of Warwick. Available at http://www.hairer.org/notes/Markov.pdf.
[8] Hairer, M. and Mattingly, J. C. (2008). Spectral gaps in Wasserstein distances and the 2D stochastic Navier–Stokes equations. Ann. Prob. 36, 20502091.
[9] Heidergott, B. and Hordijk, A. (2003). Taylor series expansions for stationary Markov chains. Adv. Appl. Prob. 35, 10461070. (Correction: 36 (2004), 1300.)
[10] Heidergott, B., Hordijk, A. and Weisshaupt, H. (2006). Measure-valued differentiation for stationary Markov chains. Math. Operat. Res. 31, 154172.
[11] Joulin, A. and Ollivier, Y. (2010). Curvature, concentration and error estimates for Markov chain Monte Carlo. Ann. Prob. 38, 24182442.
[12] Lohmiller, W. and Slotine, J.-J. E. (1998). On contraction analysis for non-linear systems. Automatica J. 34, 683696.
[13] Pflug, G. C. (1992). Gradient estimates for the performance of Markov chains and discrete event processes. Ann. Operat. Res. 39, 173194.
[14] Pflug, G. C. (1996). Optimization of Stochastic Models: The Interface Between Simulation and Optimization. Kluwer, Boston, MA.
[15] Pineda, F. J. (1988). Dynamics and architecture for neural computation. J. Complexity 4, 216245.
[16] Rumelhart, D. E., Hinton, G. E. and Williams, R. J. (1986). Learning representations by back-propagating errors. Nature 323, 533536.
[17] Russo, G., di Bernardo, M. and Sontag, E. D. (2010). Global entrainment of transcriptional systems to periodic inputs. PLoS Comput. Biol. 6, e1000739.
[18] Simpson-Porco, J. W. and Bullo, F. (2014). Contraction theory on Riemannian manifolds. Systems Control Lett. 65, 7480.
[19] Steinsaltz, D. (1999). Locally contractive iterated function systems. Ann. Prob. 27, 19521979.
[20] Stenflo, Ö. (2012). A survey of average contractive iterated function systems. J. Difference Equat. Appl. 18, 13551380.
[21] Vázquez-Abad, F. J. and Kushner, H. J. (1992). Estimation of the derivative of a stationary measure with respect to a control parameter. J. Appl. Prob. 29, 343352.
[22] Villani, C. (2009). Optimal Transport: Old and New. Springer, Berlin.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Advances in Applied Probability
  • ISSN: 0001-8678
  • EISSN: 1475-6064
  • URL: /core/journals/advances-in-applied-probability
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

MSC classification

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 55 *
Loading metrics...

Abstract views

Total abstract views: 98 *
Loading metrics...

* Views captured on Cambridge Core between 20th March 2018 - 16th August 2018. This data will be updated every 24 hours.