Hostname: page-component-77c78cf97d-hf2s2 Total loading time: 0 Render date: 2026-04-26T11:06:59.176Z Has data issue: false hasContentIssue false
  • English
  • Français

Interacting nonlinear reinforced stochastic processes: Synchronization or non-synchronization

Part of: Mathematical sociology Applications Limit theorems Sequential methods Special processes

Published online by Cambridge University Press:  01 August 2022

Irene Crimaldi ,
Pierre-Yves Louis Open the ORCID record for Pierre-Yves Louis [Opens in a new window]  and
Ida G. Minelli
Irene Crimaldi*
Affiliation:
IMT School for Advanced Studies Lucca
Pierre-Yves Louis*
Affiliation:
PAM UMR 02.102 and Institut de Mathématiques de Bourgogne
Ida G. Minelli*
Affiliation:
Università degli Studi dell’Aquila
*
*Postal address: Piazza San Ponziano 6, 55100 Lucca, Italy. Email address: irene.crimaldi@imtlucca.it
**Postal address: Université Bourgogne Franche-Comté, Institut Agro Dijon, 1 esplanade Erasme, 21000, Dijon, France.
****Postal address: Dipartimento di Ingegneria e Scienze dell’Informazione e Matematica, Università degli Studi dell’Aquila, Via Vetoio (Coppito 1), 67100 L’Aquila, Italy. Email address: idagermana.minelli@univaq.it
Get access Rights & Permissions [Opens in a new window]

Abstract

The rich-get-richer rule reinforces actions that have been frequently chosen in the past. What happens to the evolution of individuals’ inclinations to choose an action when agents interact? Interaction tends to homogenize, while each individual dynamics tends to reinforce its own position. Interacting stochastic systems of reinforced processes have recently been considered in many papers, in which the asymptotic behavior is proven to exhibit almost sure synchronization. In this paper we consider models where, even if interaction among agents is present, absence of synchronization may happen because of the choice of an individual nonlinear reinforcement. We show how these systems can naturally be considered as models for coordination games or technological or opinion dynamics.

Keywords

Interacting agentsinteracting stochastic processesreinforced stochastic processurn modelnonlinear Pólya urngeneralized Pólya urnreinforcement learningstochastic approximationgame theorytechnological dynamicsCODA models

MSC classification

Primary: 60K35: Interacting random processes; statistical mechanics type models; percolation theory 62L20: Stochastic approximation 91D30: Social networks
Secondary: 60F15: Strong theorems 62P20: Applications to economics 62P25: Applications to social sciences

Information

Type
Original Article
Information
Advances in Applied Probability , Volume 55 , Issue 1 , March 2023 , pp. 275 - 320
Check for updates
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of Applied Probability Trust

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Aleandri, M. and Minelli, I. G. (2019). Opinion dynamics with Lotka–Volterra type interactions. Electron. J. Prob. 24, 31 pp.10.1214/19-EJP373CrossRefGoogle Scholar
Aleandri, M. and Minelli, I. G. (2021). Delay-induced periodic behaviour in competitive populations. J. Statist. Phys. 185, article no. 6.10.1007/s10955-021-02820-3CrossRefGoogle Scholar
Aletti, G. and Crimaldi, I. (2022). The rescaled Pólya urn: local reinforcement and chi-squared goodness of fit test. To appear in Adv. Appl. Prob. 54.10.1017/apr.2021.56CrossRefGoogle Scholar
Aletti, G., Crimaldi, I. and Ghiglietti, A. (2017). Synchronization of reinforced stochastic processes with a network-based interaction. Ann. Appl. Prob. 27, 3787–3844.10.1214/17-AAP1296CrossRefGoogle Scholar
Aletti, G., Crimaldi, I. and Ghiglietti, A. (2019). Networks of reinforced stochastic processes: asymptotics for the empirical means. Bernoulli 25, 3339–3378.10.3150/18-BEJ1092CrossRefGoogle Scholar
Aletti, G., Crimaldi, I. and Ghiglietti, A. (2020). Interacting reinforced stochastic processes: statistical inference based on the weighted empirical means. Bernoulli 26, 1098–1138.CrossRefGoogle Scholar
Aletti, G. and Ghiglietti, A. (2017). Interacting generalized Friedman’s urn systems. Stoch. Process. Appl. 127, 2650–2678.CrossRefGoogle Scholar
Aletti, G., Ghiglietti, A. and Rosenberger, W. F. (2018). Nonparametric covariate-adjusted response-adaptive design based on a functional urn model. Ann. Statist. 46, 3838–3866.10.1214/17-AOS1677CrossRefGoogle Scholar
Aletti, G., Ghiglietti, A. and Vidyashankar, A. N. (2018). Dynamics of an adaptive randomly reinforced urn. Bernoulli 24, 2204–2255.CrossRefGoogle Scholar
Alos Ferrer, C. and Weidenholzer, S. (2008). Contagion and efficiency. J. Econom. Theory 143, 251–274.CrossRefGoogle Scholar
Arenas, A. et al. (2008). Synchronization in complex networks. Phys. Reports 469, 93–153.10.1016/j.physrep.2008.09.002CrossRefGoogle Scholar
Arthur, B., Durlauf, S. and Lane, D. A. (2015). Process and emergence in the economy. In The Economy as an Evolving Complex System II, CRC Press, Boca Raton, FL, pp. 1–14.Google Scholar
Arthur, W. B. (1999). Complexity and the economy. Science 284, 107–109.CrossRefGoogle ScholarPubMed
Arthur, W. B., Ermoliev, Y. and Kaniovski, Y. (1983). A generalized urn problem and its applications. Kibernetika 1, 49–56.Google Scholar
Arthur, W. B., Ermoliev, Y. M. and Kaniovski, Y. M. (1987). Non-linear urn processes: asymptotic behavior and applications. IIASA Working Paper WP-87-085.Google Scholar
Axelrod, R. (1997). The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration. Princeton University Press.CrossRefGoogle Scholar
Benaïm, M. (1999). Dynamics of stochastic approximation algorithms. In Séminaire de Probabilités XXXIII, Springer, Berlin, Heidelberg, pp. 1–68.Google Scholar
Benam, M., Benjamini, I., Chen, J. and Lima, Y. (2015). A generalized Pólya’s urn with graph based interactions. Random Structures Algorithms 46, 614–634.CrossRefGoogle Scholar
Bernheim, B. D. and Whinston, M. D. (1990). Multimarket contact and collusive behavior. RAND J. Econom. 21, 1–26.CrossRefGoogle Scholar
Berti, P., Crimaldi, I., Pratelli, L. and Rigo, P. (2016). Asymptotics for randomly reinforced urns with random barriers. J. Appl. Prob. 53, 1206–1220.CrossRefGoogle Scholar
Bilancini, E. and Boncinelli, L. (2019). The evolution of conventions under condition-dependent mistakes. Econom. Theory 69, 497–521.CrossRefGoogle Scholar
Blume, L. E. (1993). The statistical mechanics of strategic interaction. Games Econom. Behavior 5, 387–424.CrossRefGoogle Scholar
Bonabeau, E. (2002). Agent-based modeling: methods and techniques for simulating human systems. Proc. Nat. Acad. Sci. USA 99, 7280–7287.CrossRefGoogle Scholar
Bonacich, P. and Liggett, T. M. (2003). Asymptotics of a matrix valued Markov chain arising in sociology. Stoch. Process. Appl. 104, 155–171.CrossRefGoogle Scholar
Borkar, V. S. (2009). Stochastic Approximation: a Dynamical Systems Viewpoint. Hindustan Book Agency, New Delhi.Google Scholar
Bottazzi, G., Dosi, G., Fagiolo, G. and Secchi, A. (2007). Modeling industrial evolution in geographical space. J. Econom. Geography 7, 651–672.CrossRefGoogle Scholar
Challet, D., Marsili, M. and Zecchina, R. (2000). Statistical mechanics of systems with heterogeneous agents: minority games. Phys. Rev. Lett. 84, article no. 1824.CrossRefGoogle ScholarPubMed
Chen, J. and Lucas, C. (2014). A generalized Pólya’s urn with graph based interactions: convergence at linearity. Electron. Commun. Prob. 19, 13 pp.CrossRefGoogle Scholar
Chen, M.-R. and Kuba, M. (2013). On generalized Pólya urn models. J. Appl. Prob. 50, 1169–1186.CrossRefGoogle Scholar
Cirillo, P., Gallegati, M. and Hüsler, J. (2012). A Pólya lattice model to study leverage dynamics and contagious financial fragility. Adv. Complex Systems 15, article no. 1250069.CrossRefGoogle Scholar
Collet, F., Dai Pra, P. and Formentin, M. (2015). Collective periodicity in mean-field models of cooperative behavior. Nonlinear Differential Equat. Appl. 22, 1461–1482.CrossRefGoogle Scholar
Collet, F., Formentin, M. and Tovazzi, D. (2016). Rhythmic behavior in a two-population mean-field Ising model. Phys. Rev. E 94, article no. 042139.CrossRefGoogle Scholar
Collevecchio, A., Cotar, C. and LiCalzi, M. (2013). On a preferential attachment and generalized Pólya’s urn model. Ann. Appl. Prob. 23, 1219–1253.CrossRefGoogle Scholar
Crimaldi, I., Dai Pra, P., Louis, P.-Y. and Minelli, I. G. (2019). Synchronization and functional central limit theorems for interacting reinforced random walks. Stoch. Process. Appl. 129, 70–101.CrossRefGoogle Scholar
Crimaldi, I., Dai Pra, P. and Minelli, I. G. (2016). Fluctuation theorems for synchronization of interacting Pólya’s urns. Stoch. Process. Appl. 126, 930–947.CrossRefGoogle Scholar
Crimaldi, I., Louis, P.-Y. and Minelli, I. G. (2022). An urn model with random multiple drawing and random addition. Stoch. Process. Appl. 147:270–299, 2022.10.1016/j.spa.2022.01.014CrossRefGoogle Scholar
Dai Pra, P., Louis, P.-Y. and Minelli, I. G. (2014). Synchronization via interacting reinforcement. J. Appl. Prob. 51, 556–568.CrossRefGoogle Scholar
Delyon, B. (2000). Stochastic approximation with decreasing gain: convergence and asymptotic theory. Tech. Rep., Université de Rennes.Google Scholar
Donahue, K., Hauser, O., Nowak, M. and Hilbe, C. (2020). Evolving cooperation in multichannel games. Nature Commun. 11, article no. 3885.CrossRefGoogle ScholarPubMed
Dosi, G., Ermoliev, Y. and Kaniovski, Y. (1994). Generalized urn schemes and technological dynamics. J. Math. Econom. 23, 1–19.CrossRefGoogle Scholar
Dosi, G., Fagiolo, G. and Roventini, A. (2006). An evolutionary model of endogenous business cycles. Comput. Econom. 27, 3–34.CrossRefGoogle Scholar
Duffy, J. and Hopkins, E. (2005). Learning, information, and sorting in market entry games: theory and evidence. Games Econom. Behavior 51, 31–62.CrossRefGoogle Scholar
Duflo, M. (1990). Méthodes Récursives Aléatoires. Masson, Paris.Google Scholar
Duflo, M. (1997). Random Iterative Models. Springer, Berlin, Heidelberg.Google Scholar
Eggenberger, F. and Pólya, G. (1923). Über die Statistik verketteter Vorgänge. Z. Angew. Math. Mech. 3, 279–289.CrossRefGoogle Scholar
Eidelson, B. M. and Lustick, I. (2004). Vir-pox: an agent-based analysis of smallpox preparedness and response policy. J. Artif. Soc. Social Simul. 7.Google Scholar
Eisenbud, D. (1995). Commutative Algebra: with a View Toward Algebraic Geometry. Springer, New York.CrossRefGoogle Scholar
Ellison, G. (1993). Learning, local interaction, and coordination. Econometrica 61, 1047–1071.CrossRefGoogle Scholar
Fagiolo, G. (2005). A note on equilibrium selection in Polya-urn coordination games. Econom. Bull. 3, 1–14.Google Scholar
Fagiolo, G. (2005). Endogenous neighborhood formation in a local coordination model with negative network externalities. J. Econom. Dynamics Control 29, 297–319.CrossRefGoogle Scholar
Fagiolo, G. and Dosi, G. (2003). Exploitation, exploration and innovation in a model of endogenous growth with locally interacting agents. Struct. Change Econom. Dynamics 14, 237–273.CrossRefGoogle Scholar
Fagiolo, G., Dosi, G. and Gabriele, R. (2004). Matching, bargaining, and wage setting in an evolutionary model of labor market and output dynamics. Adv. Complex Systems 7, 157–186.CrossRefGoogle Scholar
Fort, G. (2015). Central limit theorems for stochastic approximation with controlled Markov chain dynamics. ESAIM Prob. Statist. 19, 60–80.CrossRefGoogle Scholar
Fortini, S., Petrone, S. and Sporysheva, P. (2018). On a notion of partially conditionally identically distributed sequences. Stoch. Process. Appl. 128, 819–846.CrossRefGoogle Scholar
Galam, S. (1986). Majority rule, hierarchical structures, and democratic totalitarianism: a statistical approach. J. Math. Psych. 30, 426–434.CrossRefGoogle Scholar
Ghiglietti, A., Vidyashankar, A. N. and Rosenberger, W. F. (2017). Central limit theorem for an adaptive randomly reinforced urn model. Ann. Appl. Prob. 27, 2956–3003.CrossRefGoogle Scholar
Hayhoe, M., Alajaji, F. and Gharesifard, B. (2017). A Polya urn-based model for epidemics on networks. In 2017 American Control Conference (ACC), Institute of Electrical and Electronics Engineers, Piscataway, NJ, pp. 358–363.CrossRefGoogle Scholar
Hill, B. M., Lane, D. and Sudderth, W. (1980). A strong law for some generalized urn processes. Ann. Prob. 8, 214–226.CrossRefGoogle Scholar
Kandori, M., Mailath, G. J. and Rob, R. (1993). Learning, mutation, and long run equilibria in games. Econometrica 61, 29–56.CrossRefGoogle Scholar
Kreindler, G. E. and Young, H. P. (2014). Rapid innovation diffusion in social networks. Proc. Nat. Acad. Sci. USA 111, 10881–10888.CrossRefGoogle Scholar
Kushner, H. J. and Yin, G. G. (2003). Stochastic Approximation and Recursive Algorithms and Applications, 2nd edn. Springer, New York.Google Scholar
Laruelle, S. and Pagès, G. (2013). Randomized urn models revisited using stochastic approximation. Ann. Appl. Prob. 23, 1409–1436.CrossRefGoogle Scholar
Lasmar, N., Mailler, C. and Selmi, O. (2018). Multiple drawing multi-colour urns by stochastic approximation. J. Appl. Prob. 55, 254–281.CrossRefGoogle Scholar
Lewis, D. K. (1969). Convention: a Philosophical Study. Blackwell, Oxford.Google Scholar
Lima, Y. (2016). Graph-based Pólya’s urn: completion of the linear case. Stoch. Dynamics 16, article no. 1660007.Google Scholar
Louis, P.-Y. and Minelli, I. G. (2018). Synchronization in interacting reinforced stochastic processes. In Probabilistic Cellular Automata, Springer, Cham, pp. 105–118.CrossRefGoogle Scholar
Louis, P.-Y. and Mirebrahimi, M. (2018). Synchronization and fluctuations for interacting stochastic systems with individual and collective reinforcement. Preprint. Available at https://hal.archives-ouvertes.fr/hal-01856584v3.Google Scholar
Mahmoud, H. M. (2009). Pólya Urn Models. CRC Press, Boca Raton, FL.Google Scholar
Martins, A. C. (2008). Continuous opinions and discrete actions in opinion dynamics problems. Internat. J. Modern Phys. C 19, 617–624.CrossRefGoogle Scholar
Martins, A. C. (2013). Trust in the coda model: opinion dynamics and the reliability of other agents. Phys. Lett. A 377, 2333–2339.CrossRefGoogle Scholar
Matsushima, H. (2001). Multimarket contact, imperfect monitoring, and implicit collusion. J. Econom. Theory 98, 158–178.CrossRefGoogle Scholar
Orbell, J. et al. (2004). ‘Machiavellian’ intelligence as a basis for the evolution of cooperative dispositions. Amer. Political Sci. Rev. 98, 1–15.CrossRefGoogle Scholar
Paganoni, A. M. and Secchi, P. (2004). Interacting reinforced-urn systems. Adv. Appl. Prob. 36, 791–804.CrossRefGoogle Scholar
Pemantle, R. (1990). Nonconvergence to unstable points in urn models and stochastic approximations. Ann. Prob. 18, 698–712.CrossRefGoogle Scholar
Pemantle, R. (2007). A survey of random processes with reinforcement. Prob. Surveys 4, 1–79.CrossRefGoogle Scholar
Sahasrabudhe, N. (2016). Synchronization and fluctuation theorems for interacting Friedman urns. J. Appl. Prob. 53, 1221–1239.CrossRefGoogle Scholar
Silverman, E. (2018). Methodological Investigations in Agent-Based Modelling. Springer, Cham.CrossRefGoogle Scholar
Skyrms, B. and Pemantle, R. (2009). A dynamic model of social network formation. In Adaptive Networks, Springer, Berlin, Heidelberg, pp. 231–251.CrossRefGoogle Scholar
Tracy, N. D. and Seaman, J. W. (1992). Urn models for evolutionary learning games. J. Math. Psych. 36, 278–282.CrossRefGoogle Scholar
Zhang, L.-X. (2016). Central limit theorems of a recursive stochastic algorithm with applications to adaptive designs. Ann. Appl. Prob. 26, 3630–3658.CrossRefGoogle Scholar