Abstract
This paper studies a decentralized routing problem over a network, using the paradigm of mean-field games with large number of players. Building on a state-space extension technique, we turn the problem into an optimal control one for each single player. The main contribution is an explicit expression of the optimal decentralized control which guarantees the convergence both to local and to global equilibrium points. Furthermore, we study the stability of the system also in the presence of a delay which we model using an hysteresis operator. As a result of the hysteresis, we prove existence of multiple equilibrium points and analyze convergence conditions. The stability of the system is illustrated via numerical studies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Garavello, M., Piccoli, B.: Traffic Flow on Networks. American Institute of Mathematical Sciences, Springfield (2006)
Engel, K.-J., Kramar, F.M., Nagel, R., Sikolya, E.: Vertex control of flows in networks. Netw. Heterog. Media 3(4), 709–722 (2008)
Ren, W., Beard, R.W.: Distributed Consensus in Multi-vehicle Cooperative Control. Springer, London (2008)
Bauso, D., Zhang, X., Papachristodoulou, A.: Density Flow over Networks via Mean-Field Game. IEEE Trans. Autom. Control (2016). doi:10.1109/TAC.2016.2584979
Bauso, D., Blanchini, F., Giarré, L., Pesenti, R.: The linear saturated decentralized strategy for constrained flow control is asymptotically optimal. Automatica 49(7), 2206–2212 (2013)
Como, G., Savla, K., Acemoglu, D., Dahleh, M., Frazzoli, E.: Distributed robust routing in dynamical networks-Part I: locally responsive policies and weak resilience. IEEE Trans. Autom. Control 58(2), 317–332 (2013)
Como, G., Savla, K., Acemoglu, D., Dahleh, M., Frazzoli, E.: Distributed robust routing in dynamical networks-Part II: strong resilience, equilibrium selection and cascaded failures. IEEE Trans. Autom. Control 58(2), 333–348 (2013)
Kelly, F.P., Maulloo, A.K., Tan, D.K.H.: Rate control for communication networks: shadow prices, proportional fairness and stability. J. Oper. Res. Soc. 49, 237–252 (1998)
Basna, R., Hilbert, A., Kolokoltsov, V.: An epsilon Nash equilibrium for non-linear Markov games of mean-field-type on finite spaces. Commun. Stoch. Anal. 8(4), 449–468 (2014)
Bagagiolo, F., Bauso, D.: Objective function design for robust optimality of linear control under state-constraints and uncertainty. ESAIM Control Optim. Calc. Var. 17(1), 155–177 (2011)
Carmona, R., Delarue, F., Lachapelle, A.: Control of McKean–Vlasov dynamics versus mean field games. Math. Financ. Econ. 7(2), 131–166 (2013)
Ceragioli, F., De Persis, C., Frasca, P.: Discontinuities and hysteresis in quantized average consensus. Automatica 47(9), 1916–1928 (2011)
Huang, M.Y., Caines, P.E., Malhamé, R.P.: Individual and mass behaviour in large population stochastic wireless power control problems: centralized and Nash equilibrium solutions. IEEE Conference on Decision and Control, HI, USA, pp. 98–103 (2003)
Huang, M.Y., Caines, P.E., Malhamé, R.P.: Large population cost-coupled LQG problems with non-uniform agents: individual-mass behaviour and decentralized \(\varepsilon \)-Nash equilibria. IEEE Trans. Autom. Control 52(9), 1560–1571 (2007)
Lasry, J.-M., Lions, P.-L.: Jeux \(\grave{a}\) champ moyen. I Le cas stationnaire. Comptes Rendus Math. Acad. Sci. Paris 343(9), 619–625 (2006)
Lasry, J.-M., Lions, P.-L.: Mean field games. Jpn. J. Math. 2, 229–260 (2007)
Weintraub, G.Y., Benkard, C., Van Roy, B.: Oblivious equilibrium: a mean field approximation for large-scale dynamic games. In: Advances in Neural Information Processing Systems. MIT Press (2005)
Achdou, Y., Camilli, F., Capuzzo Dolcetta, I.: Mean field games: numerical methods for the planning problem. SIAM J. Control Optim. 50(1), 77–109 (2012)
Gueant, O., Lasry, J.-M., Lions, P.-L.: Mean field games and applications. In: Paris-Princeton Lectures, pp. 1–66. Springer (2010)
Lachapelle, A., Salomon, J., Turinici, G.: Computation of mean-field equilibria in economics. Math. Models Methods Appl. Sci. 20(4), 1–22 (2010)
Bauso, D., Zhu, Q., Basar, T.: Mixed integer optimal compensation: decompositions and mean-field approximations. In: Proceeding of 2012 American control conference, Montreal, Canada (2012)
Bagagiolo, F., Bauso, D.: Mean-field games and dynamic demand management in power grids. Dyn. Games Appl. 4(2), 155–176 (2014)
Achdou, Y., Capuzzo Dolcetta, I.: Mean field games: numerical methods. SIAM J. Numer. Anal. 48(3), 1136–1162 (2010)
Casti, J.: On the general Inverse problem of optimal control theory. J. Optim. Theory Appl. 32(4), 491–497 (1980)
Chitour, Y., Jean, F., Mason, P.: Optimal control models of goal-oriented human locomotion. SIAM J. Control Optim. 50(1), 147–170 (2012)
Krasnoselskii, M.A., Pokrovskii, A.V.: Systems with Hysteresis. Springer, Berlin (1989)
Visintin, A.: Differential Models of Hysteresis. Springer, Berlin (1994)
Wardrop, J.G.: Some theoretical aspects of road traffic research. Proc. Inst. Civil Eng. II 1, 325–378 (1952)
Acknowledgements
The work has been developed within the OptHySYS project of the University of Trento that is gratefully acknowledged. This work has started during a visiting period of R. Maggistro at the Department of Automatic Control and Systems Engineering, the University of Sheffield, from January 18, 2016, to March 22, 2016.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bagagiolo, F., Bauso, D., Maggistro, R. et al. Game Theoretic Decentralized Feedback Controls in Markov Jump Processes. J Optim Theory Appl 173, 704–726 (2017). https://doi.org/10.1007/s10957-017-1078-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-017-1078-3