Abstract
This paper revisits a recent study by Posen and Levinthal (Manag Sci 58:587–601, 2012) on the exploration/exploitation tradeoff for a multi-armed bandit problem, where the reward probabilities undergo random shocks. We show that their analysis suffers two shortcomings: it assumes that learning is based on stale evidence, and it overlooks the steady state. We let the learning rule endogenously discard stale evidence, and we perform the long run analyses. The comparative study demonstrates that some of their conclusions must be qualified.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Berry D, Fristedt B (1985) Bandit problems. Chapman and Hall, London
Cesa-Bianchi N, Lugosi G (2006) Prediction, learning, and games. Cambridge University Press, New York
Duffy J (2006) Agent-based models and human subject experiments. In: Tesfatsion L, Judd KL (eds) Handbook of computational economics, vol 2. North-Holland, Amsterdam/New York, pp 949–1011
LiCalzi M, Marchiori D (2013) Pack light on the move: exploitation and exploration in a dynamic environment. Working Paper 4/2013, Department of Management, Università Ca’ Foscari Venezia,
March JG (1991) Exploration and exploitation in organizational learning. Organ Sci 1:71–87
Posen HE, Levinthal DA (2012) Chasing a moving target: exploitation and exploration in dynamic environments. Manag Sci 58:587–601
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. The MIT University Press, Cambridge
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
LiCalzi, M., Marchiori, D. (2014). Pack Light on the Move: Exploitation and Exploration in a Dynamic Environment. In: Leitner, S., Wall, F. (eds) Artificial Economics and Self Organization. Lecture Notes in Economics and Mathematical Systems, vol 669. Springer, Cham. https://doi.org/10.1007/978-3-319-00912-4_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-00912-4_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-00911-7
Online ISBN: 978-3-319-00912-4
eBook Packages: Business and EconomicsEconomics and Finance (R0)