Skip to main content
SpringerLink
Log in

An Accelerated Value/Policy Iteration Scheme for Optimal Control Problems and Games

  • Conference paper
  • First Online:
Book cover Numerical Mathematics and Advanced Applications - ENUMATH 2013

Part of the book series: Lecture Notes in Computational Science and Engineering ((LNCSE,volume 103))

  • 3241 Accesses

Abstract

We present an accelerated algorithm for the solution of static Hamilton-Jacobi-Bellman equations related to optimal control problems and differential games. The new scheme combines the advantages of value iteration and policy iteration methods by means of an efficient coupling. The method starts with a value iteration phase on a coarse mesh and then switches to a policy iteration procedure over a finer mesh when a fixed error threshold is reached. We present numerical tests assessing the performance of the scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. A. Alla, M. Falcone, D. Kalise, An efficient policy iteration algorithm for dynamic programming equations (submitted). Available at arXiv:1308.2087

    Google Scholar 

  2. M. Bardi, I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations (Birkhäuser, Boston, 1997)

    Book  MATH  Google Scholar 

  3. R. Bellman, Dynamic Programming (Princeton University Press, Princeton, 1957)

    MATH  Google Scholar 

  4. O. Bokanowski, S. Maroso, H. Zidani, Some convergence results for Howard’s algorithm. SIAM J. Numer. Anal. 47, 3001–3026 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  5. C. Chow, J. Tsitsiklis, An optimal one-way multigrid algorithm for discrete-time stochastic control. IEEE Trans. Autom. Control 36, 898–914 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  6. R.A. Howard, Dynamic Programming and Markov Processes. Technology Press of the Massachusetts Institute of technology. New York; London: J. Wiley, cop. (1960)

    Google Scholar 

  7. R. Kalaba, On nonlinear differential equations, the maximum operation and monotone convergence. J. Math. Mech. 8, 519–574 (1959)

    MATH  MathSciNet  Google Scholar 

  8. M. Pollatschek, B. Avi-Itzhak, Algorithms for stochastic games with geometrical interpretation. Manage. Sci. 15, 399–415 (1969)

    Article  MATH  MathSciNet  Google Scholar 

  9. M.L. Puterman, S.L. Brumelle, On the convergence of Policy iteration in stationary dynamic programming. Math. Oper. Res. 4, 60–69 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  10. M.S. Santos, Numerical solution of dynamic economic models, in Handbook of Macroeconomics, ed. by J.B. Taylor, M. Woodford (Elsevier Science, Amsterdam/New York, 1999), pp. 311–386

    Google Scholar 

  11. M.S. Santos, J. Rust, Convergence properties of policy iteration. SIAM J. Control Optim. 42, 2094–2115 (2004)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Hamburg, Bundesstraße 55, Hamburg, Germany

    Alessandro Alla

  2. SAPIENZA – University of Rome, Ple. Aldo Moro 2, Rome, Italy

    Maurizio Falcone

  3. RICAM, Austrian Academy of Sciences, Altenberger Straße 69, Linz, Austria

    Dante Kalise

Authors
  1. Alessandro Alla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maurizio Falcone
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dante Kalise
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dante Kalise .

Editor information

Editors and Affiliations

  1. MATHICSE-ANMC, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

    Assyr Abdulle

  2. SB MATHICSE CMCS, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

    Simone Deparis

  3. SB MATHICSE ANCHP, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

    Daniel Kressner

  4. SB MATHICSE CSQI, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

    Fabio Nobile

  5. SB MATHICSE GR-PI, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

    Marco Picasso

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Alla, A., Falcone, M., Kalise, D. (2015). An Accelerated Value/Policy Iteration Scheme for Optimal Control Problems and Games. In: Abdulle, A., Deparis, S., Kressner, D., Nobile, F., Picasso, M. (eds) Numerical Mathematics and Advanced Applications - ENUMATH 2013. Lecture Notes in Computational Science and Engineering, vol 103. Springer, Cham. https://doi.org/10.1007/978-3-319-10705-9_48

Download citation

Publish with us

Policies and ethics

Search

Navigation