Skip to main content
SpringerLink
Log in

HJB-RBF Based Approach for the Control of PDEs

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

Semi-Lagrangian schemes for the discretization of the dynamic programming principle are based on a time discretization projected on a state-space grid. The use of a structured grid makes this approach not feasible for high-dimensional problems due to the curse of dimensionality. Here, we present a new approach for infinite horizon optimal control problems where the value function is computed using radial basis functions by the Shepard moving least squares approximation method on scattered grids. We propose a new method to generate a scattered mesh driven by the dynamics and the selection of the shape parameter in the RBF using an optimization routine. This mesh will help to localize the problem and approximate the dynamic programming principle in high dimension. Error estimates for the value function are also provided. Numerical tests for high dimensional problems will show the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data Availibility

Enquiries about code availability should be directed to the authors. No data has been used in this paper.

Notes

  1. We remark that, although it is well-known how to compute feedback controls by semi-Lagrangian schemes, this is not trivial for high dimensional problems, as already mentioned earlier in the introduction.

  2. Strictly positive definite functions are such that the associated matrix \(K_X\) is positive definite.

References

  1. Alla, A., D’Elia, M., Glusa, C., Oliveria, A.H.: Control of fractional diffusion problems via dynamic programming equations. J. Peridyn. Nonlocal. Model (2023). https://doi.org/10.1007/s42102-023-00101-z

  2. Alla, A., Falcone, M., Kalise, D.: An efficient policy iteration algorithm for dynamic programming equations. SIAM J. Sci. Comput. 3(7), 181–200 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  3. Alla, A., Falcone, M., Saluzzi, L.: An efficient DP algorithm on a tree-structure for finite horizon optimal control problems. SIAM J. Sci. Comput. 41, A2384–A2406 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  4. Alla, A., Falcone, M., Volkwein, S.: Error analysis for POD approximations of infinite horizon problems via the dynamic programming approach. SIAM J. Control Optim. 55, 3091–3115 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  5. Alla, A., Saluzzi, L.: A HJB-POD approach for the control of nonlinear PDEs on a tree structure. Appl. Numer. Math. 155, 192–207 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  6. Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman Equations. Birkhäuser, Basel (1997)

    Book  MATH  Google Scholar 

  7. Benner, P., Gugercin, S., Willcox, K.: A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Rev. 57, 483–531 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bokanowski, O., Garcke, J., Griebel, M., Klompmaker, I.: An adaptive sparse grid semi-Lagrangian scheme for first order Hamilton–Jacobi Bellman equations. J. Sci. Comput. 55, 575–605 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  9. Capuzzo Dolcetta, I., Ishii, H.: Approximate solutions of the Bellman equation of deterministic control theory. Appl. Math. Optim. 11, 161–181 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  10. Capuzzo Dolcetta, I.: On a discrete approximation of the Hamilton–Jacobi equation of dynamic programming. Appl. Math. Optim. 10, 367–377 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  11. Carlini, E., Falcone, M., Ferretti, R.: An efficient algorithm for Hamilton–Jacobi equations in high dimension. Comput. Vis. Sci. 7, 15–29 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  12. Carlini, E., Ferretti, R.: A semi-Lagrangian scheme with radial basis approximation for surface reconstruction. Comput. Vis. Sci. 18, 103–112 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  13. Chen, W., Fu, Z.-J., Chen, C.-S.: Recent advances in radial basis function collocation methods. Briefs Appl. Sci. Technol. (2014). https://doi.org/10.1007/978-3-642-39572-7

    Article  MATH  Google Scholar 

  14. Chilan, C.M., Conway, B.A.: Optimal nonlinear control using Hamilton–Jacobi–Bellman viscosity solutions on unstructured grids. J. Guid. Control Dyn. 43, 30–38 (2020)

    Article  Google Scholar 

  15. Darbon, J., Osher, S.J.: Splitting enables overcoming the curse of dimensionality. In: Glowinski, R., Osher, S., Yin, W. (eds.) Splitting Methods in Communication, Imaging, Science, and Engineering. Scientific Computation. Springer, Cham (2016)

    MATH  Google Scholar 

  16. Darbon, J., Osher, S.: Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere. Res. Math. Sci. 3, 19–26 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  17. Dolgov, S., Kalise, D., Kunisch, K.: Tensor decomposition for high-dimensional Hamilton–Jacobi–Bellman equations, submitted, 2019. https://arxiv.org/pdf/1908.01533.pdf

  18. Dolgov, S., Kalise, D., Saluzzi, L.: Data-driven tensor train gradient cross approximation for Hamilton–Jacobi–Bellman equations, submitted, 2022. arxiv:2205.05109

  19. Ehring, T., Haasdonk, B.: Feedback control for a coupled soft tissue system by kernel surrogates. In: Proceedings of COUPLED 2021 (2021)

  20. Ehring, T., Haasdonk, B.: Greedy sampling and approximation for realizing feedback control for high dimensional nonlinear systems. IFAC-PapersOnLine 55, 325–330 (2022)

    Article  Google Scholar 

  21. Falcone, M., Ferretti, M.R.: Discrete time high-order schemes for viscosity solutions of Hamilton–Jacobi–Bellman equations. Numer. Math. 67, 315–344 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  22. Falcone, M., Ferretti, R.: Semi-Lagrangian Approximation Schemes for Linear and Hamilton-Jacobi equations. SIAM (2013). https://doi.org/10.1137/1.9781611973051

    Article  MATH  Google Scholar 

  23. Fasshauer, G.F.: Meshfree approximation methods with MATLAB (2007)

  24. Fasshauer, G.F., McCourt, M.: Kernel-based approximation methods using MATLAB (2015)

  25. Ferretti, G., Ferretti, R., Junge, O., Schreiber, A.: An adaptive multilevel radial basis function scheme for the HJB equation. IFAC-PapersOnLine 50, 1643–1648 (2017)

    Article  Google Scholar 

  26. Fornberg, B., Flyer, N.: Solving PDEs with radial basis functions. Acta Numer. 24, 215–258 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  27. Garcke, J., Kröner, A.: Suboptimal feedback control of PDEs by solving HJB equations on adaptive sparse grids. J. Sci. Comput. 70, 1–28 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  28. Junge, O., Schreiber, A.: Dynamic programming using radial basis functions. Discrete Contin. Dyn. Syst.- Ser. A 35, 4439–4453 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  29. Grüne, L.: An adaptive grid scheme for the discrete Hamilton–Jacobi–Bellman equation. Numerische Mathematik 75, 319–337 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  30. Kalise, D., Kunisch, K.: Polynomial approximation of high-dimensional Hamilton–Jacobi–Bellman equations and applications to feedback control of semilinear parabolic PDEs. SIAM J. Sci. Comput. 40, A629–A652 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  31. Kunisch, K., Volkwein, S., Xie, L.: HJB-POD based feedback design for the optimal control of evolution problems. SIAM J. Appl. Dyn. Syst. 4, 701–722 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  32. Leveque, R.J.: Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems. SIAM (2007). https://doi.org/10.1137/1.9780898717839

    Article  MATH  Google Scholar 

  33. McEneaney, W.M.: Convergence rate for a curse-of-dimensionality-free method for Hamilton–Jacobi–Bellman PDEs represented as maxima of quadratic forms. SIAM J. Control Optim. 48, 2651–2685 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  34. McEneaney, W.M.: A curse-of-dimensionality-free numerical method for solution of certain HJB PDEs. SIAM J. Control Optim. 46, 1239–1276 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  35. Schmidt, A., Haasdonk, B.: Data-driven surrogates of value functions and applications to feedback control for dynamical systems. IFAC-PapersOnLine 51, 307–312 (2018)

    Article  Google Scholar 

  36. Wendland, H.: Piecewise polynomial, positive definite and compactly supported radial functions of minimal degree. Adv. Comput. Math. 4, 389–396 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  37. Wendland, H.: Scattered data approximation. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

Download references

Funding

AA was supported by the CNPq research grant 3008414/2019-1 and by a research grant from PUC-Rio. HO was supported by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES).

Author information

Authors and Affiliations

  1. Dipartimento di Scienze Molecolari e Nanosistemi, Univeristà Ca’ Foscari Venezia, Venice, Italy

    Alessandro Alla

  2. Department of Mathematics, PUC-Rio, Rio de Janeiro, Brazil

    Hugo Oliveira

  3. Digital Society Center, Bruno Kessler Foundation, Trento, Italy

    Gabriele Santin

Authors
  1. Alessandro Alla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hugo Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gabriele Santin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessandro Alla.

Ethics declarations

Conflict of interest

No conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alla, A., Oliveira, H. & Santin, G. HJB-RBF Based Approach for the Control of PDEs. J Sci Comput 96, 25 (2023). https://doi.org/10.1007/s10915-023-02208-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-023-02208-3

Keywords

Mathematics Subject Classification

Search

Navigation