Optimal control deals with optimization problems in which variables steer a dynamical system, and its outcome contributes to the objective function. Two classical approaches to solving these problems are Dynamic Programming and the Pontryagin Maximum Principle. In both approaches, Hamiltonian equations offer an interpretation of optimality through auxiliary variables known as costates. However, Hamiltonian equations are rarely used due to their reliance on forward-backward algorithms across the entire temporal domain. This paper introduces a novel neural-based approach to optimal control. Neural networks are employed not only for implementing state dynamics but also for estimating costate variables. The parameters of the latter network are determined at each time step using a newly introduced local policy referred to as the time-reversed generalized Riccati equation. This policy is inspired by a result discussed in the Linear Quadratic (LQ) problem, which we conjecture stabilizes state dynamics. We support this conjecture by discussing experimental results from a range of optimal control case studies.

Betti, A., Casoni, M., Gori, M., Marullo, S., Melacci, S., Tiezzi, M. (2024). Neural Time-Reversed Generalized Riccati Equation. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (pp.7935-7942). Washington, DC, : AAAI Press [10.1609/aaai.v38i8.28630].

Neural Time-Reversed Generalized Riccati Equation

Casoni, Michele;Gori, Marco;Melacci, Stefano;
2024-01-01

Abstract

Optimal control deals with optimization problems in which variables steer a dynamical system, and its outcome contributes to the objective function. Two classical approaches to solving these problems are Dynamic Programming and the Pontryagin Maximum Principle. In both approaches, Hamiltonian equations offer an interpretation of optimality through auxiliary variables known as costates. However, Hamiltonian equations are rarely used due to their reliance on forward-backward algorithms across the entire temporal domain. This paper introduces a novel neural-based approach to optimal control. Neural networks are employed not only for implementing state dynamics but also for estimating costate variables. The parameters of the latter network are determined at each time step using a newly introduced local policy referred to as the time-reversed generalized Riccati equation. This policy is inspired by a result discussed in the Linear Quadratic (LQ) problem, which we conjecture stabilizes state dynamics. We support this conjecture by discussing experimental results from a range of optimal control case studies.
2024
978-1-57735-887-9
Betti, A., Casoni, M., Gori, M., Marullo, S., Melacci, S., Tiezzi, M. (2024). Neural Time-Reversed Generalized Riccati Equation. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (pp.7935-7942). Washington, DC, : AAAI Press [10.1609/aaai.v38i8.28630].
File in questo prodotto:
File Dimensione Formato  
28630-Article Text-32684-1-2-20240324.pdf

accesso aperto

Tipologia: PDF editoriale
Licenza: PUBBLICO - Pubblico con Copyright
Dimensione 270.7 kB
Formato Adobe PDF
270.7 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11365/1260376