The generalised discrete algebraic Riccati equation in linear-quadratic optimal control

Abstract

This paper investigates the properties of the solutions of the generalised discrete algebraic Riccati equation arising from the classic infinite-horizon linear quadratic (LQ) control problem. In particular, a geometric analysis is used to study the relationship existing between the solutions of the generalised Riccati equation and the output-nulling subspaces of the underlying system and the corresponding reachability subspaces. This analysis reveals the presence of a subspace that plays an important role in the solution of the related optimal control problem, which is reflected in the generalised eigenstructure of the corresponding extended symplectic pencil. In establishing the main results of this paper, several ancillary problems on the discrete Lyapunov equation and spectral factorisation are also addressed and solved.

Introduction

Due to their ubiquitousness in optimal control and filtering problems, as well as in linear factorisation and stochastic realisation problems, Riccati equations are universally regarded as a cornerstone of modern control theory. Several monographs have been entirely devoted to providing a general and systematic framework for the study of Riccati equations, see e.g. Abou-Kandil, Freiling, Ionescu, and Jank (2003), Ionescu, Oaraˇ, and Weiss (1999) and Lancaster and Rodman (1995).

The classic solution of the discrete-time infinite-horizon LQ problem is traditionally expressed in terms of the solution $X$ of the Riccati equation

$$X=A^{T}XA-(A^{T}XB+S){(R+B^{T}XB)}^{-1}(B^{T}XA+S^T)+Q\text{,}$$

where $A\in {\mathbb{R}}^{n\times n},B\in {\mathbb{R}}^{n\times m},Q\in {\mathbb{R}}^{n\times n},S\in {\mathbb{R}}^{n\times m}$ and $R\in {\mathbb{R}}^{m\times m}$ are such that the Popov matrix $\Pi$ satisfies

$$\Pi \stackrel{\mathrm{def}}{=}\left[\begin{array}{cc}\hfill Q\hfill & \hfill S\hfill \\ \hfill S^T\hfill & \hfill R\hfill \end{array}\right]={\Pi }^T\ge 0\text{.}$$

The set of matrices $\Sigma =(A,B;Q,R,S)$ is often referred to as the Popov triple, see e.g. Ionescu et al. (1999). Eq. (1) is the so-called Discrete Riccati Algebraic Equation $\mathrm{DARE}\left(\Sigma \right)$.

Nevertheless, an LQ problem may have solutions even if $\mathrm{DARE}\left(\Sigma \right)$ has no solutions, and the optimal control can be written in this case as a state feedback given in terms of a matrix $X$ such that $R+B^{T}XB$ is singular and satisfies the more general Riccati equation

$$X=A^{T}XA-(A^{T}XB+S){(R+B^{T}XB)}^{†}(B^{T}XA+S^T)+Q\text{,}$$$$ker(R+B^{T}XB)\subseteq ker(A^{T}XB+S)\text{,}$$

where the matrix inverse in $\mathrm{DARE}\left(\Sigma \right)$ has been replaced by the Moore–Penrose pseudo-inverse, see Rappaport and Silverman (1971). Eq. (3) is known as the generalised discrete-time algebraic Riccati equation $\mathrm{GDARE}\left(\Sigma \right)$. The $\mathrm{GDARE}\left(\Sigma \right)$ with the additional constraint (4) is sometimes referred to as the constrained generalised discrete-time algebraic Riccati equation, herein denoted by $\mathrm{CGDARE}\left(\Sigma \right)$. It is obvious that (3) is a generalisation of the classic $\mathrm{DARE}\left(\Sigma \right)$, in the sense that any solution of $\mathrm{DARE}\left(\Sigma \right)$ is also a solution of $\mathrm{GDARE}\left(\Sigma \right)$–and therefore also of $\mathrm{CGDARE}\left(\Sigma \right)$ because $ker(R+B^{T}XB)=0_m$–but the vice-versa is not true in general. Despite its generality, this type of Riccati equation has only been marginally studied in the monographs (Abou-Kandil et al., 2003, Ionescu et al., 1999, Saberi et al., 1995) and in the paper (Ferrante, 2004). The only contributions entirely devoted to the study of the solutions of this equation are Ionescu and Oaraˇ (1996) and Stoorvogel and Saberi (1998). The former investigates conditions under which the $\mathrm{GDARE}\left(\Sigma \right)$ admits a stabilising solution in terms of the deflating subspaces of the extended symplectic pencil. The latter studies the connection between the solutions of this equation and the rank-minimising solutions of the so-called Riccati linear matrix inequality. In pursuing this task, the authors of Stoorvogel and Saberi (1998) derived a series of results that shed some light on the fundamental role played by the term $R_X\stackrel{\mathrm{def}}{=}R+B^{T}XB$. An example is the important observation according to which the inertia of this matrix $R_X$ is independent of the solution $X$ of $\mathrm{CGDARE}\left(\Sigma \right)$, Stoorvogel and Saberi (1998, Theorem 2.4). Hence, (i) if $X$ is a solution of $\mathrm{DARE}\left(\Sigma \right)$, then all solutions of $\mathrm{CGDARE}\left(\Sigma \right)$ will also satisfy $\mathrm{DARE}\left(\Sigma \right)$ and, (ii) if $X$ is a solution of $\mathrm{CGDARE}\left(\Sigma \right)$ such that $R_X$ is singular, then $\mathrm{DARE}\left(\Sigma \right)$ does not admit solutions. The results presented in Stoorvogel and Saberi (1998) are established in the very general setting in which the Popov matrix $\Pi$ is not necessarily positive semidefinite as in (2).

It is often taken for granted that $\mathrm{GDARE}\left(\Sigma \right)$ generalises the standard $\mathrm{DARE}\left(\Sigma \right)$ in the solution of the infinite LQ optimal control problem in the same way in which (Rappaport & Silverman, 1971) established that the generalised Riccati difference equation generalises the standard Riccati difference equation in the solution of the finite-horizon LQ problem. However, to the best of the authors’ knowledge, this fact has never been presented in a direct, self-contained and rigorous way. Thus, the first aim of this paper is to show the connection of the $\mathrm{CGDARE}\left(\Sigma \right)$ and the solution of the standard infinite-horizon LQ optimal control problem. The second aim of this paper is to provide a geometric picture describing the structure of the solutions of the $\mathrm{CGDARE}\left(\Sigma \right)$ in terms of the output nulling subspaces and the corresponding reachability subspaces of $\Sigma$. Indeed, when $\Pi \ge 0$, the null-space of $R_X$ is independent of the solution $X$ of $\mathrm{CGDARE}\left(\Sigma \right)$, and is linked to the presence of a subspace which plays an important role in the characterisation of the solutions of $\mathrm{CGDARE}\left(\Sigma \right)$ and in the solution of the related optimal control problem. This subspace does not depend on the particular solution $X$, nor does the closed-loop matrix restricted to this subspace. This new geometric analysis reveals that the spectrum of the closed-loop system is divided into a part that depends on the solution $X$ of $\mathrm{CGDARE}\left(\Sigma \right)$, and one–coinciding with the eigenvalues of the closed-loop restricted to this subspace–which is independent of it. At first sight, this fact seems to constitute a limitation in the design of the optimal feedback, because regardless of the solution of the generalised Riccati equation chosen for the implementation of the optimal feedback, the closed-loop matrix will always present a certain fixed eigenstructure as part of its spectrum. However, when $R_X$ is singular, the set of optimal controls presents a further degree of freedom–which is also identified in Saberi et al. (1995, Remark 4.2.3)–that allows us to place all the closed-loop poles at the desired locations without changing the cost.