2.1 Preliminaries

We denote with $\mathbb{N}$ , $\mathbb{Z}$ , $\mathbb{R}$ , and $\mathbb{C}$ the set of natural, integers, real and complex numbers, respectively. We use bold letters for vectors and matrices; all vectors considered are column vectors. We denote with $\mathbf{1}$ the identity matrix, specifying its dimension only when not clear from the context. We refer to the $i$ -th row and the $j$ -th column of $\mathbf{M}$ by $\mathbf{M}(i,:)$ and $\mathbf{M}({:},j)$ . Given a matrix $\mathbf{M}\in \mathbb{R}^{p\times q}$ of rank $n$ , a rank factorization is a factorization $\mathbf{M}=\mathbf{P}\mathbf{Q}$ , where $\mathbf{P} \in \mathbb{R}^{p\times n}$ , $\mathbf{Q} \in \mathbb{R}^{n\times q}$ , and $\textrm{rank}(\mathbf{M})=\textrm{rank}(\mathbf{P})=\textrm{rank}(\mathbf{Q})=n$ . Let $\mathbf{M} \in \mathbb{R}^{p \times q}$ of rank $n$ , the compact singular value decomposition SVD of $\mathbf{M}$ is the factorization $\mathbf{M}=\mathbf{U}\mathbf{D}\mathbf{V}^{\top }$ , where $\mathbf{U}\in \mathbb{R}^{p\times n}$ , $\mathbf{D}\in \mathbb{R}^{n\times n}$ , $\mathbf{V}\in \mathbb{R}^{q \times n}$ are such that $\mathbf{U}^{\top }\mathbf{U}=\mathbf{V}^{\top }\mathbf{V}=\mathbf{1}$ , and $\mathbf{D}$ is a diagonal matrix. The columns of $\mathbf{U}$ and $\mathbf{V}$ are called left and right singular vectors, while the entries of $\mathbf{D}$ are the singular values. The Moore-Penrose pseudo-inverse $\mathbf{M}^+$ of $\mathbf{M}$ is the unique matrix such that $\mathbf{M}\mathbf{M}^+\mathbf{M}=\mathbf{M}$ , $\mathbf{M}^+\mathbf{M}\mathbf{M}^+=\mathbf{M}^+$ , with $\mathbf{M}^+\mathbf{M}$ and $\mathbf{M}\mathbf{M}^+$ Hermitian (Zhu, Reference Zhu1990). The spectral radius $\rho (\mathbf{M})$ of a matrix $\mathbf{M}$ is the largest modulus among its eigenvalues.

A Hilbert space is a complete normed vector space where the norm arises from an inner product. A linear operator $T: X \rightarrow Y$ between Hilbert spaces is bounded if it has finite operator norm, that is $\|T\|_{op} = \sup _{\|g\|_X\leq 1}\|Tg\|_Y \lt \infty$ . We denote by $\mathbf{T}$ the (infinite) matrix associated with $T$ by some (canonical) orthonormal basis on $H$ . An operator is compact if the image of the unit ball in $X$ is relatively compact. Given Hilbert spaces $X, Y$ and a compact operator $T:X \rightarrow Y$ , we denote its adjoint by $T^*$ . The singular numbers $\{\sigma _n\}_{n \geq 0}$ of $T$ are the square roots of the eigenvalues of the self-adjoint operator $T^* T$ , arranged in decreasing order. A $\sigma$ -Schmidt pair $\{\boldsymbol{\xi }, \boldsymbol{\eta }\}$ for $T$ is a couple of norm $1$ vectors such that: $\mathbf{T}\boldsymbol{\xi }=\sigma \boldsymbol{\eta }$ and $\mathbf{T}^*\boldsymbol{\eta }= \sigma \boldsymbol{\xi }$ . The Hilbert-Schmidt decomposition provides a generalization of the compact SVD for the infinite matrix of a compact operator $T$ using singular numbers and orthonormal Schmidt pairs: $\mathbf{T}\mathbf{x}=\sum _{n\geq 0}\sigma _n\langle \mathbf{x},\boldsymbol{\xi }_n \rangle \boldsymbol{\eta }_k$ (Zhu, Reference Zhu1990). The spectral norm $\|\mathbf{T}\|$ of the matrix representing the operator $T$ is the largest singular number of $T$ . Note that the spectral norm of $\mathbf{T}$ corresponds to the operator norm of $T$ .

Let $\ell ^2$ be the Hilbert space of square-summable sequences over $\Sigma ^*$ , with norm $\|f\|_2^2=\sum _{x\in \Sigma ^*}|f(x)|^2$ and inner product $\langle f, g \rangle = \sum _{x \in \Sigma ^*}f(x)g(x)$ for $f,g \in \mathbb{R}^{\Sigma ^*}$ . Let $\mathbb{T}=\{z\in \mathbb{C}: |z|=1\}$ be the complex unit circle, $\mathbb{D}=\{z\in \mathbb{C}: |z|\lt 1\}$ the (open) complex unit disc. Let $1\lt p\lt \infty$ , $\mathscr{L}^{\,p}(\mathbb{T})$ be the space of measurable functions on $\mathbb{T}$ for which the $p$ -th power of the absolute value is Lebesgue integrable. For $p=\infty$ , we denote with $\mathscr{L}^{\,\infty }(\mathbb{T})$ the space of measurable functions that are bounded, with norm $\|f\|_{\infty }=\sup \{|f(x)|: x\in \mathbb{T}\}$ .

2.2 Hankel matrix and weighted automata

Let $\Sigma$ be a fixed finite alphabet and $\Sigma ^*$ be the set of all finite strings with symbols in $\Sigma$ . We denote with $\varepsilon$ the empty string. Given $p,s \in \Sigma ^*$ , we denote with $ps$ the string obtained by their concatenation. Let $f : \Sigma ^* \to \mathbb{R}$ be a function defined on sequences, we consider the bi-infinite matrix $\mathbf{H}_f \in \mathbb{R}^{\Sigma ^* \times \Sigma ^*}$ having rows and columns indexed by strings and defined by $\mathbf{H}_f(p,s) = f(ps)$ for $p, s \in \Sigma ^*$ .

Given a Hankel matrix $\mathbf{H} \in \mathbb{R}^{\Sigma ^* \times \Sigma ^*}$ , there is a unique function $f : \Sigma ^* \to \mathbb{R}$ such that $\mathbf{H}_f = \mathbf{H}$ . Intuitively, the Hankel property tells us that each entry of the matrix only depends on the composition of the coordinates. Since rows and columns are indexed using strings, then the value stored in each entry only depends on the string obtained by concatenating the coordinates.

Weighted finite automata are a class of models defined over sequential data. A weighted finite automaton (WFA) of $n$ states over $\Sigma$ is a tuple $A = \langle \boldsymbol{\alpha }, \{ A_a\}_{a \in \Sigma }, \boldsymbol{\beta } \rangle$ , where $\boldsymbol{\alpha },$ $\boldsymbol{\beta } \in \mathbb{R}^n$ are the vector of initial and final weights, respectively, and $\mathbf{A}_a \in \mathbb{R}^{n \times n}$ is the matrix containing the transition weights associated with each symbol $a\in \Sigma$ . Every WFA $A$ with real weights realizes (or computes) a function $f_A : \Sigma ^* \to \mathbb{R}$ , that is given a string $x = x_1 \cdots x_t \in \Sigma ^*$ , it returns $f_A(x) = \boldsymbol{\alpha } ^\top \mathbf{A}_{x_1} \cdots \mathbf{A}_{x_t} \boldsymbol{\beta } = \boldsymbol{\alpha } ^\top \mathbf{A}_x \boldsymbol{\beta }$ . A function $f : \Sigma ^* \to \mathbb{R}$ is called rational if there exists a WFA $A$ that realizes it. Given $f : \Sigma ^* \to \mathbb{R}$ , we can use the Hankel matrix $\mathbf{H}_f \in \mathbb{R}^{\Sigma ^* \times \Sigma ^*}$ to recover information about the weighted automaton computing $f$ .

Given a WFA $A = \langle \boldsymbol{\alpha }, \{ A_a\}_{a \in \Sigma }, \boldsymbol{\beta } \rangle$ , the forward matrix of $A$ is the infinite matrix $\mathbf{F}_A \in \mathbb{R}^{\Sigma ^* \times n}$ given by $\mathbf{F}_A(p,{:}) = \boldsymbol{\alpha } ^\top \mathbf{A}_p$ for any $p \in \Sigma ^*$ , while the backward matrix of $A$ is $\mathbf{B}_A \in \mathbb{R}^{\Sigma ^* \times n}$ , given by $\mathbf{B}_A(s,{:}) = (\mathbf{A}_s \boldsymbol{\beta })^\top$ for any $s \in \Sigma ^*$ . Let $\mathbf{H}_f$ be the Hankel matrix of $f$ , its forward-backward (FB) factorization is: $\mathbf{H}_f = \mathbf{F} \mathbf{B}^\top$ . A WFA with $n$ states is reachable if $\textrm{rank}(\mathbf{F}_A)=n$ , while it is observable if $\textrm{rank}(\mathbf{B}_A)=n$ . A WFA is minimal if it is reachable and observable. If $A$ is minimal, the (unique) FB factorization is a rank factorization (Balle et al., Reference Balle, Carreras, Luque and Quattoni2014a).

We recall the definition of the singular value automaton, a canonical form for WFAs (Balle et al., Reference Balle, Panangaden and Precup2015).

The SVA can be computed with an efficient algorithm relying on the following matrices (Balle et al., Reference Balle, Panangaden and Precup2019).

If $A$ is in its SVA form, the Gramians associated with its FB factorization satisfy $\mathbf{P}_A = \mathbf{Q}_A = \mathbf{D}$ , where $\mathbf{D}$ is the matrix of singular values of the corresponding Hankel matrix. The Gramians can alternatively be characterized and computed (Balle et al., Reference Balle, Panangaden and Precup2019)) using fixed point equations, corresponding to Lyapunov equations when $|\Sigma |=1$ (Lyapunov, Reference Lyapunov1950).

Finally, we recall the definition of generative probabilistic automata (GPA). A WFA $A= \langle \boldsymbol{\alpha }, \{ A_a\}_{a \in \Sigma }, \boldsymbol{\beta } \rangle$ is a GPA if $f_A(x)\geq 0$ for every $x$ and $\sum _{x\in \Sigma ^*}f_A(x)=1$ , that is if $f_A$ computes a probability distribution over $\Sigma ^*$ . In general, this class of automata can contain pathological examples having states not connected to any final state. To avoid these cases, we introduce the following property on the spectral radius of the transition matrix.

2.3 AAK theory

In this section, we introduce the theory of optimal approximation for Hankel operators and complex functions known as AAK theory (Adamyan et al., Reference Adamyan, Arov and Krein1971). A comprehensive presentation of the concepts recalled in this section can be found in Nikol’Skii (Reference Nikol’Skii2002); Peller (Reference Peller2012).

We consider the space of square-integrable complex functions on the unit circle $\mathscr{L}^{\,2}(\mathbb{T})$ . To avoid any confusion with functions defined over sequences, when dealing with complex function we make explicit the dependence on the complex variable $z=e^{it}$ . Note that a function $\phi (z) \in \mathscr{L}^{\,2}(\mathbb{T})$ can be represented, using the orthonormal basis $\{z^n\}_{n \in \mathbb{Z}}$ , by means of its Fourier series: $\phi (z)=\sum _{n \in \mathbb{Z}}\widehat{\phi }(n)z^n$ , with Fourier coefficients $\widehat{\phi }(n)= \int _{\mathbb{T}}\phi (z) \bar{z}^n dz, \, n \in \mathbb{Z}$ . Thus, we can partition the function space $\mathscr{L}^{\,2}(\mathbb{T})$ into two subspaces.

Definition 7. The Hardy space $\mathscr{H}^{\,\kern1pt2}$ and the negative Hardy space $\mathscr{H}^{\kern1pt\,2}_-$ on $\mathbb{T}$ are the subspaces of $\mathscr{L}^{\,2}(\mathbb{T})$ defined as:

\begin{equation*} \mathscr{H}^{\kern1pt\,2}=\left \{ \phi (z) \in \mathscr{L}^{\,2}(\mathbb {T}) : \widehat {\phi }(n)=0, n \lt 0 \right \}, \end{equation*}

\begin{equation*} \mathscr{H}^{\kern1pt\,2}_-= \left \{ \phi (z) \in \mathscr{L}^{\,2}(\mathbb {T}) : \widehat {\phi }(n)=0, n \geq 0 \right \}. \end{equation*}

Interestingly, the elements of the Hardy space can be canonically identified with the set of functions analytic in the unit disc $\mathbb{D}$ , with the property that the square of their absolute value is integrable on $\mathbb{T}$ (a proof can be found in Nikol’Skii (Reference Nikol’Skii2002)). Thus, we will make no difference between these functions in the unit disc and their boundary value on the circle. Moreover, we remark that the definition of Hardy space can be generalized for any $p$ -th power of the functions’ absolute value, for $0\lt p\leq \infty$ .

We define Hankel operators in the Hardy spaces.

The matrix $\mathbf{H}_{\phi }$ associated with the Hankel operator $H_{\phi }:\mathscr{H}^{\kern1pt\,2} \rightarrow \mathscr{H}^{\kern1pt\,2}_-$ is:

(1) \begin{equation} \mathbf{H}_{\phi }=\left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} \widehat{\phi }(-1) & \widehat{\phi }(-2) & \widehat{\phi }(-3) & \dots \\[4pt] \widehat{\phi }(-2) & \widehat{\phi }(-3) & \widehat{\phi }(-4) &\dots \\[4pt] \widehat{\phi }(-3) & \widehat{\phi }(-4) & \widehat{\phi }(-5) &\dots \\[4pt] \vdots & \vdots &\vdots &\ddots \end{array}\right). \end{equation}

Note that this matrix satisfies the Hankel property, as each entry only depends on the composition of the corresponding coordinates.

Nehari’s theorem (Nehari, Reference Nehari1957), characterizes bounded Hankel operators and their norm.

Theorem 9 (Nehari (Reference Nehari1957)).  Let $\phi \in \mathscr{L}^{\,2}(\mathbb{T})$ be a symbol of the Hankel operator on Hardy spaces $H_{\phi }:\mathscr{H}^{\kern1pt\,2} \rightarrow \mathscr{H}^{\kern1pt\,2}_-$ . Then, $H_{\phi }$ is bounded on $\mathscr{H}^{\kern1pt\,2}$ if and only if there exists $\psi \in \mathscr{L}^{\,\infty }(\mathbb{T})$ such that $\widehat{\psi }(m)=\widehat{\phi }(m)$ for all $m\lt 0$ . If the conditions above are satisfied, then:

(2) \begin{equation} \|H_{\phi }\|=\inf \left \{\|\psi \|_{\infty }:\widehat{\psi }(m)=\widehat{\phi }(m), \, m\lt 0 \right \}. \end{equation}

As a consequence of Theorem 9, if $H_{\phi }$ is a bounded operator, we can consider without loss of generality $\phi (z) \in \mathscr{L}^{\,\infty }(\mathbb{T})$ . We remark that a Hankel operator has infinitely many different symbols, since $H_{\phi }=H_{\phi +\psi }$ for $\psi (z) \in \mathscr{H}^{\kern1pt\,\infty }$ .

The following result of Kronecker relates finite-rank infinite Hankel matrices to rational functions.

We are ready to state the main result of Adamyan et al. (Reference Adamyan, Arov and Krein1971). The theorem shows that for infinite dimensional Hankel matrices the constraint of preserving the Hankel property does not affect the achievable approximation error.

Theorem 12 (Adamyan et al. (Reference Adamyan, Arov and Krein1971)).  Let $H_{\phi }$ be a compact Hankel operator of rank $n$ , matrix $\mathbf{H}$ and singular numbers $\sigma _0 \geq \dots \geq \sigma _{n-1}\gt 0$ . Then there exists a unique Hankel operator $H_g$ with matrix $\mathbf{G}$ of rank $k\lt n$ such that:

(3) \begin{equation} \|H_{\phi } - H_g\| = \|\mathbf{H}-\mathbf{G}\|= \sigma _k. \end{equation}

We denote with $\mathscr{R}_k\subset \mathscr{H}^{\kern1pt\,\infty }_-$ the set of strictly proper rational functions of rank $k$ , and we consider the set of functions:

(4) \begin{equation} \mathscr{H}^{\kern1pt\,\infty }_k= \left \{\psi \in \mathscr{L}^{\,\infty }(\mathbb{T}): \,\,\exists g \in \mathscr{R}_k, \exists l \in \mathscr{H}^{\kern1pt\,\infty },\,\, \psi =g+l \right \}. \end{equation}

The proof of the AAK theorem is directly connected with the problem of approximating a bounded function defined on the unit circle. In fact, the theorem can be reformulated in terms of the symbols associated with the Hankel operators.

Theorem 13 (Adamyan et al. (Reference Adamyan, Arov and Krein1971)).  Let $\phi \in \mathscr{L}^{\,\infty }(\mathbb{T})$ . Then there exists a complex function $\psi \in \mathscr{H}^{\kern1pt\,\infty }_k$ such that:

(5) \begin{equation} \|\phi - \psi \|_{\infty }= \sigma _k(H_{\phi }). \end{equation}

This theorem provides us with an alternative interpretation of singular numbers, relating them to the “smoothness” of the corresponding operator (or symbol). The advantage of this second formulation is that its proof is constructive and tells us how to find the function $\psi$ . We state as a corollary the critical steps of the proof, that allows us to find the best approximating symbol.

Corollary 14. Let $\phi$ and $\{\boldsymbol{\xi }_k, \boldsymbol{\eta }_k\}$ be a symbol and a $\sigma _k$ -Schmidt pair for $H_{\phi }$ . A function $\psi \in \mathscr{L}^{\,\infty }({\mathbb{T}})$ is the best AAK approximation according to Theorem 13 , if and only if:

(6) \begin{equation} (\phi -\psi )\xi ^{+}_k=\sigma _k\eta ^{-}_k. \end{equation}

Moreover, the function $\psi$ does not depend on the particular choice of the pair $\{\boldsymbol{\xi }_k, \boldsymbol{\eta }_k\}$ .

Note that the solutions of Theorem 12 and 13 are strictly related.

In particular, this means that to find the Hankel operator $H_g$ corresponding to the optimal approximation, we can first obtain $\psi$ by applying Corollary 14. Then, we can extract the rational component $g$ of $\psi$ : this will correspond to a symbol for $\mathbf{H}_g$ .

3.1 Defining a Hankel operator: the one-letter assumption

As a first step, we want to understand whether or not a Hankel operator on the Hardy space can be associated with the Hankel matrix of a weighted automaton. To do so, we compare the Hankel matrix $\mathbf{H}_f$ of a WFA realizing a function $f$ over an alphabet $\Sigma$ , to the Hankel matrix $\mathbf{H}_{\phi }$ of a Hankel operator in the Hardy space:

(7) \begin{equation} \mathbf{H}_f = \left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} f(\varepsilon ) & f(a) & f(b) &\dots \\[4pt] f(a) & f(aa) & f(ab) &\dots \\[4pt] f(b) &f(ba) & f(bb) &\dots \\[4pt] \vdots & \vdots &\vdots &\ddots \end{array}\right) \quad \quad \mathbf{H}_{\phi }= \left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} \widehat{\phi }(-1) & \widehat{\phi }(-2) & \widehat{\phi }(-3) & \dots \\[4pt] \widehat{\phi }(-2) & \widehat{\phi }(-3) & \widehat{\phi }(-4) &\dots \\[4pt] \widehat{\phi }(-3) & \widehat{\phi }(-4) & \widehat{\phi }(-5) &\dots \\[4pt] \vdots & \vdots &\vdots &\ddots \end{array}\right). \end{equation}

We remark that the columns and rows of $\mathbf{H}_f$ are indexed using the letters of the alphabet $\Sigma$ :

\begin{equation*} \mathbf {H}_f(p,s) = f(ps) \quad \text {for}\,p,s \in \Sigma, \end{equation*}

while in the case of $\mathbf{H}_{\phi }$ , the entries are indexed using natural numbers

\begin{equation*} \mathbf {H}_{\phi }(j,k)= \widehat {\phi }(-j-k-1) \quad \text {for}\, j,k\geq 0. \end{equation*}

If we think of the intuitive definition of the Hankel property presented in the previous section, we have that it holds in both cases the entries of the matrices only depend on the composition of the coordinate. Note that “composition” means concatenation of letters in the first case, and sum of numbers in the second one. One fundamental difference is that adding natural numbers is a commutative operation, while concatenating letters is not. For example, while for the matrix corresponding to a Hankel operator in the Hardy space we have:

\begin{equation*} \mathbf {H}_{\phi }(0,1)=\widehat {\phi }(-2)=\mathbf {H}_{\phi }(1,0), \end{equation*}

in the case of the WFA’s matrix, this is not true:

\begin{equation*} \mathbf {H}_f(a,b)=f(ab)\neq f(ba)=\mathbf {H}_f(b,a). \end{equation*}

This fact reflects in the much stronger structural property satisfied by Hankel of matrices in the Hardy spaces, where the Hankel property implies that the anti-diagonals have constant entries. This property is not reflected by the matrix of an arbitrary WFA, so it is not always possible to associate a Hankel operator to an automaton over an alphabet of arbitrary size, and AAK theory cannot be generally applied. The only case in which concatenation of strings is commutative is when we are restricting our focus on alphabets of one letter. In particular, when $|\Sigma |=1$ , the set of strings $\Sigma ^*$ can be identified with $\mathbb{N}$ . Therefore, the function $f:\Sigma ^* \rightarrow \mathbb{R}$ recognized by a minimal WFA can be rewritten as $f:\mathbb{N} \rightarrow \mathbb{R}$ , and the Hankel matrix $\mathbf{H}_f$ associated with it can be interpreted as the matrix of a Hankel operator between sequences $H_f:\ell ^2\rightarrow \ell ^2$ . In this case, the Hankel matrix is defined by $\mathbf{H}(i,j)=f(i+j)$ , for $i,j\geq 0$ :

\begin{equation*} \mathbf{H}_f= \left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} f(0) & f(1) & f(2) & \dots \\[4pt] f(1) & f(2) & f(3) &\dots \\[4pt] f(2) & f(3) & f(4) &\dots \\[4pt] \vdots & \vdots &\vdots &\ddots \end{array}\right). \end{equation*}

Using the Fourier isomorphism, we can interpret $\mathbf{H}_f$ as the matrix $\mathbf{H}_{\phi }$ of a Hankel operator over Hardy spaces, associated with a complex function $\phi \in \mathscr{L}^{\,2}(\mathbb{T})$ . In particular, we can embed the sequence space $\ell ^2$ into $\ell ^2(\mathbb{Z})$ by “duplicating” each vector, that is by associating $\boldsymbol{\mu }=(\mu _0, \mu _1, \dots )\in \ell ^2$ to $\boldsymbol{\mu }^{(2)}=(\dots, \mu _1,\mu _0, \mu _1, \dots )\in \ell ^2(\mathbb{Z})$ . Then, we can use the Fourier isomorphism to map the vector $\boldsymbol{\mu }^{(2)}\in \ell ^2(\mathbb{Z})$ to the complex function space $\mathscr{L}^{\,2}(\mathbb{T})$ . In this way, each vector $\boldsymbol{\mu }\in \ell ^2$ corresponds to two functions in the Hardy spaces:

(8) \begin{align} &\mu ^-(z)=\sum _{j=0}^{\infty }\boldsymbol{\mu }_j z^{-j-1} \in \mathscr{H}^{\kern1pt\,2}_-, \\[4pt] &\mu ^+(z)=\sum _{j=0}^{\infty } \boldsymbol{\mu }_j z^{j} \in \mathscr{H}^{\kern1pt\,2}.\notag \end{align}

Moreover, we can derive the relationship between $f$ and $\phi$ :

\begin{equation*} \left(\begin {array}{c@{\quad}c@{\quad}c@{\quad}c} f_A(0) & f_A(1) & f_A(2) & \dots \\[4pt] f_A(1) & f_A(2) & f_A(3) &\dots \\[4pt] f_A(2) & f_A(3) & f_A(4) &\dots \\[4pt] \vdots & \vdots &\vdots &\ddots \end {array}\right) =\left(\begin {array}{c@{\quad}c@{\quad}c@{\quad}c} \widehat {\phi }(-1) & \widehat {\phi }(-2) & \widehat {\phi }(-3) & \dots \\[4pt] \widehat {\phi }(-2) & \widehat {\phi }(-3) & \widehat {\phi }(-4) &\dots \\[4pt] \widehat {\phi }(-3) & \widehat {\phi }(-4) & \widehat {\phi }(-5) &\dots \\[4pt] \vdots & \vdots &\vdots &\ddots \end {array}\right). \end{equation*}

from which we obtain:

(9) \begin{equation} f(n)=\widehat{\phi }(-n-1). \end{equation}

Since we know how to express the function $f$ with respect to the parameters of the WFA, we can explicitly compute the rational component of the symbol:

(10) \begin{equation} \mathbb{P}_-\phi = \sum _{k\geq 0}f(k) z^{-k-1} = \sum _{k\geq 0} \boldsymbol{\alpha }^{\top }\mathbf{A}^k \boldsymbol{\beta } z^{-k-1} = \boldsymbol{\alpha }^{\top }(z\mathbf{1}-\mathbf{A})^{-1} \boldsymbol{\beta }, \end{equation}

where the last equality holds only if $\rho (A)\lt 1$ .

The correspondence between symbol and function computed by a model allows us to reformulate the approximation problem in terms of Hankel operators and functions in the complex space and to apply AAK theory.

We consider the following example, from Balle et al. (Reference Balle, Lacroce, Panangaden, Precup, Rabusseau, Bansal, Merelli and Worrell2021).

Figure 1. Graphical representation of the generative probabilistic automaton described in Example2.

Example 2. Let $|\Sigma |=1$ , $\Sigma =\{x\}$ , we consider the WFA $A= \langle \boldsymbol{\alpha }, \mathbf{A}, \boldsymbol{\beta }\rangle$ represented in Figure 1 , with:

\begin{equation*} \mathbf {A}= \left( \begin {array}{c@{\quad}c} 0 & \frac {1}{2} \\[4pt] \frac {1}{2} & 0 \end {array}\right), \quad \boldsymbol {\alpha }= \left( \begin {array}{c} \frac {\sqrt {3}}{2} \\[4pt] 0 \end {array}\right), \quad \boldsymbol {\beta }= \left( \begin {array}{c} \frac {\sqrt {3}}{2} \\[4pt] 0 \end {array}\right), \end{equation*}

Note that $A$ is a generative probabilistic automaton. Indeed, we have that

• • $f_A(x)\geq 0$

• • $\sum _{x\in \Sigma ^*}f_A(x)=1$ ,

since the rational function realized by the WFA is defined as:

\begin{equation*} f_A(x\cdots x)=f_A(k)= \boldsymbol {\alpha }^{\top }\mathbf {A}^k\boldsymbol {\beta }= \begin {cases} 0 &\text {if $k$ is odd} \\ \frac {3}{4}2^{-k} &\text {if $k$ is even} \end {cases} \end{equation*}

where $k$ corresponds to the string where $x$ is repeated $k$ -times. We remark that $A$ is minimal and already in its SVA form, with Gramians

(11) \begin{equation} \mathbf{P}=\mathbf{Q}= \left(\begin{array}{c@{\quad}c} \frac{4}{5} & 0 \\[4pt] 0 & \frac{1}{5} \end{array}\right). \end{equation}

The corresponding Hankel matrix, with entries defined as $\mathbf{H}(i,j)=f(i+j)$ , has rank $2$ :

(12) \begin{equation} \mathbf{H}= \left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} f_A(0) & f_A(1) & f_A(2) & \dots \\[4pt] f_A(1) & f_A(2) & f_A(3) &\dots \\[4pt] f_A(2) & f_A(3) & f_A(4) &\dots \\[4pt] \vdots & \vdots &\vdots &\ddots \end{array}\right)= \left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} \frac{3}{4} & 0 & \frac{3}{16} & \dots \\[4pt] 0 & \frac{3}{16} & 0 &\dots \\[4pt] \frac{3}{16} & 0 & \frac{3}{64} &\dots \\[4pt] \vdots & \vdots &\vdots &\ddots \end{array}\right). \end{equation}

Now, we can apply the second interpretation of the Hankel matrix and look at it with respect to the symbol, using the definition $\mathbf{H}(j,k)= \widehat{\phi }(-j-k-1)$ . We have:

\begin{equation*} \mathbf {H}= \left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} \frac {3}{4} & 0 & \frac {3}{16} & \dots \\[4pt] 0 & \frac {3}{16} & 0 &\dots \\[4pt] \frac {3}{16} & 0 & \frac {3}{64} &\dots \\[4pt] \vdots & \vdots &\vdots &\ddots \end {array}\right) =\left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} \widehat {\phi }(-1) & \widehat {\phi }(-2) & \widehat {\phi }(-3) & \dots \\[4pt] \widehat {\phi }(-2) & \widehat {\phi }(-3) & \widehat {\phi }(-4) &\dots \\[4pt] \widehat {\phi }(-3) & \widehat {\phi }(-4) & \widehat {\phi }(-5) &\dots \\[4pt] \vdots & \vdots &\vdots &\ddots \end {array}\right). \end{equation*}

We can recover the rational component of a symbol, that is the projection of $\phi$ on the negative Hardy space.

\begin{equation*} \mathbb {P}_-\phi =\sum _{n \geq 0}\widehat {\phi }(-n-1)z^{-n-1}=\sum _{n \geq 0}\frac {3}{4}4^{-n}z^{-2n-1}=\frac {3z}{4z^2-1}. \end{equation*}

Note that this is a complex rational function having degree $2$ , and it has two poles inside the unit disc at $z=\pm \frac{1}{2}$ (as predicted by Theorem 11). It is important to remark that from the Hankel matrix, we can only recover the negative Fourier coefficients of $\phi$ , meaning only the component of the symbol that belongs to the negative Hardy space.

4.1 Assumptions

We briefly list and analyze the assumptions made to solve the approximate minimization problem. A class of automata that automatically satisfies the following properties is that of generative probabilistic automata.

4.2 Problem formulation

Let $A= \langle \boldsymbol{\alpha }, \mathbf{A}, \boldsymbol{\beta } \rangle$ be a minimal irredundant WFA with $n$ states and real weights, defined over a one-letter alphabet and represented in its SVA form. Let $\mathbf{H}$ be the Hankel matrix of $A$ , we denote with $\sigma _i$ , for $0\leq i \lt n$ , the singular numbers. Given a target number of states $k\lt n$ , we say that a WFA $\widehat{A}_k$ with $k$ states solves the optimal spectral-norm approximate minimization problem if the Hankel matrix $\mathbf{G}$ of $\widehat{A}_k$ satisfies:

(13) \begin{equation} \|\mathbf{H} - \mathbf{G}\|= \sigma _k(\mathbf{H}). \end{equation}

Note that the content of the “optimal spectral-norm approximate minimization” is equivalent to the problem solved by Theorem 12, with the exception that here we represent the inputs and outputs of the problem effectively by means of WFAs.

Based on the AAK theory sketched in Section 2.3, we draw the following steps:

1. 1. Compute a symbol for the WFA. Given an irredundant WFA on a one-letter alphabet, we consider its Hankel matrix $\mathbf{H}$ and the function $f$ that it is computing. We use Equation 10 to associate a complex rational function to the WFA.

2. 2. Compute the optimal symbol $\psi (z)$ using Corollary 14 . The main challenge here is to find a suitable representation for the functions $\psi (z)$ and $e(z)=\phi (z)-\psi (z)$ . We define them in terms of two auxiliary WFAs. The key point is to select constraints on their parameters to leverage the properties of weighted automata, while still keeping the formulation general.

3. 3. Extracting the rational component by solving for $g(z)$ in Corollary 15 . This step is arguably the most conceptually challenging, as it requires to identify the position of the function’s poles. In fact, we know from Theorem 11 that $g(z)$ has $k$ poles, all inside the unit disc.

4. 4. Find a WFA representation for $g(z)$ . Since in Step 2 we parametrized the functions using WFAs, the expression of $g(z)$ directly reveals the WFA $\widehat{A}_k$ .

4.3 Finding the optimal approximation

We analyze each of the steps detailed above.

4.3.1 Finding a symbol for the WFA

Let $A= \langle \boldsymbol{\alpha }, \mathbf{A}, \boldsymbol{\beta } \rangle$ be a minimal irredundant WFA with $n$ states, already represented in SVA form. $A$ realizes a function $f: \Sigma ^* \rightarrow \mathbb{R}$ , defined over a one-letter alphabet $\Sigma =\{a\}$ . Let $\mathbf{H}$ be its Hankel matrix, with corresponding bounded Hankel operator $H$ , and singular numbers $\sigma _i$ , for $0\leq i \lt n$ .

As seen in sub-section 3.1, we can associate a complex function to the WFA. In particular, since we are assuming that $A$ is irredundant, from Equation 10 we obtain an expression for the rational component of the symbol:

(14) \begin{equation} \mathbb{P}_-\phi = \boldsymbol{\alpha }^{\top }(z\mathbf{1}-\mathbf{A})^{-1} \boldsymbol{\beta }. \end{equation}

4.3.2 Finding the optimal symbol

To find the solution to Theorem 12, we need to first derive the function $\psi$ from Theorem 13. Therefore, the second step to solve the approximate minimization problem is to find a proper expression for the complex functions $\psi$ and $e=\phi -\psi$ described in Theorem 13. Since our objective is to find the WFA corresponding to the optimal approximation, we focus on representing these functions using the parameters of two auxiliary WFAs. We consider a WFA $\widehat{A}=\langle \widehat{\boldsymbol{\alpha }},\widehat{\mathbf{A}},\widehat{\boldsymbol{\beta }}\rangle$ with more than $k$ states, such that the automaton $E= \langle \boldsymbol{\alpha }_e, \mathbf{A}_e, \boldsymbol{\beta }_e \rangle$ computing the difference between $A$ and $\widehat{A}$ is minimal, with:

(15) \begin{equation} \mathbf{A}_e= \begin{pmatrix} \mathbf{A} & \mathbf{0} \\[4pt] \mathbf{0} & \widehat{\mathbf{A}} \end{pmatrix}, \quad \boldsymbol{\alpha }_e= \begin{pmatrix} \boldsymbol{\alpha } \\[4pt] -\widehat{\boldsymbol{\alpha }} \end{pmatrix}, \quad \boldsymbol{\beta }_e= \begin{pmatrix} \boldsymbol{\beta } \\[4pt] \widehat{\boldsymbol{\beta }} \end{pmatrix}. \end{equation}

Now, given $C\in \mathscr{H}^{\kern1pt\,\infty }$ , we consider the complex functions:

\begin{align*} &\psi = \widehat{\boldsymbol{\alpha }}^{\top }(z\mathbf{1}-\widehat{\mathbf{A}})^{-1} \widehat{\boldsymbol{\beta }} + C\\ &e= \phi -\psi = \boldsymbol{\alpha }_e^{\top }(z\mathbf{1}-\mathbf{A}_e)^{-1} \boldsymbol{\beta }_e - C. \end{align*}

The idea is that we want to find the parameters of $\widehat{A}$ that make $\psi$ the solution of Theorem 13. By definition, $\psi$ is the sum of two components, one that is bounded around the unit circle and one that has $k$ poles inside the unit disc (where $k$ is the size of the sought approximation). Therefore, there cannot be poles on the unit circle. By looking at the way we defined the function $\psi$ , we can see that its poles correspond to the eigenvalues of $\widehat{\mathbf{A}}$ , counted with their multiplicities. Thus, in order for $\psi$ to be the solution of Theorem 13, $1$ cannot be an eigenvalue of $\widehat{\mathbf{A}}$ , and the WFA $\widehat{A}$ needs to have at least $k$ states.

As remarked in the previous section, the parameters of the automaton $A$ only encode the negative Fourier coefficients of the symbol. We add $C$ to the definition of $\psi$ to account for the $\mathscr{H}^{\kern1pt\,\infty }$ component when considering the difference $\phi -\psi$ . In fact, while this component of the symbol does not affect the spectral norm, it plays a role in the computation of the $\mathscr{L}^{\,\infty }$ -norm (in Equation 5), so it cannot be entirely dismissed. Nonetheless, we won’t need to find the value of $C$ , as ultimately we are only interested in the WFA’s parameters.

Now that we have an expression for $\psi$ and $e$ , we can look back at Theorem 13. From this theorem, we know that by definition, $\sigma _k^{1}e$ is a unimodular function. This property of $e$ can be used to derive a set of constraints on the parameters of the WFA $E=\langle \boldsymbol{\alpha }_e, \mathbf{A}_e, \boldsymbol{\beta }_e\rangle$ . In particular, it is possible to use the maximum modulus principle, according to which the maximum modulus of an holomorphic function is attained on the boundary of the domain. To do so, we leverage a result from the control theory literature (Chui and Chen, Reference Chui and Chen1997), that can be easily applied to our setting. In fact, a parallel can be drawn between dynamical systems and automata, by noting that the impulse response of a discrete time-invariant Single-Input-Single-Output SISO system can be parametrized as a WFA over a one-letter alphabet. This allows us to apply a theorem from Chui and Chen (Chui and Chen (Reference Chui and Chen1997), Theorem 6.3) to find two matrices, $\mathbf{P}_e$ and $\mathbf{Q}_e$ , satisfying properties similar to those of the Gramians. It is important to notice that, a priori, the controllability and observability Gramians of $E$ might not be well defined.

Theorem 17 (Chui and Chen (Reference Chui and Chen1997)).  Consider the function $e= \boldsymbol{\alpha }_e^{\top }(z\mathbf{1}-\mathbf{A}_e)^{-1} \boldsymbol{\beta }_e - C$ and the corresponding minimal WFA $E=\langle \boldsymbol{\alpha }_e, \mathbf{A}_e, \boldsymbol{\beta }_e \rangle$ associated with it. $\sigma _k^{-1}e$ is unimodular if and only if there exists a unique pair of symmetric invertible matrices $\mathbf{P}_e$ and $\mathbf{Q}_e$ satisfying:

1. (1) $\mathbf{P}_e-\mathbf{A}_e \mathbf{P}_e \mathbf{A}_e^\top = \boldsymbol{\beta }_e\boldsymbol{\beta }_e^\top$

2. (2) $\mathbf{Q}_e-\mathbf{A}_e^\top \mathbf{Q}_e \mathbf{A}_e = \boldsymbol{\alpha }_e\boldsymbol{\alpha }_e^\top$

3. (3) $\mathbf{P}_e\mathbf{Q}_e=\sigma ^2_k\mathbf{1}$

We can now derive the parameters of the WFA $\widehat{A}=\langle \widehat{\boldsymbol{\alpha }},\widehat{\mathbf{A}},\widehat{\boldsymbol{\beta }}\rangle$ that make $\psi$ the solution of Theorem 13.

Theorem 18. Let $A=\langle \boldsymbol{\alpha }, \mathbf{A}, \boldsymbol{\beta } \rangle$ be a minimal WFA with $n$ states in its SVA form, and let $\phi = \boldsymbol{\alpha }^{\top }(z\mathbf{1}-\mathbf{A})^{-1}\boldsymbol{\beta }$ be a symbol for its Hankel operator $H$ . Let $\sigma _k$ be a singular number of multiplicity $r$ for $H$ , with:

(16) \begin{equation} \sigma _0 \geq \dots \gt \sigma _k=\dots =\sigma _{k+r-1}\gt \sigma _{k+r}\geq \dots \geq \sigma _{n-1}\gt 0. \end{equation}

We can partition the Gramian matrices $\mathbf{P}$ , $\mathbf{Q}$ as follows:

(17) \begin{equation} \mathbf{P}=\mathbf{Q}=\begin{pmatrix} \boldsymbol{\Sigma } & \mathbf{0} \\[4pt] \mathbf{0} & \sigma _k\mathbf{1}_r \end{pmatrix}, \end{equation}

where $\boldsymbol{\Sigma }\in \mathbb{R}^{(n-r)\times (n-r)}$ is the diagonal matrix containing the remaining singular numbers, and partition $\mathbf{A}$ , $\boldsymbol{\alpha }$ and $\boldsymbol{\beta }$ to conform with the Gramians:

(18) \begin{equation} \mathbf{A}= \begin{pmatrix} \mathbf{A}_{11} & \mathbf{A}_{12} \\[4pt] \mathbf{A}_{21} & \mathbf{A}_{22} \end{pmatrix},\quad \boldsymbol{\alpha }= \begin{pmatrix} \boldsymbol{\alpha }_1 \\[4pt] \boldsymbol{\alpha }_2 \end{pmatrix},\quad \boldsymbol{\beta }= \begin{pmatrix} \boldsymbol{\beta }_1 \\[4pt] \boldsymbol{\beta }_2 \end{pmatrix}. \end{equation}

Let $\mathbf{R}=\sigma _k^2\mathbf{1}_{n-r}-\boldsymbol{\Sigma }^2$ , we denote by $({\cdot})^{+}$ the Moore-Penrose pseudo-inverse. The function $\psi = \widehat{\boldsymbol{\alpha }}^{\top }(z\mathbf{1}-\widehat{\mathbf{A}})^{-1} \widehat{\boldsymbol{\beta }}+C$ is the best approximation of $\phi$ if and only if:

• • If $\boldsymbol{\alpha }_2 \neq \mathbf{0}$ :

  (19) \begin{equation} \begin{cases} \widehat{\boldsymbol{\beta }} = - \widehat{\mathbf{A}}\mathbf{A}_{21}^{\top }(\boldsymbol{\beta }_2^{\top })^{+} \\[4pt] \widehat{\boldsymbol{\alpha }} = \widehat{\mathbf{A}}^{\top }\mathbf{R}\mathbf{A}_{12}(\boldsymbol{\alpha }_2^{\top })^{+}\\[4pt] \widehat{\mathbf{A}}(\mathbf{A}_{11}^{\top }- \mathbf{A}_{21}^{\top }(\boldsymbol{\beta }_2^{\top })^{+}\boldsymbol{\beta }_1^{\top })=\mathbf{1} \end{cases} \end{equation}

• • If $\boldsymbol{\alpha }_2=\mathbf{0}$ :

  (20) \begin{equation} \begin{cases} \widehat{\boldsymbol{\beta }} = (\mathbf{1} -\widehat{\mathbf{A}}\mathbf{A}_{11}^{\top })(\boldsymbol{\beta }_1^{\top })^{+}\\[4pt] \widehat{\boldsymbol{\alpha }} =-(\mathbf{R} -\widehat{\mathbf{A}}^{\top }\mathbf{R}\mathbf{A}_{11})(\boldsymbol{\alpha }_1^{\top })^{+}\\[4pt] \widehat{\mathbf{A}}\mathbf{A}_{21}^{\top }=\mathbf{0} \end{cases} \end{equation}

Proof. We prove the first implication of this proof by applying Theorem 17 and by obtaining from it a set of equations allowing us to derive the parameters of the WFA $\widehat{A}$ . The other direction of the theorem can be easily proved by direct computation.

If $\psi$ is the optimal approximation of $\phi$ , the function $\sigma ^{-1}e=\phi -\psi$ is unimodular and from Theorem 17 there exist two symmetric nonsingular matrices $\mathbf{P}_e$ , $\mathbf{Q}_e$ satisfying the fixed point equations:

(21) \begin{align} \mathbf{P}_e-\mathbf{A}_e\mathbf{P}_e\mathbf{A}_e^{\top }&=\boldsymbol{\beta }_e\boldsymbol{\beta }_e^{\top } \end{align}

(22) \begin{align} \mathbf{Q}_e-\mathbf{A}_e^{\top }\mathbf{Q}_e\mathbf{A}_e&=\boldsymbol{\alpha }_e\boldsymbol{\alpha }_e^{\top } \end{align}

and such that $\mathbf{P}_e\mathbf{Q}_e=\sigma ^2_k\mathbf{1}$ . We can partition $\mathbf{P}_e$ and $\mathbf{Q}_e$ according to the definition of $\mathbf{A}_e$ (see Equation 15):

\begin{equation*} \mathbf {P}_e= \begin {pmatrix} \mathbf {P}_{11} & \mathbf {P}_{12} \\ \mathbf {P}_{12}^{\top } & \mathbf {P}_{22} \end {pmatrix},\quad \mathbf {Q}_e= \begin {pmatrix} \mathbf {Q}_{11} & \mathbf {Q}_{12} \\ \mathbf {Q}_{12}^{\top } & \mathbf {Q}_{22} \end {pmatrix}. \end{equation*}

From Equations 21 and 22, we note that $\mathbf{P}_{11}$ and $\mathbf{Q}_{11}$ correspond to the controllability and observability Gramians of $A$ :

\begin{equation*} \mathbf {P}_{11}=\mathbf {Q}_{11}=\mathbf {P}=\begin {pmatrix} \boldsymbol {\Sigma } & \mathbf {0} \\ \mathbf {0} & \sigma _k\mathbf {1} \end {pmatrix}. \end{equation*}

Moreover, since $\mathbf{P}_e\mathbf{Q}_e=\sigma _k^2\mathbf{1}$ , we get $\mathbf{P}_{12}\mathbf{Q}_{12}^{\top }=\sigma _k^2\mathbf{1}-\mathbf{P}^2$ . It follows that $\mathbf{P}_{12}\mathbf{Q}_{12}^{\top }$ has rank $n-r$ . Without loss of generality we can set $\dim{\widehat{\mathbf{A}}}=j=n-r$ , and choose an appropriate basis for the state space such that $\mathbf{P}_{12}=\begin{pmatrix} \mathbf{1} & \mathbf{0} \end{pmatrix} ^{\top }$ and $\mathbf{Q}_{12}=\begin{pmatrix} \mathbf{R} & \mathbf{0} \end{pmatrix} ^{\top }$ , with $\mathbf{R}=\sigma _k^2\mathbf{1}-\boldsymbol{\Sigma }^2$ . Once $\mathbf{P}_{12}$ and $\mathbf{Q}_{12}$ are fixed, the values of $\mathbf{P}_{22}$ and $\mathbf{Q}_{22}$ are automatically determined. We obtain:

(23) \begin{equation} \mathbf{P}_e= \begin{pmatrix} \boldsymbol{\Sigma } & \mathbf{0} & \mathbf{1}\\ \mathbf{0} & \sigma _k\mathbf{1} & \mathbf{0}\\ \mathbf{1} & \mathbf{0} & -\boldsymbol{\Sigma } \mathbf{R}^{-1} \end{pmatrix},\quad \mathbf{Q}_e= \begin{pmatrix} \boldsymbol{\Sigma } & \mathbf{0} & \mathbf{R}\\ \mathbf{0} & \sigma _k\mathbf{1} & \mathbf{0}\\ \mathbf{R} & \mathbf{0} & -\boldsymbol{\Sigma } \mathbf{R} \end{pmatrix}. \end{equation}

Now that we have an expression for the matrices $\mathbf{P}_e$ and $\mathbf{Q}_e$ of Theorem 17, we can rewrite the fixed point equations to derive the parameters $\widehat{\boldsymbol{\alpha }}$ , $\widehat{\mathbf{A}}$ and $\widehat{\boldsymbol{\beta }}$ . We obtain the following systems:

(24) \begin{equation} \begin{cases} \mathbf{P} -\mathbf{A} \mathbf{P} \mathbf{A}^{\top }= \boldsymbol{\beta }\boldsymbol{\beta }^{\top }\\[4pt] \mathbf{N} -\mathbf{A} \mathbf{N} \widehat{\mathbf{A}}^{\top }= \boldsymbol{\beta }\widehat{\boldsymbol{\beta }}^{\top }\\[4pt] -\boldsymbol{\Sigma } \mathbf{R}^{-1} +\widehat{\mathbf{A}}\boldsymbol{\Sigma } \mathbf{R}^{-1}\widehat{\mathbf{A}}^{\top }={\widehat{\boldsymbol{\beta }}}{\widehat{\boldsymbol{\beta }}}^{\top } \end{cases} \quad \begin{cases} \mathbf{P} -\mathbf{A}^{\top } \mathbf{P} \mathbf{A}= \boldsymbol{\alpha }\boldsymbol{\alpha }^{\top }\\[4pt] \mathbf{M} -\mathbf{A}^{\top } \mathbf{M} \widehat{\mathbf{A}}= -\boldsymbol{\alpha }\widehat{\boldsymbol{\alpha }}^{\top }\\[4pt] -\boldsymbol{\Sigma } \mathbf{R} +\widehat{\mathbf{A}}^{\top }\boldsymbol{\Sigma } \mathbf{R}\widehat{\mathbf{A}}={\widehat{\boldsymbol{\alpha }}}{\widehat{\boldsymbol{\alpha }}}^{\top } \end{cases} \end{equation}

where $\mathbf{N}= \begin{pmatrix} \mathbf{1} \\ \mathbf{0} \end{pmatrix}$ and $\mathbf{M}= \begin{pmatrix} \mathbf{R} \\ \mathbf{0} \end{pmatrix}$ .

We can rewrite the second equation of each system as follows:

(25) \begin{equation} \begin{cases} \mathbf{1} -\mathbf{A}_{11} \widehat{\mathbf{A}}^{\top }= \boldsymbol{\beta }_1\widehat{\boldsymbol{\beta }}^{\top }\\[4pt] -\mathbf{A}_{21}\widehat{\mathbf{A}}^{\top }=\boldsymbol{\beta }_2\widehat{\boldsymbol{\beta }}^{\top } \end{cases} \quad \begin{cases} \mathbf{R} -\mathbf{A}_{11}^{\top } \mathbf{R} \widehat{\mathbf{A}}=-\boldsymbol{\alpha }_1\widehat{\boldsymbol{\alpha }}^{\top }\\[4pt] \widehat{\mathbf{A}}^{\top }\mathbf{R}\mathbf{A}_{12}=\widehat{\boldsymbol{\alpha }}\boldsymbol{\alpha }_2^{\top } \end{cases} \end{equation}

If $\boldsymbol{\alpha }_2 \neq \mathbf{0}$ , then also $\boldsymbol{\beta }_2 \neq \mathbf{0}$ (recall that $\boldsymbol{\alpha }_i=\operatorname{sgn}(\lambda _i)\boldsymbol{\beta }_i$ ), and we have:

(26) \begin{equation} \begin{cases} \widehat{\boldsymbol{\beta }} = - \widehat{\mathbf{A}}\mathbf{A}_{21}^{\top }(\boldsymbol{\beta }_2^{\top })^{+} \\[4pt] \widehat{\boldsymbol{\alpha }} = \widehat{\mathbf{A}}^{\top }\mathbf{R}\mathbf{A}_{12}(\boldsymbol{\alpha }_2^{\top })^{+}\\[4pt] \widehat{\mathbf{A}}(\mathbf{A}_{11}^{\top }- \mathbf{A}_{21}^{\top }(\boldsymbol{\beta }_2^{\top })^{+}\boldsymbol{\beta }_1^{\top })=\mathbf{1} \end{cases} \end{equation}

with $(\boldsymbol{\alpha }_2^{\top })^{+}=\frac{\boldsymbol{\alpha }_2}{\boldsymbol{\alpha }_2^{\top }\boldsymbol{\alpha }_2}$ and $(\boldsymbol{\beta }_2^{\top })^{+}=\frac{\boldsymbol{\beta }_2}{\boldsymbol{\beta }_2^{\top }\boldsymbol{\beta }_2}$ .

If $\boldsymbol{\alpha }_2=\mathbf{0}$ , we have $\widehat{\mathbf{A}}\mathbf{A}_{21}^{\top }=\mathbf{0}$ , in which case we obtain the following set of solutions:

(27) \begin{equation} \begin{cases} \widehat{\boldsymbol{\beta }} = (\mathbf{1} -\widehat{\mathbf{A}}\mathbf{A}_{11}^{\top })(\boldsymbol{\beta }_1^{\top })^{+}\\[4pt] \widehat{\boldsymbol{\alpha }} =-(\mathbf{R} -\widehat{\mathbf{A}}^{\top }\mathbf{R}\mathbf{A}_{11})(\boldsymbol{\alpha }_1^{\top })^{+}\\[4pt] \widehat{\mathbf{A}}\mathbf{A}_{21}^{\top }=\mathbf{0} \end{cases}. \end{equation}

This completes the proof of the first implication. The other direction of the proof can be verified by direct computation by first obtaining the matrices $\mathbf{P}_e$ and $\mathbf{Q}_e$ , and then showing that they are symmetric, invertible, and satisfy the set of equations of Theorem 17, hence proving that $\sigma _k^{-1}e$ is unimodular.

We remark that, when $\boldsymbol{\alpha }_2=0$ , the solution returned by the algorithm will depend on the size of the original automaton and the target approximation. Specifically, $\widehat{\mathbf{A}}$ has size $(n-r)\times (n-r)$ , while $\mathbf{A}_{21}^{\top }$ is $(n-r)\times r$ , so the system of equations corresponding to $\widehat{\mathbf{A}}\mathbf{A}_{21}^{\top }=\mathbf{0}$ is underdetermined if $r\lt \frac{n}{2}$ , in which case we obtain the solutions in Equation 20, with $\widehat{\mathbf{A}}\neq \mathbf{0}$ . On the other hand, if $r\geq \frac{n}{2}$ , that is if the multiplicity of the singular number $\sigma _k$ is more than half the size of the original WFA, the system might not have any solution unless $\widehat{\mathbf{A}}=\mathbf{0}$ (or unless $\mathbf{A}_{21}$ was zero to begin with). In this setting, the method proposed returns $\widehat{\mathbf{A}}=\mathbf{0}$ . In the (rare) case in which the algorithm returns $\widehat{\mathbf{A}}=\mathbf{0}$ , an alternative and preferable approach is to search for an approximation of size $k-1$ or $k+1$ . This way, the multiplicity $r$ of the singular number $\sigma _k$ is such that $r\lt \frac{n}{2}$ , and the system in Equation 27 is underdetermined.

Theorem 18 provides us with a way to compute the coefficients of the function $\psi$ solving Theorem 13. It is important to notice that the WFA $A_k$ is not necessarily the best approximation we are looking for. Intuitively, the problem is that it might be too big, as irredundancy is not guaranteed by the system of equations (while we know from AAK theory that the best approximation corresponds to a bounded operator). Therefore, in these cases we need to “extract” from $A_k$ a smaller WFA of size $k$ . We do this by extracting the component of the function $\psi$ that belongs to the negative Hardy space.

4.3.3 Extracting the rational component

The objective of this section is to “isolate” the function $g\in \mathscr{R}_k$ , that is the rational component of $\psi$ . To do this, we study the position of the poles of $\psi$ . In fact, we know from Theorem 11 that the poles of a strictly proper rational function lie inside the unit disc. As noted before, the key to solving our problem is the way we parametrized the functions. We defined $\psi$ so that its poles correspond to the eigenvalues of $\widehat{A}$ . Therefore, we study the eigenvalues of $\widehat{\mathbf{A}}$ using the following auxiliary result from Ostrowski and Schneider (Reference Ostrowski and Schneider1962). A proof of this theorem can be found in Wimmer (Reference Wimmer1973).

Theorem 19 (Ostrowski and Schneider (Reference Ostrowski and Schneider1962)).  Let $|\Sigma |=1$ , and let $\mathbf{P}$ be a solution to the fixed point equation $X-\mathbf{A} X\mathbf{A}^{\top }=\boldsymbol{\beta }\boldsymbol{\beta }^{\top }$ for the WFA $A=\langle \boldsymbol{\alpha }, \mathbf{A}, \boldsymbol{\beta } \rangle$ . If $A$ is reachable, then:

• • The number of eigenvalues $\lambda$ of $\mathbf{A}$ such that $|\lambda |\lt 1$ is equal to the number of positive eigenvalues of $\mathbf{P}$ .

• • The number of eigenvalues $\lambda$ of $\mathbf{A}$ such that $|\lambda |\gt 1$ is equal to the number of negative eigenvalues of $\mathbf{P}$ .

After a change of basis (that we detail in Section 5 with the approximation algorithm), we can rewrite $\widehat{\mathbf{A}}$ in block-diagonal form:

(28) \begin{equation} \widehat{\mathbf{A}}= \begin{pmatrix} \widehat{\mathbf{A}}_+ & \mathbf{0} \\ \mathbf{0} & \widehat{\mathbf{A}}_- \end{pmatrix} \end{equation}

where the modulus of the eigenvalues of $\widehat{\mathbf{A}}_+$ (resp. $\widehat{\mathbf{A}}_-$ ) is smaller (resp. greater) than one. We then apply the same change of coordinates on $\widehat{\boldsymbol{\alpha }}$ and $\widehat{\boldsymbol{\beta }}$ .

We can finally find the rational component of the function $\psi$ , that is the function $g$ from Corollary 15 necessary to solve that approximate minimization problem.

Theorem 20. Let $\widehat{\mathbf{A}}_+$ be as in Equation 28 , and $\widehat{\boldsymbol{\alpha }}_+, \widehat{\boldsymbol{\beta }}_+$ obtained applying the same change of basis. The rational component of $\psi$ is the function $g= \widehat{\boldsymbol{\alpha }}_+^{\top }(z\mathbf{1}-\widehat{\mathbf{A}}_+)^{-1}\widehat{\boldsymbol{\beta }}_+$ .

Proof. Clearly $\psi =g+ l$ , with $l=\widehat{\boldsymbol{\alpha }}_-^{\top }(z\mathbf{1}-\widehat{\mathbf{A}}_-)^{-1}\widehat{\boldsymbol{\beta }}_-$ , $l \in \mathscr{H}^{\kern1pt\,\infty }$ . To conclude the proof we need to show that $g$ has $k$ poles inside the unit disc, and that therefore it has rank $k$ . We do this by studying the modulus of the eigenvalues of $\widehat{\mathbf{A}}_+$ .

Since $E$ is minimal, $\widehat{A}$ is reachable by definition, so we can use Theorem 19 and solve the problem by directly examining the eigenvalues of $-\boldsymbol{\Sigma } \mathbf{R}$ . From the proof of Theorem 18, we have $-\boldsymbol{\Sigma } \mathbf{R}=\boldsymbol{\Sigma }(\boldsymbol{\Sigma }^2-\sigma ^2_k\mathbf{1})$ , where $\boldsymbol \Sigma$ is the diagonal matrix having as elements the singular numbers of $H$ different from $\sigma _k$ . It follows that $-\boldsymbol{\Sigma } \mathbf{R}$ has only $k$ strictly positive eigenvalues, and $\widehat{\mathbf{A}}$ has $k$ eigenvalues with modulus smaller than $1$ . Thus, $\widehat{\mathbf{A}}_+$ has $k$ eigenvalues, corresponding to the poles of $g$ .

4.3.4 Solving the approximation problem

Now that we have found the rational function $g$ , a symbol for the operator that solves Theorem 12, we need to find the parameters of $\widehat{A}_k$ , the WFA corresponding to the optimal approximation. These are directly revealed by the expression of $g$ , due to the function’s parametrization.

Theorem 21. Let $A= \langle \boldsymbol{\alpha }, \mathbf{A}, \boldsymbol{\beta } \rangle$ be a minimal WFA with $n$ states over a one-letter alphabet. Let $A$ be in its SVA form. The optimal spectral-norm approximation of rank $k$ is given by the WFA $\widehat{A}_k= \langle \widehat{\boldsymbol{\alpha }}_+, \widehat{\mathbf{A}}_+, \widehat{\boldsymbol{\beta }}_+ \rangle$ .

Proof. From Corollary 15, we know that $g$ is the rational function associated with the Hankel matrix of the best approximation. Given the correspondence between the Fourier coefficients of $g$ and the entries of the matrix, we have:

(29) \begin{equation} g= \widehat{\boldsymbol{\alpha }}_+^{\top }(z\mathbf{1}-\widehat{\mathbf{A}}_+)^{-1}\widehat{\boldsymbol{\beta }}_+ = \sum _{k\geq 0} \widehat{\boldsymbol{\alpha }}_+^{\top }\widehat{\mathbf{A}}_+^k \widehat{\boldsymbol{\beta }}_+ z^{-k-1} =\sum _{k\geq 0}\bar{f}(k) z^{-k-1} \end{equation}

where $\bar{f}:\Sigma ^* \rightarrow \mathbb{R}$ is the function computed by $\widehat{A}_k$ and $\widehat{\boldsymbol{\alpha }}_+, \widehat{\mathbf{A}}_+, \widehat{\boldsymbol{\beta }}_+$ are the parameters.

4.4 Error analysis

Thanks to the use of AAK theory, the method outlined in the previous sections is guaranteed to return the rank $k$ optimal spectral-norm approximation of a WFA satisfying our assumptions, and the singular number $\sigma _k$ provides the error. As noticed before, since the Hankel matrix has finite rank and we can derive the Gramian matrices of the WFA, the singular number corresponding to the error can be computed precisely, even though the Hankel matrix is infinite.

Similarly to the case of SVA truncation (Balle et al., Reference Balle, Panangaden and Precup2019), owing to the ordering of the singular numbers, the error decreases when $k$ increases, meaning that allowing $\widehat{A}_k$ to have more states guarantees a better approximation of $A$ . Note that the solution we propose is optimal in the spectral norm, but it might not be the case in other norms. Nonetheless, we have the following bound between $\ell ^2$ norm and spectral norm.

Theorem 22. Let $A$ be a minimal WFA computing $f:\Sigma ^* \rightarrow \mathbb{R}$ , with matrix $\mathbf{H}$ . Let $\widehat{A}_k$ be its optimal spectral-norm approximation, computing $g:\Sigma ^* \rightarrow \mathbb{R}$ , with matrix $\mathbf{G}$ . Then:

(30) \begin{equation} \|f-g\|_{\ell ^2} \leq \|\mathbf{H} - \mathbf{G}\| = \sigma _k. \end{equation}

Proof. Let $\mathbf{e}_0=\begin{pmatrix} 1 & 0 & \cdots \end{pmatrix}^{\top }$ , $f:\Sigma ^*\rightarrow \mathbb{R}$ , $g:\Sigma ^*\rightarrow \mathbb{R}$ with Hankel matrices $\mathbf{H}$ and $\mathbf{G}$ , respectively. We have:

\begin{align*} \|f-g\|_{\ell ^2} &=\left (\sum _{n=0}^{\infty }|f_n-g_n|^2 \right )^{1/2}\\ &=\|(\mathbf{H}-\mathbf{G}) \mathbf{e}_0\|_{\ell ^2}\\ &\leq \sup _{\|\mathbf{x}\|_{\ell ^2}=1}\|(\mathbf{H}-\mathbf{G})\mathbf{x}\|_{\ell ^2}\\ &=\|\mathbf{H}-\mathbf{G}\|=\sigma _k \end{align*}

where the second equation follows by definition and by observing that matrix difference is computed entry-wise.

5.1 Block diagonalization

The algorithm involves a call to Algorithm2, BlockDiagonalize. This algorithm corresponds to the steps necessary to derive the WFA $\widehat{A}_k$ associated with the rational function $g$ . One way to solve the problem is to compute the Jordan form of the matrix. Unfortunately, this problem is ill-conditioned, so it is not suitable for our algorithmic purposes. Following the steps of Glover (Reference Glover1984), we compute the Schur decomposition, that is we find an orthogonal matrix $\mathbf{U}$ such that the matrix $\mathbf{U}^{\top }\widehat{\mathbf{A}}\mathbf{U}$ is upper triangular, with the eigenvalues of $\widehat{\mathbf{A}}$ on the diagonal. We obtain:

(31) \begin{equation} \mathbf{T}=\mathbf{U}^{\top }\widehat{\mathbf{A}}\mathbf{U}= \begin{pmatrix} \widehat{\mathbf{A}}_{+} & \widehat{\mathbf{A}}_{12} \\ \mathbf{0} & \widehat{\mathbf{A}}_{-} \end{pmatrix} \end{equation}

where the eigenvalues are arranged in increasing order of modulus, and the modulus of those in $\widehat{\mathbf{A}}_{+}$ (resp. $\widehat{\mathbf{A}}_{-}$ ) is smaller (resp. greater) than one. To transform this upper triangular matrix into a block-diagonal one, we use the following result.

Theorem 23 (Roth (Reference Roth1952)).  Let $\mathbf{T}$ be the matrix defined in Equation 31 . The matrix $\mathbf{X}$ is a solution of the equation $\widehat{\mathbf{A}}_{+}\mathbf{X}- \mathbf{X}\widehat{\mathbf{A}}_{-} +\widehat{\mathbf{A}}_{12}=\mathbf{0}$ if and only if the matrices

(32) \begin{equation} \mathbf{M}=\begin{pmatrix} \mathbf{1} & \mathbf{X} \\ \mathbf{0} & \mathbf{1} \end{pmatrix}, \quad \text{and} \quad \mathbf{M}^{-1}=\begin{pmatrix} \mathbf{1} & -\mathbf{X} \\ \mathbf{0} & \mathbf{1} \end{pmatrix} \end{equation}

satisfy:

(33) \begin{equation} \mathbf{M}^{-1}\mathbf{T}\mathbf{M}=\begin{pmatrix} \widehat{\mathbf{A}}_{+} & \mathbf{0} \\ \mathbf{0} & \widehat{\mathbf{A}}_{-} \end{pmatrix}, \end{equation}

where $\mathbf{T}$ is the matrix defined in Equation 31 .

Setting $\boldsymbol{\Gamma }=\begin{pmatrix}\mathbf{1}_k & \mathbf{0} \end{pmatrix}$ we can now derive the rational component of the WFA:

(34) \begin{align} &\widehat{\mathbf{A}}_+= \boldsymbol{\Gamma }\mathbf{M}^{-1}\mathbf{U}^{\top }\widehat{\mathbf{A}}\mathbf{U}\mathbf{M}\boldsymbol{\Gamma }^{\top } \end{align}

(35) \begin{align} &\widehat{\boldsymbol{\alpha }}_+= \boldsymbol{\Gamma } \mathbf{M}^{\top }\mathbf{U}^{\top }\widehat{\boldsymbol{\alpha }} \end{align}

(36) \begin{align} &\widehat{\boldsymbol{\beta }}_+= \boldsymbol{\Gamma }\mathbf{M}^{-1}\mathbf{U}^{\top }\widehat{\boldsymbol{\beta }}. \end{align}

The algorithm BlockDiagonalize corresponds to the implementation of this procedure, and Step $5$ can be performed using the Bartels-Stewart algorithm (Bartels and Stewart, Reference Bartels and Stewart1972).

Algorithm 2: BlockDiagonalize

5.2 Computational cost

The running time of BlockDiagonalize with input a WFA $\widehat{A}$ with $(n-r)$ states is thus in $O((n-r)^3)$ , where $r$ is the multiplicity of the singular value considered. The running time of AAKapproximation for an input WFA $\widehat{A}$ with $n$ states is in $O((n-r)^3)$ . In particular, it is possible to analyze the cost associated with each step of the algorithms (Trefethen and Bau III, Reference Trefethen and Bau1997):

• • The product of two $n\times n$ matrices can be computed in time $O(n^3)$ using a standard iterative algorithm.

• • The inversion of a $n\times n$ matrix can be computed in time $O(n^3)$ using Gauss-Jordan elimination.

• • The computation of the Schur decomposition of a $n\times n$ matrix can be done with a two-step algorithm, where each step takes $O(n^3)$ , using the Hessenberg form of the matrix.

• • The Bartels-Stewart algorithm applied to upper triangular matrices to find a matrix of size $m\times n$ takes $O(mn^2+nm^2)$ .

8.1 Removing the finite-rank assumption

The proof of Theorem 12 is constructive for any compact Hankel operator. In the setting of this paper, compactness is guaranteed, as the operator corresponding to an irredundant WFA has finite rank and is bounded. While boundness is necessary for compactness, the finite-rank hypothesis is not. Therefore, an interesting extension of this work is to investigate other classes of models by relaxing the finite-rank (or finite state) assumption. An example of models corresponding to infinite-rank Hankel matrices is recurrent neural networks (RNNs) (Hochreiter and Schmidhuber, Reference Hochreiter and Schmidhuber1997). Recently, particular attention has been given to the problem of extracting, from an RNN, a weighted finite automaton (Ayache et al., Reference Ayache, Eyraud and Goudian2018; Rabusseau et al., Reference Rabusseau, Li, Precup, Chaudhuri and Sugiyama2019; Weiss et al., Reference Weiss, Goldberg and Yahav2019; Theertha Suresh et al., Reference Theertha Suresh, Roark, Riley and Schogol2021; Okudono et al., Reference Okudono, Waga, Sekiyama and Hasuo2020; Eyraud and Ayache Reference Eyraud and Ayache2024; Zhang et al., Reference Zhang, Du, Xie, Ma, Liu and Sun2021). In this sense, the knowledge distillation task (Hinton et al., Reference Hinton, Vinyals and Dean2015) is very similar to an approximate minimization problem, since WFAs are a less expensive alternative to RNNs, while still being expressive and suited for sequence modeling and prediction (Cortes et al., Reference Cortes, Haffner and Mohri2004; Denis and Esposito, Reference Denis and Esposito2008). In Lacroce et al. (Reference Lacroce, Panangaden, Rabusseau, Chandlee, Eyraud, Heinz, Jardine and van Zaanen2021), we investigated the use of AAK theory on black-box models trained for language modeling on sequential data. In particular, we showed that compactness is automatically respected by black boxes for language modeling, and proposed an algorithm for the one-letter setting, based on AAK theory. This particular extension of the method presented in this paper constitutes a first fundamental step towards developing provable approximation algorithms for black-box models.

8.2 Removing the spectral radius assumption

One could consider a WFA over a one-letter alphabet with $\rho (\mathbf{A})\neq 1$ , that is not necessarily irredundant. In this case, the method proposed in the previous sections can be extended and the quality of the approximation can be estimated, but the result is not optimal in the spectral norm. Once again, we draw inspiration from the control theory literature, where some theoretical work has been done to study an analogous approach for continuous time systems and their approximation error (Glover, Reference Glover1984).

The key idea is to block-diagonalize $\mathbf{A}$ like we did in Section 4.3.3. This way, we obtain two components, $\mathbf{A}_+$ and $\mathbf{A}_-$ , with the property that $\rho \lt 1$ and $\rho \gt 1$ , respectively. We tackle each component separately. The case of $A_+=\langle \boldsymbol{\alpha }_+, \mathbf{A}_+, \boldsymbol{\beta }_+ \rangle$ , the component having $\rho (\mathbf{A})\lt 1$ , can be dealt with in the way presented in the previous sections. This means that we can find an optimal spectral-norm approximation of the desired size for $A_+$ . Then, we can consider the second component, $A_-=\langle \boldsymbol{\alpha }_-, \mathbf{A}_-, \boldsymbol{\beta }_- \rangle$ . In this case, we apply the transformation

\begin{equation*} z^{j-1} \mapsto z^{-j} \quad \text {for}\,\, j\geq 1 \end{equation*}

to the symbol $\phi ^{\prime}(z)$ associated to $A_-$ . Then, the function

\begin{equation*} \phi ^{\prime}(z^{-1})= \sum _{k\geq 0} \boldsymbol {\alpha }_-^{\top }\mathbf {A}_-^k z^k \boldsymbol {\beta }_- = \boldsymbol {\alpha }_-^{\top }(\mathbf {1}-z\mathbf {A}_-)^{-1}\boldsymbol {\beta }_- \end{equation*}

is well defined, as the series converges for $z$ with small enough modulus. The use of this transformation allows us to obtain a function having poles only inside the unit disc, and to apply the method presented in this chapter. We remark that in this case, an important choice to make is the size of the target approximation of $A_-$ , as it can influence the quality of the result. Analyzing the effects of this parameter on the approximation error is an interesting direction for future work, both on the theoretical and experimental side.

8.3 Removing the one-letter assumption

The most pressing direction for future work is undoubtedly to extend our results to a multi-letter setting. The work of Adamyan, Arov and Krein provides us with a powerful theory connecting sequences to the study of complex functions. Unfortunately, this approach cannot be directly generalized to the multi-letter case, when $\Sigma ^*$ is a noncommutative monoid, as it requires to generalize standard harmonic analysis results to the non-abelian case. A recent line of work in multivariable operator theory has been centered around extending results of standard operator theory to the case of noncommutative operators defined on Fock spaces Frazho (Reference Frazho1982); Bunce (Reference Bunce1984); Arias and Popescu (Reference Arias and Popescu1995); Popescu (Reference Popescu1989, Reference Popescu1992, Reference Popescu1993, Reference Popescu1995, Reference Popescu2003, Reference Popescu2006, Reference Popescu2010, Reference Popescu2013); Ball and Bolotnikov (Reference Ball and Bolotnikov2021); Jury et al. (Reference Jury, Martin and Shamovich2021). In particular, a noncommutative definition of Hankel operator, and a noncommutative version of the AAK theorem are presented in a recent work of Popescu Popescu (Reference Popescu2003), but its proof is not constructive. Therefore, solving the approximate minimization problem for multi-letter alphabets using AAK theory comes with two distinct challenges:

• • Finding a noncommutative Hankel operator: given a WFA and its Hankel matrix, we need to find a way to reformulate the approximation problem using multivariable operators. In particular, we need to find a noncommutative analogue of the Hardy space and of the symbol.

• • Making AAK constructive: the proof of the noncommutative version of the AAK theorem does not provide us with an expression for the optimal approximation. An interesting direction would be to explore ways to extend the proof to a constructive one.

In Lacroce et al. (Reference Lacroce, Panangaden and Rabusseau2022), we proposed a framework to associate a noncommutative Hankel operator (defined on a noncommutative version of the Hardy space) and a noncommutative rational function to the Hankel matrix computed by a model on sequential data, solving the first point listed above. In the one-letter case, obtaining the framework allowed us to reformulate the approximation problem in terms of functional analysis, and to solve it using the constructive proof of AAK theorem. In Lacroce (Reference Lacroce2022), we tried to address the question of whether or not the proof of the noncommutative AAK theorem can be made constructive. While we did not manage to provide a definitive answer, we laid out possible approaches that can be used to tackle the problem of making the proof of the noncommutative version of AAK theorem constructive.