1. Introduction
The famous Hopfield model of associative memory [Reference Hopfield13] provides basic ideas for neural computing and has attracted a lot of interest. Neurone in the network takes discrete value (e.g.,
$-1$
or
$1$
) to reflect a binary pattern, and a set of patterns is stored such that when a new pattern is presented, the network responds by producing a stored pattern that most closely resembles the new pattern. This is the basic mechanism for the binary pattern recognition using an associative-memory network, and the memories are reflected by dynamically stable attractors [Reference Aoyagi1, Reference Aonishi2, Reference Hoppensteadt and E.9, Reference Heger and Krischer10]. One advantage of this type of model is that it can be implemented using variety of small devices, such as phase-locked loop circuits [Reference Hoppensteadt and E.9].
As a paradigm for synchronisation of coupled oscillators, Kuramoto model [Reference Kuramoto14] was proposed in the 1970s and has attracted much attention owing to its significance in mathematical physics and various applications in different fields such as neuroscience and engineering [Reference Choi, Ha, Jung and Kim3–Reference Dorfler and Bullo5, Reference Ha, Noh and Park7]. Recent findings suggest that the system of coupled Kuramoto-type oscillators with Hebbian learning rule provides a simple model to implement the associative memory for error-free binary pattern retrieval [Reference Follmann, Macau and Rosa6, Reference Hölzel and Krischer11, Reference Hölzel and Krischer12, Reference Li, Zhao and Xue16, Reference Zhao, Li and Xue19, Reference Zhao, Li and Xue20]. The purpose of this paper is to continue the earlier study in refs. [Reference Li, Zhao and Xue16, Reference Zhao, Li and Xue19] regarding the following system for binary pattern retrieval problems:
\begin{equation} {\dot \varphi }_i =\frac{1}{N}\sum _{j=1}^{N} C_{ij}\sin\!(\varphi _{j} - \varphi _i)+\frac{\varepsilon }{N}\sum _{j=1}^{N}\sin 2(\varphi _{j}-\varphi _{i}) \end{equation}
where
$\varphi _i$
is a function of time and denotes the phase of
$i$
-th oscillator,
$N$
is the size of the network and
$\varepsilon \gt 0$
is the strength of the second-order Fourier term. The coupling matrix
$C=[C_{ij}]$
, reflecting the so-called Hebbian rule [Reference Hebb8], is set as
$C_{ij}\;:\!=\;\sum _{k=1}^{M}\xi ^k_i\xi ^k_j$
, where
$\{\xi ^1, \xi ^2,\dots, \xi ^M\}\subseteq \{-1,1\}^N$
is the set of so-called memorised binary patterns stored in the network. To the knowledge of the authors, this model was proposed in ref. [Reference Nishikawa, Lai and Hoppensteadt18] by Nishikawa, Lai, and Hoppensteadt. Notice that
$\xi ^k$
and
$-\xi ^k$
can be regarded as the same pattern; in fact, the coupling coefficient
$C_{ij}$
does not change if
$\xi ^k$
is replaced by
$-\xi ^k$
.
For any binary pattern
$\xi \in \{-1,1\}^N$
, there is a unique (up to constant translation) phase-locked state
$\varphi ^*(\xi )=(\varphi _1^*(\xi ),\varphi _2^*(\xi ),\dots,\varphi _N^*(\xi ))$
corresponding to the pattern
$\xi$
, which is characterised by
\begin{align*} \left |\varphi _{i}^*(\xi )-\varphi _{j}^*(\xi )\right |=\begin{cases}0 \quad \mathrm{mod}\,\, 2\pi, &\xi _{i}=\xi _{j};\\[5pt] \pi \quad \mathrm{mod} \,\,2\pi, &\xi _{i}\ne \xi _{j}, \end{cases} \end{align*}
where
$i,j=1,2,\dots,N$
and
$i\neq j$
. The coefficient
$\varepsilon$
can be regarded as an adjustable parameter that controls the stability/instability of equilibria and leads to rich dynamical properties. The symmetry of the matrix
$C$
guarantees that (1.1) can be written as a gradient system
$\dot{\varphi }=-\nabla f(\varphi )$
with
$\varphi =(\varphi _1,\varphi _2,\dots,\varphi _N)$
and potential
\begin{equation*}f(\varphi )=-\frac {1}{2N}\sum _{i=1}^{N}\sum _{j=1}^{N}C_{ij}\cos (\varphi _j-\varphi _i) -\frac {\varepsilon }{4N}\sum _{i=1}^{N}\sum _{j=1}^{N}\cos 2(\varphi _j-\varphi _i).\end{equation*}
For any initial data, the solution of the network (1.1) will evolve towards an equilibrium which locally minimises the potential [Reference Li and Xue15]. Let
$\xi ^{\mathrm{def}}$
be a defective copy of one of the memorised patterns and
$\varphi (0)$
be an initial data reflecting
$\xi ^{\mathrm{def}}$
, the solution of (1.1) will converge to a phase-locked state corresponding to a memorised pattern that most closely resembles the defective pattern. For notational convenience, we denote this process or the problem by
$\mathrm{PR}(\xi ^1, \xi ^2,\dots, \xi ^M; \xi ^{\mathrm{def}})$
. The overlap
\begin{equation*} m(\xi )=\bigg |\frac {1}{N} \sum _{i=1}^{N}\xi _{i} e^{\sqrt {-1}\varphi _i(t)}\bigg | \end{equation*}
measures the closeness of the solution
$\varphi (t)$
to the pattern
$\xi$
. Due to the global phase shift invariance in (1.1),
$m(\xi )$
is invariant under global rotations.
Let
$\{\eta ^1, \eta ^2,\dots, \eta ^M\}$
be a given set of standard binary patterns, and
$\eta ^{\mathrm{def}}$
be a defective copy of one of the standard patterns, our task is to recognise the standard pattern closest to the defective one. For the systems solving the above problems, a principal issue concerns the stability/instability of equilibria corresponding to binary patterns. The stability of standard patterns
$\{\xi ^k\}_{k=1}^{M}$
and instability of others
$\{-1, 1\}^N\setminus \{\xi ^k\}_{k=1}^{M}$
are desired, so that the standard patterns can be produced in the evolution rather than others. When we simply set the memorised patterns identical to the standard ones, the stability was studied by linearisation in refs. [Reference Nishikawa, Hoppensteadt and Lai17, Reference Nishikawa, Lai and Hoppensteadt18] where the stability/instability is conditional to the eigenspectrum of Jacobian. For system (1.1), the mutual orthogonality in memorised patterns was incorporated [Reference Li, Zhao and Xue16, Reference Zhao, Li and Xue19], and the authors proposed a concept that a binary pattern
$\eta$
is called
$\varepsilon$
-independently (asymptotically) stable if
$\varphi ^*(\eta )$
is (asymptotically) stable for any
$\varepsilon \gt 0$
. Then the benefits are at last two-fold. First, easy conditions rather than the eigenvalues for stability/instability are presented. Second, the memorised patterns are
$\varepsilon$
-independently stable, which enables us to distinguish the memorised patterns and most others.
However, it is highly desired to solve problems with standard patterns lacking the orthogonality in practical situations since the orthogonality usually does not hold. For this aim, a unified approach and the corresponding algorithm are proposed in ref. [Reference Zhao, Li and Xue20], which combines the strategy of subgrouping and orthogonal lifting of two general patterns, where the strategy of orthogonal lifting enables us to exploit the advantages of mutual orthogonality, see Subsection 2.2. A natural question then arises:
Can we use the orthogonal lifting for subgroups containing more than two patterns so that the times of retrieval processes can be reduced evidently?
An example in ref. [Reference Zhao, Li and Xue20, Subsection III.C] shows that the natural extension of the orthogonal lift for three patterns, which quadruples the dimension of patterns, typically fails to identify the appropriate memorised patterns. In this paper, we propose a least orthogonal lift for a group of three patterns, which enables us to transform the general problem into subproblems with three mutually orthogonal memorised patterns in lowest possible dimension. Moreover, we investigate the critical strength of the second-order Fourier term for any set of three mutually orthogonal memorised patterns, which gives a criterion for the set-up of the parameter making the memorised patterns stable and all other patterns unstable. This strategy guarantees error-free retrieval as well, and it reduces the number of subproblems for solving the pattern retrieval problem in comparison with [Reference Zhao, Li and Xue20]. Simulations are presented to illustrate the approach.
Organisation of paper. In Section 2, we review the Kuramoto-type system and present some existing results. In Section 3, we consider the model with three memorised binary patterns, present the least orthogonal lift and estimate the range of the parameter for achieving error-free retrieval. In Section 4, an approach for general standard patterns and simulations are presented. Section 5 is devoted to a brief conclusion.
2. Preliminaries
In this section, we give a brief review of some known results and an approach for pattern retrieval. Consider the dynamical Equations (2.1), i.e.,
\begin{equation} {\dot \varphi }_i =\frac{1}{N}\sum _{j=1}^{N} \sum _{k=1}^{M}\xi _{i}^{k}\xi _{j}^{k}\sin\!(\varphi _{j} - \varphi _i)+\frac{\varepsilon }{N}\sum _{j=1}^{N}\sin 2(\varphi _{j}-\varphi _{i}), \; i =1, 2, \dots, N, \end{equation}
with memorised binary patterns
$\{\xi ^1,\xi ^2,\dots,\xi ^M\}\subseteq \{-1, 1\}^N$
.
2.1. Mutually orthogonal memorised patterns
Suppose
$\{\xi ^1, \xi ^2,\dots, \xi ^M\}\subseteq \{-1,1\}^{N}$
be a set of mutually orthogonal memorised patterns, and
$\xi ^{\mathrm{def}}\in [\!-\!1,1]^{N}$
be a defective pattern. In refs. [Reference Li, Zhao and Xue16, Reference Zhao, Li and Xue19], the authors introduced the notion of
$\varepsilon$
-independently (asymptotical) stability to find criteria in parameter
$\varepsilon$
to guarantee the stability of memorised patterns and instability of most others.
Let
$\mathscr{P}\{-1,1\}^N$
denote the power set of
$\{-1,1\}^N$
. In ref. [Reference Zhao, Li and Xue19], the mapping
\begin{equation*}G_N \;:\; \mathscr{P}\{-1,1\}^N\to \mathscr{P}\{-1,1\}^N\end{equation*}
was introduced, which carries a set of memorised binary patterns to the corresponding set of
$\varepsilon$
-independently asymptotically stable binary patterns for system (2.1). In other words, for a given set of binary patterns
$\{\xi ^1,\xi ^2,\dots,\xi ^M \}$
, the image is
\begin{equation*}G_N(\xi ^1,\xi ^2,\dots,\xi ^M )\;:\!=\;\left \{\eta \in \{-1,1\}^N\,| \, \eta \; \text {is $\varepsilon $-independently asymptotically stable}\right \}.\end{equation*}
For mutually orthogonal memorised patterns, the following result was obtained. Note that
$\xi \cdot \eta$
represents the standard inner product of
$\xi$
and
$\eta$
in
$\mathbb{R}^N$
, i.e.,
$\xi \cdot \eta =\xi _1\eta _1+\xi _2\eta _2+\dots +\xi _N\eta _N$
.
Lemma 2.1. [Reference Li, Zhao and Xue16, Reference Zhao, Li and Xue19] Consider the network (2.1) with a set of mutually orthogonal memorised binary patterns
$\{\xi ^1,\xi ^2,\dots,\xi ^M \}\subseteq \{-1,1\}^N$
, and let
$\eta \in \{-1,1\}^N$
be a binary pattern. The following three statements are equivalent:
-
(1)
$\eta \in G_N\left (\left \{\xi ^1,\xi ^2,\dots,\xi ^M\right \}\right )$
; -
(2)
$\eta$
satisfies
$\sum _{k=1}^{M}(\xi ^{k}\cdot \eta )^2= N^{2}$
; -
(3) There exists
$(a_{1},a_{2},\dots,a_{M})\in \mathbb{R}^{M}$
such that
$\eta =\sum _{l=1}^{M}a_{l}\xi ^l$
. In other words,
$\eta$
is in the linear span
$L(\xi ^1,\xi ^2,\dots,\xi ^M)$
.
If
$\eta \notin G_N\left (\left \{\xi ^1,\xi ^2,\dots,\xi ^M\right \}\right )$
, then there exists
$\varepsilon _\eta ^*\gt 0$
such that
$\eta$
is stable for
$\varepsilon \in (\varepsilon _\eta ^*,+\infty )$
and unstable for
$\varepsilon \in (0, \varepsilon _\eta ^*)$
. Moreover, a lower bound is given by
\begin{equation} \varepsilon _\eta ^*\geq \max _{l\in \{1,2,\dots,M\}}\frac{N^{2}-\sum _{k=1}^{M}(\xi ^{k}\cdot \eta )^2}{2(N^{2}-(\xi ^{l}\cdot \eta )^2)}\;=\!:\;\varepsilon _{\eta }, \end{equation}
which means that when
$\varepsilon \in (0,\varepsilon _{\eta }),$
$\eta$
is unstable.
By Lemma 2.1
$(2)$
, when
$\xi ^1,\xi ^2,\dots,\xi ^M$
are mutually orthogonal, we have
\begin{equation*}\{\xi ^1,\xi ^2,\dots,\xi ^M \}\subseteq G_N\{\xi ^1,\xi ^2,\dots,\xi ^M \}.\end{equation*}
In ref. [Reference Li, Zhao and Xue16], it is shown that the equality always holds for
$M=2$
and
$M=3$
. That is, if
$\xi ^1,\xi ^2$
and
$\xi ^3$
are orthogonal to each other, then
\begin{equation*}G_N(\{\xi^1,\xi^2 \})=\{\pm\xi^1,\pm\xi^2 \} \quad \text{and} \quad G_N(\{\xi^1,\xi^2,\xi^3 \})=\{\pm\xi^1,\pm\xi^2,\pm\xi^3 \} \end{equation*}
The equality can fail for
$M\geq 4$
, see [Reference Li, Zhao and Xue16, Example 4.1].
2.2. An approach to general problems
Let
$N_1\in \mathbb{N}$
and
$\{\xi ^1, \xi ^2,\dots, \xi ^M\}\subseteq \{-1,1\}^{N_1}$
be a set of general standard patterns that can be orthogonal or nonorthogonal to each other. Let
$\xi ^{\mathrm{def}}\in [\!-\!1,1]^{N_1}$
be a defective pattern. Here,
${\xi }^{\mathrm{def}}$
can be a non-binary pattern, which represents a grey-scale image. In ref. [Reference Zhao, Li and Xue20], an approach was developed by subgrouping the standard patterns in pairs which converts the problem to a series of subproblems. In each subproblem, an orthogonal lift is employed to obtain an associated problem with orthogonal memorised patterns. Here, an (orthogonal) lift of a pattern set
$\{\xi ^1, \xi ^2,\dots, \xi ^M\}\subseteq \{-1,1\}^{N_1}$
means a set of (mutually orthogonal) patterns
$\{\tilde \xi ^1, \tilde \xi ^2,\dots, \tilde \xi ^M\}\subseteq \{-1,1\}^{N_1+N_2}$
with
$N_2\in \mathbb{N}$
and
$\tilde \xi ^k_i=\xi ^k_i$
for
$i=1,2,\dots,N_1$
and
$k=1,2,\dots, M$
. The procedure for solving the pattern retrieval task is proposed as follows, referred to as procedure
$\mathscr{A}$
here.
-
(A1) Subgroup the standard patterns so that each one contains two patterns. A subgroup containing only one pattern is allowed when
$M$
is odd. Then we have
$\lceil \frac{M}{2}\rceil$
subgroups, where
$\lceil x\rceil$
denotes the smallest integer not less than
$x$
. -
(A2) For each subgroup containing two patterns
$\{\xi ^k,\xi ^l\}$
, we construct an orthogonal lift of the subgroup and a lift of defective pattern as follows:or
\begin{equation*}\{\xi ^k, \xi ^l\} \longrightarrow \left \{\Big [\xi ^k, \xi ^k\Big ], \Big [\xi ^l, -\xi ^l\Big ]\right \}\;=\!:\;\left \{\tilde {\xi }^k,\tilde {\xi }^l\right \}, \quad \xi ^{\mathrm {def}} \longrightarrow \Big [\xi ^{\mathrm {def}}, \,\,\frac {1}{2}(\xi ^k-\xi ^l)\Big ]\;=\!:\;\tilde {\xi }^{\mathrm {def}}, \end{equation*}
where
\begin{equation*}\{\xi ^k, \xi ^l\} \longrightarrow \left \{\Big [\xi ^k, -\xi ^k\Big ], \Big [\xi ^l, \xi ^l\Big ]\right \}\;=\!:\;\left \{\tilde {\xi }^k,\tilde {\xi }^l\right \},\quad \xi ^{\mathrm {def}} \longrightarrow \Big [\xi ^{\mathrm {def}}, \,\,\frac {1}{2}(-\xi ^k+\xi ^l)\Big ]\;=\!:\;\tilde {\xi }^{\mathrm {def}},\end{equation*}
$\left [\cdot,\cdot \right ]$
denotes the concatenation of two vectors. Obviously,
$\{\tilde{\xi }^k,\tilde{\xi }^l\}$
is an orthogonal lift of
$\{{\xi }^k,{\xi }^l\}$
with dimension
$2N_1$
. We identify the problem
$\mathrm{PR}\{\xi ^k, \xi ^l; \xi ^{\mathrm{def}}\}$
with the lifted problem
$\mathrm{PR}\{\tilde{\xi }^k, \tilde{\xi }^l; \tilde{\xi }^{\mathrm{def}}\}$
to retrieve one of
$\tilde{\xi }^k$
and
$\tilde{\xi }^l$
by system (2.1), which means that
$\xi ^{\mathrm{def}}$
recognises one of
${\xi }^k$
and
${\xi }^l$
. For subgroup containing a single pattern, it is retrieved immediately. We collect the retrieved patterns, denoted by
$\{\xi ^{(1,1)}, \xi ^{(1,2)}, \dots, \xi ^{(1,\lceil \frac{M}{2}\rceil )}\}$
.
-
(A3) Use
$\big \{\xi ^{(1,1)}, \xi ^{(1,2)}, \dots, \xi ^{(1,\lceil \frac{M}{2}\rceil )}\big \}$
as a new set of standard patterns and perform the same process in Step
$\textbf{(A1)}$
and Step
$\textbf{(A2)}$
. After finite iterations, a final one of the standard patterns is retrieved.
In the above procedure, we need to build subsystems with dimension
$2N_1$
and carry out the pattern retrieval processes for
$M-1$
times. In fact, one standard pattern is excluded in each process and therefore the total times should be
$M-1$
.
3. The theory for three memorised patterns
In this section, we consider the case of three memorised binary patterns. We explore the least orthogonal lift and the critical strength of the second-order term.
3.1. Least orthogonal lift
Let
$\xi ^1, \xi ^2$
and
$\xi ^3$
be three memorised binary patterns in
$\{-1,1\}^{N_1}$
with
$N_1\in \mathbb N$
. We first introduce some notation. Denote
\begin{align*} S_{123}&\;:\!=\;\left \{ i\in [N_1] \, : \,{\xi }_i^1=1,{\xi }_i^2=1,{\xi }_i^3=1\; \,\text{or} \; \,{\xi }_i^1=-1,{\xi }_i^2=-1,{\xi }_i^3=-1 \right \},\\[5pt] S_{23}&\;:\!=\;\left \{ i \in [N_1] \, : \,{\xi }_i^1=-1,{\xi }_i^2=1,{\xi }_i^3=1\; \,\text{or} \,\;{\xi }_i^1=1,{\xi }_i^2=-1,{\xi }_i^3=-1 \right \},\\[5pt] S_{13}&\;:\!=\;\left \{ i \in [N_1] \, : \,{\xi }_i^1=1,{\xi }_i^2=-1,{\xi }_i^3=1\; \,\text{or}\, \;{\xi }_i^1=-1,{\xi }_i^2=1,{\xi }_i^3=-1 \right \},\\[5pt] S_{12}&\;:\!=\;\left \{ i \in [N_1] \, : \,{\xi }_i^1=1,{\xi }_i^2=1,{\xi }_i^3=-1\;\, \text{or}\, \;{\xi }_i^1=-1,{\xi }_i^2=-1,{\xi }_i^3=1 \right \}, \end{align*}
and let
\begin{equation} n_0\;:\!=\;|S_{123}|,\quad n_1\;:\!=\; |S_{23}|,\quad n_2 \;:\!=\;|S_{13}|,\quad n_3\;:\!=\; |S_{12}|, \end{equation}
where
$[N]\;:\!=\;\{1,2,\dots,N\}$
for
$N\in \mathbb N$
, and
$|\cdot |$
represents the number of elements in a set.
Suppose
$N_2\in \mathbb N$
and let
$\{\tilde \xi ^1, \tilde \xi ^2, \tilde \xi ^3\}\subseteq \{-1,1\}^{N_1+N_2}$
be a lift of
$\{\xi ^1, \xi ^2, \xi ^3\}$
, we denote
\begin{align*} L_{123}&\;:\!=\;\left \{ i\in [N_1+N_2] \, : \, \tilde{\xi }_i^1=1, \tilde{\xi }_i^2=1,\tilde{\xi }_i^3=1\;\, \text{or} \,\; \tilde{\xi }_i^1=-1, \tilde{\xi }_i^2=-1,\tilde{\xi }_i^3=-1 \right \},\\[5pt] L_{23}&\;:\!=\;\left \{ i \in [N_1+N_2] \, : \, \tilde{\xi }_i^1=-1, \tilde{\xi }_i^2=1,\tilde{\xi }_i^3=1\; \,\text{or}\, \; \tilde{\xi }_i^1=1, \tilde{\xi }_i^2=-1,\tilde{\xi }_i^3=-1 \right \},\\[5pt] L_{13}&\;:\!=\;\left \{ i \in [N_1+N_2] \, : \, \tilde{\xi }_i^1=1, \tilde{\xi }_i^2=-1,\tilde{\xi }_i^3=1\; \,\text{or} \,\; \tilde{\xi }_i^1=-1, \tilde{\xi }_i^2=1,\tilde{\xi }_i^3=-1 \right \},\\[5pt] L_{12}&\;:\!=\;\left \{ i \in [N_1+N_2] \, : \, \tilde{\xi }_i^1=1, \tilde{\xi }_i^2=1,\tilde{\xi }_i^3=-1\;\, \text{or} \,\; \tilde{\xi }_i^1=-1, \tilde{\xi }_i^2=-1,\tilde{\xi }_i^3=1 \right \}, \end{align*}
and
\begin{equation} x_0\;:\!=\; |L_{123}\setminus [N_1]|,\quad \, x_1\;:\!=\; |L_{23}\setminus [N_1]|,\quad \, x_2\;:\!=\;|L_{13}\setminus [N_1]|, \quad \; x_3\;:\!=\; |L_{12}\setminus [N_1]|. \end{equation}
Then we have the following relations:
\begin{align} & n_{0}+n_{1}+n_{2}+n_{3}=N_1, \end{align}
\begin{align} & x_{0}+x_{1}+x_{2}+x_{3}=N_2, \end{align}
\begin{align} & n_{0}+n_{3}-n_{1}-n_{2}=\xi ^1\cdot \xi ^2, \end{align}
\begin{align} & n_{0}+n_{2}-n_{1}-n_{3}=\xi ^1\cdot \xi ^3, \end{align}
\begin{align} & n_{0}+n_{1}-n_{2}-n_{3}=\xi ^2\cdot \xi ^3. \end{align}
Our aim is to find an orthogonal lift of
$\{\xi ^1, \xi ^2, \xi ^3\}$
.
Theorem 3.1 (Orthogonal lift). Let
$\{\xi ^1,\xi ^2, \xi ^3\}\subseteq \{-1,1\}^{N_1}$
be a set of three memorised binary patterns with
$N_1\in \mathbb{N}$
. Then there exists an orthogonal lift
$\{\tilde \xi ^1, \tilde \xi ^2, \tilde \xi ^3\}\subseteq \{-1,1\}^{N_1+N_2}$
for
$N_2\in \mathbb N$
, if and only if
$N_2$
satisfies conditions
\begin{align} (N_1+N_2)\mod 4=0 \quad \text{and }\quad N_1+N_2\ge \max \limits _{i=0,1,2,3}4n_i. \end{align}
Moreover, for any orthogonal lift, we have
\begin{equation} x_i=\frac{N_1+N_2}{4}-n_i, \; \quad i=0,1,2,3, \end{equation}
Proof.
Sufficiency. If (3.8) holds, then we set
$y_i=\frac{N_1+N_2}{4}-n_i$
for
$i=0,1,2,3,$
where
$n_i$
is given as in (2.1). We construct a lift in
$\{-1,1\}^{N_1+N_2}$
as follows:
\begin{align} \begin{aligned} \tilde \xi ^1= \left [\xi ^1, \, \textbf{1}^{y_{0}},\, -\textbf{1}^{y_{1}}, \,\textbf{1}^{y_{2}}, \,\textbf{1}^{y_{3}}\right ],\,\, \tilde \xi ^2= \left [\xi ^2, \,\textbf{1}^{y_{0}}, \,\textbf{1}^{y_{1}}, \,-\textbf{1}^{y_{2}}, \,\textbf{1}^{y_{3}}\right ],\,\, \tilde \xi ^3= \left [\xi ^3, \,\textbf{1}^{y_{0}}, \,\textbf{1}^{y_{1}}, \,\textbf{1}^{y_{2}}, \,-\textbf{{1}}^{y_{3}}\right ] \end{aligned} \end{align}
where
$\left [\cdot,\cdot,\cdot,\cdot,\cdot \right ]$
denotes the concatenation of vectors, and
$\textbf{1}^{y} \,(y\in \mathbb N)$
denotes the
$y$
-dimensional vector
$[1, 1, \dots,1]$
. We can see, for the lift (3.10), the index
$x_i$
defined in (3.2) coincides with
$y_i$
for
$i=0,1,2,3$
. Combining (3.5) and (3.10) with the above settings of
$y_i$
, we obtain
\begin{equation*} \tilde {\xi }^1\cdot \tilde {\xi }^2=\xi ^1\cdot \xi ^2+y_0-y_1-y_2+y_3=(n_0+n_3-n_1-n_2)+y_0-y_1-y_2+y_3=0. \end{equation*}
Similarly,
$\tilde{\xi }^1\cdot \tilde{\xi }^3=0$
and
$\tilde{\xi }^2\cdot \tilde{\xi }^3=0$
can be derived. This shows that
$\{\tilde \xi ^1, \tilde \xi ^2, \tilde \xi ^3\}$
is an orthogonal lift of
$\{\xi ^1,\xi ^2,\xi ^3\}$
.
Necessity. Let
$\{\tilde \xi ^1,\tilde \xi ^2, \tilde \xi ^3\}$
be a lift of
$\{\xi ^1, \xi ^2, \xi ^3\}$
. Following the notation in (3.1) and (3.2), it is an orthogonal lift if and only if
\begin{align} \begin{split} \left \{ \begin{array}{l} (n_{0}+x_{0})+ (n_{3}+x_{3})-(n_{1}+x_{1})- (n_{2}+x_{2})=0,\\[5pt] (n_{0}+x_{0})+ (n_{2}+x_{2})-(n_{1}+x_{1})- (n_{3}+x_{3})=0,\\[5pt] (n_{0}+x_{0})+ (n_{1}+x_{1})-(n_{2}+x_{2})- (n_{3}+x_{3})=0. \end{array} \right . \end{split} \end{align}
By (3.4)–(3.7), the above relation (3.11) is equivalent to
\begin{align} \begin{split} \left \{ \begin{array}{l} x_{0}+x_{1}+x_{2}+x_{3}=N_2,\\[5pt] x_{0}-x_{1}-x_{2}+x_{3}=-\xi ^1\cdot \xi ^2,\\[5pt] x_{0}-x_{1}+x_{2}-x_{3}=-\xi ^1\cdot \xi ^3,\\[5pt] x_{0}+x_{1}-x_{2}-x_{3}=-\xi ^2\cdot \xi ^3. \end{array} \right . \end{split} \end{align}
This is a system of non-homogeneous linear equations. Owing to Cramer’s rule, since the coefficient determinant
\begin{align*} D=\left |\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} 1 & 1 & 1 & 1\\[5pt] 1 & -1 & -1 & 1\\[5pt] 1 & -1 & 1 & -1\\[5pt] 1 & 1 & -1 & -1\\[5pt] \end{array}\right |=16\neq 0, \end{align*}
the system of Equations (3.12) has a unique solution
\begin{equation*}x_{0}=\frac {D_0}{D},\quad x_{1}=\frac {D_1}{D},\quad x_{2}=\frac {D_2}{D},\quad x_{3}=\frac {D_3}{D},\end{equation*}
where
\begin{align*} D_0=\left |\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} N_2 & 1 & 1 & 1\\[5pt] -\xi ^1\cdot \xi ^2 & -1 & -1 & 1\\[5pt] -\xi ^1\cdot \xi ^3 & -1 & 1 & -1\\[5pt] -\xi ^2\cdot \xi ^3 & 1 & -1 & -1\\[5pt] \end{array}\right |,\quad D_1=\left |\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} 1 & N_2 & 1 & 1\\[5pt] 1 & -\xi ^1\cdot \xi ^2 & -1 & 1\\[5pt] 1& -\xi ^1\cdot \xi ^3 & 1 & -1\\[5pt] 1 &-\xi ^2\cdot \xi ^3 & -1 & -1\\[5pt] \end{array}\right |, \end{align*}
\begin{align*} D_2=\left |\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} 1 &1& N_2 & 1\\[5pt] 1 &-1 & -\xi ^1\cdot \xi ^2 & 1\\[5pt] 1& -1 & -\xi ^1\cdot \xi ^3 & -1\\[5pt] 1 &1 & -\xi ^2\cdot \xi ^3 & -1\\[5pt] \end{array}\right |,\quad D_3=\left |\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} 1 &1&1& N_2\\[5pt] 1 &-1 &-1 & -\xi ^1\cdot \xi ^2\\[5pt] 1& -1 &1 & -\xi ^1\cdot \xi ^3\\[5pt] 1 &1 & -1 &-\xi ^2\cdot \xi ^3\\[5pt] \end{array}\right |. \end{align*}
It is easy to see that
\begin{align*} D_0&=\left |\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} N_2 & 1 & 0 & 0\\[5pt] -\xi ^1\cdot \xi ^2 & -1 & 0 & 2\\[5pt] -\xi ^1\cdot \xi ^3 & -1 & 2 & 0\\[5pt] -\xi ^2\cdot \xi ^3 & 1 & -2 & -2\\[5pt] \end{array}\right |=\left |\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} N_2 & 1 & 0 & 0\\[5pt] -\xi ^1\cdot \xi ^2 & -1 & 0 & 2\\[5pt] -\xi ^1\cdot \xi ^3 & -1 & 2 & 0\\[5pt] -(\xi ^1\cdot \xi ^2+\xi ^2\cdot \xi ^3 )& 0 & -2 & 0\\[5pt] \end{array}\right |\\[5pt] &=2\cdot (\!-\!1)^6\left |\begin{array}{c@{\quad}c@{\quad}c} N_2 & 1 & 0 \\[5pt] -\xi ^1\cdot \xi ^3 & -1 & 2\\[5pt] -(\xi ^1\cdot \xi ^2+\xi ^2\cdot \xi ^3 )& 0 & -2\\[5pt] \end{array}\right | =2\left |\begin{array}{c@{\quad}c@{\quad}c} N_2 & 1 & 0 \\[5pt] N_2-\xi ^1\cdot \xi ^3 & 0 & 2\\[5pt] -(\xi ^1\cdot \xi ^2+\xi ^2\cdot \xi ^3 )& 0 & -2\\[5pt] \end{array}\right |\\[5pt] &=2\cdot (\!-\!1)^3 \left |\begin{array}{c@{\quad}c} N_2-\xi ^1\cdot \xi ^3 & 2\\[5pt] -(\xi ^1\cdot \xi ^2+\xi ^2\cdot \xi ^3 )&-2\\[5pt] \end{array}\right |=-2[-2(N_2-\xi ^1\cdot \xi ^3)+2(\xi ^1\cdot \xi ^2+\xi ^2\cdot \xi ^3)]\\[5pt] &=4(N_2+N_1-4n_0), \end{align*}
where we used (3.5)–(3.7) in the last equality. Similarly, we derive
\begin{align*} D_1=4(N_1+N_2-4n_{1}), \; D_2=4(N_1+N_2-4n_{2}), \; D_3=4(N_1+N_2-4n_{3}). \end{align*}
So we have
\begin{equation*} x_{0}=\frac {N_1+N_2}{4}-n_{0},\; x_{1}=\frac {N_1+N_2}{4}-n_{1},\; x_{2}=\frac {N_1+N_2}{4}-n_{2},\; x_{3}=\frac {N_1+N_2}{4}-n_{3}.\end{equation*}
Now,
$x_i\in \mathbb{N}\cup \{0\}$
and
$N_2\in \mathbb{N}\cup \{0\}$
mean that
\begin{equation*}(N_1+N_2)\mod 4=0,\quad \text {and}\quad N_1+N_2\ge \max _{i=0,1,2,3}\{4n_i\}.\end{equation*}
Remark 1. In the above theorem, the relation (3.8) describes the dimension, while (3.9) gives the construction, of an orthogonal lift, for a given set of three patterns. For computational simplicity, it is reasonable to seek an orthogonal lift with the minimal dimension. For a set of three patterns, an orthogonal lift with the minimal dimension is called a least orthogonal lift of the set.
Following Theorem 3.1, we now present the least orthogonal lift.
Theorem 3.2 (Least orthogonal lift). Let
$\{\xi ^1,\xi ^2,\xi ^3\}\subseteq \{-1,1\}^{N_1}, N_1\in \mathbb{N}$
be a set of three memorised binary patterns. Then the dimension of the least orthogonal lift is
\begin{equation*} N_1+N_2=4\bar n, \quad \text {with}\quad \bar n\;:\!=\;\max \limits _{i=0,1,2,3}\{n_i\}. \end{equation*}
Remark 2. The lift
$\{\tilde \xi ^1, \tilde \xi ^2, \tilde \xi ^3\}$
given in (3.10), with
$N_2$
in Theorem 3.2, is a least orthogonal lift of
$\{\xi ^1, \xi ^2,\xi ^3\}$
. To construct this lift, we set
\begin{align} x_{0}=\bar n-n_{0},\quad x_{1}=\bar n -n_{1},\quad x_{2}=\bar n -n_{2},\quad x_{3}=\bar n -n_{3}, \end{align}
and
\begin{align} \begin{aligned} \tilde \xi ^1= \left [\xi ^1, \, \textbf{1}^{x_{0}},\, -\textbf{1}^{x_{1}}, \,\textbf{1}^{x_{2}}, \,\textbf{1}^{x_{3}}\right ],\,\, \tilde \xi ^2= \left [\xi ^2, \,\textbf{1}^{x_{0}}, \,\textbf{1}^{x_{1}}, \,-\textbf{1}^{x_{2}}, \,\textbf{1}^{x_{3}}\right ],\,\, \tilde \xi ^3= \left [\xi ^3, \,\textbf{1}^{x_{0}}, \,\textbf{1}^{x_{1}}, \,\textbf{1}^{x_{2}}, \,-\textbf{1}^{x_{3}}\right ]. \end{aligned} \end{align}
Remark 3. For subgroup of three memorised patterns lacking mutual orthogonality, the orthogonal lifting is necessary, since the nice property in Lemma 2.1 does not hold true. The following example illustrates this point. Let
$\{\xi ^1,\xi ^2, \xi ^3\}$
be the three memorised patterns with
$44$
rows and
$22$
columns
$(N=968)$
indicated as in Figure 1a–1c. By calculating the eigenvalues, we see that none of
$\xi ^1,\xi ^2$
and
$\xi ^3$
is
$\varepsilon$
-independently stable; however, the binary pattern
$\xi$
in Figure 1d is
$\varepsilon$
-independently stable. This means that
$\xi ^1,\xi ^2$
and
$\xi ^3$
cannot be distinguished from other patterns by selecting
$\varepsilon$
.
3.2. Set-up of the parameter
$\varepsilon$
In this subsection, we consider the criterion for selecting the parameter
$\varepsilon$
to make the orthogonal memorised patterns stable and other binary patterns unstable, for system (3.1) with three mutually orthogonal memorised patterns.
Figure 1. Three memorised patterns
$\{\xi ^1,\xi ^2,\xi ^3\}$
and a binary pattern
$\xi$
.
Theorem 3.3.
Consider the system (3.1) with a set of mutually orthogonal memorised patterns
$\{\xi ^1, \xi ^2,\xi ^3\}\subseteq \{-1,1\}^{N}, N\in \mathbb{N}$
.
-
(1) The memorised patterns are
$\varepsilon$
-independently stable. More precisely, the eigenvalues of the Jacobian at the corresponding equilibria
$\{\varphi ^*(\xi ^l)\}_{l=1}^3$
are as follows:
\begin{equation*}\underbrace {-1-2\varepsilon,\dots, -1-2\varepsilon }_{N-3},-2\varepsilon,-2\varepsilon,0.\end{equation*}
-
(2) Any binary pattern
$\eta \in \{-1,1\}^N\setminus \{\pm \xi ^1,\pm \xi ^2,\pm \xi ^3\}$
is not
$\varepsilon$
-independently asymptotically stable and the critical strength
$\varepsilon _\eta ^*$
has a lower bound of
$\frac{1}{6}$
, i.e.,
In other words, if
\begin{equation*}\varepsilon _\eta ^*\geq \frac {1}{6}.\end{equation*}
$\varepsilon \in (0,\frac{1}{6})$
, then only
$\{\pm \xi ^1,\pm \xi ^2,\pm \xi ^3\}$
are asymptotically stable and other binary patterns are all unstable.
Proof.
$(1)$
The result can be directly obtained through [Reference Zhao, Li and Xue19, Theorem 4.1].
$(2)$
Fix
$\varepsilon \in (0,\frac{1}{6})$
. For any
$\eta \in \{-1,1\}^N\setminus \{\pm \xi ^1,\pm \xi ^2,\pm \xi ^3\}$
, we denote for
$l=1,2,3$
,
\begin{align*} & [\xi ^l=\eta ]\;:\!=\;\{ j\in \{1,2,\dots,N\}\;| \;\xi ^l_j=1, \eta _j=1\; \text{or} \; \xi ^l_j=-1, \eta _j=-1 \},\\[5pt] & [\xi ^l\neq \eta ]\;:\!=\;\{ j\in \{1,2,\dots,N\}\;| \;\xi ^l_j=1, \eta _j=-1\; \text{or} \; \xi ^l_j=-1, \eta _j=1 \}. \end{align*}
The instability of
$\eta$
will be discussed in three cases.
Case 1:
$\frac{N}{4}\le \big |[\xi ^l\neq \eta ]\big |\le \frac{3N}{4}$
for
$ l=1,2,3$
. Then we have
\begin{equation*}(\xi ^l\cdot \eta )^2=(N-2\big |[\xi ^l\neq \eta ]\big |)^2\le \frac {N^2}{4},\; l=1,2,3,\end{equation*}
and it can be seen that
\begin{align*} \frac{N^{2}-\sum _{k=1}^{3}(\xi ^{k}\cdot \eta )^2}{2[N^{2}-(\xi ^{1}\cdot \eta )^2]}&=\frac{[N^2-(\xi ^1\cdot \eta )^2]-[(\xi ^2\cdot \eta )^2+(\xi ^3\cdot \eta )^2]}{2[N^2-(\xi ^1\cdot \eta )^2]} =\frac{1}{2}-\frac{(\xi ^2\cdot \eta )^2+(\xi ^3\cdot \eta )^2}{2[N^2-(\xi ^1\cdot \eta )^2]}\\[5pt] &\ge \frac{1}{2}-\frac{\frac{N^2}{4}+\frac{N^2}{4}}{2[N^2-(\xi ^1\cdot \eta )^2]} =\frac{1}{2}-\frac{N^2}{4[N^2-(\xi ^1\cdot \eta )^2]}\\[5pt] &\ge \frac{1}{2}-\frac{N^2}{4[N^2-\frac{N^2}{4}]} =\frac{1}{2}-\frac{1}{3}=\frac{1}{6}. \end{align*}
From (3.2), we obtain
\begin{equation*} \varepsilon _{\eta }=\max _{l\in \{1,2,3\}}\frac {N^{2}-\sum _{k=1}^{3}(\xi ^{k}\cdot \eta )^2}{2(N^{2}-(\xi ^{l}\cdot \eta )^2)}\ge \frac {N^{2}-\sum _{k=1}^{3}(\xi ^{k}\cdot \eta )^2}{2[N^{2}-(\xi ^{1}\cdot \eta )^2]} \ge \frac {1}{6}. \end{equation*}
Lemma 2.1 tells us that
$\eta$
is unstable if
$\varepsilon \in (0,\frac{1}{6})$
.
Case 2: There is an
$l\in \{1,2,3\}$
such that
$1\le \big |[\xi ^l\neq \eta ]\big |\le \frac{N}{4}-1.$
For convenience, let’s say
$l=1$
, i.e.,
\begin{equation*}1\le \big |[\xi ^1\neq \eta ]\big |\le \frac {N}{4}-1.\end{equation*}
Then by
$\xi ^1\cdot \xi ^2=0$
and
$\xi ^1\cdot \xi ^3=0$
, we have
\begin{equation*}\sum _{j\in [\xi ^1=\eta ]}\xi ^2_j\xi ^1_j=-\sum _{j\in [\xi ^1\neq \eta ]}\xi ^2_j\xi ^1_j\quad \text {and}\quad \sum _{j\in [\xi ^1= \eta ]}\xi ^3_j\xi ^1_j=-\sum _{j\in [\xi ^1\neq \eta ]}\xi ^3_j\xi ^1_j,\end{equation*}
which yields
\begin{align*} (\xi ^1\cdot \eta )^2&=(N-2\big |[\xi ^1\neq \eta ]\big |)^2=N^2-4N\big |[\xi ^1\neq \eta ]\big |+4\big |[\xi ^1\neq \eta ]\big |^2,\\[5pt] (\xi ^2\cdot \eta )^2&=\left (\sum _{j=1}^{N}\xi ^2_j\eta _j\right )^2=\left (\sum _{j\in [\xi ^1=\eta ]}\xi ^2_j\eta _j+\sum _{j\in [\xi ^1\neq \eta ]}\xi ^2_j\eta _j\right )^2\\[5pt] &=\left (\sum _{j\in [\xi ^1=\eta ]}\xi ^2_j\xi ^1_j-\sum _{j\in [\xi ^1\neq \eta ]}\xi ^2_j\xi ^1_j\right )^2=\left (-\sum _{j\in [\xi ^1\neq \eta ]}\xi ^2_j\xi ^1_j-\sum _{j\in [\xi ^1\neq \eta ]}\xi ^2_j\xi ^1_j\right )^2\\[5pt] &=4\left (\sum _{j\in [\xi ^1\neq \eta ]}\xi ^2_j\xi ^1_j\right )^2\le 4|[\xi ^1\neq \eta ]|^2, \\[5pt] (\xi ^3\cdot \eta )^2&=4\left (\sum _{j\in [\xi ^1\neq \eta ]}\xi ^3_j\xi ^1_j\right )^2\le 4|[\xi ^1\neq \eta ]|^2. \end{align*}
Then
\begin{align*} \frac{N^{2}-\sum _{k=1}^{3}(\xi ^{k}\cdot \eta )^2}{2[N^{2}-(\xi ^{1}\cdot \eta )^2]}&=\frac{[N^2-(\xi ^1\cdot \eta )^2]-[(\xi ^2\cdot \eta )^2+(\xi ^3\cdot \eta )^2]}{2[N^2-(\xi ^1\cdot \eta )^2]} =\frac{1}{2}-\frac{(\xi ^2\cdot \eta )^2+(\xi ^3\cdot \eta )^2}{2[N^2-(\xi ^1\cdot \eta )^2]}\\[5pt] &\ge \frac{1}{2}-\frac{4|[\xi ^1\neq \eta ]|^2+4|[\xi ^1\neq \eta ]|^2}{2[N^2-(\xi ^1\cdot \eta )^2]} =\frac{1}{2}-\frac{8|[\xi ^1\neq \eta ]|^2}{2[4N|[\xi ^1\neq \eta ]|-4|[\xi ^1\neq \eta ]|^2]}\\[5pt] &=\frac{1}{2}-\frac{|[\xi ^1\neq \eta ]|}{N-|[\xi ^1\neq \eta ]|} =\frac{3}{2}-\frac{N}{N-|[\xi ^1\neq \eta ]|}. \end{align*}
Note that
$1\le |[\xi ^1\neq \eta ]|\le \frac{N}{4}-1$
, then
\begin{equation*}\frac {N^{2}-\sum _{k=1}^{3}(\xi ^{k}\cdot \eta )^2}{2[N^{2}-(\xi ^{1}\cdot \eta )^2]}\ge \frac {3}{2}-\frac {N}{N-(\frac {N}{4}-1)}=\frac {N+12}{2(3N+4)}.\end{equation*}
So we obtain
\begin{equation*} \varepsilon _{\eta }=\max _{l\in \{1,2,3\}}\frac {N^{2}-\sum _{k=1}^{3}(\xi ^{k}\cdot \eta )^2}{2(N^{2}-(\xi ^{l}\cdot \eta )^2)}\ge \frac {N^{2}-\sum _{k=1}^{3}(\xi ^{k}\cdot \eta )^2}{2[N^{2}-(\xi ^{1}\cdot \eta )^2]}\ge \frac {N+12}{2(3N+4)} \gt \frac {1}{6}. \end{equation*}
By Lemma 2.1,
$\eta$
is unstable if
$\varepsilon \in (0,\frac{1}{6})$
.
Case 3: There is an
$l\in \{1,2,3\}$
such that
$\frac{3N}{4}+1\le \big |[\xi ^l\neq \eta ]\big |\le N-1.$
Note that the stability of
$\eta$
and
$-\eta$
is the same. In this case, we can consider
$-\eta$
and find that
\begin{equation*}\big |[\xi ^l\neq -\eta ]\big |=\big |[\xi ^l= \eta ]\big | =N-\big |[\xi ^l\neq \eta ]\big |\in \Big [1,\frac {N}{4}-1\Big ].\end{equation*}
Then the desired result can be obtained through Case 2.
Remark 4. Comparing to the bound given in Equation (2.2), the lower bound of critical strength,
$\frac{1}{6}$
, is independent of the patterns
$\xi ^1,\xi ^2,\xi ^3$
and
$\eta$
, and independent of the dimension
$N$
.
Remark 5. We acknowledge that the bound
$\frac{1}{6}$
is still conservative. According to Lemma 2.1, the critical strength which distinguishes orthogonal memorised patterns
$\{\xi ^1,\xi ^2,\xi ^3\}$
from all other binary patterns is given by
\begin{equation*}\min \limits _{\eta \notin \{\pm \xi ^1,\pm \xi ^2,\pm \xi ^3\}}\varepsilon ^*_{\eta }.\end{equation*}
Here, we consider the mutually orthogonal memorised patterns
$\{\xi ^{1},\xi ^{2},\xi ^{3}\}$
shown in Figure 2 and examine the stability of binary patterns by computing the eigenvalues of Jacobians.
Figure 2. The mutually orthogonal memorised patterns
$\xi ^{1},\xi ^{2},\xi ^{3}$
.
For given
$\varepsilon$
, we count the number of binary patterns at which the Jacobian matrix has
$(N-1)$
negative eigenvalues (0 is an eigenvalue due to global phase shift invariance). Table 1 shows, as the strength
$\varepsilon$
increases, the number of (asymptotically) stable binary patterns is gradually increased. It’s easy to see that
\begin{equation*}\min \limits _{\eta \notin \{\pm \xi ^1,\pm \xi ^2,\pm \xi ^3\}}\varepsilon ^*_{\eta }\ge 0.20\gt \frac {1}{6}.\end{equation*}
4. An approach for general standard patterns and simulations
In this section, we propose an approach for the pattern retrieval problem with general standard binary patterns that can be orthogonal or nonorthogonal to each other. Consider a set of standard binary patterns
$\{\eta ^1,\eta ^2,\dots,\eta ^9,\eta ^0\}$
shown in Figure 3a which are symbols of numbers
$\{1,2,\dots,9,0\}$
. The dimension
$N_1$
of each pattern is
$968$
with
$44$
rows and
$22$
columns. A defective pattern
$\eta^{\mathrm{def}}$
is shown in Figure 3b, which is generated by incorporating normally distributed noises with mean value
$\mu =0.5$
and standard deviation
$\sigma =0.45$
into each pixel of the symbol
$6$
.
Table 1. The number of stable binary patterns for different values of
$\varepsilon$
. Here, the six stable patterns for small
$\varepsilon$
are
$\pm \xi ^1,\pm \xi ^2$
and
$\pm \xi ^3$
.
Figure 3. The standard patterns
$\{\eta ^1, \dots,\eta ^9,\eta ^0\}$
and the defective pattern
$\eta ^{\mathrm{def}}$
.
4.1. The case of three standard patterns:
$M=3$
In this part, we present a simulation for a subproblem with three standard patterns. Let
$\{\xi ^1, \xi ^2,\xi ^3\}$
be the set of standard binary patterns without orthogonality and
$\xi ^{\mathrm{def}}$
be a defective pattern. Following (3.13)–(3.14), we construct a least orthogonal lift of standard patterns and a lift of defective pattern as follows:
\begin{align} \begin{aligned} \xi ^1\; &\longrightarrow \; \left [\xi ^1, \quad \textbf{1}^{x_{0}},\quad -\textbf{1}^{x_{1}}, \quad \textbf{1}^{x_{2}}, \quad \textbf{1}^{x_{3}}\right ]\;=\!:\;\tilde{\xi }^1,\\[5pt] \xi ^2\; &\longrightarrow \; \left [\xi ^2, \quad \textbf{1}^{x_{0}}, \quad \textbf{1}^{x_{1}}, \quad -\textbf{1}^{x_{2}}, \quad \textbf{1}^{x_{3}}\right ]\;=\!:\;\tilde{\xi }^2,\\[5pt] \xi ^3\; &\longrightarrow \; \left [\xi ^3, \quad \textbf{1}^{x_{0}}, \quad \textbf{1}^{x_{1}}, \quad \textbf{1}^{x_{2}}, \quad -\textbf{1}^{x_{3}}\right ]\;=\!:\;\tilde{\xi }^3,\\[5pt] \xi ^{\mathrm{def}}\; &\longrightarrow \; \Big [\xi ^{\mathrm{def}}, \quad \textbf{1}^{x_{0}}, \quad \frac{1}{3}\textbf{1}^{x_{1}}, \quad \frac{1}{3} \textbf{1}^{x_{2}}, \quad \frac{1}{3}\textbf{1}^{x_{3}}\Big ]\;=\!:\;\tilde{\xi }^{\mathrm{def}}, \end{aligned} \end{align}
where
$x_i,i=0,1,2,3$
are defined as in (3.13). We now summarise the process for solving the problem
$\mathrm{PR}\{\xi ^1, \xi ^2,\xi ^3; \xi ^{\mathrm{def}}\}$
as follows, referred to as procedure
$\mathscr{B}$
:
-
(B1) Construct the lift of standard patterns
$\{\xi ^1, \xi ^2,\xi ^3\}$
and defective pattern
$\xi ^{\mathrm{def}}$
as in (4.1). Identify the original problem
$\mathrm{PR}\{\xi ^1, \xi ^2,\xi ^3; \xi ^{\mathrm{def}}\}$
with the lifted problem
$\mathrm{PR}\{\tilde{\xi }^1, \tilde{\xi }^2,\tilde{\xi }^3; \tilde{\xi }^{\mathrm{def}}\}$
. -
(B2) Solve the lifted problem
$\mathrm{PR}\{\tilde{\xi }^1, \tilde{\xi }^2,\tilde{\xi }^3; \tilde{\xi }^{\mathrm{def}}\}$
using system (2.1).
For the system (2.1) memorising
$\{\tilde{\xi }^1, \tilde{\xi }^2,\tilde{\xi }^3\}$
, only the three memorised patterns are
$\varepsilon$
-independently asymptotically stable when
$\varepsilon \in (0,\frac{1}{6})$
by Theorem 3.3.
Figure 4. Lifted mutually orthogonal memorised patterns
$\{\tilde{\xi }^1,\tilde{\xi }^2,\tilde{\xi }^3\}$
and lifted defective pattern
$\tilde{\xi }^{\mathrm{def}}$
.
Figure 5. Number
$6$
is successfully retrieved with mutually orthogonal memorised patterns
$\{\tilde{\xi }^1,\tilde{\xi }^2,\tilde{\xi }^3\}$
and
$\varepsilon =0.12$
.
We present a simulation to illustrate procedure
$\mathscr{B}$
. Consider the problem
$\mathrm{PR}\{\xi ^1,\xi ^2,\xi ^3; \xi ^{\mathrm{def}}\}$
with
\begin{equation*}\xi ^1=\eta ^4, \quad \xi ^2=\eta ^5,\quad \xi ^3=\eta ^6, \quad \xi ^{\mathrm {def}}=\eta ^{\mathrm {def}},\end{equation*}
where
$\eta ^4,\eta ^5$
and
$\eta ^6$
are given in Figure 3a and
$\eta ^{\mathrm{def}}$
in Figure 3b. According to procedure
$\mathscr{B}$
, we construct the lift shown in Figure 4a–4d with
\begin{equation*} N_1+N_2=1524, \; N_2=556, \; x_{0}=0,\; x_{1}=179, \;x_{2}=188, \;x_{3}=189. \end{equation*}
Figure 5 shows that the number symbol
$6$
is successfully retrieved with
$\varepsilon =0.12$
and initial data
$\varphi (0)=\arccos (\tilde{\xi }^{\mathrm{def}})$
.
4.2. The case of general standard patterns:
$M\ge 4$
Combining the procedure
$\mathscr{A}$
and
$\mathscr{B}$
, we use an idea of subgrouping to solve a general pattern retrieval problem. Let
$\{\xi ^1,\xi ^2,\dots,\xi ^M\}$
be a set of standard patterns with
$M\geq 4$
and let
$\xi ^{\mathrm{def}}$
be a defective pattern. The procedure is as follows, referred as procedure
$\mathscr{C}$
:
-
(C1) Subgroup the standard patterns so that each subgroup contains three patterns. A subgroup containing one or two patterns is allowed. We have
$\lceil \frac{M}{3}\rceil$
subgroups. -
(C2) For each subgroup containing three patterns, we apply procedure
$\mathscr{B}$
to retrieve one from the subgroup for the defective pattern
$\xi ^{\mathrm{def}}$
. For the subgroup with two patterns, we apply
$\textbf{(A2)}$
in procedure
$\mathscr{A}$
to retrieve one from the subgroup. For the subgroup containing only one pattern, this pattern is retrieved immediately. Finally we retrieve
$\lceil \frac{M}{3}\rceil$
patterns, denoted by
$\xi ^{(1,1)}, \xi ^{(1,2)}, \dots, \xi ^{(1,\lceil \frac{M}{3}\rceil )}$
. -
(C3) Collect the retrieved patterns
$\big \{\xi ^{(1,1)}, \xi ^{(1,2)}, \dots, \xi ^{(1,\lceil \frac{M}{3}\rceil )}\big \}$
. Use it as a new set of standard patterns and perform the same process in Step
$\textbf{(C1)}$
and Step
$\textbf{(C2)}$
. After finite iterations, a final one of the standard patterns is retrieved.
In each retrieval process, two patterns are excluded. Therefore, the total time of the retrieval processes required in the task is
$\lceil \frac{ M-1}{2}\rceil$
, where
$M$
is the number of standard patterns. The algorithm realising procedure
$\mathscr{C}$
is given in Algorithm 1.
Next, we present a simulation to illustrate the application of Algorithm 1. The retrieval processes for the defective symbol
$6$
in Figure 3b are described in detail. First, the mutually nonorthogonal binary patterns
$\{1,2,\dots,9,0\}$
are partitioned into four subgroups:
$\{1,2,3\}$
,
$\{4,5,6\}$
,
$\{7,8,9\}$
and
$\{0\}$
. We perform Step (C2) for each subgroup, and
$1$
,
$6$
,
$7$
,
$0$
are retrieved, respectively. We collect them and redo Steps (C1)–(C2). We repeat the above process until the set of retrieved patterns contains only one pattern. Figure 6a and 6b–6f shows that the final output is
$6$
, which means that the proper pattern is successfully retrieved.
Figure 7. The retrieval processes for the gray-scale defective symbol
$6$
in Figure 3b.
In this simulation, we terminate the programme in each subproblem if one of the overlaps is greater than
$0.95$
. We explore the CPU time consumed in Algorithm 1, and it is
$47.5318$
seconds. In contrast, it takes
$106.8977$
seconds for the procedure
$\mathscr{A}$
in ref. [Reference Zhao, Li and Xue20]. This suggests that Algorithm 1 can save time.
In Figure 7, we consider a different way of subgrouping the number symbols
$\{1,2,\dots,9,0\}$
and it shows that the defective symbol
$6$
is successfully recognised again.
5. Conclusion
In this paper, we investigated the Hebbian network of Kuramoto oscillators with applications in binary pattern retrieval. We focused on the system with three memorised patterns and developed a theory of least orthogonal lift, which converts the problems with three general memorised patterns to problems with mutually orthogonal memorised patterns. Meanwhile, we provided an estimate for the critical strength of second-order coupling, which tells us how to set-up the parameter
$\varepsilon$
such that the memorised patterns are stable while other binary patterns are all unstable in each subproblem. In this sense, the error-free retrieval can be guaranteed. This estimate for the critical strength is independent of the memorised patterns and dimensions, therefore, it is extremely convenient for applications. Combining this theory with the idea of subgrouping, we designed an approach and an algorithm for solving the problems with general standard patterns.
Acknowledgements
The authors thank the editor and the referees for their useful comments.
Funding statement
The first author was supported by the Natural Science Foundation of China (Grants No. 12201156), the China Postdoctoral Science Foundation (Grant No. 2021M701013) and the Heilongjiang Postdoctoral Science Foundation (Grant No. LBH-Z21158). The second author was supported by the Natural Science Foundation of China (Grants No. 12371509), the Heilongjiang Provincial Natural Science Foundation of China (Grant No. YQ2022A007) and the Fundamental Research Funds for the Central Universities (Grant No. HIT.BRET.2022006). This work was also supported in part by the Science Center Program of National Natural Science Foundation of China (Grant No. 62188101), and in part by the Heilongjiang Touyan Team Program.
Competing interests
The authors declare none.
Open access
