2 Problem formulation

2.2 Model description

Recall that there are three types of agents in the graph $\mathcal{G}(t)$ : cooperative agents in $\mathcal{C}$ , Byzantine agents in $\mathcal{B}$ and the leader $\ell$ satisfying $\mathcal{V}=\{\ell\}\cup\mathcal{C}\cup\mathcal{B}$ . Byzantine is a traditional name that is used in computer science and control theory [Reference Lamport, Shostak and Pease17]. These agents are able to arbitrarily update their data and hence can potentially be malicious. Formally, Byzantine agents can include both faulty agents, which are due to random errors, and malicious agents, which may be controlled by some malicious attackers. Hence, Byzantine agents in $\mathcal{B}$ are potentially harmful and pose a serious threat to the collective control objective of the multi-agent systems [Reference LeBlanc, Zhang, Koutsoukos and Sundaram18, Reference Mitra, Richards, Bagchi and Sundaram19, Reference Shang25, Reference Shang27]. They can collude with other Byzantine agents, and their identity and number are not available to the cooperative agents. Cooperative agents in $\mathcal{C}$ on the other hand are those we have control upon. They usually take a diffusive agent dynamics, and the aim of resilient (tracking) consensus problem is to design appropriate distributed protocols for these cooperative agents so that they can reach an agreement in some sense despite the obstruction from potential Byzantine agents. The leader $\ell$ is also a cooperative agent by nature, but it does not receive information from other agents, namely, $\mathcal{N}_{\ell}(t)=\emptyset$ for all t. We will adopt a linear system design to enable the cooperative agents to track the leader’s trajectory in the sense of almost sure convergence.

Formally, the dynamics of any agent $i\in\mathcal{V}$ takes the following form

(2.1) \begin{align}x_i(t+1)=Ax_i(t)+Bu_i(t),\qquad t\in\mathbb{N},\end{align}

where $A\in\mathbb{R}^{n\times n}$ , $B\in\mathbb{R}^n$ , $x_i(t)\in\mathbb{R}^n$ means the state of agent i and $u_i(t)\in\mathbb{R}$ is its control input. Here, the linear system design enables us to accommodate vector state spaces (see e.g. [Reference Shang26]), which is more general than most of the existing work on resilient consensus problems. The control input $u_i(t)$ for cooperative agents $i\in\mathcal{C}$ will be designed below in (2.5). For a Byzantine agent $i\in\mathcal{B}$ , $u_i(t)$ can take any value [Using the unifying framework (2.1) for Byzantine agents is for the ease of presentation; essentially, no restriction is imposed on the behavior of Byzantine dynamics]. The leader agent $\ell$ assumes $u_{\ell}(t)=f(t)$ for some function $f:\mathbb{N}\rightarrow\mathbb{R}$ satisfying the following assumption.

Assumption 1. We assume $\lim_{t\rightarrow\infty}x_{\ell}(t)=x_f$ for some $x_f\in\mathbb{R}^n$ , and $\|x_{\ell}(t)-x_f\|=o\big(t^{-1}\big)$ as $t\rightarrow\infty$ . Here, $\|\cdot\|$ is the Euclidean norm in $\mathbb{R}^n$ .

Remark 1. The convergence here is more general compared to the constant reference states required in [Reference Usevitch and Panagou29, Reference Usevitch and Panagou30]. Under Assumption 1, the state $x_{\ell}(t)$ is obviously bounded. Although the convergence assumption is more restrictive compared with the boundedness requirement in [Reference Razaee, Parisini and Polycarpou22], the bounds information has to be shared with all cooperative agents therein, which is difficult to realize in a distributed manner in practice. It is also worth noting that the convergence here is non-probabilistic because $\ell$ is not influenced by other agents through random edges.

Remark 2. A simple example that satisfies Assumption 1 is $f(t)\equiv0$ . In fact, the dynamics for the leader reduces to $x_{\ell}(t+1)=Ax_{\ell}(t)$ by (2.1). It follows from the Jordan normal form, if any eigenvalue $\lambda$ of A has norm less than 1, Assumption 1 is met. The convergence is exponentially fast.

Denote by $\operatorname{det}(\lambda I_n-A)=\lambda^n+\alpha_1\lambda^{n-1}+\alpha_2\lambda^{n-2}+\cdots+\alpha_n$ the characteristic polynomial of A. If the pair (A, B) is controllable, we define the transformation matrix

(2.2) \begin{align} \Gamma=\left(B, AB, A^2B, \cdots, A^{n-1}B\right)\cdot\left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c@{\quad}c} 1&\alpha_1&\alpha_2&\cdots&\alpha_{n-1}\\[2pt] 0&1&\alpha_1&\cdots&\alpha_{n-2}\\[2pt] 0&0&1&\cdots&\alpha_{n-3}\\[-1pt] \vdots&\vdots&\vdots&\ddots&\vdots\\[2pt] 0&0&0&\cdots&1 \end{array}\right)\in\mathbb{R}^{n\times n}.\end{align}

Here, the controllability of (A, B) means the matrix $\left(B, AB, A^2B,\cdots, A^{n-1}B\right)$ has rank n [Reference Chen6]. The system (2.1) can be recast as the following canonical form [Reference Wang, Wang and Kong32] by using the similarity transformation $x_i(t)=\Gamma y_i(t)$ for $i\in\mathcal{V}$ :

(2.3) \begin{align}y_i(t+1)=\,\hat{\!A}y_i(t)+\hat{B}u_i(t),\qquad t\in\mathbb{N},\end{align}

where $\,\hat{\!A}=\Gamma^{-1}A\Gamma=\left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c@{\quad}c}-\alpha_1&-\alpha_2&-\alpha_3&\cdots&-\alpha_n\\[1pt]1&0&0&\cdots&0\\[1pt]0&1&0&\cdots&0\\[1pt]\vdots&\vdots&\vdots&\ddots&\vdots\\[1pt]0&0&0&\cdots&0\end{array}\right)\in\mathbb{R}^{n\times n}$ and $\hat{B}=\Gamma^{-1}B=$ $(1,0,0,\cdots,0)^{\operatorname{T}}\in\mathbb{R}^n$ , where $\operatorname{T}$ is the matrix transpose.

Write $y_i(t)=\left(y_{i,n-1}(t),y_{i,n-2}(t),\cdots,y_{i,0}(t)\right)^{\operatorname{T}}$ . Instead of transmitting vector states between agents in $\mathcal{G}(t)$ , we define a scalar value $z_i(t)$ for each agent $i\in\mathcal{V}$ :

(2.4) \begin{align}z_i(t)=y_{i,n-1}(t)+\sum_{k=1}^{n-1}\beta_ky_{i,n-1-k}(t),\end{align}

which encodes the components of $y_i(t)$ . We will see in the next section the information agents exchange over $\mathcal{G}(t)$ will be expressed by $z_i(t)$ . Here, $\beta_k\in\mathbb{R}$ for $k=1,2,\cdots, n-1$ are chosen such that the polynomial $\lambda^{n-1}+\beta_1\lambda^{n-2}+\beta_2\lambda^{n-3}+\cdots+\beta_{n-1}$ becomes Schur stable, meaning that its roots are in the open unit disk. A quick sufficient condition for Schur stability is, for example, $0<\beta_{n-1}<\beta_{n-2}<\cdots<\beta_1<1$ [Reference Bhattacharyya, Datta and Keel3].

2.3 Resilient tracking consensus protocol

In this subsection, we introduce the resilient tracking consensus strategy for cooperative agents, which is a purely distributed protocol.

Fix a parameter $r\in\mathbb{N}$ . For each $i\in\mathcal{C}$ , at time step $t\in\mathbb{N}$ , the agent i uses the encoded state information $z_j(t)$ $\big(\,j\in\mathcal{N}^{\mathrm{in}}_i(t)\big)$ from its in-neighbors and arranges the list $z_{\big(|\mathcal{N}^{\mathrm{in}}_i(t)|\big)}(t)\leqslant z_{\big(|\mathcal{N}^{\mathrm{in}}_i(t)|-1\big)}(t)\leqslant\cdots\leqslant z_{(1)}(t)$ . If at least r values in this list are larger than $z_i(t)$ , we remove the corresponding vertices j of the largest r values (except for $j=\ell$ ). If there are less than r such values, all of their corresponding vertices are removed (except for $j=\ell$ ). Analogously, if at least r values in this sorted list are smaller than $z_i(t)$ , we remove the corresponding vertices j of the smallest r values (except for $j=\ell$ ). If there are less than r such values, all of their corresponding vertices are removed (except for $j=\ell$ ). All removed vertices are recorded in the set $\mathcal{R}_i(t)$ . The strategy is shown in Tab. 1 below. The control input in (2.1) for agent i is taken as

(2.5) \begin{align}u_i(t)=&\sum_{k=1}^n\alpha_ky_{i,n-k}(t)-\sum_{k=1}^{n-1}\beta_ky_{i,n-k}(t)+z_i(t)\nonumber\\&+\gamma_i(t)\cdot\sum_{j\in\mathcal{N}^{\mathrm{in}}_i(t)\backslash\mathcal{R}_i(t)}a_{ij}(t)(z_j(t)-z_i(t)),\end{align}

where $0<\gamma_i(t)<\big(\sum_{j\in\mathcal{N}^{\mathrm{in}}_i(t)}a_{ij}(t)\big)^{-1}$ . This control function $u_i(t)$ can be further generalized to accommodate a nonlinear diffusion; see Section 3.

Table 1. Processing for a cooperative agent $i\in \mathcal{C}$ .

Remark 3. The removal of extreme values is a key feature of W-MSR-type algorithms. The intuition behind is that the Byzantine agents might manipulate the system states by taking some extremely large or small values to deviate the states of cooperative agents. If these extreme values are removed, the system is more likely to achieve consensus. This rationale is also in line with our daily wisdom in many scoring systems in sports and competitions, where the top and bottom scores are removed to ensure a fair judgment. The meaning of the parameter r will be explained below.

Remark 4. In the above algorithm, the leader takes the role of a trustworthy node [Reference Abbas, Laszka and Koutsoukos1], whose value will always be trusted by a cooperative agent i. The idea of trustworthy nodes has been employed in the study of resilient leaderless consensus to effectively mitigate the influence of Byzantine agents [Reference Abbas, Laszka and Koutsoukos1, Reference Fu, Qin, Shi, Zheng and Kang11, Reference Huang, Wu, Chang, Tao and He15]. This assumption is also necessary in resilient tracking consensus because if the leader is not trusted, it plays a role essentially the same as a malicious agent and leader-following may not be feasible. A stronger condition that requires a leader being identifiable has been proposed in [Reference Razaee, Parisini and Polycarpou22], where the value of a leader is fed into the protocol differently from a cooperative agent (A convergence rate is estimated as a result of this).

Remark 5. Although the entry $a_{ij}(t)$ of the adjacency matrix is a random variable for any given time t, $\gamma_i(t)$ can be prescribed as a deterministic number. For example, if we know $a_{ij}(t)\leqslant M(t)$ holds for some $M(t)>0$ and all $i,j\in\mathcal{V}$ , $\gamma_i(t)$ can be chosen in the range of $(0,M(t)|\mathcal{N}^{\mathrm{in}}_i(t)|)$ .

The above strategy is pertinent to a parameter r, which we will use to bound the maximum number of Byzantine agents in the neighborhood of a cooperative agent. In particular, a vertex set $\mathcal{S}\subseteq\mathcal{V}$ is called r-local if for any $i\not\in\mathcal{S}$ , $|\mathcal{N}^{\mathrm{in}}_i(t)\cap\mathcal{S}|\leqslant r$ for any $t\in\mathbb{N}$ . If $\mathcal{B}$ is r-local, clearly any cooperative agent will have at most r Byzantine in-neighbors.

With the above resilient tracking consensus strategy, we will show that the cooperative agents in $\mathcal{C}$ can follow the leader $\ell$ in the sense of almost sure convergence as $t\rightarrow\infty$ . Specifically, we say the resilient tracking consensus is achieved for the multi-agent system (2.1) if $\|x_i(t)-x_{\ell}(t)\|\rightarrow0$ as $t\rightarrow\infty$ almost surely for any $i\in\mathcal{C}$ and initial configuration $\{x_i(0)\}_{i\in\mathcal{V}}$ .

3 Convergence analysis

In this section, we investigate the resilient tracking consensus of the linear system (2.1) in the presence of Byzantine agents in a dynamic random graph $\mathcal{G}(t)$ . Our main result is the following.

Proof. For any agent $i\in\mathcal{C}$ , by (2.3) and (2.5), we have

(3.1) \begin{align}y_i(t+1)=&\left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c@{\quad}c}-\alpha_1&-\alpha_2&-\alpha_3&\cdots&-\alpha_n\\1&0&0&\cdots&0\\0&1&0&\cdots&0\\\vdots&\vdots&\vdots&\ddots&\vdots\\0&0&0&\cdots&0\end{array}\right)y_i(t)+\left(\begin{array}{c}1\\[2pt]0\\[2pt]0\\[2pt]\vdots\\[2pt]0\end{array}\right)\cdot\left(\sum_{k=1}^n\alpha_ky_{i,n-k}(t)\right.\nonumber\\&\left.-\sum_{k=1}^{n-1}\beta_ky_{i,n-k}(t)+z_i(t)+\gamma_i(t)\cdot\sum_{j\in\mathcal{N}^{\mathrm{in}}_i(t)\backslash\mathcal{R}_i(t)}a_{ij}(t)(z_j(t)-z_i(t))\right).\end{align}

Noting that $y_i(t)=(y_{i,n-1}(t),y_{i,n-2}(t),\cdots,y_{i,0}(t))^{\operatorname{T}}$ for $t\in\mathbb{N}$ , we obtain

(3.2) \begin{align}y_{i, n-1}(t+1)=&-\sum_{k=1}^{n-1}\beta_ky_{i,n-k}(t)+z_i(t)\nonumber\\& +\gamma_i(t)\cdot\sum_{j\in\mathcal{N}^{\mathrm{in}}_i(t)\backslash\mathcal{R}_i(t)}a_{ij}(t)(z_j(t)-z_i(t)),\end{align}

and

(3.3) \begin{align} y_{i,n-2}(t+1)=y_{i,n-1}(t), \quad y_{i,n-3}(t+1)=y_{i,n-2}(t), \quad \cdots,\quad y_{i,0}(t+1)=y_{i,1}(t).\end{align}

It follows from the definition of $z_i(t)$ in (2.4) and (3.2) that for $i\in\mathcal{C}$ ,

(3.4) \begin{align} z_i(t+1)=z_i(t)+\gamma_i(t)\cdot\sum_{j\in\mathcal{N}^{\mathrm{in}}_i(t)\backslash\mathcal{R}_i(t)}a_{ij}(t)(z_j(t)-z_i(t)).\end{align}

Since $0<\gamma_i(t)<\big(\!\sum_{j\in\mathcal{N}^{\mathrm{in}}_i(t)}a_{ij}(t)\big)^{-1}$ , $z_i(t+1)$ in (3.4) can be viewed as a convex combination of $z_i(t)$ and $z_j(t)$ for $j\in\mathcal{N}^{\mathrm{in}}_i(t)\backslash\mathcal{R}_i(t)$ . Define $\overline{z}(t)=\max\{\{z_i(t)\}_{i\in\mathcal{C}},z_{\ell}(t)\}$ and $\underline{z}(t)=\min\{\{z_i(t)\}_{i\in\mathcal{C}},z_{\ell}(t)\}$ . Fix $t\in\mathbb{N}$ . For any agent $i\in\mathcal{C}$ , since $\mathcal{B}$ is r-local and the convex combination is in effect, there are at most r in-neighbors (excluding possibly $\ell$ ) taking values outside the range $[\underline{z}(t),\overline{z}(t)]$ at time $t+1$ . By our resilient tracking consensus strategy, any such neighbors will be removed in the update for agent i at time step $t+1$ with two possible exceptions: (i) a neighbor j has state $z_j(t+1)$ satisfying $\overline{z}(t)<z_j(t+1)\leqslant z_{\ell}(t+1)$ or (ii) a neighbor j has state $z_j(t+1)$ satisfying $\underline{z}(t)>z_j(t+1)\geqslant z_{\ell}(t+1)$ . In both cases, $\ell\in\mathcal{N}^{\mathrm{in}}_i(t)$ . Employing Assumption 1, we know there exists $0\leqslant\varepsilon(t)=o\big(t^{-1}\big)$ such that

(3.5) \begin{align}\underline{z}(t)-\varepsilon(t)\leqslant\underline{z}(t+1)\leqslant\overline{z}(t+1)\leqslant\overline{z}(t)+\varepsilon(t)\end{align}

for $t\in\mathbb{N}$ . This relationship holds regardless of the randomness of edges in $\mathcal{G}(t)$ .

For any cooperative agent $i\in\mathcal{C}$ , write $w_i(t)=(y_{i,n-2}(t), y_{i, n-3}(t),\cdots,y_{i,0}(t))^{\operatorname{T}}\in\mathbb{R}^{n-1}$ . Therefore, $y_i(t)=(y_{i,n-1}(t),w_i(t)^{\operatorname{T}})^{\operatorname{T}}$ . In view of (3.3), we obtain a new linear system

(3.6) \begin{align}w_i(t+1)=\,\tilde{\!A}w_i(t)+\tilde{B}z_i(t),\end{align}

where $\,\tilde{\!A}=\left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c@{\quad}c}-\beta_1&-\beta_2&-\beta_3&\cdots&-\beta_{n-1}\\[2pt]1&0&0&\cdots&0\\[2pt]0&1&0&\cdots&0\\[2pt]\vdots&\vdots&\vdots&\ddots&\vdots\\[2pt]0&0&0&\cdots&0\end{array}\right)\in\mathbb{R}^{(n-1)\times(n-1)}$ and $\tilde{B}=(1, 0, 0, \cdots,0)^{\operatorname{T}}$ $\in$ $\mathbb{R}^{n-1}$ , where $z_i(t)$ is given by (2.4).

Define $b(t)=\overline{z}(t)-\underline{z}(t)-2t\varepsilon(t)$ for $t\in\mathbb{N}$ and let $\mathcal{F}(t)$ be the $\sigma$ -algebra generated by $\{\{z_i(0)\}_{i\in\mathcal{V}},\{z_i(1)\}_{i\in\mathcal{V}}, \cdots,\{z_i(t)\}_{i\in\mathcal{V}}\}$ . By (3.5), we obtain $\mathbb{E}|b(t)|<\infty$ for $t\in\mathbb{N}$ and $\mathbb{E}(b(t)|\mathcal{F}(t-1))\leqslant b(t-1)$ for $t\geqslant1$ . Thus, b(t) is a super-martingale relevant to the filtration $\mathcal{F}(t)$ . There is some $b\geqslant0$ such that

(3.7) \begin{align}\lim_{t\rightarrow\infty}b(t)=\lim_{t\rightarrow\infty}(\overline{z}(t)-\underline{z}(t))=b,\end{align}

almost surely by employing the martingale convergence theorem [Reference Hall and Heyde14, p. 2] and our assumption of $\varepsilon(t)$ .

Next, we will show the limit $b=0$ . In fact, if this is not the case we define three random sets $\mathcal{I}_1(t)=\{i\in\mathcal{V}\backslash\mathcal{B}:z_i(t)=\overline{z}(t)\}\not=\emptyset$ , $\mathcal{I}_2(t)=\{i\in\mathcal{V}\backslash\mathcal{B}:z_i(t)=\underline{z}(t)\}\not=\emptyset$ , and $\mathcal{I}_3(t)=(\mathcal{V}\backslash\mathcal{B})\backslash(\mathcal{I}_1(t)\cup\mathcal{I}_2(t))$ . Clearly, these three sets partition $\mathcal{V}\backslash\mathcal{B}$ . We consider three cases depending on the location of leader $\ell$ : (i) $\ell\in\mathcal{I}_1(t)$ , (ii) $\ell\in\mathcal{I}_2(t)$ , and (iii) $\ell\in\mathcal{I}_3(t)$ .

(i) Suppose $\mathcal{I}_2(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)\not=\emptyset$ . We have, say, $i\in\mathcal{I}_2(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)$ . By our resilient tracking consensus strategy, $z_{\ell}(t)$ will be used in the update for agent i at time step $t+1$ via (3.4). Since $\mathcal{B}$ is r-local, it is easy to see our protocol guarantees any value outside $[\underline{z}(t),\overline{z}(t)]$ will not be used in the update for agent i at time step $t+1$ in this situation. Because of the convex combination in (3.4), agent i must take a larger value at the next time step, that is, $z_i(t+1)>z_i(t)$ .

If $\mathcal{I}_2(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)=\emptyset$ , by our assumption that $\mathcal{G}(t)$ is leader-follower $(2r+1)$ -robust with positive probability, we know $\mathcal{I}_2(t)$ is $(2r+1)$ -robust with positive probability. Hence, with positive probability there exists $i\in\mathcal{I}_2(t)$ such that i has at least $2r+1$ in-neighbors outside the set $\mathcal{I}_2(t)$ . As $\mathcal{B}$ is r-local, i has at least $r+1$ cooperative in-neighbors outside $\mathcal{I}_2(t)$ . By our algorithm, at least one of such neighbors will be used in the update for agent i at time step $t+1$ . Using the fact that $\mathcal{B}$ is r-local again, we know that any value outside $[\underline{z}(t),\overline{z}(t)]$ will not be used by i in the update at time $t+1$ . Therefore, via the convex combination in (3.4), we obtain $z_i(t+1)\geqslant z_i(t)$ and with positive probability $z_i(t+1)>z_i(t)$ .

(ii) This case can be shown analogously as in (i) by considering two situations: $\mathcal{I}_1(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)\not=\emptyset$ and $\mathcal{I}_1(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)=\emptyset$ . Consequently, we know that with positive probability there exists some agent $i\in\mathcal{I}_1(t)$ such that $z_i(t+1)\leqslant z_i(t)$ and with positive probability $z_i(t+1)<z_i(t)$ .

(iii) We first consider two situations $\mathcal{I}_2(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)\not=\emptyset$ and $\mathcal{I}_2(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)=\emptyset$ and proceed similarly as in case (i) to conclude there is some agent $i\in\mathcal{I}_2(t)$ satisfying $z_i(t+1)\geqslant z_i(t)$ and with positive probability $z_i(t+1)>z_i(t)$ . Then, we consider two situations $\mathcal{I}_1(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)\not=\emptyset$ and $\mathcal{I}_1(t)\cap\mathcal{N}^{\mathrm{out}}_{\ell}(t)=\emptyset$ and proceed similarly as in case (ii) to derive there exists agent $j\in\mathcal{I}_1(t)$ satisfying $z_j(t+1)\leqslant z_j(t)$ and with positive probability $z_j(t+1)<z_j(t)$ .

Combining (i), (ii), (iii) and noting that $\mathcal{G}(t)$ is a finite graph, we conclude that $\overline{z}(t)-\underline{z}(t)$ tends to zero in probability as $t\rightarrow\infty$ . But this is at odds with our assumption that $b>0$ in (3.7). Therefore, we have shown that almost surely

(3.8) \begin{align}\lim_{t\rightarrow\infty}(\overline{z}(t)-\underline{z}(t))=0.\end{align}

Using Assumption 1, (3.5) and (3.8), we obtain that there is some $c\in\mathbb{R}$ satisfying $\lim_{t\rightarrow\infty}z_i(t)=c$ almost surely for all $i\in\mathcal{C}\cup\{\ell\}$ . Moreover, by (2.4) we know that c is determined via the following

(3.9) \begin{align}c=\left(\Gamma^{-1}x_f\right)_{n-1}+\sum_{k=1}^{n-1}\beta_k\!\left(\Gamma^{-1}x_f\right)_{n-1-k},\end{align}

where $\Gamma^{-1}x_f=\left(\left(\Gamma^{-1}x_f\right)_{n-1},\left(\Gamma^{-1}x_f\right)_{n-2},\cdots,\left(\Gamma^{-1}x_f\right)_0\right)$ .

With the convergence of $z_i(t)$ at hand, we now move onto the convergence of $w_i(t)$ for $i\in\mathcal{C}$ . In doing so, define an error vector $\delta_i(t)=w_i(t)-c/\left(1+\sum_{k=1}^{n-1}\beta_k\right)1_{n-1}$ , where $1_{n-1}\in\mathbb{R}^{n-1}$ is an all one vector. Using (3.6), we arrive at

(3.10) \begin{align} \delta_i(t+1)& =\,\tilde{\!A}w_i(t)+\tilde{B}z_i(t)-\frac{c}{1+\sum_{k=1}^{n-1}\beta_k}1_{n-1}\nonumber\\[3pt] & =\,\tilde{\!A}\delta_i(t)+\,\tilde{\!A}\frac{c}{1+\sum_{k=1}^{n-1}\beta_k}1_{n-1}+ \tilde{B}z_i(t)-\frac{c}{1+\sum_{k=1}^{n-1}\beta_k}1_{n-1}\nonumber\\[3pt] & =\,\tilde{\!A}\delta_i(t)+\left[\left(\begin{array}{c@{\quad}c@{\quad}c@{\quad}c@{\quad}c}-\beta_1&-\beta_2&-\beta_3&\cdots&-\beta_{n-1}\\[1pt] 1&0&0&\cdots&0\\[1pt] 0&1&0&\cdots&0\\ \vdots&\vdots&\vdots&\ddots&\vdots\\[1pt] 0&0&0&\cdots&0\end{array}\right)\cdot\left(\begin{array}{c}1\\[1pt] 1\\[1pt] 1\\[1pt] \vdots\\ 1\end{array}\right)-\left(\begin{array}{c}1\\[1pt] 1\\[1pt] 1\\ \vdots\\[1pt] 1\end{array}\right)\right]\nonumber\\ &\qquad \cdot\frac{c}{1+\sum_{k=1}^{n-1}\beta_k}+\tilde{B}z_i(t)\nonumber\\[5pt] & =\,\tilde{\!A}\delta_i(t)-\tilde{B}c+\tilde{B}z_i(t).\end{align}

Denote by $\rho_i(t)=z_i(t)-c$ for $i\in\mathcal{C}$ and hence $\lim_{t\rightarrow\infty}\rho_i(t)=0$ almost surely. From (3.10), we have

(3.11) \begin{align}\delta_i(t+1)=\,\tilde{\!A}\delta_i(t)+\tilde{B}\rho_i(t).\end{align}

It is easy to see that the matrix $\,\tilde{\!A}$ is Schur stable because the characteristic polynomial $\operatorname{det}(\lambda I_{n-1}-\,\tilde{\!A})$ is a Schur stable polynomial as defined at the end of Section 2.2. Thanks to the Lyapunov stability theory [Reference Chen6, Theorem 5.D5], we obtain for any positive definite matrix $Q\in\mathbb{R}^{(n-1)\times(n-1)}$ there is a positive definite matrix $P\in\mathbb{R}^{(n-1)\times(n-1)}$ such that $P-Q=\tilde{A}^{\operatorname{T}}P\,\tilde{\!A}$ . For any cooperative agent $i\in\mathcal{C}$ , define the Lyapunov candidate function $\Theta_i(t)=\delta_i(t)^{\operatorname{T}}P\delta_i(t)$ and along the solution of (3.11) it follows

(3.12) \begin{align}\Theta_i(t+1)-\Theta_i(t)&=-\delta_i(t+1)^{\operatorname{T}}P\delta_i(t+1)-\delta_i(t)^{\operatorname{T}}P\delta_i(t)\nonumber\\&=-\delta_i(t)^{\operatorname{T}}Q\delta_i(t)+2\delta_i(t)^{\operatorname{T}}\tilde{A}^{\operatorname{T}}P\tilde{B}\rho_i(t)+\rho_i(t)^2\tilde{B}^{\operatorname{T}}P\tilde{B}.\end{align}

Recall that $\,\tilde{\!A}$ is stable. Applying the input-to-state stability result [Reference Jiang and Wang16, Example 3.4, Lemma 3.5] to the linear system (3.6), we derive that $w_i(t)$ is bounded for all $t\in\mathbb{N}$ . In light of the definition of $\delta_i(t)$ , we know $\delta_i(t)$ is also bounded. Since $\lim_{t\rightarrow\infty}\rho_i(t)=0$ almost surely, we obtain $\lim_{t\rightarrow\infty}|2\delta_i(t)^{\operatorname{T}}\tilde{A}^{\operatorname{T}}P\tilde{B}\rho_i(t)+\rho_i(t)^2\tilde{B}^{\operatorname{T}}P\tilde{B}|=0$ almost surely. By (3.12), it is not difficult to see $\lim_{t\rightarrow\infty}\Theta_i(t)=0$ almost surely for any cooperative agent $i\in\mathcal{C}$ . Indeed, if this is not true, $\delta_i(t)\not\rightarrow0$ as $n\rightarrow\infty$ by the definition of $\Theta_i(t)$ and the positive definiteness of P. Using (3.12) and the positive definiteness of Q, for any $t\in\mathbb{N}$ there are $\varepsilon>0$ and $\hat{t}\geqslant t$ such that $\Theta_i(\hat{t}+1)-\Theta_i(\hat{t})\leqslant-\varepsilon$ . This is in conflict with the fact that $\Theta_i(t)$ for all $t\in\mathbb{N}$ is lower bounded by zero.

Using the definition of $\Theta_i(t)$ again, we have $\lim_{t\rightarrow\infty}\delta_i(t)=0$ almost surely for $i\in\mathcal{C}$ . Thereby, $\lim_{t\rightarrow\infty}w_i(t)=c/\left(1+\sum_{k=1}^{n-1}\beta_k\right)1_{n-1}$ almost surely. Applying (3.3), we know $\lim_{t\rightarrow\infty}y_i(t)=c/\left(1+\sum_{k=1}^{n-1}\beta_k\right)1_n$ almost surely for all cooperative agents $i\in\mathcal{C}$ . As $x_i(t)=\Gamma y_i(t)$ , it follows that

(3.13) \begin{align}\lim_{t\rightarrow\infty}x_i(t)& =\frac{c}{\left(1+\sum_{k=1}^{n-1}\beta_k\right)}\Gamma1_n\nonumber\\& =\Gamma 1_n\frac{(1,\beta_1, \beta_2,\cdots,\beta_{n-1})\Gamma^{-1}x_f}{\left(1+\sum_{k=1}^{n-1}\beta_k\right)}\nonumber\\& =\Gamma\Gamma^{-1}x_f\nonumber\\& =x_f\end{align}

almost surely by using (3.9). This means all cooperative agents will track the evolution of the leader agent $\ell$ by Assumption 1. The proof is complete. $\Box$

Remark 8. The above linear system analysis method can be applied to a more general control function $u_i(t)$ given as follows

(3.14) \begin{align}u_i(t)=&\sum_{k=1}^n\alpha_ky_{i,n-k}(t)-\sum_{k=1}^{n-1}\beta_ky_{i,n-k}(t)+z_i(t) +\gamma_i(t)\cdot\sum_{j\in\mathcal{N}^{\mathrm{in}}_i(t)\backslash\mathcal{R}_i(t)}a_{ij}(t)g_{ij}(z_j(t),z_i(t)),\end{align}

where $\gamma_i(t)$ is chosen as in (2.5) and the nonlinear function $g_{ij}:\mathbb{R}\times\mathbb{R}\rightarrow\mathbb{R}$ satisfies (i) $g_{ij}(z_1,z_2)=0$ if and only if $z_1=z_2$ and (ii) there exist two constants $0<q_i\leqslant q^{\prime}_i\leqslant1$ such that

(3.15) \begin{align}q_i\leqslant\frac{g_{ij}(z_1,z_2)}{z_1-z_2}\leqslant q^{\prime}_i\end{align}

for any $z_1\not=z_2$ . The control function $u_i(t)$ in (2.5) is a special case of (3.14) by taking $g_{ij}(z_1,z_2)=z_1-z_2$ . It is worth mentioning that the two bounds in (3.15) do not depend on any node j as we can always take the maximum and minimum over all nodes of $\mathcal{G}(t)$ (which is a finite graph) without loss of generality.

With this coupling function, Theorem 1 can be proved along the same line. In fact, when (2.5) is replaced by (3.14), the update rule (3.4) for an agent $i\in\mathcal{C}$ becomes

(3.16) \begin{align}z_i(t+1)=z_i(t)+\gamma_i(t)\cdot\sum_{j\in\mathcal{N}^{\mathrm{in}}_i(t)\backslash\mathcal{R}_i(t)}a_{ij}(t)g_{ij}(z_j(t),z_i(t)).\end{align}

Although $z_i(t+1)$ is no longer a convex combination of $z_i(t)$ and $\{z_j(t)\}_{j\in\mathcal{N}^{\mathrm{in}}_i(t)\backslash\mathcal{R}_i(t)}$ , it is bounded by two convex combinations of them from the above and the below.

4 Numerical simulations

In this section, we present two examples to illustrate our theory.

Example 1. Consider a directed graph $\mathcal{G}$ over $N=8$ vertices with $\ell=1$ , $\mathcal{B}=\{2\}$ and $\mathcal{C}=\{3,4,\cdots,8\}$ ; see Figure 1. The time-dependent random graph $\mathcal{G}(t)$ is built upon $\mathcal{G}$ by assuming binary edge weights, namely, $a_{ij}(t)$ takes 1 or 0. The edge probabilities $p_{ij}(t)\equiv0.5$ and the corresponding weights $a_{ij}(t)$ are independent for all i, j, and t. It is straightforward to check that $\mathcal{G}$ is leader-follower 3-robust. Hence, $\mathcal{G}(t)$ is leader-follower 3-robust with positive probability for any $t\in\mathbb{N}$ ; c.f. Remark 6.

Figure 1. A leader-follower 3-robust graph $\mathcal{G}$ with $N=8$ agents: leader $\ell=1$ , Byzantine $\mathcal{B}=\{2\}$ and cooperative $\mathcal{C}=\{3,4,5,6,7,8\}$ .

The agents’ dynamics for $i\in\mathcal{V}$ and $t\in\mathbb{N}$ and are taken as

(4.1) \begin{align}x_i(t+1)=\left(\begin{array}{c@{\quad}c}0.5&1\\[3pt]0&0.5\end{array}\right)x_i(t)+\left(\begin{array}{c}0\\[3pt]1\end{array}\right)u_i(t),\end{align}

where $u_1(t)=f(t)\equiv0$ , $u_2(t)=0.2\cos(t/5)$ , and $u_i(t)$ for $i\in\mathcal{C}$ is given by (2.5) with $\alpha_1=-1$ , $\alpha_2=0.25$ , $\beta_1=0.5$ , $\gamma_i(t)\equiv0.2$ . It is easy to check that the pair (A, B) is stable, $x_f=(0,0)^{\operatorname{T}}$ , and all conditions in Theorem 1 are satisfied. Let $x_i(t)=(x_{i,1}(t),x_{i,2}(t))^{\operatorname{T}}$ for $i\in\mathcal{C}$ .

By choosing initial states randomly within $[-4,4]$ , we display the dynamical evolution of the multi-agent system in Figure 2. We observe that both components of the states of cooperative agents track the leader agent 1 and converge to the equilibrium $x_f$ as expected.

Figure 2. Resilient tracking consensus over the 2-dimensional state space with the leader agent 1 (red) and the Byzantine agent 2 (blue).

Figure 3. The animal interaction network of $N=12$ macaca fuscata. It has a leader $\ell=1$ , Byzantine $\mathcal{B}=\{2\}$ and cooperative $\mathcal{C}=\{3,4,\cdots,12\}$ .

In this example, the Byzantine agent $2\in\mathcal{N}^{\mathrm{out}}_1(t)$ with probability 0.5 and it influences half of the cooperative agents in $\mathcal{C}$ . Moreover, the Byzantine agent possesses a dynamics repeatedly oscillating around the equilibrium. All this represents a highly adverse scenario to the collective consensus task. Remarkably, our simple distributed protocol enables the cooperative agents to converge towards the leader’s equilibrium relatively quickly.

Let $n=1$ . The agents’ dynamics for $i\in\mathcal{V}$ and $t\in\mathbb{N}$ and are chosen as

(4.2) \begin{align}x_i(t+1)=\frac12 x_i(t)+u_i(t),\end{align}

where $u_1(t)=f(t)\equiv0$ , $u_2(t)=0.1\cos(t/20)$ , and $u_i(t)$ for $i\in\mathcal{C}$ is given by (2.5) with $\alpha_1=-0.5$ , $\gamma_i(t)\equiv 1/12$ . It is clear that $x_f=0,$ and all conditions in Theorem 1 are satisfied.

The initial configuration is chosen as $x_1(0)=-0.8$ , $x_2(0)=0.3$ , $x_3(0)=-0.1$ , $x_4(0)=-0.5$ , $x_5(0)=1$ , $x_6(0)=-0.7$ , $x_7(0)=0.2$ , $x_8(0)=0.6$ , $x_9(0)=0.7$ , $x_{10}(0)=-1$ , $x_{11}(0)=0.1$ , $x_{12}(0)=-0.3$ . In Figure 4(a) and (b), we show the state evolution for different edge probability $p=0.1$ and $p=0.8$ , respectively. In both cases, the followers are able to track the leader and a denser network tends to give rise to a faster convergence.

Figure 4. Resilient tracking consensus for the animal interaction network with the leader agent 1 (red) and the Byzantine agent 2 (blue) with edge density (a) $p=0.1$ and (b) $p=0.8$ .