2.1. Definition and main result

Let $a,b \in \mathbb{R}$ and $\lambda > 0$ . We consider a discrete-time process $(\tilde{X}_n)_{n \geq 1}$ with initial condition $(\tilde{X}_0, \tilde{X}_{-1})$ such that the following holds for all $n\ge 1$ :

\begin{equation*} \textrm{conditioned on}\ \tilde{X}_{-1},\ldots,\tilde{X}_{n-1}: \quad \tilde{X}_n\sim\mathcal{P}((a\tilde{X}_{n-1} + b\tilde{X}_{n-2} + \lambda)_+), \end{equation*}

where $(\!\cdot\!)_+$ is the ReLU function defined on $\mathbb{R}$ by $(x)_+ \;:\!=\; \max(0,x)$ .

As we said previously, some papers have already dealt with the linear version of this process: if a and b are non-negative, the parameter of the Poisson random variable in (1) is also non-negative, and the ReLU function vanishes. In this case, [Reference Ferland, Latour and Oraichi6, Proposition 1] states that the process is a second-order stationary process if $a+b<1$ . This weak stationarity ensures that the mean, variance, and autocovariance are constant with time.

Let us define the function

\begin{align*} b_\textrm{c}(a) = \begin{cases} 1 & a \le 0, \\[5pt] 1-a & a\in(0,2), \\[5pt] -\dfrac{a^2}{4} & a\ge 2, \end{cases}\end{align*}

and define the following sets (see Figure 1):

(2) \begin{equation} \mathcal R = \{(a,b)\in\mathbb{R}^2\colon b<b_\textrm{c}(a)\}, \qquad \mathcal T = \{(a,b)\in\mathbb{R}^2\colon b>b_\textrm{c}(a)\}.\end{equation}

Our main result is the following.

If $(a,b) \in \mathcal{T}$ , then the sequence $(\tilde X_n)_{n\ge0}$ satisfies, almost surely, $\tilde X_n+\tilde X_{n+1}\underset{n\to\infty}{\longrightarrow}+\infty$ .

Figure 1. The partition of the parameter space described in Theorem 2. The green region corresponds to $\mathcal{R}$ , while the red region corresponds to $\mathcal T$ . The smaller figures are typical trajectories of the Markov chain $(X_n)_{n\ge0}$ for each region of the parameter space. In all the simulations, we chose $\lambda=1$ . The region delineated by the dashed triangular line corresponds to the region of parameter space for which the linear recurrence equation $y_{n} = ay_{n-1} + by_{n-2} + \lambda$ is bounded for all $n \in \mathbb{N}$ , for any given $y_0, y_1 \in \mathbb{R}$ . See Appendix A for more details.

This result derives from studying the natural Markov chain associated with $\tilde X_n$ that is defined by

(3) \begin{equation} X_n \;:\!=\; (\tilde X_{n}, \tilde X_{n-1}), \qquad n\ge0.\end{equation}

Before giving more details about the behaviour of $(X_n)_{n\ge0}$ , let us comment on Theorem 1. In particular, we stress that the condition for convergence in law is not symmetrical in a and b. More precisely, for any $a\in\mathbb{R}$ , the sequence $(\tilde X_n)$ can be tight provided that b is chosen small enough, but the converse is not true as soon as $b>1$ . This induces inhibition having a stronger regulating effect when it occurs after an excitation rather than before.

The question of the critical behaviour of the process on the boundary $\{b=b_\textrm{c}(a)\}$ remains open and presents a difficult question for further work.

2.2. The associated Markov chain

As mentioned, the main part of this article is devoted to studying a Markov chain $(X_n)$ which encodes the time dependency of $(\tilde X_n)$ . We rely on the recent treatment in [Reference Douc, Moulines, Priouret and Soulier5] for results about Markov chains. In particular, we use their notion of irreducibility, which is weaker than the usual notion of irreducibility typically found in textbooks on Markov chains (on a discrete state space). Thus, a Markov chain is called irreducible if there exists an accessible state, i.e. a state that can be reached with positive probability from any other state. Following [Reference Douc, Moulines, Priouret and Soulier5], we refer to the usual notion of irreducibility (i.e. every state is accessible) as strong irreducibility.

The transition matrix of the Markov chain $(X_n)_{n\ge0}$ defined in (3) is thus given for $(i,j,k,l)\in\mathbb{N}^4$ by

\begin{equation*} P((i,j), (k,\ell)) = \delta_{i\ell}\frac{\textrm{e}^{-s_{ij}}s_{ij}^k}{k!},\end{equation*}

where $s_{ij} \;:\!=\; (ai+bj+\lambda)_+$ and

\begin{align*}\delta_{ij} \;:\!=\;\begin{cases} 1 & \textrm{if}\; i=j, \\[5pt] 0 & \textrm{otherwise}.\end{cases}\end{align*}

In other words, starting from a state (i, j), the next step of the Markov chain will be (k, i) where $k \in \mathbb{N}$ is the realization of a Poisson random variable with parameter $s_{ij}$ . In particular, if $s_{ij} = 0$ , then the next step of the Markov chain is (0,i).

Since the probability that a Poisson random variable is zero is strictly positive, it is possible to reach the state (0,0) with positive probability from any state in two steps. In particular, the state (0,0) is accessible and the Markov chain is irreducible. Furthermore, the Markov chain is aperiodic [Reference Douc, Moulines, Priouret and Soulier5, Section 7.4], since $P((0,0),(0,0)) = \textrm{e}^{-\lambda} > 0$ . Note that strong irreducibility may not hold (see Proposition 2).

Recall the definition of the sets $\mathcal{R}$ and $\mathcal{T}$ in (2).

Theorem 1 is a simple consequence of this result. Indeed, in the case of $(a,b)\in\mathcal{R}$ , the convergence in law of $\tilde{X}_n$ simply derives from the convergence in law of $X_n$ since $\tilde{X}_n$ is the first coordinate of $X_n$ . In the transient case, $(a,b)\in\mathcal{T}$ , the result in Theorem 1 simply derives from the fact that $||X_n||_1\to_{n\to\infty}\infty$ almost surely.

The rest of the article is devoted to the proof of Theorem 2. We first focus on the recurrent case in Section 3, then on the transient case in Section 4. Throughout, we provide typical trajectories of the cases considered. For the sake of clarity we have plotted the realizations of $(X_n)_{n=0,\dots,N}$ by connecting its successive realizations. Unless otherwise stated, for coherence purposes we always set $X_0 = (0,0)$ and $\lambda = 1$ for our plots.

3.1. Foster–Lyapounov drift criteria

Drift criteria are powerful tools that were introduced in [Reference Foster8], and deeply studied and popularized in [Reference Meyn and Tweedie13], among others. These drift criteria allow us to prove convergence to the invariant measure of Markov chains and yield explicit rates of convergence. Here we use the treatment from [Reference Douc, Moulines, Priouret and Soulier5], which is influenced by [Reference Meyn and Tweedie13], but is more suitable for Markov chains that are irreducible but not strongly irreducible.

A set of states $C\subset \mathbb{N}^2$ is called petite [Reference Douc, Moulines, Priouret and Soulier5, Definition 9.4.1] if there exists a state $x_0\in \mathbb{N}^2$ and a probability distribution $(p_n)_{n\in\mathbb{N}}$ on $\mathbb{N}$ such that $\inf_{x\in C}\sum_{n\in\mathbb{N}} p_n P^n(x,x_0) > 0$ , where we recall that $P^n(x,x_0)$ is the n-step transition probability from x to $x_0$ . Since the Markov chain $(X_n)_{n\ge0}$ is irreducible, any finite set is petite (take $x_0$ to be the accessible state) and any finite union of petite sets is petite [Reference Douc, Moulines, Priouret and Soulier5, Proposition 9.4.5].

Let $V \colon \mathbb{N}^2 \to [1,\infty)$ be a function, $\varepsilon\in(0,1]$ , $K<\infty$ , and $C\subset \mathbb{N}^2$ a set of states. We say that the drift condition $D(V,\varepsilon,K,C)$ is satisfied if

\begin{equation*} \Delta V(x) \;:\!=\; \mathbb{E}_x [V(X_1)-V(X_0)] \leq -\varepsilon V(x)+K\boldsymbol 1_C,\end{equation*}

where $\mathbb{E}_x[\cdot] = \mathbb{E}[\,\cdot \mid X_0 = x]$ . It is easy to see that this condition implies the condition $D_g(V,\lambda,b,C)$ from [Reference Douc, Moulines, Priouret and Soulier5, Definition 14.1.5], with $\lambda = 1-\varepsilon$ and $b = K$ .

Proposition 1. Assume that the drift condition $D(V,\varepsilon,K,C)$ is verified for some V, $\varepsilon$ , K, and C as above, and assume that C is petite. Then there exists $\beta > 1$ and a probability measure $\pi$ on $\mathbb{N}^2$ such that, for every initial state $x\in \mathbb{N}^2$ ,

\begin{align*} \beta^n\times \sum_{y\in \mathbb{N}^2} V(y)|\mathbb{P}_x(X_n = y)-\pi(y)| \to 0, \qquad n\to\infty. \end{align*}

In particular, for every initial state $x\in \mathbb{N}^2$ , $\beta^n d_\textrm{TV}(\operatorname{Law}(X_n),\pi)\to 0$ as $n\to\infty$ , and $\pi$ is an invariant probability measure for the Markov chain $(X_n)_{n\ge0}$ .

We consider separately the following ranges of the parameters:

\begin{align*} \mathcal R_1 &= \{(a,b)\in \mathbb{R}^2\colon a,b < 1\textrm{ and }a+b<1\}; \\[5pt] \mathcal R_2 &= \{a>0\textrm{ and }a^2 + 4b < 0\}; \\[5pt] \mathcal R_3 &= \{1\le a<2\textrm{ and } {-}1 < b < 1-a\}.\end{align*}

We then have $\mathcal R = \mathcal R_1\cup\mathcal R_2\cup \mathcal R_3$ ; see Figure 2.

Figure 2. Illustration of the three zones of parameters on which the proof of ergodicity will be carried.

3.2. Case $\mathcal R_1$

This case is the natural extension of the results that have been already proved for the linear process (see [Reference Ferland, Latour and Oraichi6, Proposition 1]).

Let $V \colon \mathbb{N}^2 \to \mathbb{R}_+$ be the function defined by $V ( i, j ) \;:\!=\; \alpha i + \beta j + 1$ , where $\alpha,\beta >0$ are parameters to be chosen later. Then $V ( i, j ) \geq 1$ for all $( i, j ) \in \mathbb{N}^2$ . We look for $\varepsilon > 0$ such that $\Delta V(x) + \varepsilon V(x) \leq 0$ except for a finite number of $x \in \mathbb{N}^2$ .

Let $\varepsilon > 0$ to be properly chosen later. Then,

\begin{align*} \Delta V ( i, j ) + \varepsilon V ( i, j ) & = \sum_{k \in \mathbb{N}}\frac{\textrm{e}^{-s_{ij}}s_{ij}^k}{k!}(\alpha k + \beta i + 1) - (\alpha i + \beta j + 1) + \varepsilon(\alpha i + \beta j + 1) \\[5pt] & = \alpha s_{ij} + i(\beta - \alpha + \alpha\varepsilon) + j(\beta\varepsilon - \beta) + \varepsilon . \end{align*}

Note that $s_{ij} = 0$ or $s_{ij} = ai + bj + \lambda > 0$ . In both cases, $\Delta V + \varepsilon V$ is a linear function of $( i, j ) \in \mathbb{N}^2$ . We thus choose $\alpha, \beta$ such that the coefficients of $\Delta V + \varepsilon V$ are negative, so there will be only a finite number of (i, j) that satisfy $\Delta V ( i, j ) + \varepsilon V ( i, j ) \geq 0$ .

Let us first consider couples (i, j) such that $s_{ij}=0$ . According to the above, it is sufficient to have

\begin{align*}\left\{ \begin{array}{ll} \beta - \alpha + \alpha\varepsilon < 0 \\[5pt] \beta\varepsilon - \beta < 0 \end{array} \right. \quad \Longleftrightarrow \quad\left\{ \begin{array}{ll} \beta < \alpha(1-\varepsilon) \\[5pt] \varepsilon < 1 \end{array} \right. \end{align*}

In what follows, we impose $\varepsilon <1$ .

If $s_{ij} = ai+bj+\lambda > 0$ , then

\begin{align*}\Delta V( i, j )+ \varepsilon V ( i, j ) = i(\alpha a - \alpha + \beta + \alpha \varepsilon) + j(\alpha b + \beta \varepsilon - \beta) + \lambda \alpha + \varepsilon.\end{align*}

For the same reasons as before, it is sufficient to have $\alpha,\beta > 0$ such that

\begin{align*}\left\{ \begin{array}{ll} \alpha a - \alpha + \beta + \alpha\varepsilon < 0 \\[5pt] \alpha b + \beta\varepsilon - \beta < 0 \end{array} \right. \quad \Longleftrightarrow \quad\left\{ \begin{array}{ll} \beta < \alpha (1 - a - \varepsilon) \\[5pt] \beta > \dfrac{\alpha b}{1-\varepsilon} \qquad\quad (\mbox{since } \varepsilon < 1) \end{array}\right.\end{align*}

Let $\alpha \;:\!=\; 1$ . With the above statements we thus want to choose $\beta, \varepsilon > 0$ such that

\begin{align*}\left\{ \begin{array}{ll} \dfrac{b}{1-\varepsilon} < \beta < 1-a-\varepsilon, \\[5pt] \beta < 1-\varepsilon; \end{array} \right. \quad \textrm{i.e.} \quad \dfrac{b}{1-\varepsilon} < \beta < \min\{ 1-a-\varepsilon, 1 - \varepsilon\}.\end{align*}

Recall that $a+b<1$ , so it is possible to find $\varepsilon_0 \in (0,1)$ small enough that, for all $\tilde{\varepsilon} \leq \varepsilon_0$ ,

\begin{align*}\frac{b}{1-\tilde{\varepsilon}} < 1 -a-\tilde{\varepsilon}.\end{align*}

If $a \geq 0$ , then $\min\{1-a-\varepsilon, 1 - \varepsilon\} = 1 - a - \varepsilon$ and, since $a < 1$ , we can choose $\varepsilon \leq \varepsilon_0$ small enough that ${b}/({1-\varepsilon}) < \beta < \min \{ 1-a-\varepsilon, 1 - \varepsilon\}$ on one hand, and $1-a-\varepsilon > 0$ on the other hand. It is thus possible to choose $\beta > 0$ such that

\begin{align*}\dfrac{b}{1-\varepsilon} < \beta < \min \{ 1-a-\varepsilon, 1 - \varepsilon \}.\end{align*}

If $a < 0$ , then $\min \{ 1-a-\varepsilon, 1 - \varepsilon \} = 1 - \varepsilon$ . Since $b<1$ , it is possible to set $\varepsilon \leq \varepsilon_0$ small enough that $b < (1 - \varepsilon)^2$ . Hence, we have ${b}/({1-\varepsilon}) < 1 - \varepsilon$ , so that it is possible to choose $\beta > 0$ that satisfies our constraints.

Note that $\Delta V( 0, 0 ) = \lambda > 0$ . Hence, with $\alpha, \beta, \varepsilon > 0$ chosen as above, $\Delta V( i, j )\le -\varepsilon V ( i, j )$ except for a finite number of states $( i, j ) \in \mathbb{N}^2$ . This proves that a drift condition $D(V,\varepsilon,C)$ holds for a finite set C, which yields the result.

3.3. Case $\mathcal R_2$

In this section, we assume that $a>0$ and $a^2+4b<0$ . The Lyapounov function we will consider is the following:

\begin{align*}\textrm{for all } ( i, j ) \in \mathbb{N}^2, \quad V ( i, j ) = \dfrac{i}{j+1}+1.\end{align*}

Before getting into the details, a remark about this function. While we initially discovered it by trial and error, it has an interesting geometric interpretation. As shown in Figure 3, for the case of $\mathcal R_2$ the macroscopic trajectories of the Markov chain tend to turn counterclockwise until they hit the j-axis and eventually get pulled back to (0, 0). This provides a heuristic understanding of why V should be a Lyapounov function. Indeed, it is an increasing function of the angle between the vector (i, j) and the j-axis, and therefore $V(X_n)$ should have a tendency to decrease whenever $X_n$ is far away from the j-axis.

Figure 3. An illustration of the case of $\mathcal R_2$ . Here, the parameters are $a = 3$ , $b = -2.5$ , and $N=1000$ . The red region indicates the set A of couples (i, j) such that $s_{ij}=s_{0i}=0$ .

We now turn to the details. We will need to distinguish the region A of the states (i, j) where $s_{ij}=0$ (shown in red in Figure 3):

\begin{equation*} A \;:\!=\; \{(i, j) \in \mathbb{N}^2 \colon s_{ij} = 0\} = \{(i, j) \in \mathbb{N}^2 \colon ai+bj+\lambda \le 0\}.\end{equation*}

We have the following lemma.

Proof. By the definition of A, we have $s_{ij} = 0$ for all $(i,j)\in A$ , and hence $P((i,j),(0,i)) = 1$ . Furthermore, for every $i\in \mathbb{N}$ , since $b<-a^2/4 < 0$ ,

\begin{align*} P((0,i),(0,0)) = \textrm{e}^{-s_{0i}} = \textrm{e}^{-(\lambda+bi)_+} \ge \textrm{e}^{-\lambda}. \end{align*}

It follows that $\inf_{(i,j)\in A} P^2((i,j),(0,0)) \ge \textrm{e}^{-\lambda} > 0$ , which shows that A is petite.

Proof. Since ${a^2}/{4}+b < 0$ , there exists $\varepsilon \in (0,1)$ small enough that

\begin{align*}\frac{(a+\varepsilon)^2}{4}+b(1-\varepsilon)<0.\end{align*}

Consider $(i,j) \not \in A$ , and compute

\begin{align*} \Delta V ( i, j ) + \varepsilon V(i,j) = \dfrac{(\varepsilon-1)i^2+bj^2+(a+\varepsilon)ij + L_1(i,j)}{(i+1)(j+1)}, \end{align*}

where $L_1(i,j)$ is a polynomial of degree 1. In the numerator we recognize a quadratic form, and as ${(a+\varepsilon)^2}/{4}+b(1-\varepsilon)<0$ , this quadratic form is negative definite. Thus, there are only a finite number of $(i,j) \not \in A$ such that $\Delta V(i,j) +\varepsilon V(i,j) > 0$ . We define $C \subset \mathbb{N}^2 \setminus A$ to be the finite set of such (i, j).

Note that, for every $(i,j)\in A$ , $\Delta V(i, j) + \varepsilon V(i,j) \le \mathbb E_{(i,j)}[V(X_1)] = V(0,i) = 1$ . Hence, setting $K = 1\vee \max_{x\in C} \mathbb E_x[V(X_1)]\in [1,\infty)$ , the finiteness of K following from the fact that C is finite, the drift condition $D(V,\varepsilon,K,A\cup C)$ is satisfied.

Figure 4 illustrates the cutting of the state space that we just described.

In the case of $\mathcal{R}_2$ , by Lemma 1 and Lemma 2 we can now apply Proposition 1. Note that $A\cup C$ is petite because A is petite (Lemma 1), C is finite, hence petite, and the union of two petite sets is again petite. This yields the proof of case $\mathcal{R}_2$ of Theorem 2.

Figure 4. Graphical representation of the sets A and C described in the proof of Lemma 2.

3.4. Case $\mathcal R_3$

To finish the proof of Theorem 2, it suffices to consider parameters a and b such that $1\le a<2$ and $-{a^2}/{4} < b < 1-a$ . However, for the sake of conciseness, we will prove the ergodicity of the Markov chain on a larger space, namely $\mathcal R_3$ . As a consequence, this case will cover some parameter sets which have already been considered in case $\mathcal R_2$ . Note that this does not represent any issue in our proof strategy. The choice of $\mathcal{R}_3$ will become clearer later on.

We thus assume here that $1\le a<2$ and $-1<b<1-a$ . Let us denote by V the function, for all $(i,j) \in \mathbb{N}^2$ ,

\begin{align*}V ( i, j ) \;:\!=\; 1 + \bigg(i^2 - aij + \dfrac{b^2+1}{2} j^2\bigg)\textbf{1}_{A^\textrm{c}}(i,j) .\end{align*}

First, notice that the quadratic form in V is positive definite. Indeed, if $1\leq a < 2$ , then $b^2 > (1-a)^2$ and

\begin{align*}4 \times \dfrac{b^2+1}{2} - a^2 > 2(1-a)^2+2-a^2 = (a-2)^2 > 0.\end{align*}

Thus, the function V satisfies $V \geq 1$ .

Compute, for $(i,j) \not \in A$ and $\varepsilon \in (0,1)$ to be properly chosen later,

\begin{align*} \Delta V(i,j) + \varepsilon V(i,j) & = \sum_{k=0}^\infty \dfrac{\textrm{e}^{-s_{ij}} s_{ij}^k}{k!} V(k,i) + (\varepsilon-1)V(i,j) \\[5pt] & \leq \sum_{k=0}^\infty \dfrac{\textrm{e}^{-s_{ij}} s_{ij}^k}{k!} \bigg(1+\bigg(k^2 - aki + \dfrac{b^2+1}{2}i^2\bigg)\bigg) + (\varepsilon-1)V(i,j) \\[5pt] & = s_{ij}(s_{ij}+1)-ais_{ij}+\dfrac{b^2+1}{2}i^2+(\varepsilon - 1)V(i,j) + 1\\[5pt] & = \bigg(\dfrac{b^2-1}{2}+\varepsilon\bigg)i^2 + a(b+1-\varepsilon)ij + \bigg(\dfrac{b^2(1+\varepsilon)+\varepsilon-1}{2}\bigg)j^2 + L_2(i,j),\end{align*}

where $L_2(i,j)$ is a polynomial of degree 1.

We want to choose $\varepsilon \in (0,1)$ such that the above quadratic form is negative definite, i.e. such that

(4) \begin{equation} \dfrac{b^2-1}{2}+\varepsilon < 0, \qquad \bigg(\dfrac{b^2-1}{2}+\varepsilon\bigg)\bigg(\frac{b^2(1+\varepsilon)+\varepsilon-1}{2}\bigg) - \dfrac{a^2}{4}(b+1-\varepsilon)^2 > 0.\end{equation}

On the one hand, we have $b^2-1<0$ . On the other hand, the second inequality in (4) can be written as $(b^2-1)^2-a^2(b+1)^2 + k_{\varepsilon, a,b} > 0$ , where $k_{\varepsilon,a,b} \in \mathbb{R}$ satisfies $k_{\varepsilon,a,b} \underset{\varepsilon \to 0}{\longrightarrow} 0$ .

In addition, note that $(a,b) \in \mathcal{R}_3 \Longrightarrow (b^2-1)^2 - a^2(b+1)^2 > 0$ . We can therefore deduce that there exists $\varepsilon \in (0,1)$ small enough that both conditions of (4) are satisfied. Thus, there are only a finite number of $(i,j) \not \in A$ such that $\Delta V(i,j) +\varepsilon V(i,j) > 0$ . We define $C \subset \mathbb{N}^2 \setminus A$ to be the finite set of such (i, j).

Finally, similarly to Lemma 1, the set A is petite, because $b <1-a \leq 0$ . Furthermore, similarly to the case $\mathcal R_2$ , for all $(i,j)\in A$ , $\mathbb E_{(i,j)}(V(X_1))=V(0,i)$ is bounded, since $(0,i)\in A$ except for a finite number of i. Since the set C is finite, $A \cup C$ is a petite set and, up to an adequate choice of K, the drift condition $D(V,\varepsilon,K, A\cup C)$ is satisfied.

4.1. Case T1

In this region of parameters, the Markov chain eventually reaches the i and j axes. Indeed, since $a<0$ , if $(X_n)$ hits a state (i, 0) with $i \geq -{\lambda}/{a}$ , as $s_{i0} = (ai+\lambda)_+ = 0$ , the next step of the Markov chain will be (0, i). Afterwards, the Markov chain will hit the state $( \mathcal{P}(bi+\lambda), 0 )$ , with $bi+\lambda > i$ . Consequently, to follow the example, if we focus on the i axis, starting from (k, 0) with k big enough, the Markov chain will return in two steps to a state (k’, 0) belonging to the i-axis that satisfies $k' > k$ with high probability. This behaviour is illustrated in Figure 5.

Figure 5. Log-log plot of a typical trajectory of $(X_n)$ , to make the erratic behaviour of the first points of the Markov chain more visible. Here, the parameters are $a = -0.3$ , $b=1.2$ , and $N=100$ .

In order to formalize these observations, it is very natural to consider the Markov chain induced by the transition matrix $P^2$ , namely $(X_{2n+1})_{n\geq 0}$ . For $i \geq {-\lambda}/{a}$ , $s_{i0}=0$ and thus

(6) \begin{equation} \mathbb{P}\bigg(X_{2n+1} = (k, 0) \mid X_{2n-1} = (i, 0), i \geq \frac{-\lambda}{a}\bigg) = \frac{\textrm{e}^{-s_{0 i}}s_{0 i}^k}{k!} = \frac{\textrm{e}^{-(bi+\lambda)} (bi+\lambda)^k}{k!}.\end{equation}

Note that if $a \leq -\lambda$ , this result holds for $i \in \mathbb{N}$ .

Equation (6) means that if $\tilde X_{2n-1} \geq -{\lambda}/{a}$ and $\tilde X_{2n-2} = 0$ , then $\tilde X_{2n} = 0$ , and $\tilde X_{2n+1}$ is a Poisson random variable with parameter $b\tilde X_{2n-1} + \lambda$ .

Let us now prove our statement.

Proof of the transience of $(X_n)$ when $a<0$ and $b>1$ . Fix $r\in (1,b)$ . We wish to apply Lemma 3 with $m_n = 2n-1$ , $n\ge 1$ , and $S_n = \{(i,0)\in \mathbb{N}\colon i \ge r^n\}$ . We verify that assumptions (i)–(iii) from Lemma 3 hold. For the first assumption, note that if $X_{2n-1} = (i,0) \in S_n$ , then $X_{2n} = (j,i)$ for some j, hence $X_{2n-1} \ne (0,0)$ and $X_{2n} \ne (0,0)$ since $i\ge1$ . In particular, assumption (i) holds.

We now verify that the second assumption holds. For states $x,y\in \mathbb{N}^2$ , write $x\to_1 y$ if $\mathbb{P}_x(X_1 = y) > 0$ . Furthermore, for $S\subset\mathbb{N}^2$ , write $x\to_1 S$ if $x\to_1 y$ for some $y\in S$ . Note that $(0,0) \to_1 (i,0)$ for every $i\in\mathbb{N}$ , so that $(0,0)\to_1 S_1$ . Now, for every $i\in \mathbb{N}$ , we have $(i,0)\to_1 (0,i)$ , and then, because $b>0$ , $(0,i)\to_1 (j,0)$ for every $j\in\mathbb{N}$ . In particular, from every $x\in S_n$ , we can indeed reach $S_{n+1}$ in two steps. Hence, the second assumption is verified as well.

We now prove the third assumption. We claim that there exists $n_0 \in\mathbb{N}$ such that

(7) \begin{equation} \textrm{for all } n\geq n_0 \textrm{ and } x\in S_n: \quad \mathbb{P}_x(X_2\in S_{n+1}) \ge 1 - \dfrac{b}{(b-r)^2r^{n}}. \end{equation}

To prove (7), first note that, according to the earlier remark on (6), if $n_0$ is chosen such that $r^{n_0} \geq -\lambda/a$ then, starting from a state (i,0) with $i \geq r^{n_0}$ , we have $\tilde{X}_1 = 0$ almost surely and $\tilde{X}_{2}\sim \mathcal{P}(bi + \lambda)$ . Therefore, if $n\ge n_0$ and $i \ge r^n \ge r^{n_0}$ ,

\begin{align*} 1 - \mathbb{P}_{(i,0)}(\tilde X_2 \ge r^{n+1}, \tilde X_1 = 0) & = \mathbb{P}_{(i,0)}(\tilde X_2 < r^{n+1}) \\[5pt] & \le \mathbb{P}(\mathcal{P}(bi + \lambda)<r^{n+1}) \\[5pt] & \leq \mathbb{P}(\mathcal{P}(br^{n}) < r^{n+1}) \\[5pt] & = \mathbb{P}(\mathcal{P}(br^{n})-br^{n} < r^{n}(r-b)) \\[5pt] & \leq \mathbb{P}(|\mathcal{P}(br^{n})-br^{n}| > r^{n}(b-r)) \\[5pt] & \leq \dfrac{b}{(b-r)^2r^{n}}, \end{align*}

by the Bienaymé–Chebychev inequality. This proves (7). Now, (7) implies that, for all $x\in S_n$ ,

\begin{align*} \mathbb{P}_x(X_2\in S_{n+1}) \ge p_n \;:\!=\; \bigg(1 - \dfrac{b}{(b-r)^2r^{n}}\bigg)_+, \end{align*}

and

\begin{align*} \sum_{n\ge 1} (1-p_n) \le \sum_{n\ge1} \dfrac{b}{(b-r)^2r^{n}} < \infty. \end{align*}

This proves that the third assumption of Lemma 3 holds. The lemma then shows that the Markov chain is transient.

4.2. Case T2

For this case, we will take benefit from the comparison between the stochastic process $(\tilde X_n)$ and its linear deterministic version. Namely, let us consider the linear recurrence relation defined by $y_0, y_1 \in \mathbb{N}$ and

(8) \begin{equation} \textrm{for all } n \geq 0, \quad y_{n+2} = ay_{n+1} + by_n + \lambda.\end{equation}

The solutions to this equation are determined by the eigenvalues and eigenvectors of the matrix $\big(\begin{smallmatrix}0 & b \\ 1 & a\end{smallmatrix}\big)$ , which is the companion matrix of the polynomial $X^2 - aX - b$ (see Appendix A for more details). An easy calculation shows that in case T2, we have $a^2+4b > 0$ , and hence the eigenvalues are simple and real-valued. We denote the largest eigenvalue by

\begin{align*}\theta \;:\!=\; \dfrac{a+\sqrt{a^2+4b}}{2}.\end{align*}

In case T2, as can be easily verified,

(9) \begin{align} \theta & > 1, \end{align}

(10) \begin{align} \theta^2 + b & > 0. \end{align}

In fact, we can check that case T2 exactly corresponds to the region in the space of parameters a, b where $\theta>1$ , meaning that the sequence $(y_{n+1},y_n)_{n\ge0}$ , with $(y_n)_{n\ge0}$ the solution to (8), grows exponentially inside the positive quadrant, along the direction of the eigenvector $(\theta,1)$ .

In what follows, we fix $1 < r < \theta$ such that

(11) \begin{equation} r^2-ar-b < 0,\end{equation}

where we use the fact that $\theta > 1$ is the largest root of the polynomial $X^2 - aX - b$ .

We split our study into two different subcases depending on the sign of b.

Subcase T2a: $b \geq 0$

In this case, we have $a\tilde X_n + b \tilde X_{n-1} + \lambda > 0$ for all $n\in\mathbb{N}$ , and so $\tilde X_{n+1} \sim \mathcal{P}(a\tilde X_n + b\tilde X_{n-1} + \lambda)$ , i.e. no truncation is necessary. Classically, in this case $\tilde X_n$ grows exponentially in n almost surely, but we provide a simple proof for completeness.

We therefore apply Lemma 3 with the sequence $m_n=n$ and

\begin{align*}S_n=\{ (i,j)\in\mathbb{N}^2, i\ge r^n, j\ge r^{n-1}\}.\end{align*}

With this notations, assumption (i) is automatically satisfied. Assumption (ii) is also satisfied, because $(i,j)\to_1 (k,i)$ for every $i,j,k\in\mathbb{N}$ , since $ai+bj+\lambda > 0$ for every $i,j\in\mathbb{N}$ , as explained above.

In order to prove assumption (iii), let us consider $n \in \mathbb{N}$ and let $(i,j)\in S_n$ . By definition, starting from (i, j), $\tilde X_{1} \sim \mathcal{P}(a i + bj + \lambda)$ . Thus,

\begin{align*} \mathbb{P}_{(i,j)} (\tilde X_1 < r^{n+1}) & = \mathbb{P}(\mathcal{P}(ai + bj + \lambda) < r^{n+1}) \\[5pt] & \leq \mathbb{P}(\mathcal{P}(ar^{n} + b r^{n-1}) < r^{n+1}) \\[5pt] & = \mathbb{P}(\mathcal{P}(ar^{n} + b r^{n-1}) - (ar^{n} + br^{n-1}) < r^{n-1}(r^2-ar - b)).\end{align*}

Recall that $r^2-ar-b < 0$ by (11), which implies

\begin{align*} \mathbb{P}_{(i,j)} (\tilde X_1 < r^{n+1}) & \leq \mathbb{P}(|\mathcal{P}(ar^n + b r^{n-1}) - (ar^n + br^{n-1})| > -r^{n-1}(r^2-ar - b)) \\[5pt] & \leq \dfrac{(a+b)r^2}{r^{n}(r^2-ar-b)^2} ,\end{align*}

where we again used the Bienaymé–Chebychev inequality. Thus,

\begin{align*}\mathbb{P}_{(i,j)} (X_1\in S_{n+1}) \ge \bigg(1-\dfrac{(a+b)r^2}{r^{n}(r^2-ar-b)^2}\bigg)_+ =: p_n.\end{align*}

This allows us to conclude the proof with Lemma 3, as in the previous case.

Subcase T2b: $b \lt 0$

In this case, because of the negativity of b it is more difficult to find an adequate lower bound of $a\tilde X_n + b\tilde X_{n-1}$ . We thus prove a stronger result, which is illustrated in Figure 6: asymptotically, the process $(\tilde X_n)$ grows exponentially and the ratio $\tilde X_{n+1}/\tilde X_n$ is close to $\theta$ .

From (11) and (10), we can choose $\varepsilon > 0$ small enough that

(12) \begin{align} r^2 - a(r-\varepsilon)-b & < 0, \end{align}

(13) \begin{align} \theta^2-\theta\varepsilon+b & > 0. \end{align}

We use Lemma 3 using $m_n = n$ and, for $n \in \mathbb{N}^*$ ,

\begin{align*}S_n=\bigg\{(i,j)\in\mathbb{N}^2,i\ge r^n,j\ge r^{n-1},\bigg|\frac{i}{j}-\theta\bigg|\leq\varepsilon\bigg\}.\end{align*}

Note that assumption (i) from Lemma 3 is again automatically verified. Assumption (ii) is also verified since, for $(i,j)\in S_n$ ,

(14) \begin{equation} a i + b j + \lambda > (a(\theta - \varepsilon)+b) j \geq (a(r-\varepsilon)+b)j > 0 \end{equation}

by (12), and so $(i,j)\to_1 (k,i)$ for every $k\in \mathbb{N}$ .

Figure 6. Log-log plot of a typical trajectory of $(X_n)$ , with $a = 1.5$ , $b=-0.3$ , and $N=100$ .

We now show that assumption (iii) from Lemma 3 is verified. Let $n\in \mathbb{N}$ and $(i,j) \in S_n$ . Then

(15) \begin{equation} \mathbb{P}_{(i,j)}(X_{1}\notin S_{n+1}) \leq \mathbb{P}_{(i,j)}(\tilde X_{1} < r^{n+1}) + \mathbb{P}_{(i,j)}\bigg(\bigg|\dfrac{\tilde X_{1}}{i}-\theta\bigg| > \varepsilon\bigg).\end{equation}

We first bound the first term on the right-hand side of (15). By (14), we have

\begin{align*} \mathbb{P}_{(i,j)}(\tilde X_{1} < r^{n+1}) & = \mathbb{P}(\mathcal{P}(ai + b j + \lambda) < r^{n+1}) \\[5pt] & \leq \mathbb{P}(\mathcal{P}([a(r-\varepsilon)+b]r^{n-1}) < r^{n+1}) \\[5pt] & = \mathbb{P}(\mathcal{P}([a(r-\varepsilon)+b]r^{n-1}) \\[5pt] & \quad - [a(r-\varepsilon)+b]r^{n-1} < r^{n-1}[r^2 - a(r-\varepsilon)-b]).\end{align*}

Furthermore, using (12) and applying the Bienaymé–Chebychev inequality,

(16) \begin{align} \mathbb{P}_{(i,j)}(\tilde X_{1} < r^{n+1}) & \leq \mathbb{P}\big(|\mathcal{P}([a(r-\varepsilon)+b]r^{n-1}) \nonumber \\[5pt] & \qquad - [a(r-\varepsilon) + b]r^{n-1}| > -r^{n-1}[r^2 - a(r-\varepsilon)-b]\big) \nonumber \\[5pt] & \leq \dfrac{[a(r-\varepsilon)+b]r^{n-1}}{[r^{n-1}[r^2 - a(r-\varepsilon)- b]]^2} \nonumber \\[5pt] & = \dfrac{[a(r-\varepsilon)+b]r}{r^n[r^2 - a(r-\varepsilon)- b]^2} = \dfrac{C_1}{r^n}, \end{align}

where $C_1$ is a constant that does not depend on n.

We now bound the second term on the right-hand side of (15). Let us write

\begin{equation*} \bigg|\dfrac{\tilde X_{1}}{i}-\theta\bigg| = \bigg|\dfrac{\tilde X_{1}-\mathbb{E}_{(i,j)}[\tilde X_{1}]}{i}\bigg| + \bigg|\dfrac{\mathbb{E}_{(i,j)}[\tilde X_{1}]}{i}-\theta\bigg|. \end{equation*}

First, notice that, for any $(i,j) \in S_n$ ,

(17) \begin{align} \bigg|\dfrac{\mathbb{E}_{(i,j)}[\tilde X_{1}]}{i}-\theta\bigg| = \bigg|\dfrac{ai+bj+\lambda}{i}-\theta\bigg| & = \bigg|a+b\dfrac{j}{i} + \dfrac{\lambda}{i}-\theta\bigg| \nonumber \\[5pt] & \leq \underbrace{\bigg|a+\frac{b}{\theta}-\theta\bigg|}_{= 0} + |b|\bigg|\dfrac{j}{i} - \frac{1}{\theta}\bigg| + \frac{\lambda}{i} \nonumber \\[5pt] & < \dfrac{|b|}{\theta(\theta-\varepsilon)}\varepsilon + \frac{\lambda}{i}, \end{align}

where we used that if $|x-\theta|<\varepsilon$ and $\varepsilon < \theta$ , then

\begin{align*}\bigg|\frac 1 x - \frac 1 \theta\bigg| =\frac{|\theta-x|}{x\theta} < \frac{\varepsilon}{\theta(\theta-\varepsilon)}.\end{align*}

To prove that

\begin{align*}\mathbb{P}_{(i,j)}\bigg(\bigg|\dfrac{\tilde X_{1}}{i}-\theta\bigg| > \varepsilon\bigg) \leq \dfrac{C_2}{r^n},\end{align*}

where $C_2$ is a constant that does not depend on n, we deduce from (17) that it is sufficient to show that

\begin{align*}\mathbb{P}_{(i,j)}\bigg(\dfrac{|\tilde X_{1}-\mathbb{E}_{(i,j)}[\tilde X_{1}]|+\lambda}{i} >\delta\varepsilon\bigg) \leq \frac{C_2}{r^n},\end{align*}

where, by (13),

\begin{align*}\delta \;:\!=\; 1- \dfrac{|b|}{\theta(\theta-\varepsilon)} > 0.\end{align*}

Furthermore, since $b<0$ and $(i,j)\in S_n$ , $ai + bj +\lambda \leq ai + \lambda \leq (a+\lambda)i$ . We finally have, using the Bienaymé–Chebychev inequality,

\begin{align*} \mathbb{P}_{(i,j)}\bigg(\dfrac{|\tilde X_{1}-\mathbb{E}_{(i,j)}[\tilde X_{1}]|+\lambda}{i} > \delta\varepsilon\bigg) & = \mathbb{P}_{(i,j)}\big(|\tilde X_{1}-\mathbb{E}_{(i,j)}[\tilde X_{1}]| > \delta\varepsilon i-\lambda\big) \\[5pt] & \leq \dfrac{(a+\lambda)i}{(\delta \varepsilon i-\lambda)^2} \\[5pt] & = \dfrac{a+\lambda}{(\delta\varepsilon\sqrt{i} - \lambda/\sqrt{i})^2} \\[5pt] & \leq \dfrac{a+\lambda}{(\delta\varepsilon r^{n/2}-\lambda r^{-n/2})^2}.\end{align*}

The last inequality holds for a sufficiently large n. Indeed, since $i \geq r^n$ , we always have $\delta \varepsilon \sqrt{i} - \lambda/ \sqrt{i} > \delta \varepsilon r^{n/2}-\lambda r^{-n/2}$ , and for n large enough we have $\delta \varepsilon \sqrt{i}- {\lambda}/{\sqrt{i}} > 0$ . This yields, for some constant $C_2<\infty$ ,

(18) \begin{equation} \mathbb{P}_{(i,j)}\bigg(\dfrac{|\tilde X_{1}-\mathbb{E}_{(i,j)}[\tilde X_{1}]|+\lambda}{i} > \bigg(1-\dfrac{|b|}{\theta(\theta-\varepsilon)}\bigg)\varepsilon\bigg) \leq \frac{C_2}{r^n}.\end{equation}

Combining (16) and (18), we have

\begin{align*}\mathbb{P}_{(i,j)}(X_{1}\in S_{n+1}) \ge \bigg(1-\dfrac{C_1+C_2}{r^n}\bigg)_+ \;=\!:\; p_n,\end{align*}

which will finally lead us to the result, by using Lemma 3 as before.

5.1. Critical behavior

In the case of linear Hawkes processes, it is well known that, at criticality, the process achieves fractal-like, i.e. heavy-tail, behaviour related to critical branching processes. It is tempting to believe that this should remain true on the whole boundary between the phases $\mathcal R$ and $\mathcal T$ , but the fractal exponents might differ.

For the sake of completeness, we offer a numerical study of the various critical cases of the model considered, which indicates different behaviour depending on whether $a<2$ or $a>2$ . We present realizations of the process $(\tilde X_n)$ , as we believe it is simpler to visualize the behavioural differences compared to showing realizations of the Markov chain in $\mathbb N^2$ . Given the diversity of the process behaviours, we anticipate the need for various probabilistic tools to describe the process evolution over long time spans. We consider the same setting as for the previous figures: an initial condition $\tilde X_{-1}, \tilde X_0 = 0$ and $\lambda = 1$ . The number N describes the number of simulated steps.

In Figure 7, we observe linear growth of the discrete-time process $\widetilde{X}_n$ , with oscillations to 0 when $a<0$ and $b=1$ (left panel) and without oscillations in the case $a+b=1$ . The situation seems to be different for $a\ge2$ and $b=-a^2/4$ . When $a>2$ we observe exponential growth (Figure 8 (left)), similar to the transient regime, while the case $a=2$ presents large excursions away from 0, but deciphering transient or recurrent behaviour is difficult. These simulations show that the study of these critical cases is an interesting research topic for the future.

Figure 7. Trajectories of $(\widetilde{X}_n)_{0\le n\le 1000}$ for critical parameters $(a,b)=(\!-\!1,1)$ on the left and $(a,b)=(0.5,0.5)$ on the right. We observe linear growth of the process as could be expected in critical cases.

Figure 8. Trajectories of $(\widetilde{X}_n)_{0\le n\le 1000}$ for critical parameters $(a,b)=(4,-4)$ on the left and $(a,b)=(2,-1)$ on the right.

5.2. Generalization of the model

As explained at the beginning of the article, the results obtained here should be seen as a starting point for the search for necessary and sufficient conditions for the stability of Hawkes processes with inhibition, in discrete or continuous time.

We believe that obtaining a similar classification in the cases $p>2$ or $p=\infty$ is a very difficult problem. It should be closely related to the study of the asymptotic behaviour of certain deterministic equations, such as the non-linear recurrence equation $x_n = (a_1x_{n-1}+\cdots+a_px_{n-p})_+$ . It seems that the algebraic structures underlying these equations are intricate and, to this date, unknown. Understanding these structures seems crucial for the study of the asymptotic behaviour of the solutions to these equations.