1. Introduction
Random walks are widely used in applied mathematics, physics, and computer science and much attention is devoted to the analysis of performance measures of random walks in graphs. Examples of such performance measures are the cover time which is the time needed to visit at least once all the nodes of the graph or the hitting time of a particular subset of nodes which is the time needed to reach one node of the subset. Many applications of random walks on graphs are proposed in [Reference Aldous2] as for instance the analysis of electrical networks which are detailed in [Reference Doyle and Snell10]. Very interesting surveys on random walks on graphs are [Reference Aldous and Fill3] and [Reference Lovasz16] while [Reference Motowani and Raghavan17] additionally contains many applications in both computer science and networks. In telecommunication systems, random walks are used to explore the network or to collect information. For instance, in the context of graph theory for social networks, a notion of centrality has been used to assess the relative importance of nodes in a given network topology and a novel form of centrality based on the second-order moments of return times in random walks was proposed in [Reference Kermarrec, Le Merrer, Sericola and Trédan15].
It has been shown in [Reference Feige12] and [Reference Feige11] that the expected cover time in a random walk with n states is at least 
$(1 - o(1))n \ln(n)$ and at most 
$(4/27+o(1))n^3$. In the case of a regular graph, for which each vertex has the same number of neighbors, it is at most 
$2n^2$, see [Reference Feige11]. In [Reference Banderier and Dobrow5], the authors consider an extension of the cover time called the marking time defined as follows. When the random walk reaches node i, a coin is flipped and with probability pi the node i is marked. The marking time is then the time needed by the walk to mark all its nodes. They give general formulas for the expected marking time of a random walk and provide asymptotics when the pi’s are small. In [Reference Ikeda, Kubo and Yamashita14], the authors first prove that for any transition probability matrix, both the expected hitting time and the expected cover time of the path graph is 
$\Omega(n^2)$. They also prove that for some particular Markov chains called degree-biased random walks, the expected hitting time is 
$O(n^2)$ and the expected cover time is 
$O(n^2\ln(n))$. The cover time of a discrete-time homogenous Markov chain is analyzed in [Reference Sericola19] where both its distribution and its moments are considered with applications to particular graphs namely the generalized cycle graph, the complete graph, and the generalized path graph.
In this paper, we consider hitting times on the lollipop graph which consists of a complete graph (a clique) on m vertices and a path graph on n − m vertices. One end of the path graph is connected to one vertex, denoted by m, of the clique, and its other end is the target vertex n for the hitting times. Such a graph has been denoted by 
$L_m^n$ for 
$n,m \in \mathbb{N}$, with 
$2 \leq m \leq n$. It is represented in Figure 1 for m = 8 and n = 14. Observe that when m = n we obtain the well-known complete graph Kn.
Figure 1. The lollipop graph 
$L_8^{14}$.
The expected hitting time of the path end node n, when starting from a node 
$i \in \{1,\ldots,m-1\}$, has been obtained in [Reference Brightwell and Winkler9] where the authors prove that the maximum expected hitting time among all the n-vertex graphs is obtained for the lollipop graph 
$L_m^n$ when 
$m = \lceil 2(n-1)/3 \rceil$, with value 
$(4/27+o(1))n^3$. This major result is obviously quite important and shows that a detailed analysis of the lollipop graph is relevant.
This is the goal of our paper. The novelty of the proof techniques lies in the way we obtain detailed recurrence formulas for the calculation of all moments of the hitting times. These formulas then make it possible to exhibit the asymptotic behavior of all the moments. This phase is clearly the hardest to come by. From these asymptotic moments, we obtain the asymptotic distribution of the hitting times using a standard argument of complex analysis.
The paper is organized as follows. In Section 2, we give general formulas for the distribution and for all the moments of the hitting times in a general Markov chain. We use these results in Section 3 to derive the mean and the variance of the hitting times on the lollipop graph. Our proof for obtaining the expected hitting times on the lollipop graph differs from the one of [Reference Brightwell and Winkler9] but we need it to obtain higher order moments. We also obtain in Section 3 the moment generating function of the hitting times and we propose a simple method to compute the convergence domain of this function which consists in evaluating the eigenvalues and in particular the spectral radius of the sub-stochastic matrix arising in the distribution expression of the hitting time. We propose in Section 4 a general recursive way to obtain all the moments of the hitting time Tn of node n and we use this recursion to obtain the asymptotic behavior of the distribution of Tn when n tends to infinity for various values of m. As a particular case we obtain that, when 
$m = \lceil 2(n-1)/3 \rceil$, the asymptotic distribution of 
$27T_n/(4n^3)$ is exponential with rate 1. Section 5 concludes the paper.
2. General results
Consider a homogeneous discrete-time Markov chain 
$X=\{X_n,\; n \in \mathbb{N}\}$ over a finite state space S. We denote by P the transition probability matrix of X. We suppose that X is irreducible and, for every 
$j \in S$, we define the hitting time Tj of state j as the first instant at which state j is reached by X, that is
\begin{equation*}T_{j} = \inf\{n \geq 0 \mid X_n = j\}.\end{equation*} The irreducibility of X implies that, for every 
$j \in S$, Tj is finite almost surely. Let A be a non-empty subset of states of S. We denote by TA the hitting time of subset A, defined by the time needed to reach A, that is
\begin{equation*}T_A = \inf\{n \geq 0 \mid X_n \in A\}.\end{equation*} The complementary subset of A in S, 
$S \setminus A$, is denoted simply by Ac. The partition 
$\{A,A^c\}$ of S induces a decomposition of P into four submatrices as
\begin{equation*}P = \left( \begin{array}{cc}
P_{A} & P_{A,A^c}\\
P_{A^c,A} & P_{A^c}
\end{array}
\right), \end{equation*} where matrix PA (resp. 
$P_{A^c}$) contains the transition probabilities between states of A (resp. Ac) and matrix 
$P_{A,A^c}$ (resp. 
$P_{A^c,A}$) contains the transition probabilities from states of A (resp. Ac) to states of Ac (resp. A). We also denote by 
$\mathbb{1}$ the column vector with all its entries equal to 1, its dimension being specified by the context. With these notations, we have, for all 
$i \in A^c$ and 
$k \geq 1$, see for instance [Reference Sericola18],
\begin{equation}
\mathbb{P}\{T_A = k \mid X_0 = i\} = (P_{A^c}^{k-1}P_{A^c,A} \mathbb{1})_i.
\end{equation} The r th moment of the hitting time of subset A starting from state 
$i \in A^c$ is then given by
\begin{equation}
\mathbb{E}(T_A^r \mid X_0 = i) = \sum_{k=1}^\infty k^r (P_{A^c}^{k-1}P_{A^c,\,A} \mathbb{1})_i.
\end{equation} The following theorem gives a finite expression of this r-th moment which involves only the successive powers of matrix 
$\left(I - P_{A^c}\right)^{-1}$.
Theorem 2.1. For every 
$r \geq 1$, we have
\begin{equation*}\mathbb{E}(T_A^r \mid X_0 = i) =
\sum_{\ell=1}^{r} c_{r,\ell}\left(\left(I - P_{A^c}\right)^{-\ell}\mathbb{1}\right)_i\end{equation*} where, for every 
$\ell = 1,\ldots,r$,
\begin{equation*}c_{r,\ell} = (-1)^r\sum_{j=1}^{\ell} (-1)^j \binom{\ell}{j}\, j^r.\end{equation*}Proof. For 
$r \geq 0$ and 
$x \in (0,1]$, we introduce the notation 
${\displaystyle G_r(x) = \sum_{k=1}^\infty k^r x^{k-1}P_{A^c}^{k-1}}$. We prove by induction that, for 
$r \geq 1$, we have
\begin{equation*}G_{r}(x) = \sum_{\ell=1}^{r} c_{r,\ell} \left(I - xP_{A^c}\right)^{-(\ell+1)}
\mbox{ where }
c_{r,\ell} = (-1)^r\sum_{j=1}^{\ell} (-1)^j \binom{\ell}{j}\, j^r.\end{equation*}The result is true for r = 1. Indeed, for r = 1, we have
\begin{equation*}G_1(x) = \sum_{k=1}^\infty k x^{k-1}P_{A^c}^{k-1},\end{equation*}which gives
\begin{equation*}\int_0^x G_1(t)dt =
\sum_{k=1}^\infty k \left(\int_0^x t^{k-1}dt\right) P_{A^c}^{k-1} =
\sum_{k=1}^\infty x^{k} P_{A^c}^{k-1} = xG_{0}(x) = x(I-xP_{A^c})^{-1}.\end{equation*}By taking the derivative with respect to x, we get
\begin{equation*}G_1(x) = (I-xP_{A^c})^{-1} + xP_{A^c}(I-xP_{A^c})^{-2}
= \left(I-xP_{A^c} + xP_{A^c}\right)(I-xP_{A^c})^{-2} = (I-xP_{A^c})^{-2}.\end{equation*} On the other hand, the equality is satisfied since 
$c_{1,1} = 1$.
Suppose that the relation is true for index r − 1, that is, suppose that
\begin{equation*}G_{r-1}(x) = \sum_{\ell=1}^{r-1} c_{r-1,\ell} \left(I - xP_{A^c}\right)^{-(\ell+1)}
\mbox{ where }
c_{r-1,\ell} = (-1)^{r-1}\sum_{j=1}^{\ell} (-1)^j \binom{\ell}{j} j^{r-1}.\end{equation*}We then have
\begin{equation*}\int_0^x G_r(t)dt = \sum_{k=1}^\infty k^{r-1} x^{k} P_{A^c}^{k-1} = xG_{r-1}(x).\end{equation*}By taking the derivative with respect to x, we get
\begin{equation*}G_r(x) = G_{r-1}(x) + xG'_{r-1}(x).\end{equation*}Using the induction hypothesis, we obtain
\begin{equation*}G_r(x) = \sum_{\ell=1}^{r-1} c_{r-1,\ell}\left(I - xP_{A^c}\right)^{-(\ell+1)}
+ x P_{A^c}\sum_{\ell=1}^{r-1} (\ell+1)c_{r-1,\ell}\left(I - xP_{A^c}\right)^{-(\ell+2)}.\end{equation*} Observing that 
$xP_{A^c}(\ell+1)c_{r-1,\ell}
= -(\ell+1)c_{r-1,\ell}(I-xP_{A^c}) + (\ell+1)c_{r-1,\ell}I$, we get
\begin{align*}
G_r(x)
& = \sum_{\ell=1}^{r-1} c_{r-1,\ell}\left(I - xP_{A^c}\right)^{-(\ell+1)} - \sum_{\ell=1}^{r-1} (\ell+1)c_{r-1,\ell}\left(I - xP_{A^c}\right)^{-(\ell+1)}
+ \sum_{\ell=1}^{r-1} (\ell+1)c_{r-1,\ell}\\ &\quad\times\left(I - xP_{A^c}\right)^{-(\ell+2)} \\
& = - \sum_{\ell=1}^{r-1} \ell c_{r-1,\ell}\left(I - xP_{A^c}\right)^{-(\ell+1)}
+ \sum_{\ell=2}^{r} \ell c_{r-1,\ell-1}\left(I - xP_{A^c}\right)^{-(\ell+1)} \\
& = - c_{r-1,1}\left(I - xP_{A^c}\right)^{-2} + \sum_{\ell=2}^{r-1}
\ell (c_{r-1,\ell-1}-c_{r-1,\ell})\left(I - xP_{A^c}\right)^{-(\ell+1)}
+ rc_{r-1,r-1}\left(I - xP_{A^c}\right)^{-(r+1)}.
\end{align*} Note that 
$-c_{r-1,1}=(-1)^{r+1}=c_{r,1}$. Moreover, since it is well-known that 
$c_{r-1,r-1}=(r-1)!$, we have 
$rc_{r-1,r-1}=r!=c_{r,r}$. Finally, for 
$\ell=2,\ldots,r-1$, we obtain
\begin{align*}
\ell (c_{r-1,\ell-1}-c_{r-1,\ell})
& = \ell\left[(-1)^{r-1}\sum_{j=1}^{\ell-1} (-1)^j \binom{\ell-1}{j} j^{r-1}
- (-1)^{r-1}\sum_{j=1}^{\ell} (-1)^j \binom{\ell}{j} j^{r-1}\right] \\
& = (-1)^{r-1}\sum_{j=1}^{\ell-1}
(-1)^j \ell \left[\binom{\ell-1}{j} - \binom{\ell}{j}\right] j^{r-1}
- (-1)^{r-1}(-1)^\ell \ell^{r} \\
& = (-1)^{r-1}\sum_{j=1}^{\ell-1} (-1)^j (-j)\binom{\ell}{j}j^{r-1}
- (-1)^{r-1}(-1)^\ell \ell^{r} \\
& = (-1)^{r}\sum_{j=1}^{\ell-1} (-1)^j \binom{\ell}{j}j^{r} + (-1)^{r}(-1)^\ell \ell^{r} \\
& = c_{r,\ell}.
\end{align*}We thus recover
\begin{equation*}G_{r}(x) = \sum_{\ell=1}^{r} c_{r,\ell}\left(I - xP_{A^c}\right)^{-(\ell+1)},\end{equation*}which completes the proof by induction. Using relation (2), we obtain
\begin{equation*}\mathbb{E}(T_A^r \mid X_0 = i) = (G_r(1)P_{A^c,\,A} \mathbb{1})_i
= \sum_{\ell=1}^{r} c_{r,\ell}\left(\left(I - P_{A^c}\right)^{-\ell}
\left(I - P_{A^c}\right)^{-1}P_{A^c,\,A} \mathbb{1}\right)_i.\end{equation*} Matrix P being stochastic we have 
$\left(I - P_{A^c}\right)^{-1}P_{A^c,\,A} \mathbb{1} = \mathbb{1}$, which completes the proof.
As a byproduct of Theorem 2.1, the reader may note that taking r = 1 in the theorem we get the expected hitting time of subset A, that is
\begin{equation}
\mathbb{E}(T_A \mid X_0 = i) = \left((I-P_{A^c})^{-1}\mathbb{1}\right)_i,
\end{equation}while taking r = 2 we get the second-order moment of TA, that is
\begin{equation}
\mathbb{E}(T_A^2 \mid X_0 = i) = -\left(\left(I - P_{A^c}\right)^{-1}\mathbb{1}\right)_i
+ 2\left(\left(I - P_{A^c}\right)^{-2}\mathbb{1}\right)_i.
\end{equation}In this spirit, the following section is devoted to the analysis of all moments of the hitting time on the lollipop graph.
3. The lollipop graph
Recall that for 
$n,m \in \mathbb{N}$, with 
$2 \leq m \lt n$, the n-vertex lollipop graph, 
$L_m^n$, consists of a clique on m vertices and a path graph on n − m vertices. The m vertices of the clique are numbered 
$1,\ldots,m$ and the n − m vertices of the path are numbered 
$m+1, \ldots,n$, vertex m being the one joining the clique and the path as shown in Figure 1.
We consider a random walk on the vertices of a lollipop graph 
$L_m^n$, that is a Markov chain 
$X = \{X_k, \; k \in \mathbb{N}\}$, for which the transition probability from a state i to each neighbor j of i occurs with equal probability. More precisely, the state space of X is the set 
$S = \{1,\ldots,m-1,m,m+1,\ldots,n\}$ with transition probability matrix P which non-zero entries are given by
\begin{equation*}\left\{\begin{array}{lcl}
P_{i,\,j}={\displaystyle \frac{1}{m-1}} & \mbox{for } & i,j \in \{1,\ldots,m-1\} \mbox{ and } i \neq j,\\\\
P_{i,\,m}={\displaystyle \frac{1}{m-1}} & \mbox{for } & i \in \{1,\ldots,m-1\}, \\\\
P_{m,\,j}={\displaystyle \frac{1}{m}} & \mbox{for } & j \in \{1,\ldots,m-1,m+1\}, \\\\
P_{i,\,i-1}=P_{i,\,i+1}={\displaystyle \frac{1}{2}} & \mbox{for } & i \in \{m+1,\ldots,n-1\}, \\\\
P_{n,\,n-1}={\displaystyle 1}. & &
\end{array}\right.\end{equation*}For example, this matrix is represented below for m = 5 and n = 9
\begin{equation}
P = \left(\begin{array}{cccc|c|ccc|c}
0 & 1/4 & 1/4 & 1/4 & 1/4 & 0 & 0 & 0 & 0 \\
1/4 & 0 & 1/4 & 1/4 & 1/4 & 0 & 0 & 0 & 0 \\
1/4 & 1/4 & 0 & 1/4 & 1/4 & 0 & 0 & 0 & 0 \\
1/4 & 1/4 & 1/4 & 0 & 1/4 & 0 & 0 & 0 & 0 \\
\hline
1/5 & 1/5 & 1/5 & 1/5 & 0 & 1/5 & 0 & 0 & 0 \\
\hline
0 & 0 & 0 & 0 & 1/2 & 0 & 1/2 & 0 & 0 \\
0 & 0 & 0 & 0 & 0 & 1/2 & 0 & 1/2 & 0 \\
0 & 0 & 0 & 0 & 0 & 0 & 1/2 & 0 & 1/2 \\
\hline
0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0
\end{array}\right).
\end{equation} The hitting time on the lollipop graph is the time needed to reach state n when the initial state is state 
$i = 1,\ldots,n-1$. According to the notation of the previous section, it is given by the random variable Tn. As usual, we use the convention that a sum of terms 
$\sum_a^b (...)$ is set to 0 when a > b. Concerning the rth moment of Tn, we will write 
$\mathbb{E}_i(T_{n}^r)$ for 
$\mathbb{E}(T_{n}^r \mid X_0=i)$, 
$\mathbb{V}_i(T_{n})$ for the variance 
$\mathbb{V}(T_{n} \mid X_0=i)$ and for any event E, 
$\mathbb{P}_i(E)$ for 
$\mathbb{P}(E \mid X_0=i)$.
3.1. Mean hitting time
Using relation (3) with 
$A=\{n\}$, we obtain for 
$i = 1,\ldots,n-1$,
\begin{equation*}\mathbb{E}_i(T_{n}) = \left((I - P_{\{1,\ldots,n-1\}})^{-1}\mathbb{1}\right)_i.\end{equation*} Matrix 
$P_{\{1,\ldots,n-1\}}$ is the submatrix obtained from P by deleting row n and column n. As suggested by the example above, we decompose matrix 
$P_{\{1,\ldots,\,n-1\}}$ through the partition 
$\{1,\ldots,m-1\}$, 
$\{m\}$, 
$\{m+1,\ldots,n-1\}$ of the set of states 
$\{1,\ldots,n-1\}$ as follows
\begin{equation}
P_{\{1,\ldots,n-1\}} = \left(\begin{array}{ccc}
Q & {\displaystyle \frac{1}{m-1}}\mathbb{1} & 0 \\\\
{\displaystyle \frac{1}{m}}\mathbb{1}^{\top} & 0 & {\displaystyle \frac{1}{m}}c^\top \\\\
0 & {\displaystyle \frac{1}{2}}c & R
\end{array}\right),
\end{equation} where 
$\top$ denotes the transpose operator, Q is the 
$(m-1,m-1)$ matrix given, for i ≠ j, by 
$Q_{i,j}=1/(m-1)$, c is the unit column vector defined by 
$c=(1,0,\ldots,0)^\top$ with dimension 
$n-m-1$ and R is the 
$(n-m-1,n-m-1)$ matrix, of which non-zero entries are given by 
$R_{i,\,i-1}=R_{i,\,i+1}=1/2$ for 
$i = m+1,\ldots,n-1$.
We then have the following explicit expression of the mean hitting time of state n when starting from state i, with i ≠ n. A part of this result has already been obtained in [Reference Brightwell and Winkler9] when 
$i \in \{1,\ldots,m-1\}$ using another proof. The proof used here is relevant because it is the starting point for the obtention of higher order moments for which we need in addition the mean hitting time of state n when starting from state 
$i \in \{m,\ldots,n-1\}$.
Theorem 3.1. For every 
$n,m \in \mathbb{N}$, with 
$2 \leq m \leq n$, we have
\begin{equation*}\begin{array}{rclcl}
\mathbb{E}_i(T_{n}) & = & (n-m)(m^2-2m+n)+m-1 & \mbox{for } & i = 1,\ldots,m-1 \\\\
\mathbb{E}_i(T_{n}) & = & (n-m)(m^2-2m+n) - (i-m)(m^2-2m+i) & \mbox{for } & i = m,\ldots,n.
\end{array}\end{equation*}Proof. We introduce the column vector 
$V = (v_1, \ldots,v_{n-1})^\top$ defined by 
$V = (I - P_{\{1,\ldots,n-1\}})^{-1}\mathbb{1}$. We decompose vector V following the partition used in (6), by writing
\begin{equation*}V = \left(\begin{array}{c}
V_1 \\
v_m \\
V_2
\end{array}\right), \mbox{ with }
V_1 = \left(\begin{array}{c}
v_{1} \\
\vdots \\
v_{m-1}
\end{array}\right) \mbox{ and }
V_2 = \left(\begin{array}{c}
v_{m+1} \\
\vdots \\
v_{n-1}
\end{array}\right).\end{equation*} We then have 
$\mathbb{E}_i(T_{n})=v_i$ with
\begin{equation*}V = (I - P_{\{1,\ldots,n-1\}})^{-1}\mathbb{1} \Longleftrightarrow
(I - P_{\{1,\ldots,n-1\}}) V = \mathbb{1} \Longleftrightarrow
\left\{\begin{array}{l}
V_1 - Q V_1 - {\displaystyle \frac{v_m}{m-1}}\mathbb{1} = \mathbb{1} \\\\
v_m - {\displaystyle \frac{1}{m}}\mathbb{1}^\top V_1 - {\displaystyle \frac{1}{m}}c^\top V_2 = 1 \\\\
V_2 - {\displaystyle \frac{v_m}{2}}c - RV_2 = \mathbb{1}.
\end{array}\right.\end{equation*} Observing, by symmetry, that 
$\mathbb{E}_i(T_{n})$ has the same value for every 
$i \in \{1,\ldots,m-1\}$, we have 
$v_1=\cdots=v_{m-1}$. We denote this common value by v. We then have
\begin{align*}V_1 & = v \mathbb{1}, \;\;\; QV_1 = v Q\mathbb{1} = \frac{(m-2)v}{m-1}\mathbb{1}, \;\;\; \mathbb{1}^\top V_1 = (m-1)v,\\ \;\;\; & c^\top V_2 = v_{m+1},\end{align*}which leads to
\begin{equation}
\left\{\begin{array}{l}
v_m = v - m+1 \\\\
v_m - {\displaystyle \frac{(m-1)v}{m}} - {\displaystyle \frac{v_{m+1}}{m}} = 1 \\\\
v_i - {\displaystyle \frac{1}{2}}v_{i-1} - {\displaystyle \frac{1}{2}}v_{i+1} = 1, \mbox{ for } i = m+1, \ldots, n-1,
\end{array}\right.
\end{equation} where we have defined 
$v_n=\mathbb{E}_n(T_{n})=0$. Using the first relation of (7) in the second one of (7), we obtain 
$v_{m+1} = v - m^2$. Defining 
$z_i = v_i - v_{i+1}$ for 
$i=m,\ldots,n-1$, the third relation of (7) gives 
$z_i = z_{i-1} + 2$, for 
$i=m+1, \ldots,n-1$ with 
$z_m = v_m - v_{m+1} = v - m+1 - (v - m^2) = m^2-m+1$. It follows that 
$z_{m+i} = z_m + 2i = m^2-m+1 +2i$ and thus we obtain by telescoping
\begin{equation*}v_m = \sum_{i=m}^{n-1} z_i = \sum_{i=0}^{n-m-1} z_{m+i} = (n-m)(m^2-2m+n).\end{equation*} We then get the value of v using 
$v = v_m + m -1$. In the same way, we have
\begin{equation*}v_m - v_{m+i} = \sum_{\ell=m}^{m+i-1} z_\ell = \sum_{\ell=0}^{i-1} z_{m+\ell}
= i(m^2-m+1)+i(i-1) = i(m^2-m+i),\end{equation*} which gives for 
$i=0,\ldots,n-m$,
\begin{equation*}v_{m+i} = v_m - i(m^2-m+i) = (n-m)(m^2-2m+n) - i(m^2-m+i),\end{equation*} that is 
$v_{i} = (n-m)(m^2-2m+n) - (i-m)(m^2-2m+i)$, for 
$i=m,\ldots,n$.
3.2. Variance of the hitting time
To evaluate the variance of the hitting time Tn, we first calculate the second-order moment 
$\mathbb{E}(T_{n}^2)$. Using relation (4) with 
$A=\{n\}$, we obtain for 
$i = 1,\ldots,n-1$,
\begin{equation*}\mathbb{E}_i(T_{n}^2) = - \left((I - P_{\{1,\ldots,n-1\}})^{-1}\mathbb{1}\right)_i +
2\left((I - P_{\{1,\ldots,n-1\}})^{-2}\mathbb{1}\right)_i.\end{equation*}Theorem 3.2. For every 
$n,m \in \mathbb{N}$, with 
$2 \leq m \leq n$, we have, for 
$i = 1,\ldots,m-1$
\begin{equation*}\mathbb{E}_i(T_{n}^2) =
2(n-m)[(m^2-m+1)v - m(m-1)] + (2m-3)v + 4\sum_{\ell=m+1}^{n-1} (n-\ell)v_{\ell}\end{equation*} and for 
$i = m,\ldots,n$
\begin{equation*}\mathbb{E}_i(T_{n}^2) = 2(n-i)[(m^2-m+1)v - m(m-1)] - v_i + 4\left(\sum_{\ell=m+1}^{n-1} (n-\ell)v_{\ell} - \sum_{\ell=m+1}^{i-1} (i-\ell)v_{\ell}\right),\end{equation*} where the values of v and 
$v_\ell$, which are given by Theorem 3.1, are 
$v = (n-m)(m^2-2m+n)+m-1$ and 
$v_\ell = (n-m)(m^2-2m+n) - (\ell-m)(m^2-2m+\ell)$, for 
$\ell = m,\ldots,n-1$.
Proof. The proof is close to the one of Theorem 3.1. We first introduce the column vectors 
$V = (v_1, \ldots,v_{n-1})^\top$ and 
$W = (w_1, \ldots,w_{n-1})^\top$ defined by
\begin{equation*}V = (I - P_{\{1,\ldots,n-1\}})^{-1}\mathbb{1}, \;\; W = (I - P_{\{1,\ldots,n-1\}})^{-2}\mathbb{1}.\end{equation*} Vector W is thus given by 
$W = (I - P_{\{1,\ldots,n-1\}})^{-1}V$, where vector V has been obtained in Theorem 3.1. As we did for vector V, we decompose vector W following the partition used in (6), by writing
\begin{equation*}W = \left(\begin{array}{c}
W_1 \\
w_m \\
W_2
\end{array}\right), \mbox{ with }
W_1 = \left(\begin{array}{c}
w_{1} \\
\vdots \\
w_{m-1}
\end{array}\right) \mbox{ and }
W_2 = \left(\begin{array}{c}
w_{m+1} \\
\vdots \\
w_{n-1}
\end{array}\right).\end{equation*}We first evaluate the vector W which is given by
\begin{equation*}W = (I - P_{\{1,\ldots,n-1\}})^{-1}V \Longleftrightarrow
(I - P_{\{1,\ldots,n-1\}}) W = V \Longleftrightarrow
\left\{\begin{array}{l}
W_1 - Q W_1 - {\displaystyle \frac{w_m}{m-1}}\mathbb{1} = V_1 \\\\
w_m - {\displaystyle \frac{1}{m}}\mathbb{1}^\top W_1 - {\displaystyle \frac{1}{m}}c^\top W_2 = v_m \\\\
W_2 - {\displaystyle \frac{w_m}{2}}c - RW_2 = V_2.
\end{array}\right.\end{equation*} Observing, by symmetry, that 
$\mathbb{E}_i(T_{n}^2)$ has the same value for every 
$i = 1,\ldots,m-1$, we have 
$w_1=\cdots=w_{m-1}$. We denote this common value by w. We then have
\begin{equation*}W_1 = w \mathbb{1}, \;\;\; QW_1 = w Q\mathbb{1} = \frac{(m-2)w}{m-1}\mathbb{1}, \;\;\; \mathbb{1}^\top W_1 = (m-1)w, \;\;\; c^\top W_2 = w_{m+1},\end{equation*}which leads to
\begin{equation}
\left\{\begin{array}{l}
w - {\displaystyle \frac{(m-2)w}{m-1}} - {\displaystyle \frac{w_m}{m-1}} = v \\\\
w_m - {\displaystyle \frac{(m-1)w}{m}} - {\displaystyle \frac{w_{m+1}}{m}} = v_m \\\\
w_i - {\displaystyle \frac{1}{2}}w_{i-1} - {\displaystyle \frac{1}{2}}w_{i+1} = v_i, \mbox{for } i = m+1, \ldots, n-1,
\end{array}\right.
\end{equation} where we have defined 
$w_n=\mathbb{E}_n(T_{n}^2)=0$. The first relation of (8) gives 
$w_m = w - (m-1)v$. Using this result in the second relation of (8) and the fact that 
$v_m = v - (m - 1)$, we obtain 
$w_{m+1} = w - m^2v + m(m-1)$.
Defining 
$z_i = w_i - w_{i+1}$ for 
$i=m,\ldots,n-1$, the third relation of (8) gives 
$z_i = z_{i-1} + 2v_i$, for 
$i=m+1, \ldots,n-1$ with 
$z_m = w_m - w_{m+1} = (m^2-m+1)v - m(m-1)$. It follows that
\begin{equation*}z_{m+i} = z_m + 2\sum_{\ell=1}^i v_{m+\ell}
= (m^2-m+1)v - m(m-1) + 2\sum_{\ell=m+1}^{m+i}v_{\ell},\end{equation*}and thus
\begin{equation*}w_m = \sum_{i=m}^{n-1} z_i = \sum_{i=0}^{n-m-1} z_{m+i} = (n-m)[(m^2-m+1)v - m(m-1)] + 2\sum_{\ell=m+1}^{n-1} (n-\ell)v_{\ell}.\end{equation*} We then get the value of w using 
$w = w_m + (m -1)v$. In the same way, we have
\begin{equation*}w_m - w_{m+i} = \sum_{\ell=m}^{m+i-1} z_\ell = \sum_{\ell=0}^{i-1} z_{m+\ell}
= i[(m^2-m+1)v - m(m-1)] + 2\sum_{\ell=m+1}^{m+i-1} (m+i-\ell)v_{\ell},\end{equation*} which gives for 
$i=1,\ldots,n-m$,
\begin{align*}
w_{m+i}
& = w_m - i[(m^2-m+1)v - m(m-1)] - 2\sum_{\ell=m+1}^{m+i-1} (m+i-\ell)v_{\ell} \\
& = (n-m-i)[(m^2-m+1)v - m(m-1)] + 2\left(\sum_{\ell=m+1}^{n-1} (n-\ell)v_{\ell}
- \sum_{\ell=m+1}^{m+i-1} (m+i-\ell)v_{\ell}\right),
\end{align*} that is, for 
$i=m+1,\ldots,n$,
\begin{equation}
w_{i} = (n-i)[(m^2-m+1)v - m(m-1)] + 2\left(\sum_{\ell=m+1}^{n-1} (n-\ell)v_{\ell}
- \sum_{\ell=m+1}^{i-1} (i-\ell)v_{\ell}\right).
\end{equation} Since 
$\mathbb{E}_i(T_{n}^2) = 2 w_i - v_i$, we obtain for 
$i=1,\ldots,m-1$, 
$\mathbb{E}_i(T_{n}^2) = 2w-v = 2w_m + (2m-3)v$, that is
\begin{equation*}\mathbb{E}_i(T_{n}^2) =
2(n-m)[(m^2-m+1)v - m(m-1)] + (2m-3)v + 4\sum_{\ell=m+1}^{n-1} (n-\ell)v_{\ell}\end{equation*} and, for 
$i=m,\ldots,n-1$,
\begin{equation*}\mathbb{E}_i(T_{n}^2) = 2(n-i)[(m^2-m+1)v - m(m-1)] - v_i + 4\left(\sum_{\ell=m+1}^{n-1} (n-\ell)v_{\ell}
- \sum_{\ell=m+1}^{i-1} (i-\ell)v_{\ell}\right),\end{equation*}which completes the proof.
The variance of Tn, when the initial state is state i, is then given by 
$\mathbb{V}_i(T_{n}) = \mathbb{E}_i(T_{n}^2) - \mathbb{E}_i(T_{n})^2$.
3.3. Moments generating function of the hitting time
Using relation (1), the probability mass function of Tn is given, for every 
$k \geq 1$, by
\begin{equation*}\mathbb{P}_i\{T_{n} = k\} =
\left(P_{\{1,\ldots,n-1\}}^{k-1}P_{\{1,\ldots,n-1\},\{n\}} \mathbb{1}\right)_i.\end{equation*} The column vector 
$B = (b_1,\ldots,b_{n-1})^\top$ defined by 
$B = P_{\{1,\ldots,n-1\},\{n\}} \mathbb{1}$ has all its entries equal to 0, except the last one, that is, its 
$(n-1)$th entry which is equal to 
$1/2$. We then have for every 
$k \geq 1$,
\begin{equation*}\mathbb{P}_i\{T_{n} = k\} =
\left(P_{\{1,\ldots,n-1\}}^{k-1}B\right)_i,\end{equation*} with 
$B = (0,\ldots,0,1/2)^\top$. The matrix 
$P_{\{1,\ldots,n-1\}}$ being an irreducible sub-stochastic matrix, its spectral radius ρn satisfies 
$0 \lt \rho_n \lt 1$. The moment generating function of the hitting time Tn when the initial state is state i is given by the function
\begin{equation*}M_i(t) = \mathbb{E}_i\left(e^{tT_{n}}\right) =
\sum_{k=0}^\infty e^{tk}\mathbb{P}_i\{T_{n}=k\}
= \sum_{k=1}^\infty e^{tk}\left(P_{\{1,\ldots,\,n-1\}}^{k-1}B\right)_i = e^t\left(\sum_{k=0}^\infty \left(e^{t}P_{\{1,\ldots,\,n-1\}}\right)^{k}B\right)_i.\end{equation*} This series converges for all real numbers t such that 
$e^t\rho_n \lt 1$, that is for all t such that 
$t \lt -\ln(\rho_n)$. For these values of t, we obtain
\begin{equation*}M_i(t) = e^t\left(\left(I - e^{t}P_{\{1,\ldots,n-1\}}\right)^{-1}B\right)_i.\end{equation*} An explicit expression of the moment generating function 
$M_i(t)$ is given by the following theorem.
Theorem 3.3. For every 
$n,m \in \mathbb{N}$, with 
$2 \leq m \leq n$ and for all t such that 
$t \lt -\ln(\rho_n)$, we have
\begin{equation}
\begin{array}{rclcl}
M_i(t) & = & \frac{{\displaystyle 1}}{{\displaystyle \sum_{\ell=0}^{n-m+1} a_{n-m,\ell} e^{-\ell t}}} & \mbox{for } & i = 1,\ldots,m-1 \\\\
M_{i}(t) & = & \frac{{\displaystyle \sum_{\ell=0}^{i-m+1} a_{i-m,\ell} e^{-\ell t}}}
{{\displaystyle \sum_{\ell=0}^{n-m+1} a_{n-m,\ell} e^{-\ell t}}} & \mbox{for } & i = m,\ldots,n,
\end{array}
\end{equation} where the 
$a_{i,\ell}$ are given by
\begin{equation*}a_{i,\ell} = \left\{\begin{array}{lcl}
{\displaystyle (-1)^{\frac{i-\ell+2}{2}}(m-2)2^{\ell-1}}
{\displaystyle \left[\binom{\frac{i+\ell-2}{2}}{\frac{i-\ell}{2}}m +
2\binom{\frac{i+\ell-2}{2}}{\frac{i-\ell-2}{2}}\right]}
& \mbox{if } & i-\ell \,\mbox{is even} \\\\
{\displaystyle (-1)^{\frac{i-\ell+1}{2}}(m-1)2^{\ell-2}}
\left[{\displaystyle \binom{\frac{i+\ell-3}{2}}{\frac{i-\ell+1}{2}}m} +
{\displaystyle 2\binom{\frac{i+\ell-3}{2}}{\frac{i-\ell-1}{2}}} +
{\displaystyle 4\binom{\frac{i+\ell-1}{2}}{\frac{i-\ell-1}{2}}}\right]
& \mbox{if } & i-\ell \,\mbox{is odd}
\end{array}\right.\end{equation*} with the convention that 
${\displaystyle \binom{b}{a} = 0}$ if 
$a \notin \{0,\ldots,b-1\}$ and 
${\displaystyle \binom{b}{b} = 1}$ for all b < 0.
Proof. We introduce the column vector 
$M(t) = (M_1(t), \ldots,M_{n-1}(t))^\top$ defined by
\begin{equation*}M(t) = e^t\left(I - e^{t}P_{\{1,\ldots,n-1\}}\right)^{-1}B.\end{equation*}As we did in the proofs of the previous theorems, we decompose vectors M(t) and B following the partition used in (6), by writing
\begin{equation*}M(t) = \left(\begin{array}{c}
M^{(1)}(t) \\
M_m(t) \\
M^{(2)}(t)
\end{array}\right), \mbox{with }
M^{(1)}(t) = \left(\begin{array}{c}
M_{1}(t) \\
\vdots \\
M_{m-1}(t)
\end{array}\right) \mbox{and }
M^{(2)}(t) = \left(\begin{array}{c}
M_{m+1}(t) \\
\vdots \\
M_{n-1}(t)
\end{array}\right).\end{equation*}
\begin{equation*}B = \left(\begin{array}{c}
B^{(1)} \\
0 \\
B^{(2)}
\end{array}\right), \mbox{with }
B^{(1)} = \left(\begin{array}{c}
0 \\
\vdots \\
0
\end{array}\right) \mbox{and }
B^{(2)} = \left(\begin{array}{c}
0 \\
\vdots \\
0 \\
1/2
\end{array}\right).\end{equation*}We then have
\begin{align*}
M(t) = e^t\left(I - e^{t}P_{\{1,\ldots,n-1\}}\right)^{-1}B
& \Longleftrightarrow
(I - e^{t}P_{\{1,\ldots,n-1\}}) M(t) = e^tB \\
& \Longleftrightarrow \left\{\begin{array}{l}
M^{(1)}(t) - e^tQ M^{(1)}(t) - {\displaystyle \frac{e^tM_m(t)}{m-1}}\mathbb{1} = 0 \\\\
M_m(t) - {\displaystyle \frac{e^t}{m}}\mathbb{1}^\top M^{(1)}(t) - {\displaystyle \frac{e^t}{m}}c^\top M^{(2)}(t) = 0 \\\\
M^{(2)}(t) - {\displaystyle \frac{e^tM_m(t)}{2}}c - e^tRM^{(2)}(t) = e^tB_2.
\end{array}\right.
\end{align*} Observing, by symmetry, that 
$\mathbb{E}_i\left(e^{tT_{n}}\right)$ has the same value for every 
$i \in \{1,\ldots,m-1\}$, we have 
$M_1(t)=\cdots=M_{m-1}(t)$. We denote this common value by 
$\mu(t)$. Hence
\begin{align*}M^{(1)}(t) &= \mu(t) \mathbb{1}, \;\; QM^{(1)}(t) = \mu(t) Q\mathbb{1} = \frac{(m-2)\mu(t)}{m-1}\mathbb{1}, \;\; \mathbb{1}^\top M^{(1)}(t) = (m-1)\mu(t), \;\; c^\top M^{(2)}(t)= M_{m+1}(t),\end{align*}which leads to
\begin{equation}
\left\{\begin{array}{l}
\mu(t) - {\displaystyle \frac{(m-2)e^t\mu(t)}{m-1}} - {\displaystyle \frac{e^tM_m(t)}{m-1}} = 0 \\\\
M_m(t) - {\displaystyle \frac{(m-1)e^t\mu(t)}{m}} - {\displaystyle \frac{e^tM_{m+1}(t)}{m}} = 0 \\\\
M_i(t) - {\displaystyle \frac{e^t}{2}}M_{i-1}(t) - {\displaystyle \frac{e^t}{2}}M_{i+1}(t) = 0,
\mbox{for } i = m+1, \ldots, n-1,
\end{array}\right.
\end{equation} where we have used the fact that 
$M_n(t)=\mathbb{E}_n\left(e^{tT_{n}}\right)=1$. The first relation of (11) gives
\begin{equation*}M_m(t) = \left[(m-1)e^{-t} - (m-2)\right]\mu(t).\end{equation*} Using this result in the second relation of (11), we obtain 
$M_{m+1}(t) = me^{-t}M_m(t) - (m-1)\mu(t)$, that is
\begin{equation*}M_{m+1}(t) = \left[m(m-1)e^{-2t} - m(m-2)e^{-t} - (m-1)\right]\mu(t).\end{equation*}The third relation of (11) can be written as
\begin{equation*}M_{i}(t) = 2e^{-t}M_{i-1}(t) - M_{i-2}(t), \mbox{for } i = m+2, \ldots, n.\end{equation*} This second-order recurrence relation together with the expressions of 
$M_m(t)$ and 
$M_{m+1}(t)$ implies that 
$M_{m+i}(t)$ can be written as
\begin{equation}
M_{m+i}(t) = \left[\sum_{\ell=0}^{i+1} a_{i,\ell} e^{-\ell t}\right]\mu(t),
\mbox{ for } i = 0, \ldots, n-m.
\end{equation} We then must have for 
$i = 2, \ldots, n-m$,
\begin{equation*}\sum_{\ell=0}^{i+1} a_{i,\ell} e^{-\ell t}
= \sum_{\ell=0}^{i} 2a_{i-1,\ell} e^{-(\ell+1) t} -\sum_{\ell=0}^{i-1} a_{i-2,\ell} e^{-\ell t}
= \sum_{\ell=1}^{i+1} 2a_{i-1,\ell-1} e^{-\ell t} -\sum_{\ell=0}^{i-1} a_{i-2,\ell} e^{-\ell t}.\end{equation*} It follows that the coefficients 
$a_{i,\ell}$ must satisfy, for 
$i=2,\ldots,n-m$,
\begin{equation*}
\left\{\begin{array}{lcl}
a_{i,0} & = & -a_{i-2,0} \\
a_{i,\ell} & = & 2a_{i-1,\ell-1} - a_{i-2,\ell} \mbox{ for } \ell=1,\ldots i-1 \\
a_{i,i} & = & 2a_{i-1,i-1} \\
a_{i,i+1} & = & 2a_{i-1,i}
\end{array}\right.
\end{equation*} with 
$a_{0,0} = -(m-2)$, 
$a_{0,1} = m-1$, 
$a_{1,0} = -(m-1)$, 
$a_{1,1} = -m(m-2)$, and 
$a_{1,2} = m(m-1)$. These initial values suggest that the coefficients 
$a_{i,\ell}$ can be expressed, for 
$i=2,\ldots,n-m$, as
\begin{equation*}
a_{i,\ell} = \left\{\begin{array}{lcl}
{\displaystyle (-1)^{\frac{i-\ell+2}{2}}(m-2)a'_{i,\ell}} & \mbox{if } & i-\ell \,\mbox{is even} \\\\
{\displaystyle (-1)^{\frac{i-\ell+1}{2}}(m-1)a'_{i,\ell}} & \mbox{if } & i-\ell \,\mbox{is odd},
\end{array}\right.
\end{equation*}where
\begin{equation}
\left\{\begin{array}{lcl}
a'_{i,0} & = & a'_{i-2,0} \\
a'_{i,\ell} & = & 2a'_{i-1,\ell-1} + a_{i-2,\ell} \,\mbox{for } \ell=1,\ldots i-1 \\
a'_{i,i} & = & 2a'_{i-1,i-1} \\
a'_{i,i+1} & = & 2a'_{i-1,i}
\end{array}\right.
\end{equation} with 
$a'_{0,0} = 1$, 
$a'_{0,1} = 1$, 
$a'_{1,0} = 1$, 
$a_{1,1} = m$, and 
$a_{1,2} = m$. It is then easily checked that the following explicit expression of the 
$a'_{i,\ell}$ satisfies relations (13)
\begin{equation*}a'_{i,\ell} = \left\{\begin{array}{lcl}
{\displaystyle 2^{\ell-1}}{\displaystyle \left[\binom{\frac{i+\ell-2}{2}}{\frac{i-\ell}{2}}m +
2\binom{\frac{i+\ell-2}{2}}{\frac{i-\ell-2}{2}}\right]}
& \mbox{if } & i-\ell \,\mbox{is even} \\\\
{\displaystyle 2^{\ell-2}}\left[{\displaystyle \binom{\frac{i+\ell-3}{2}}{\frac{i-\ell+1}{2}}m} +
{\displaystyle 2\binom{\frac{i+\ell-3}{2}}{{\frac{i-\ell-1}{2}} }} +
{\displaystyle 4\binom{\frac{i+\ell-1}{2}}{{\frac{i-\ell-1}{2}} }}\right]
& \mbox{if } & i-\ell \,\mbox{is odd}
\end{array}\right.\end{equation*} with the convention that 
${\displaystyle \binom{b}{a} = 0}$ if 
$a \notin \{0,\ldots,b-1\}$ and 
${\displaystyle \binom{b}{b} = 1}$ for all b < 0.
Since 
$M_n(t)=1$, we get by taking 
$i=n-m$ in relation (12),
\begin{equation*}\mu(t) = \frac{1}{{\displaystyle \sum_{\ell=0}^{n-m+1} a_{n-m,\ell} e^{-\ell t}}}
\mbox{ and }
M_{m+i}(t) = \frac{{\displaystyle \sum_{\ell=0}^{i+1} a_{i,\ell} e^{-\ell t}}}
{{\displaystyle \sum_{\ell=0}^{n-m+1} a_{n-m,\ell} e^{-\ell t}}},
\mbox{for } i = 0, \ldots, n-m\end{equation*}which can be written as
\begin{equation*}M_{i}(t) = \frac{{\displaystyle \sum_{\ell=0}^{i-m+1} a_{i-m,\ell} e^{-\ell t}}}
{{\displaystyle \sum_{\ell=0}^{n-m+1} a_{n-m,\ell} e^{-\ell t}}},
\mbox{for } i = m, \ldots, n.\end{equation*}This completes the proof.
Observe that, since 
$M_i(0)=1$ for every 
$i=1,\ldots,n$, we have
\begin{equation*}\sum_{\ell=0}^{i-m+1} a_{i-m,\ell} = 1, \mbox{for all } i = m,\ldots,n.\end{equation*} The moment generating function 
$M_i(t)$ is defined only for 
$t \lt -\ln(\rho_n)$ and it is well-known from the Perron–Frobenius theorem that ρn is an eigenvalue of matrix 
$P_{\{1,\ldots,n-1\}}$. That is why we study the eigenvalues of matrix 
$P_{\{1,\ldots,n-1\}}$ in the following subsection.
3.4. Eigenvalues
We analyze in this section the eigenvalues of matrix 
$P_{\{1,\ldots,n-1\}}$. For fixed 
$2 \leq m \leq n$ and 
$k = 1,\ldots,n-1$, we consider submatrix 
$P_{\{1,\ldots,k\}}$ of matrix P which consists in removing the 
$n-1-k$ last lines and columns of matrix 
$P_{\{1,\ldots,n-1\}}$. We denote by 
$R_{k}(x)$ the characteristic polynomial of matrix 
$P_{\{1,\ldots,k\}}$ which is the determinant of matrix 
$xI - P_{\{1,\ldots,k\}}$, that is
\begin{equation*}R_{k}(x) = \left|xI - P_{\{1,\ldots,k\}}\right|.\end{equation*} The following theorem gives the characteristic polynomial 
$R_{k}(x)$ of matrix 
$P_{\{1,\ldots,k\}}$, for 
$k=1,\ldots,n-1$.
Theorem 3.4. We have
\begin{align*}
R_{k}(x)
& = \frac{(m-1)x - (k-1)}{m-1}\left(x + \frac{1}{m-1}\right)^{k-1}, \mbox{ for }
k = 1,\ldots,m-1 \\
R_{m}(x)
& = \frac{m(m-1)x^2 - m(m-2)x - (m-1)}{m(m-1)}\left(x + \frac{1}{m-1}\right)^{m-2},
\\
R_{m+1}(x)
& = xR_{m}(x) - \frac{1}{2m}R_{m-1}(x) \\
R_{k}(x)
& = xR_{k-1}(x) - \frac{1}{4}R_{k-2}(x), \mbox{ for } k=m+2,\ldots,n-1.
\end{align*} For 
$k=1,\ldots,n-m+1$, we have
\begin{equation*}R_{m+k-2}(x) = S_k(x)\left(x + \frac{1}{m-1}\right)^{m-2},\end{equation*} where the k-th order polynomial 
$S_k(x)$ has k distinct real roots which interlace with the k − 1 distinct real roots of 
$S_{k-1}(x)$, for 
$k \geq 2$.
Matrix 
$P_{\{1,\ldots,\,n-1\}}$ has n − 1 real eigenvalues : 
$-1/(m-1)$ with multiplicity m − 2 and 
$n-m+1$ other distinct real eigenvalues with multiplicity 1. All its eigenvalues belong to the interval 
$(-1,1)$ with the greatest one which is its spectral radius ρn belongs to the interval 
$[1/2,1)$.
Proof. For every 
$k = 1,\ldots,m-1$ and 
$x \in \mathbb{R}$, the matrix 
$xI - P_{\{1,\ldots,k\}}$ is given by 
$(xI - P_{\{1,\ldots,k\}})_{i,\,i}=x$ and 
$(xI - P_{\{1,\ldots,k\}})_{i,j}=-1/(m-1)$, for i ≠ j. We also consider the k × k matrix 
$M(k,x)$ defined by 
$M_{1,1}(k,x)=-1/(m-1)$, 
$M_{i,i}(k,x)=x$, for 
$i \geq 2$ and 
$M_{i,j}(k,x)=-1/(m-1)$, for i ≠ j.
The determinants 
$R_{k}(x)$ and 
$D_k(x)$ of matrices 
$xI - P_{\{1,\ldots,k\}}$ and 
$M(k,x)$, respectively, can then be written recursively, for 
$k \geq 2$ as
\begin{align*}
R_k(x)
& = x R_{k-1}(x) + \frac{k-1}{m-1} D_{k-1}(x) \\
D_k(x)
& = \frac{-1}{m-1} R_{k-1}(x) + \frac{k-1}{m-1} D_{k-1}(x).
\end{align*} Indeed, we performed the expansion of the determinants 
$R_{k}(x)$ and 
$D_k(x)$ using the first row. The first term is obtained from the expansion of the first column and the second term is obtained from the k − 1 identical terms derived from the expansion of the second to the 
$(k-1)$th column. The initial values of these relations are
\begin{equation*}R_{2}(x) = \left(x - \frac{1}{m-1}\right)\left(x + \frac{1}{m-1}\right)
\mbox{and }
D_2(x) = -\frac{1}{m-1}\left(x + \frac{1}{m-1}\right).\end{equation*} It is easily checked by induction that the solution to these equations is given, for 
$k = 1,\ldots,m-1$, by
\begin{equation*}R_{k}(x) = \left(x - \frac{k-1}{m-1}\right)\left(x + \frac{1}{m-1}\right)^{k-1}
\mbox{and }
D_k(x) = -\frac{1}{m-1}\left(x + \frac{1}{m-1}\right)^{k-1}.\end{equation*} We consider now the characteristic polynomial of matrix 
$P_{\{1,\ldots,m\}}$. The matrix 
$xI - P_{\{1,\ldots,m\}}$ is given by 
$(xI - P_{\{1,\ldots,m\}})_{i,i}=x$, 
$(xI - P_{\{1,\ldots,m\}})_{i,j}=-1/(m-1)$, for 
$i =1,\ldots,m-1$ with i ≠ j and 
$(xI - P_{\{1,\ldots,m\}})_{m,j}=-1/m$, for j ≠ m. The characteristic polynomial 
$R_m(x)$ of matrix 
$P_{\{1,\ldots,m\}}$ is then given by
\begin{equation*}R_m(x) = x R_{m-1}(x) + \frac{m-1}{m} D_{m-1}(x).\end{equation*} Here we performed the expansion of the determinants using the last row. The first term is obtained from the expansion of the last column and the second term is obtained from the m − 1 identical terms derived from the expansion of the 
$(m-1)$th column to the second column. Replacing 
$R_{m-1}(x)$ and 
$D_{m-1}(x)$ by their values obtained above, we get
\begin{equation*}R_m(x) = \frac{m(m-1)x^2 - m(m-2)x - (m-1)}{m(m-1)}\left(x + \frac{1}{m-1}\right)^{m-2}.\end{equation*} Concerning the characteristic polynomials 
$R_{m+1}(x),\ldots,R_{n-1}(x)$, we observe that (see (6) and the example (5)) the submatrix obtained from matrix 
$P_{\{1,\ldots,\,n-1\}}$ by deleting its m − 1 first rows and columns is a tridiagonal matrix. We thus easily get
\begin{equation*}R_{m+1}(x) = xR_{m}(x) - \frac{1}{2m}R_{m-1}(x)\end{equation*} and, for 
$k=m+2,\ldots,n-1$,
\begin{equation*}R_{k}(x) = xR_{k-1}(x) - \frac{1}{4}R_{k-2}(x).\end{equation*} For 
$k=1,\ldots,n-m+1$, we introduce the notation
\begin{equation*}S_k(x) = \frac{R_{m+k-2}(x)}{{\displaystyle \left(x + \frac{1}{m-1}\right)^{m-2}}}.\end{equation*}We then have
\begin{equation*}S_{1}(x) = \frac{(m-1)x - (m-2)}{m-1},\end{equation*}
\begin{equation*}S_{2}(x) = \frac{m(m-1)x^2 - m(m-2)x - (m-1)}{m(m-1)},\end{equation*}
\begin{equation}
S_{3}(x) = xS_{2}(x) - \frac{1}{2m}S_{1}(x)
\end{equation} and, for 
$k=4,\ldots,n-m+1$,
\begin{equation}
S_{k}(x) = xS_{k-1}(x) - \frac{1}{4}S_{k-2}(x).
\end{equation} For 
$k=1,\ldots,n-m+1$, Sk is a k-th order polynomial which satisfies
\begin{equation}
\lim_{x \longrightarrow -\infty} S_{k}(x) = \left\{\begin{array}{lcl}
-\infty & \mbox{if } & k\, \mbox{is odd} \\
+\infty & \mbox{if } & k\, \mbox{is even}
\end{array}\right.
\mbox{and }
\lim_{x \longrightarrow \infty} S_{k}(x) = +\infty.
\end{equation} We denote by 
$\gamma^{(k)}_1,\ldots,\gamma^{(k)}_{k}$ the k roots of Sk. For k = 1, we easily have
\begin{equation*}\gamma^{(1)}_1 = \frac{m-2}{m-1}.\end{equation*}For k = 2, S 2 is a second-order polynomial with 2 real roots which are
\begin{equation*}\gamma^{(2)}_1 = \frac{m(m-2) - \sqrt{m(m^3-4m+4)}}{2m(m-1)}
\mbox{and }
\gamma^{(2)}_2 = \frac{m(m-2) + \sqrt{m(m^3-4m+4)}}{2m(m-1)}.\end{equation*}It is easily checked that we have
\begin{equation*}\gamma^{(2)}_1 \lt 0 \leq \gamma^{(1)}_1 \lt \gamma^{(2)}_2.\end{equation*} Consider now relations (14) and (15), and fix an integer 
$k=3,\ldots,n-m+1$. Let γ be a root of 
$S_{k-1}$. We then have 
$S_{k-1}(\gamma)=0$ which gives by both (14) and (15), 
$S_{k}(\gamma) S_{k-2}(\gamma) \leq 0$. Suppose that 
$S_{k}(\gamma)=0$. Since 
$S_{k-1}(\gamma)=0$, 
$S_{k}(\gamma)=0$ is equivalent to 
$S_{k-2}(\gamma) = 0$. Writing (15) for integer k − 1, this implies that 
$S_{k-3}(\gamma) = 0$, which in turn implies recursively that 
$S_{k-4}(\gamma) = 0, \ldots, S_2(\gamma) = 0$. Using now (14), this implies that 
$S_{1}(\gamma) = 0$, which gives 
$\gamma = \gamma^{(1)}_1$, which is wrong because the 2 roots 
$\gamma^{(2)}_1$ and 
$\gamma^{(2)}_2$ of S 2 are different from 
$\gamma^{(1)}_1$.
It follows that if γ is a root of 
$S_{k-1}$ then 
$S_{k}(\gamma) S_{k-2}(\gamma) \lt 0$, for all 
$k=3,\ldots,n-m+1$.
We have seen above that the roots of S 1 and S 2 are real and interlace, that is, that the root of S 1 is in between the two real roots of S 2. Suppose that the roots of 
$S_{k-1}$ and Sk are all real and interlace, that is, that there is always a root of 
$S_{k-1}$ in between two successive roots of Sk. We thus have
\begin{equation*}\gamma^{(k)}_1 \lt \gamma^{(k-1)}_1 \lt \gamma^{(k)}_2 \lt \cdots \lt \gamma^{(k)}_{k-1} \lt \gamma^{(k-1)}_{k-1} \lt \gamma^{(k)}_{k}.\end{equation*} It follows, using (16), that for 
$\ell = 1,\ldots,k$,
\begin{equation*}S_{k-1}(\gamma^{(k)}_{\ell}) \gt 0 \,\mbox{if } k+\ell \,\mbox{is even and }
S_{k-1}(\gamma^{(k)}_{\ell}) \lt 0 \,\mbox{if } k+\ell \,\mbox{is odd}.\end{equation*} Since 
$S_{k+1}(\gamma^{(k)}_{\ell}) S_{k-1}(\gamma^{(k)}_{\ell}) \lt 0$, we deduce that for 
$\ell = 1,\ldots,k$,
\begin{equation*}S_{k+1}(\gamma^{(k)}_{\ell}) \lt 0 \,\mbox{if } k+\ell \,\mbox{is even and }
S_{k+1}(\gamma^{(k)}_{\ell}) \gt 0 \,\mbox{if } k+\ell \,\mbox{is odd}.\end{equation*} This implies that in each open interval 
$(\gamma^{(k)}_{\ell-1},\gamma^{(k)}_{\ell})$ there is a root of 
$S_{k+1}$. Moreover, since 
$S_{k+1}(\gamma^{(k)}_{k}) \lt 0$ and 
$\lim_{x \longrightarrow \infty} S_{k}(x) = +\infty$, we have 
$\gamma^{(k+1)}_{k+1} \gt \gamma^{(k)}_{k}$. In the same way, if k is even then since 
$S_{k+1}(\gamma^{(k)}_{1}) \gt 0$ and 
$\lim_{x \longrightarrow -\infty} S_{k+1}(x) = -\infty$, we have 
$\gamma^{(k+1)}_{1} \lt \gamma^{(k)}_{1}$. If k is odd then since 
$S_{k+1}(\gamma^{(k)}_{1}) \lt 0$ and 
$\lim_{x \longrightarrow -\infty} S_{k+1}(x) = \infty$, we also have 
$\gamma^{(k+1)}_{1} \lt \gamma^{(k)}_{1}$. The rest of the proof is due to the Perron–Frobenius Theorem.
This result allows us to compute recursively by dichotomy the spectral radius ρn of matrix 
$P_{\{1,\ldots,\,n-1\}}$ for any precision ɛ given in advance. From Theorem 3.4, for 
$k \geq m$, ρk is the greatest root of polynomial 
$S_{k-m+1}$ defined in the proof of this theorem. The recursion then begins with
\begin{equation*}\rho_m = \frac{m-2}{m-1} \mbox{and }
\rho_{m+1} = \frac{m(m-2) + \sqrt{m(m^3-4m+4)}}{2m(m-1)}.\end{equation*} From Theorem 3.4, we have 
$\rho_k \in (\rho_{k-1},1)$ and thus the algorithm starts with 
$a := \rho_{k-1}$ and 
$b:= 1$. We then compute 
$S_{k-m+1}((a+b)/2)$. If 
$S_{k-m+1}((a+b)/2) \gt 0$ then we set 
$b := (a+b)/2$ and if 
$S_{k-m+1}((a+b)/2) \lt 0$ then we set 
$a := (a+b)/2$. We repeat these instructions until 
$\left| S_{k-m+1}((a+b)/2)\right| \leq \varepsilon$, which finally leads to 
$\rho_k := (a+b)/2$ with a precision equal to ɛ. The algorithm shown in Table 1 computes the spectral radius ρk of matrix 
$P_{\{1,\ldots,k-1\}}$ obtained by removing the n − k last rows and columns of matrix 
$P_{\{1,\ldots,\,n-1\}}$, for 
$k=2,\ldots,n$.
Table 1. Algorithm computing the spectral radius ρk of matrix 
$P_{\{1,\ldots,k-1\}}$ obtained by removing the n − k last rows and columns of matrix 
$P_{\{1,\ldots,n-1\}}$, for 
$k=2,\ldots,n$ and a fixed precision ɛ.
We give below some numerical values of ρk for m = 14, n = 23 and 
$\varepsilon=10^{-10}$. We get 
$\rho_k = (k-2)/13$ for 
$k=2,\ldots,13$ and
\begin{equation*}\rho_{14} = 0.923076923,
\rho_{15} = 0.994873556,
\rho_{16} = 0.997361532,
\rho_{17} = 0.998227892,
\rho_{18} = 0.998668330,\end{equation*}
\begin{equation*}\rho_{19} = 0.998934946,
\rho_{20} = 0.999113674,
\rho_{21} = 0.999241811,
\rho_{22} = 0.999338174,
\rho_{23} = 0.999413270.\end{equation*}4. Asymptotic analysis
We analyze in this section the asymptotic behavior of both the moments and the distribution of the hitting time Tn when n tends to infinity.
4.1. Maximum expected hitting time and corresponding moments
From Theorem 3.1, we easily see, as expected, that the maximum expected hitting time on the lollipop graph is obtained when the initial state is any state 
$i \in \{1,\ldots,m-1\}$. The value of m which gives the maximum expected hitting time together with the corresponding maximum was obtained in [Reference Brightwell and Winkler9]. We recall this result in the following corollary of Theorem 3.1 and we give, for this maximal value of m, the expected hitting time on the lollipop graph for the values i = m and 
$i=n-1$ of the initial state.
Corollary 4.1. For every 
$n \geq 3$ and 
$i \in \{1,\ldots,m-1\}$, we have
\begin{equation*}\max_{m=2,\ldots,n-1} \mathbb{E}_i(T_{n}) = \left\{
\begin{array}{lcl}
{\displaystyle \frac{4}{27}n^3 - \frac{1}{9}n^2 + \frac{2}{3}n - 1} & \mbox{if } & n = 0 \mod{3} \\\\
{\displaystyle \frac{4}{27}n^3 - \frac{1}{9}n^2 + \frac{4}{9}n - \frac{13}{27}} & \mbox{if } & n = 1 \mod{3} \\\\
{\displaystyle \frac{4}{27}n^3 - \frac{1}{9}n^2 + \frac{2}{3}n - \frac{29}{27}} & \mbox{if } & n = 2 \mod{3},
\end{array}\right.\end{equation*} the maximum being reached for 
$m = \lceil 2(n-1)/3 \rceil$. For this value of m, we have
\begin{equation*}\mathbb{E}_m(T_{n}) \mathop{\sim}_{n\longrightarrow \infty} \frac{4n^3}{27} \mbox{ and }
\mathbb{E}_{n-1}(T_{n}) \mathop{\sim}_{n\longrightarrow \infty} \frac{4n^2}{9}.\end{equation*}Proof. From Theorem 3.1, we have to find, for a fixed 
$n \geq 3$, the maximum of the sequence u(m) defined by
\begin{equation*}u(m) = (n-m)(m^2-2m+n)+m-1 = -m^3 + (n + 2)m^2 - (3n -1)m + n^2 -1.\end{equation*} The difference 
$u(m+1)-u(m)$ gives
\begin{align*}
u(m+1)-u(m)
& = -(m+1)^3 + m^3 + (n + 2)((m+1)^2-m^2) - (3n -1)(m+1-m) \\
& = -3m^2 + (2n+1)m -2(n-1) \\
& = -3(m-1)\left(m-\frac{2(n-1)}{3}\right).
\end{align*} It is thus easily checked that u(m) is maximal when 
$m= \lceil 2(n-1)/3 \rceil$. The maximal mean hitting time is then given by 
$u(\lceil 2(n-1)/3 \rceil)$.
\begin{equation*}\begin{array}{llcl}
\mbox{If } n = 0 \mod{3} \mbox{ then } {\displaystyle \left\lceil \frac{2(n-1)}{3} \right\rceil} & = &
{\displaystyle \frac{2n}{3}} &
\mbox{and } {\displaystyle u\left(\frac{2n}{3}\right) = \frac{4}{27}n^3 - \frac{1}{9}n^2 + \frac{2}{3}n - 1} \\\\
\mbox{If } n = 1 \mod{3} \mbox{ then } {\displaystyle \left\lceil \frac{2(n-1)}{3} \right\rceil} & = & {\displaystyle \frac{2(n-1)}{3}} &
\mbox{and } {\displaystyle u\left(\frac{2(n-1)}{3}\right) = \frac{4}{27}n^3 - \frac{1}{9}n^2 + \frac{4}{9}n - \frac{13}{27}} \\\\
\mbox{If } n = 2 \mod{3} \mbox{ then } {\displaystyle \left\lceil \frac{2(n-1)}{3} \right\rceil} & = &
{\displaystyle \frac{2n-1}{3}} &
\mbox{and } {\displaystyle u\left(\frac{2n-1}{3}\right) = \frac{4}{27}n^3 - \frac{1}{9}n^2 + \frac{2}{3}n - \frac{29}{27}}.
\end{array}\end{equation*} When the initial state 
$i \in \{m,\ldots,n-1\}$ its enough to use Theorem 3.1.
The following corollary of Theorem 3.2 gives the second-order moment of Tn when 
$m= \lceil 2(n-1)/3 \rceil$ and the initial state is any state 
$i \in \{1,\ldots,m-1\}$, that is when the mean hitting time is maximal.
Corollary 4.2. For every 
$n \geq 3$ with 
$m= \lceil 2(n-1)/3 \rceil$ and 
$i=1,\ldots,m-1$, we have
\begin{equation*}\mathbb{E}_i(T_{n}^2) = \left\{
\begin{array}{lcl}
{\displaystyle \frac{32}{729}n^6 - \frac{56}{729}n^5 + \frac{35}{81}n^4 - \frac{112}{81}n^3 +
\frac{22}{9}n^2 - 4n + 3} & \mbox{if } & n = 0 \mod{3} \\\\
{\displaystyle \frac{32}{729}n^6 - \frac{56}{729}n^5 + \frac{187}{729}n^4 - \frac{76}{81}n^3 +
\frac{1010}{729}n^2 + \frac{182}{729}n - \frac{671}{729}} & \mbox{if } & n = 1 \mod{3} \\\\
{\displaystyle \frac{32}{729}n^6 - \frac{56}{729}n^5 + \frac{299}{729}n^4 - \frac{1024}{729}n^3 +
\frac{1654}{729}n^2 - \frac{2320}{729}n + \frac{635}{243}} & \mbox{if } & n = 2 \mod{3}.
\end{array}\right.\end{equation*}Proof. It consists simply in setting 
$m = \lceil 2(n-1)/3 \rceil$ in Theorem 3.2, that is by taking successively 
$m=2n/3$, 
$m=2(n-1)/3$ and 
$m= (2n-1)/3$.
In order to deal with the other moments, we first give a recursion to compute them for all the values of m and n with 
$2 \leq m \leq n$. This recursion follows the same lines that have been used in the proofs of Theorems 3.1 and 3.2.
While Theorem 2.1 gives a vector expression of the moments of Tn, Theorem 4.3 gives, using this vector expression, a recurrence relation to compute the coordinates of this vector which are given in relations (17) and (18). These two relations are quite important because they are used in Theorems 4.4 and 4.6 to get the asymptotic behavior of the moments of Tn.
From Theorem 2.1, to get the rth moment of Tn, we need to evaluate the coordinates of the column vector 
$\left(I - P_{\{1,\ldots,n-1\}}\right)^{-r}\mathbb{1}$. We introduce the notation
\begin{equation*}v^{(r)}_i = \left(\left(I - P_{\{1,\ldots,n-1\}}\right)^{-r}\mathbb{1}\right)_i.\end{equation*}Theorem 4.3. For every 
$n,m \in \mathbb{N}$, with 
$2 \leq m \leq n$ and for every 
$r \geq 1$, we have
for 
$i=1,\ldots,m-1$, the 
$v^{(r)}_i$ are all equal to 
$v^{(r)}$ which are recursively given by
\begin{equation}
v^{(r)} = (n-m)\left[(m^2-m+1)v^{(r-1)} - m(m-1)v^{(r-2)}\right] + (m-1)v^{(r-1)}
+ 2\sum_{\ell=m+1}^{n-1} (n-\ell)v^{(r-1)}_{\ell}
\end{equation} and for 
$i=m,\ldots,n$,
\begin{align}
v^{(r)}_{i} = (n-i)\left[(m^2-m+1)v^{(r-1)} - m(m-1)v^{(r-2)}\right]
+ 2\left(\sum_{\ell=m+1}^{n-1} (n-\ell)v^{(r-1)}_{\ell}
- \sum_{\ell=m+1}^{i-1} (i-\ell)v^{(r-1)}_{\ell}\right)\nonumber\\
\end{align} with initial conditions 
$v^{(-1)} = 0$ and 
$v^{(0)}_i = 1$, for all 
$i=1,\ldots,n$.
Proof. Let 
$V^{(r)}$ the column vector defined, for 
$r \geq 0$, by
\begin{equation*}V^{(r)} = \left(I- P_{\{1,\ldots,\,n-1\}}\right)^{-r}\mathbb{1}.\end{equation*} We then have 
$V^{(0)} = 1$ and for all 
$r \geq 1$,
\begin{equation*}V^{(r)} = (I - P_{\{1,\ldots,n-1\}})^{-1}V^{(r-1)} \Longleftrightarrow
(I - P_{\{1,\ldots,n-1\}}) V^{(r)} = V^{(r-1)}.\end{equation*} As we did in the proof of Theorems 3.1 and 3.2, we decompose vector 
$V^{(r)}$ following the partition used in (6), by writing
\begin{equation*}V^{(r)} = \left(\begin{array}{c}
V^{(r)}_1 \\
v^{(r)}_m \\
V^{(r)}_2
\end{array}\right), \mbox{ with }
V^{(r)}_1 = \left(\begin{array}{c}
v^{(r)}_{1} \\
\vdots \\
v^{(r)}_{m-1}
\end{array}\right) \mbox{ and }
V^{(r)}_2 = \left(\begin{array}{c}
v^{(r)}_{m+1} \\
\vdots \\
v^{(r)}_{n-1}
\end{array}\right).\end{equation*} For 
$r \geq 1$, we have
\begin{equation*}(I - P_{\{1,\ldots,n-1\}}) V^{(r)} = V^{(r-1)} \Longleftrightarrow
\left\{\begin{array}{l}
V^{(r)}_1 - Q V^{(r)}_1 - {\displaystyle \frac{v^{(r)}_m}{m-1}}\mathbb{1} = V^{(r-1)}_1 \\\\
v^{(r)}_m - {\displaystyle \frac{1}{m}}\mathbb{1}^\top V^{(r)}_1 - {\displaystyle \frac{1}{m}}c^\top V^{(r)}_2
= v^{(r-1)}_m \\\\
V^{(r)}_2 - {\displaystyle \frac{v^{(r)}_m}{2}}c - RV^{(r)}_2 = V^{(r-1)}_2.
\end{array}\right.\end{equation*} Observing, by symmetry, that 
$\mathbb{E}_i(T_{n}^r)$ has the same value for every 
$i \in \{1,\ldots,m-1\}$, we have 
$v^{(r)}_1=\cdots=v^{(r)}_{m-1}$. We denote this common value by 
$v^{(r)}$. We then have
\begin{equation*}V^{(r)}_1 = v^{(r)} \mathbb{1}, \;\;\;
QV^{(r)}_1 = v^{(r)} Q\mathbb{1} = \frac{(m-2)v^{(r)}}{m-1}\mathbb{1}, \;\;\;
\mathbb{1}^\top V^{(r)}_1 = (m-1)v^{(r)}, \;\;\;
c^\top V^{(r)}_2 = v^{(r)}_{m+1},\end{equation*}which leads to
\begin{equation}
\left\{\begin{array}{l}
v^{(r)} - {\displaystyle \frac{(m-2)v^{(r)}}{m-1}} - {\displaystyle \frac{v^{(r)}_m}{m-1}} = v^{(r-1)}
\\\\
v^{(r)}_m - {\displaystyle \frac{(m-1)v^{(r)}}{m}} - {\displaystyle \frac{v^{(r)}_{m+1}}{m}} = v^{(r-1)}_m \\\\
v^{(r)}_i - {\displaystyle \frac{1}{2}}v^{(r)}_{i-1} - {\displaystyle \frac{1}{2}}v^{(r)}_{i+1} = v^{(r-1)}_i,
\mbox{ for } i = m+1, \ldots, n-1,
\end{array}\right.
\end{equation} where we have defined 
$v^{(r)}_n=\mathbb{E}_n(T_{n}^r)=0$. The first relation of (19) gives
\begin{equation}
v^{(r)}_m = v^{(r)} - (m-1)v^{(r-1)}.
\end{equation}Using this result in the second relation of (19), we obtain
\begin{equation*}v^{(r)}_{m+1} = v^{(r)} - m^2v^{(r-1)} + m(v^{(r-1)} - v^{(r-1)}_m).\end{equation*} From (20), we have 
$v^{(r-1)} - v^{(r-1)}_m = (m-1)v^{(r-2)}$, which is equal to 0 for r = 1, since have 
$V^{(0)}=1$. We thus introduce the notation 
$V^{(-1)} = 0$ which is the null vector. It follows that for all 
$r \geq 1$, we have
\begin{equation*}v^{(r)}_{m+1} = v^{(r)} - m^2v^{(r-1)} + m(m-1)v^{(r-2)}.\end{equation*} Defining 
$z_i = v^{(r)}_{i} - v^{(r)}_{i+1}$ for 
$i=m,\ldots,n-1$, the third relation of (19) gives 
$z_i = z_{i-1} + 2v^{(r-1)}_{i}$, for 
$i=m+1, \ldots,n-1$ with 
$z_m = v^{(r)}_{m} - v^{(r)}_{m+1} = (m^2-m+1)v^{(r-1)} - m(m-1)v^{(r-2)}$. It follows that
\begin{equation*}z_{m+i} = z_m + 2\sum_{\ell=1}^i v^{(r-1)}_{m+\ell}
= (m^2-m+1)v^{(r-1)} - m(m-1)v^{(r-2)} + 2\sum_{\ell=m+1}^{m+i} v^{(r-1)}_{\ell},\end{equation*}and thus
\begin{equation*}v^{(r)}_m = \sum_{i=m}^{n-1} z_i = \sum_{i=0}^{n-m-1} z_{m+i} =
(n-m)\left[(m^2-m+1)v^{(r-1)} - m(m-1)v^{(r-2)}\right]
+ 2\sum_{\ell=m+1}^{n-1} (n-\ell)v^{(r-1)}_{\ell}.\end{equation*} We then get the value of 
$v^{(r)}$ using 
$v^{(r)} = v^{(r)}_m + (m-1)v^{(r-1)}$. In the same way, we have
\begin{align*}
v^{(r)}_m - v^{(r)}_{m+i}
& = \sum_{\ell=m}^{m+i-1} z_\ell = \sum_{\ell=0}^{i-1} z_{m+\ell}
= i\left[(m^2-m+1)v^{(r-1)} - m(m-1)v^{(r-2)}\right]
+ 2\sum_{\ell=0}^{i-1} \sum_{j=m+1}^{m+\ell} v^{(r-1)}_{j} \\
& = i\left[(m^2-m+1)v^{(r-1)} - m(m-1)v^{(r-2)}\right]
+ 2 \sum_{\ell=m+1}^{m+i-1} (m+i-\ell)v^{(r-1)}_{\ell}
\end{align*} which gives for 
$i=1,\ldots,n-m$,
\begin{align*}
v^{(r)}_{m+i}
& = v^{(r)}_m - i\left[(m^2-m+1)v^{(r-1)} - m(m-1)v^{(r-2)}\right]
- 2\sum_{\ell=m+1}^{m+i-1} (m+i-\ell)v^{(r-1)}_{\ell} \\
& = (n-m-i)\left[(m^2-m+1)v^{(r-1)} - m(m-1)v^{(r-2)}\right] \\
& \;\;\; + 2\left(\sum_{\ell=m+1}^{n-1} (n-\ell)v^{(r-1)}_{\ell}
- \sum_{\ell=m+1}^{m+i-1} (m+i-\ell)v^{(r-1)}_{\ell}\right),
\end{align*} which can also be written, for 
$i=m+1,\ldots,n$, as
\begin{equation*}v^{(r)}_{i} = (n-i)\left[(m^2-m+1)v^{(r-1)} - m(m-1)v^{(r-2)}\right]
+ 2\left(\sum_{\ell=m+1}^{n-1} (n-\ell)v^{(r-1)}_{\ell}
- \sum_{\ell=m+1}^{i-1} (i-\ell)v^{(r-1)}_{\ell}\right).\end{equation*}This concludes the proof.
4.2. Asymptotic moments and distribution
Armed with Theorem 4.3, we are now in position to tackle the asymptotic analysis of the various moments 
$\mathbb{E}_i(T_n^r)$ for 
$r \geq 1$ as 
$n\longrightarrow \infty$. Naturally, the result strongly depends on the asymptotic behavior of the parameter m along with n. We consider in this section the two following cases. The first case is the most natural one, namely the case when
\begin{equation*}
\frac{m}{n} \mathop{\longrightarrow}_{n\to\infty} a \text{ for some } 0 \lt a \lt 1.
\end{equation*} Theorem 4.4 and Corollary 4.5 give a complete description of the asymptotic behavior of both all moments and distribution of Tn in this case. The second case is the complementary situation when 
$m\ll n$. More technically, we investigate the case when
\begin{equation*}\frac{m}{\sqrt{n}} \mathop{\longrightarrow}_{n\to\infty} 0.\end{equation*}Theorem 4.6 and Corollary 4.7 give a complete description of the asymptotic behavior of both all moments and distribution of Tn in this case.
Let us begin with the situation when 
$m/n \longrightarrow a$. The following theorem gives the asymptotic behavior of the rth moment of Tn in this situation.
Theorem 4.4. Assume there is an 
$a \in (0,1)$ such that for every 
$n \geq 3$, we have
\begin{equation*}m = an\left(1 + \varepsilon(n)\right),
\text{ for some sequence } \varepsilon(n) \text{ satisfying }
\varepsilon(n) \mathop{\longrightarrow}_{n \to \infty} 0.\end{equation*} Then, for every given value of 
$r \geq 1$, there exists a r-dependent sequence 
$\varepsilon_{r}(n)$ such that for any 
$n\geq 3$ and any 
$i = 1,\ldots,m-1$, we have
\begin{equation*}\mathbb{E}_i(T_n^r) = r! a^{2r}(1-a)^rn^{3r}\left(1 + \varepsilon_{r}(n)\right),\end{equation*} where the sequence 
$\varepsilon_{r}(n)$ goes to zero as 
$n \longrightarrow \infty$. Note that as in Theorem 4.3 both sides of this relation are independent of i.
Proof. The proof is based on the asymptotic analysis of the 
$v^{(r)}$ and 
$v^{(r)}_{i}$ obtained in Theorem 4.3.
From now on, and in order to keep reasonably light notation, any function denoted either by 
$\varepsilon_r(n)$, or 
$\varepsilon_r(n,i)$ for 
$i=1,\ldots,m-1$, or 
$\varepsilon_r(n,i)$ for 
$i=m,\ldots,n-1$, or simply by 
$\varepsilon(n)$ if there is no dependence on r, denotes a sequence which goes to zero as n goes to infinity uniformly in i, in the sense that
\begin{align}
\varepsilon_{r}(n) \mathop{\longrightarrow}_{n \to \infty} 0,
\quad
\max_{i=1,\ldots,\,m-1} |\varepsilon_{r}(n,i)| \mathop{\longrightarrow}_{n \to \infty} 0,
\quad
\max_{i=m,\ldots,n-1} |\varepsilon_{r}(n,i)| \mathop{\longrightarrow}_{n \to \infty} 0,
\quad
\varepsilon(n) \mathop{\longrightarrow}_{n \to \infty} 0.
\end{align}The precise value of such functions may change from line to line in the computations below.
First step
In this step, we prove by induction on r that for each r, there exists sequences 
$\varepsilon_r(n)$ and 
$\varepsilon_{r}(n,i)$ such that
\begin{align}
\left\{
\begin{array}{l}
v^{(r)} = \left(a^2(1-a)\right)^{r} n^{3r}\left(1 + \varepsilon_{r}(n)\right),
\\\\
v_i^{(r)} = a^{2r}(1-a)^{r-1}n^{3r-1}(n-i) + n^{3r}\varepsilon_{r}(n,i),
\quad
\mbox{ for } i=m,\ldots,n-1,
\end{array}
\right.
\end{align} where 
$\varepsilon_r(n)$ and 
$\varepsilon_{r}(n,i)$, for 
$i=m,\dots, n-1$, go to zero as 
$n \longrightarrow \infty$ in the sense defined in (21).
If (22) is true for some given r, then we have
\begin{align}
\sum_{\ell=m+1}^{n-1} (n-\ell)v^{(r)}_{\ell}
& = \sum_{\ell=m+1}^{n-1} (n-\ell)\left[
a^{2r}(1-a)^{r-1}n^{3r-1}(n-\ell)+ n^{3r}\varepsilon_{r}(n,\ell)\right] \nonumber \\
& = a^{2r}(1-a)^{r-1}n^{3r-1}\left(\sum_{\ell=1}^{n-m-1} \ell^2\right)
+ n^{3r}\varepsilon_r(n)\left(\sum_{\ell=1}^{n-m-1} \ell \right) \nonumber \\
& = a^{2r}(1-a)^{r-1}n^{3r-1}\frac{(n-m)^3}{3}\left(1 + \varepsilon(n-m)\right)
+ n^{3r}\varepsilon_r(n)\frac{(n-m)^2}{2}\left(1 + \varepsilon(n-m)\right) \nonumber \\
& = a^{2r}(1-a)^{r-1}n^{3r-1}\frac{(1-a)^3n^3}{3}\left(1 + \varepsilon(n)\right)
+ n^{3r}\varepsilon_r(n)\frac{(1-a)^2n^2}{2}\left(1 + \varepsilon(n)\right) \nonumber \\
& = n^{3(r+1)}\varepsilon_r(n).
\end{align} Here we used the fact that 
$n-m = (1-a)n (1+ \varepsilon(n)).$ We also used the variable change 
$\ell := n-\ell$ in the original sums over 
$\ell$ and the well-known asymptotic behavior
\begin{equation}
\sum_{\ell=1}^k \ell^\alpha ~\mathop{\sim}_{k \longrightarrow \infty}~ \frac{k^{\alpha +1}}{\alpha +1}, \mbox{~~for any~~ } \alpha \geq 0.
\end{equation} In the same spirit, if (22) is true for some given r, then we also have, for any 
$i=m,\ldots,n-1$,
\begin{align}
\sum_{\ell=m+1}^{i-1} (i-\ell)v^{(r)}_{\ell}
& = \sum_{\ell=m+1}^{i-1} (i-\ell)
\left[a^{2r}(1-a)^{r-1}n^{3r-1}(n-\ell) + n^{3r}\varepsilon_r(n,\ell)\right] \nonumber \\
& = a^{2r}(1-a)^{r-1}n^{3r-1}\left(\sum_{\ell=1}^{i-m-1} \ell(n-i+\ell)\right)
+ n^{3r}\varepsilon_r(n,i)\left(\sum_{\ell=1}^{i-m-1} \ell \right) \nonumber \\
& = a^{2r}(1-a)^{r-1}n^{3r-1}\left(\frac{(n-i)(i-m)^2}{2}(1+\varepsilon(i-m))
+\frac{(i-m)^3}{3}(1+\varepsilon(i-m)) \right) \nonumber \\
& \qquad\qquad
+ n^{3r}\varepsilon_r(n,i)\frac{(i-m)^2}{2}(1 + \varepsilon(i-m)) \nonumber \\
& = n^{3(r+1)}\varepsilon_r(n,i),
\end{align} where we used the variable change 
$\ell := n-\ell$ in the original sums over 
$\ell$ and the fact that our assumptions on m and i imply 
$(n-i)(i-m)^2= n^4 \varepsilon(n,i)$, together with 
$(i-m)^3= n^4 \varepsilon(n,i)$ and 
$(i-m)^2= n^3 \varepsilon(n,i)$.
Now, using the expression of 
$v^{(r)}$ given by relation (17), we may deduce that, if (22) is true for integers r and r − 1, then we have
\begin{align*}
v^{(r+1)}
& = (n-m)\left[(m^2-m+1)v^{(r)} - m(m-1)v^{(r-1)}\right] + (m-1)v^{(r)}
+ 2\sum_{\ell=m+1}^{n-1} (n-\ell)v^{(r)}_{\ell} \\
& = \left[(1-a)n + n\varepsilon(n)\right]\left[a^2 n^2 + n^2\varepsilon(n)\right]
\left[v^{(r)} - v^{(r-1)}\right] + \left[an + n\varepsilon(n)\right]v^{(r)}
+ n^{3(r+1)}\varepsilon_r(n) \\
& = a^2(1-a) n^3(1+\varepsilon(n))\left[ v^{(r)} - v^{(r-1)}\right] + n^{3(r+1)}\varepsilon_r(n) \\
& = a^2 (1-a) n^3(1+\varepsilon(n)) \left[\left(a^2 (1-a)\right)^r n^{3r} + n^{3r}\varepsilon_r(n)\right]
+ n^{3(r+1)}\varepsilon_r(n) \\
& = \left(a^2 (1-a)\right)^{r+1} n^{3(r+1)} + n^{3(r+1)}\varepsilon_r(n),
\end{align*} where we have used 
$m=an(1+\varepsilon(n))$, together with relation (22) for the values r and r − 1, as well as the necessary relation (23). Similarly, using the expression of 
$v^{(r)}_i$ given by relation (18), we obtain that, if (22) is true for integers r and r − 1, then we have for 
$i=m,\ldots,n-1$,
\begin{align*}
v^{(r+1)}_{i}
& = (n-i)\left[(m^2-m+1)v^{(r)} - m(m-1)v^{(r-1)}\right]
+ 2\left(\sum_{\ell=m+1}^{n-1} (n-\ell)v^{(r)}_{\ell}
- \sum_{\ell=m+1}^{i-1} (i-\ell)v^{(r)}_{\ell}\right) \\
& = (n-i)a^2 n^2(1+\varepsilon(n))\left[ v^{(r)} - v^{(r-1)}\right] + n^{3(r+1)}\varepsilon_r(n,i) \\
& = (n-i)a^2 n^2(1+\varepsilon(n))\left[\left( a^2 (1-a)\right)^r n^{3 r} + n^{3 r}\varepsilon_r(n)\right]
+ n^{3(r+1)}\varepsilon_r(n,i) \\
& = a^{2(r+1)} (1-a)^{r}n^{3r+2}(n-i) + n^{3(r+1)} \varepsilon_r(n,i)
\end{align*} where we have used 
$m=an(1+\varepsilon(n))$, together with relation (22) for the values r and r − 1, as well as the necessary relations (23) and (25).
We have proved that if (22) is true for integers r and r − 1, then the same relation holds for integer r + 1.
To conclude this first step, there remains to prove that (22) is true for integers r = 1 and r = 2.
For r = 1, since 
$v^{(0)}_i = 1$ and 
$v^{(-1)}_i = 0$, for any i, using relation (17) in the case r = 1, we have
\begin{align*}
v^{(1)}
& = (n-m)(m^2-m+1) + (m-1) + 2 \sum_{\ell=m+1}^{n-1} (n-\ell) \\
& = a^2(1-a)n^3 + n^3\varepsilon(n).
\end{align*} Similarly, using relation (18) in the case r = 1, we have for 
$i=m,\ldots,n-1$,
\begin{align*}
v^{(1)}_i
& = (n-i) (m^2-m+1) + 2\left(\sum_{\ell=m+1}^{n-1} (n-\ell) - \sum_{\ell=m+1}^{i-1} (i-\ell)\right) \\
& = (n-i)\left(a^2n^2 + n^2\varepsilon(n)\right) + n^3 \varepsilon(n,i)\\
&= a^2n^2(n-i) + n^3\varepsilon(n,i).
\end{align*}Thus, relations (22) are true for r = 1.
In the same way, for r = 2 using now relations (23) and (25) in the case r = 2, which is legitimate due to the previous verification, and inserting these relations in (17) and (18) in the case r = 2, we get successively
\begin{align*}
v^{(2)}
& = (n-m) \left[(m^2-m+1) v^{(1)} - m (m-1)\right] + (m-1) v^{(1)}
+ 2 \sum_{\ell=m+1}^{n-1} (n-\ell) v_{\ell}^{(1)} \\
& = \left(a^2(1-a)\right)^2 n^6 + n^6\varepsilon(n),
\end{align*} and, for 
$i=m,\ldots,n-1$,
\begin{align*}
v^{(2)}_i
& = (n-i) \left[ (m^2-m+1) v^{(1)} - m(m-1)\right]
+ 2 \left(\sum_{\ell=m+1}^{n-1} (n-\ell) v_\ell^{(1)}
- \sum_{\ell=m+1}^{i-1} (i-\ell) v_\ell^{(1)}\right) \\
& = (n-i)\left[a^4 (1-a) n^5 + n^5 \varepsilon(n)\right] + n^6\varepsilon(n,i)
= a^4(1-a)n^5(n-i) + n^6\varepsilon(n,i).
\end{align*}Thus relations (22) are valid for r = 2. This concludes our recursion.
Second step
Inserting the above results in the formulae obtained in Theorem 4.3, we have for every 
$i=1,\ldots,$
$m-1$,
\begin{align*}
\mathbb{E}_i(T_n^r)
& = \sum_{\ell=1}^{r} c_{r,\ell}v^{(\ell)}
= \sum_{\ell=1}^{r} c_{r,\ell}
\left[\left(a^2(1-a)\right)^{\ell} n^{3\ell} + n^{3\ell}\varepsilon_\ell(n)\right] \\
& = c_{r,r} \left[\left(a^2(1-a)\right)^{r} n^{3r} + n^{3r}\varepsilon_r(n)\right] \\
& = r!\left(a^2(1-a)\right)^{r} n^{3r} + n^{3r}\varepsilon_r(n),
\end{align*}which completes the proof.
Theorem 4.4 now allows us to determine the asymptotic behavior of the distribution of the hitting time Tn, when 
$m/n \longrightarrow a$ as 
$n \longrightarrow \infty$.
Corollary 4.5. Assume there is an 
$a \in (0,1)$ such that for every 
$n \geq 3$, we have
\begin{equation*}m = an\left(1+\varepsilon(n)\right),
\text{ for some sequence } \varepsilon(n) \text{ satisfying }
\varepsilon(n) \mathop{\longrightarrow}_{n \to \infty} 0.\end{equation*} Then, for all 
$t \geq 0$, and every 
$i = 1,\ldots,m-1$
\begin{equation*}\lim_{n \longrightarrow \infty} \mathbb{P}_i\left\{\frac{T_n}{a^{2}(1-a)n^{3}} \gt t\right\}
= e^{-t}.\end{equation*}Proof. Let Z be a random variable exponentially distributed with rate 1. It is well-known that for every 
$r \geq 0$, we have 
$\mathbb{E}(Z^r) = r!$. Thus, the power series 
$\sum_{r \geq 0} \mathbb{E}(Z^r) x^r/r!$ has a positive radius (it is equal to 1). This implies, from Theorem 30.1 of [Reference Billingsley7], that the random variable Z is uniquely determined by its moments. From Theorem 4.4, we have for every 
$r \geq 1$,
\begin{equation*}\lim_{n \longrightarrow \infty} \mathbb{E}_i\left(\left(\frac{T_n}{a^2(1-a)n^3}\right)^r\right)
= r! = \mathbb{E}(Z^r).\end{equation*} It follows from Theorem 30.2 of [Reference Billingsley7] that, for all 
$i = 1,\ldots,m-1$
\begin{equation*}\lim_{n \longrightarrow \infty} \mathbb{P}_i\left\{\frac{T_n}{a^{2}(1-a)n^{3}} \gt t\right\}
= \mathbb{P}\{Z \gt t\} = e^{-t},\end{equation*}which completes the proof.
In the particular case where
\begin{equation*}m = \lceil 2(n-1)/3 \rceil,\end{equation*} which corresponds to the maximum expected hitting time as shown in Corollary 4.1, we obtain by taking 
$a=2/3$ in Theorem 4.4 and Corollary 4.5
\begin{equation*}\mathbb{E}_i(T_n^r) = r! \left(\frac{4n^3}{27}\right)^{r} + n^{3r}\varepsilon(n)
\mbox{ and }
\lim_{n \longrightarrow \infty} \mathbb{P}\left\{\frac{27T_n}{4n^3} \gt t\right\} = e^{-t}.\end{equation*} Observe, as expected, that the maximum of 
$a^2(1-a)$ in 
$(0,1)$ is obtained for 
$a=2/3$ and is equal to 
$4/27$.
Note also that when
\begin{equation*}m=n,\end{equation*} the lollipop graph 
$L_n^n$ is the complete graph Kn. In that case, we easily obtain that Tn has a geometric distribution with parameter 
$1/(n-1)$. We thus get
\begin{equation*}\mathbb{P}\left\{T_n \gt k\right\} = \left(1 - \frac{1}{n-1}\right)^k.\end{equation*} We deduce easily that for the complete graph we have, for all 
$t \geq 0$,
\begin{equation*}\lim_{n \longrightarrow \infty} \mathbb{P}\left\{\frac{T_n}{n-1} \gt t\right\} = e^{-t}.\end{equation*} This ends our discussion of the case 
$m/n\to a$ as 
$n \longrightarrow \infty$.
Let us now investigate the complementary case when 
$m \ll n$. More technically, we now discuss the case when
\begin{equation*}
\frac{m}{\sqrt{n}} \mathop{\longrightarrow}_{n\to\infty} 0.
\end{equation*} We stress here that the more general case where the parameter m is sublinear with respect to n, that is, the case when 
$\displaystyle m\mathop{\sim}_{n\longrightarrow\infty} a n^\beta$ for some 
$\beta \in (0,1)$ and some a > 0, is much more complicated when 
$\beta\geq 1/2$, and will be the subject of further research. We thus somehow restrict our attention to the case 
$\beta \lt 1/2$ below.
Theorem 4.6. Assume that for every 
$n \geq 3$, we have
\begin{equation*}m = \sqrt{n}\varepsilon(n), \text{ for some sequence } \varepsilon(n) \text{ satisfying }
\varepsilon(n) \mathop{\longrightarrow}_{n \to \infty} 0.\end{equation*} Then for every 
$r \geq 1$, there is a sequence 
$\varepsilon_r(n)$ such that for any 
$i = 1,\ldots,m-1$ we have
\begin{equation*}\mathbb{E}_i(T_n^r) = \frac{\lvert E_{2r} \rvert 2^rr!}{(2r)!}n^{2r}\left( 1 + \varepsilon_r(n)\right),\end{equation*} where 
$E_{2r}$ are the well-known Euler numbers.
Proof. As we did for Theorem 4.4, the proof is based on the asymptotic analysis of the 
$v^{(r)}$ and 
$v^{(r)}_{i}$ obtained in Theorem 4.3. Set 
$m=\sqrt{n}\varepsilon(n)$, and use the notation defined in (21).
We prove by induction that for every 
$n \geq 1$, we have
\begin{align}
\left\{\begin{array}{l}
\displaystyle
v^{(r)} = \frac{2^{r}\lvert E_{2r}\rvert}{(2r)!} n^{2r} \left(1 + \varepsilon_r(n)\right) \\\\
\displaystyle
v^{(r)}_{i} = (-1)^r2^r \left(\sum_{k=0}^r \frac{E_{2k}}{(2k)!(2(r-k))!} i^{2(r-k)}n^{2k}\right)
+ n^{2r}\varepsilon_r(n,i), \mbox{~~for~~ } i=m+1,\ldots,n-1.
\end{array}\right.
\end{align}If (26) is true for a fixed r, then we have
\begin{align*}
\sum_{\ell=m+1}^{n-1} (n-\ell)v^{(r)}_{\ell}
& = \sum_{\ell=m+1}^{n-1} (n-\ell)
\left[(-1)^r2^r \left(\sum_{k=0}^r \frac{E_{2k}}{(2k)!(2(r-k))!} \ell^{2(r-k)}n^{2k}\right)
+ n^{2r}\varepsilon_r(n,\ell)\right] \\
& = (-1)^r2^r \left( \sum_{k=0}^r \frac{E_{2k}}{(2k)!(2(r-k))!} n^{2k}
\left[n\sum_{\ell=m+1}^{n-1} \ell^{2(r-k)} - \sum_{\ell=m+1}^{n-1} \ell^{2(r-k)+1}\right]
\right) \\
& \quad + n^{2r} \varepsilon_r(n)\left[\sum_{\ell=m+1}^{n-1} (n-\ell)\right].
\end{align*}We now observe, using (24), that
\begin{align*}
\sum_{\ell=m+1}^{n-1} \ell^{2(r-k)}
& = \sum_{\ell=1}^{n-1} \ell^{2(r-k)} - \sum_{\ell=1}^{m} \ell^{2(r-k)} \\
& = \frac{n^{2(r-k)+1}}{2(r-k)+1}\left(1 + \varepsilon_{r-k}(n)\right)
- \frac{m^{2(r-k)+1}}{2(r-k)+1}\left(1 + \varepsilon_{r-k}(m)\right) \\
& = \frac{n^{2(r-k)+1}}{2(r-k)+1} \left(1 + \varepsilon_{r-k}(n)\right),
\end{align*} where we have used that 
$m =\sqrt{n} \varepsilon(n)$. Similarly, we observe
\begin{align*}
&\sum_{\ell=m+1}^{n-1} \ell^{2(r-k)+1} = \frac{n^{2(r-k)+2}}{2(r-k)+2}
\left(1 + \varepsilon_{r-k}(n)\right)
\mbox{~~and~~ }
\sum_{\ell=m+1}^{n-1} (n-\ell) = \sum_{\ell=1}^{n-m-1} \ell =
\frac{n^2}{2} \left(1 + \varepsilon(n)\right).
\end{align*}This provides
\begin{align*}
\sum_{\ell=m+1}^{n-1} (n-\ell)v^{(r)}_{\ell}
& = (-1)^r2^r \sum_{k=0}^r \frac{E_{2k}}{(2k)!(2(r-k))!}n^{2r+2}
\left(1 + \varepsilon_{r-k}(n)\right) \left[\frac{1}{2(r-k)+1} - \frac{1}{2(r-k)+2}\right]
\\
& \quad + n^{2r+2} \varepsilon_r(n) \\
& = (-1)^r2^r n^{2(r+1)} \left(1 + \varepsilon_r(n)\right)
\left(\sum_{k=0}^r \frac{E_{2k}}{(2k)!(2(r+1-k))!}\right) + n^{2r+2} \varepsilon_r(n) \\
& = \frac{(-1)^r2^r n^{2(r+1)}}{(2(r+1))!}
\left(\sum_{k=0}^r \binom{2(r+1)}{2k} E_{2k}\right) + n^{2r+2} \varepsilon_r(n).
\end{align*}Using now the well-known relations
\begin{equation*}E_{2(r+1)} = - \sum_{k=0}^r \binom{2(r+1)}{2k} E_{2k}
\mbox{ and } (-1)^{r+1}E_{2(r+1)} = \lvert E_{2(r+1)}\rvert,\end{equation*}we conclude that
\begin{align}
\sum_{\ell=m+1}^{n-1} (n-\ell)v^{(r)}_{\ell}
& = \frac{2^r \left|E_{2(r+1)}\right|}{(2(r+1))!} n^{2(r+1)} + n^{2(r+1)} \varepsilon_r(n).
\end{align} Following the same lines, if (26) is true for a fixed r then we have, for 
$i=m,\ldots,n-1$,
\begin{align}
\sum_{\ell=m+1}^{i-1} (i-\ell)v^{(r)}_{\ell}
& = (-1)^r2^r \sum_{k=0}^r \frac{E_{2k}}{(2k)!(2(r-k))!}n^{2k}
\left[i \left(\sum_{\ell=m+1}^{i-1} \ell^{2(r-k)}\right)
- \left(\sum_{\ell=m+1}^{i-1} \ell^{2(r-k)+1} \right)\right] \nonumber \\
& \quad + n^{2r}\sum_{\ell=m+1}^{i-1} (i-\ell) \varepsilon_r(n,\ell) \nonumber \\
& = (-1)^r2^r \sum_{k=0}^r \frac{E_{2k}}{(2k)!(2(r-k))!}n^{2k}
\left[i \frac{i^{2(r-k)+1}}{2(r-k)+1} - \frac{i^{2(r-k)+2}}{2(r-k)+2}\right]
\left(1+\varepsilon_{r-k}(i)\right) \nonumber \\
& \quad + n^{2r} \varepsilon_r(n,i) \sum_{\ell=1}^{i-m-1} \ell \nonumber \\
& = (-1)^r2^r \left( 1+\varepsilon_{r}(n,i)\right)
\left(\sum_{k=0}^r \frac{E_{2k}}{(2k)!(2(r+1-k))!}n^{2k}i^{2(r-k)+2}\right)
+ n^{2r+2} \varepsilon_r(n,i) \nonumber \\
& = (-1)^r 2^r \left(\sum_{k=0}^r \frac{E_{2k}}{(2k)!(2(r+1-k))!}n^{2k}i^{2(r+1-k)}\right)
+ n^{2(r+1)} \varepsilon_r(n,i).
\end{align}We are now in position to prove relations (26) by induction.
Suppose relations (26) are true for integers r and r − 1. We then have, using (27) together with (17)
\begin{align*}
v^{(r+1)}
& = (n-m) \left[(m^2-m+1) v^{(r)} - m (m-1) v^{(r-1)}\right] + (m-1) v^{(r)}
+ 2 \sum_{\ell=m+1}^{n-1} (n-\ell) v_\ell^{(r)} \\
& = n \left[n \varepsilon(n)\right]\frac{2^{r}\lvert E_{2r}\rvert}{(2r)!} n^{2r}
\left(1+\varepsilon_r(n)\right) + n^{2r+1/2}\varepsilon_r(n)
+ 2\frac{2^{r}\lvert E_{2(r+1)}\rvert}{(2(r+1))!} n^{2(r+1)}\left(1+\varepsilon_r(n)\right) \\
& = n^{2(r+1)}\varepsilon_r(n) + 2 n^{2(r+1)} 2^r \frac{\lvert E_{2(r+1)}\rvert}{(2(r+1))!}
+ n^{2(r+1)}\varepsilon_r(n) \\
& = \frac{2^{r+1}\lvert E_{2(r+1)} \rvert}{(2(r+1))!}n^{2(r+1)} + n^{2(r+1)}\varepsilon(n).
\end{align*} Similarly, if relations (26) are true for integers r and r − 1, we then have, using (27) and (28) together with formula (18), for 
$i=m,\ldots,n-1$,
\begin{align*}
v^{(r+1)}_i
& = (n-i) \left[(m^2-m+1) v^{(r)} - m (m-1) v^{(r-1)}\right]
+ 2 \left(\sum_{\ell=m+1}^{n-1} (n-\ell) v_\ell^{(r)}
- \sum_{\ell=m+1}^{i-1} (i-\ell) v_\ell^{(r)}\right) \\
& = n \left[n\varepsilon(n)\right] \frac{2^{r}\lvert E_{2r}\rvert}{(2r)!} n^{2r}
\left(1+\varepsilon_r(n)\right) + 2\frac{2^{r}\lvert E_{2(r+1)}\rvert}{(2(r+1))!} n^{2(r+1)}
\left(1+\varepsilon_r(n)\right) \\
& \quad -2(-1)^r 2^r
\left(\sum_{k=0}^r \frac{E_{2k}}{(2k)!(2(r+1-k))!}n^{2k}i^{2(r+1-k)}\right)
+ n^{2(r+1)} \varepsilon_r(n,i) \\
& = 2\frac{2^{r}\lvert E_{2(r+1)}\rvert}{(2(r+1))!} n^{2(r+1)}
- 2(-1)^r 2^r \left(\sum_{k=0}^r \frac{E_{2k}}{(2k)!(2(r+1-k))!}n^{2k}i^{2(r+1-k)}\right)
+ n^{2(r+1)} \varepsilon_r(n,i) \\
& = (-1)^{r+1} 2^{r+1} \left(\sum_{k=0}^{r+1}
\frac{E_{2k}}{(2k)!(2(r+1-k))!}n^{2k}i^{2(r+1-k)}\right) + n^{2(r+1)} \varepsilon_r(n,i),
\end{align*} where we used the well-known fact that 
$|E_{2(r+1)}| = (-1)^{r+1}E_{2(r+1)}$.
To complete our recursion, there remains to check that relations (26) are valid for r = 1 and r = 2.
For r = 1, since 
$v^{(0)}=v^{(0)}_i= 1$ and 
$v^{(-1)}=v^{(-1)}_i = 0$ for any i, we have, using relation (17)
\begin{align*}
v^{(1)}
& = (n-m) (m^2-m+1) + (m-1) + 2 \sum_{\ell=m+1}^{n-1} (n-\ell) \\
& = n \left[n \varepsilon(n)\right] + n^{1/2} \varepsilon(n) + 2 \sum_{\ell=1}^{n-m-1} \ell \\
& = n^2 \varepsilon(n) + n^2 \left(1 + \varepsilon(n)\right) = \frac{2 |E_2|}{2!} n^2 + n^2 \varepsilon(n),
\end{align*} since 
$|E_2|=1$, and, whenever 
$i=m,\ldots, n-1$, using formula (18), we also have
\begin{align*}
v^{(1)}_i
& = (n-i) (m^2-m+1) + 2 \left(\sum_{\ell=m+1}^{n-1} (n-\ell)
- \sum_{\ell=m+1}^{i-1} (i-\ell) \right) \\
& = n\left[n \varepsilon(n) \right] + 2 \left( \sum_{\ell=1}^{n-m-1} \ell\right)
- 2 \left(\sum_{\ell=1}^{i-m-1} \ell\right) \\
& = n^2 \varepsilon(n) + n^2 \Big(1+\varepsilon(n)\Big) - (i-m)^2 \Big(1+\varepsilon(i-m)\Big) \\
& = n^2-(i-m)^2 + n^2 \varepsilon(n,i) = n^2 - i^2 + n^2 \varepsilon(n,i) \\
& = (-1)^{1} 2^{1} \left(\sum_{k=0}^{1} \frac{E_{2k}}{(2k)!(2(1-k))!} n^{2k}i^{2(1-k)}\right)
+ n^{2} \varepsilon(n,i),
\end{align*} since 
$E_0=1$ and 
$E_2=1$. Thus, relations (26) are true for r = 1.
In the same way, for r = 2, using relation (27) for r = 1 in (17), which is legitimate thanks to the previous verification, provides
\begin{align*}
v^{(2)}
& = (n-m) \left[(m^2-m+1) v^{(1)}-m(m-1)\right]+(m-1) v^{(1)}
+ 2 \sum_{\ell=m+1}^{n-1}(n-\ell) v^{(1)}_\ell \\
& = n\left[n\varepsilon(n)\right]n^2 \left( 1+\varepsilon(n)\right)
+ \left[n^{1/2}\varepsilon(n)\right]n^2 \left(1+\varepsilon(n)\right)
+ \frac{2 |E_4|}{4!}n^4 + n^4 \varepsilon(n) \\
& = \frac{2 n^4 |E_4|}{4!} + n^4 \varepsilon(n)
\quad \left(= \frac{5}{6}n^4 + n^{4}\varepsilon(n)\text{ since } E_4=5\right),
\end{align*} and, using relations (27) and (28) for r = 1 in (18), which is legitimate thanks to the previous verification, provides for 
$i=m,\ldots,n-1$,
\begin{align*}
v^{(2)}_i
& = (n-i) \left[(m^2-m+1) v^{(1)} - m(m-1)\right] + (m-1) v^{(1)}
+ 2 \left(\sum_{\ell=m+1}^{n-1}(n-\ell) v^{(1)}_\ell
- \sum_{\ell=m+1}^{i-1}(i-\ell) v^{(1)}_\ell\right)\\
& = (n-i) \left[\left[n \varepsilon(n) \right] n^2 \left(1+\varepsilon(n)\right) - n\varepsilon(n)\right]
+ \left[n^{1/2} \varepsilon(n) \right]n^2 \Big(1+\varepsilon(n)\Big) \\
& \quad + 2 \left(\frac{2n^{4}}{4!} \left|E_{4}\right| + n^{4} \varepsilon(n)
- (-1) 2 \left(\sum_{k=0}^1 \frac{E_{2k}}{(2k)!(2(2-k))!}n^{2k}i^{2(2-k)}\right)
+ n^{4} \varepsilon(n,i)\right) \\
& = 2 \frac{2n^{4}}{4!} E_{4} + 4\frac{E_{0}}{4!}i^{4} + 4 \frac{E_{2}}{2!2!}n^{2}i^{2}
+ n^{4} \varepsilon(n,i) \\
& = (-1)^22^2 \left( \sum_{k=0}^2 \frac{E_{2k}}{(2k)!(2(2-k))!}i^{2(2-k)}n^{2k}\right)
+ n^{4}\varepsilon_r(n,i)
\qquad
\left(= \frac{5}{6}n^4 - i^2n^2 + \frac{1}{6}i^4 + n^4\varepsilon(n,i)\right).
\end{align*}Thus relations (26) are also valid for r = 2.
This completes the proof of relations (26) by induction.
Now armed with relation (26), we use Theorem 2.1, to deduce that, for every 
$i=1,\ldots,m-1$, we have
\begin{align*}
\mathbb{E}_i(T_n^r)
& = \sum_{\ell=1}^{r} c_{r,\ell}v^{(\ell)} = \sum_{\ell=1}^{r} c_{r,\ell}
\left[\frac{2^{\ell}\lvert E_{2\ell}\rvert}{(2\ell)!} n^{2\ell}
+ n^{2\ell}\varepsilon_\ell(n)\right] \\
& = c_{r,r} \left[\frac{2^{r}\lvert E_{2r}\rvert}{(2r)!} n^{2r} + n^{2r}\varepsilon_r(n)\right] \\
& = \frac{2^{r}\lvert E_{2r}\rvert r!}{(2r)!} n^{2r} + n^{2r}\varepsilon_r(n),
\end{align*}which completes the proof.
Theorem 4.6 allows us to determine the asymptotic behavior of the distribution of the hitting time Tn, when 
$m/\sqrt{n}$ tends to 0 when n tends to infinity.
Corollary 4.7. Assume that for every 
$n \geq 3$, we have
\begin{equation*}m = \sqrt{n}\varepsilon(n), \text{ for some sequence } \varepsilon(n) \text{ satisfying }
\varepsilon(n) \mathop{\longrightarrow}_{n \to \infty} 0.\end{equation*} Then for all 
$t \geq 0$ and for every 
$i = 1,\ldots,m-1$, we have
\begin{equation*}\lim_{n \longrightarrow \infty} \mathbb{P}_i\left\{\frac{T_n}{2n^2} \gt t\right\}
= \frac{4}{\pi} \sum_{k=0}^\infty \frac{(-1)^k}{(2k+1)} e^{-\lambda_k t},
\mbox{ where }\lambda_k = \frac{\pi^2(2k+1)^2}{4}.\end{equation*}Proof. Introducing the notation 
$Y_n = T_n/(2n^2)$, we deduce from Theorem 4.6 that, for every 
$i=1, \ldots,m-1$,
\begin{equation*}\lim_{n \longrightarrow \infty} \mathbb{E}_i\left(Y_n^r\right)
= \frac{\lvert E_{2r} \rvert r!}{(2r)!}.\end{equation*}Consider the sequence of positive reals numbers sr defined by
\begin{equation*}s_r = \frac{\lvert E_{2r} \rvert r!}{(2r)!}.\end{equation*} The first values of this sequence are 
$s_0=1$, 
$s_1=1/2$, 
$s_2=5/12$, 
$s_3=61/120$, 
$s_4=277/336$. Our task is now to identify the probability measure µ associated with the asymptotic moments given by the sequence sr, that is, such that for any r,
\begin{equation*}s_r=\int_0^\infty t^rd\mu(t).\end{equation*} This problem is well-known and is called the Stieltjes moment problem. If such a measure exists it is called a normalized (because 
$s_0=1$) Stieltjes moment sequence. If the measure is unique then the sequence sr is said to be determinate, see for instance [Reference Berg and Durán6].
First, a sufficient condition for determinacy is given by the following relation, called Carleman’s condition
\begin{equation}
\sum_{r=1}^\infty s_r^{-1/2r} = + \infty.
\end{equation}To prove (29), we use the well-known fact (see for instance [Reference Borwein, Borwein and Dilcher8]) that
\begin{equation*}\frac{\lvert E_{2r}\rvert}{(2r)!}
\displaystyle \mathop{\sim}_{r\longrightarrow\infty} \frac{4^{r+1}}{\pi^{2r+1}}.\end{equation*} Using the Stirling formula for 
$r!$, this leads to
\begin{equation*}s_r \displaystyle \mathop{\sim}_{r\longrightarrow\infty}
\frac{4^{r+1}}{\pi^{2r+1}}\left(\frac{r}{e}\right)^r\sqrt{2\pi r}.\end{equation*} Since 
$1/2r$ tends to 0 when r tends to infinity, we can write
\begin{equation*}s_r^{1/2r} \displaystyle \mathop{\sim}_{r\longrightarrow\infty}
\frac{2}{\pi\sqrt{e}}\sqrt{r}\left(\frac{4\sqrt{2r}}{\sqrt{\pi}}\right)^{1/2r}
\displaystyle \mathop{\sim}_{r\longrightarrow\infty} \frac{2}{\pi\sqrt{e}}\sqrt{r}.\end{equation*}Thus
\begin{equation*}s_r^{-1/2r} \displaystyle \mathop{\sim}_{r\longrightarrow\infty}
\frac{\pi\sqrt{e}}{2}\frac{1}{\sqrt{r}}.\end{equation*}This series being divergent the Carleman condition (29) is satisfied.
Second, in order to determine the probability measure µ, we use the following relation, which can be found in [Reference Abramowitz and Stegun1] or in [Reference Amigo4]
\begin{equation*}\sum_{k=0}^\infty \frac{(-1)^k}{(2k+1)^{2r+1}} =
\frac{(\pi/2)^{2r+1}\lvert E_{2r} \rvert}{2(2r)!} \mbox{~~for every~~ } r \geq 0.\end{equation*}Introducing the notation
\begin{equation*}\lambda_k = \frac{\pi^2(2k+1)^2}{4} \mbox{~~for every~~ } k \geq 0,\end{equation*}the moments sr can then be written as
\begin{equation*}s_r = \frac{\lvert E_{2r} \rvert r!}{(2r)!}
= \frac{4}{\pi} \sum_{k=0}^\infty \frac{(-1)^k}{(2k+1)} \frac{r!}{\lambda_k^r}.\end{equation*} Observing that 
$r!/\lambda_k^r$ is the rth order moment of the exponential distribution with rate λk, we obtain that the probability measure µ is given by the density
\begin{equation*}f(t) = \left\{
\begin{array}{ccl}
0 & \mbox{if } & t \leq 0 \\\\
{\displaystyle \frac{4}{\pi} \sum_{k=0}^\infty \frac{(-1)^k}{(2k+1)} \lambda_k e^{-\lambda_k t}}
& \mbox{if } & t \gt 0.
\end{array}\right.\end{equation*}We then obtain, using the Fréchet–Shohat theorem, see [Reference Frechet and Shohat13] or [Reference Sodin20],
\begin{equation*}\lim_{n \longrightarrow \infty} \mathbb{P}_i\left\{\frac{T_n}{2n^2} \gt t\right\}
= \int_t^\infty f(x)dx = \frac{4}{\pi} \sum_{k=0}^\infty \frac{(-1)^k}{(2k+1)} e^{-\lambda_k t},\end{equation*}which completes the proof.
Note that this result applies in the particular case m = 2, for which the lollipop graph 
$L_2^n$ is nothing but the classical path graph.
Corollaries 4.5 and 4.7 show that Tn, properly normalized, converges in distribution to an exponential distribution and to a linear combination of exponential distributions respectively. The fact that these limits involve exponential distributions seems to be intuitively clear because Tn is reached after a geometric number of attempts. Nevertheless, observe that the normalization term which leads to the convergence in distribution is of the order of 
$\mathbb{E}(T_n)$. Indeed, when m = an, Corollary 4.5 tells us that 
$T_n/\mathbb{E}(T_n)$ converges in distribution. In the same way, for instance when m is constant, Corollary 4.7 tells us that 
$T_n/\mathbb{E}(T_n)$ converges in distribution. This means that the convergence of 
$T_n/\mathbb{E}(T_n)$ should occur for more general graphs.
5. Conclusion
We considered in this paper both the distribution and the moments of the hitting time on the lollipop graph 
$L_m^n$ which has the maximum expected hitting time among all the n vertex graph. We obtained recurrence relations for all order moments and we used these relations to analyze the asymptotic behavior of the hitting time distribution when n tends to infinity. The main difficulty was to exhibit the asymptotic behavior of all order moments which has led to the asymptotic behavior of the distribution of the hitting times Tn. We observed that for several values of m depending on n, the random variable 
$T_n/\mathbb{E}(T_n)$ converges in distribution. Further research will thus be to analyze more general graphs to see if this convergence still occurs. A first step will be to consider other particular graphs such as the barbell graph obtained by connecting two copies of a complete graph by a bridge or the tadpole graph obtained by joining a cycle graph to a path graph. Another quite interesting question would be to wonder whether the lollipop graph maximizes some of the higher moments among all n vertex graphs.
Competing interest
The authors declare none.
Open access
