2.1. Mathematical model

The discrete-time model consists of m risky assets and a bank account paying a constant interest rate r. For convenience, the evolution of the deposit of one dollar after t periods in the bank account is described in terms of $e^{tr}$ instead of $(1+r)^t$ . The evolution of the risky assets is modeled through a stochastic kernel $v(dz|x,y)$ , taking into account the influence of an endogenous factor process $\{X_t\}$ , which determines the performance of the market and evolves as a finite-state Markov chain in S, according to the given one-step transition probabilities $Q_{xy}$ . In continuous-time models, this endogenous process describes a stochastic volatility environment; in [Reference Fouque, Papanicolaou and Sircar14] the authors present interesting technical arguments to introduce this type of environment of uncertainty. In our case we are introducing an extra uncertainty through the process $\{X_t\}$ , incorporating possible variations in the economical environment. Hence, the m-dimensional random vector $Z_{t+1}$ describing the relative prices of the risky assets between t and $t+1$ has conditional distribution $v(dz|X_t=x,\;X_{t+1}=y)$ . More precisely, if we denote the price at time t of the ith asset by $P^{i}_t$ , $Z_t$ is interpreted as being the m-dimensional vector with ith component $P^i_{t+1}/P_t^i$ .

Let $A\subset \mathbb{R}^m$ be the constraint set for the investor. The vector $a=(a^1,\ldots,a^m)\in A$ denotes an investment action, with $a^i$ representing the proportion of wealth invested in the ith risky asset, while the sum $1-\sum_{i=1}^{m}a^i$ is the proportion of wealth invested in the bank account. Thus, if the vector $a_t\in A$ represents the trading strategy at time t and $V_t$ denotes the value of the portfolio at that period of time, then

(2.1) \begin{equation}V_{t+1}=V_t[e^r+a_t\cdot(Z_{t+1}-e^r \textbf{1})]\end{equation}

represents its evolution at the next period of time $t+1.$ Here $\cdot $ denotes the inner product in $\mathbb{R}^m,$ and $ \textbf{1}$ stands for the m-dimensional vector of 1s. For simplicity of notation, we define for $a\in A$ and $z\in \mathbb{R}^m$

\begin{equation*}F(a,z)\;:\!=\;[e^r+a\cdot(z-e^r \textbf{1})],\end{equation*}

and abbreviate $F(a_t,Z_{t+1})$ by $F_{t+1}$ when the context is clear.

The drawdown constraint has two components:

1. 1. The fixed growth rate $\lambda \in \, ]0,r]$ represents an estimate of the inflation rate obtained by the supervisor, who imposes the restriction that any trading strategy followed by the fund manager should outperform inflation.

2. 2. Let $M_0$ be a fixed constant larger than the initial wealth $V_0,$ and let $\alpha \in \, ]0,1[$ be such that $\frac{V_0}{M_0}\geq \alpha e^{-\lambda}$ . Then the last recorded maximum process associated with $\lambda$ up to time t is defined as

   \begin{align*}M_t\;:\!=\;\max\{M_0e^{\lambda t}; V_se^{\lambda(t-s)}, s \leq t \};\end{align*}
   it is not difficult to see that $M_t=\max\{e^{\lambda}M_{t-1},V_t\}$ for all $t \in \mathbb{N}.$ We say that a trading strategy $\{a_t\}$ satisfies the drawdown constraint if the value process cannot drop below a fraction $\alpha$ of the last recorded maximum, i.e.
   (2.2) \begin{align}V_t \geq \alpha M_{t-1}, \ \ \ \ \forall\; t \in \mathbb{N}, \ \ \text{almost surely.}\end{align}

Remark 2.1. The value of the drawdown parameter $\alpha$ depends on the financial industry. The drawdown constraint imposed on the fund manager by his supervisor commits to other users a proportion $\alpha$ of the profits made by the manager, when the last recorded maximum is reached. This constraint also reflects the fact that the supervisor is reluctant to let the wealth fund fall below a fraction of its maximum to date (see [Reference Guasoni and Obloj16], [Reference Grossman and Zhou15]); in the investment management industry $\alpha$ ranges from 75% to 90%.

In order to describe precisely the set of admissible trading strategies, it is convenient to define the quotient process as

(2.3) \begin{align}W_0\;:\!=\;\frac{V_0}{M_0} \ \ \ \textrm{and} \ \ \ W_t\;:\!=\;\frac{V_t}{M_t}, \ \ \ \forall \ t \in \mathbb{N}.\end{align}

When the value process $\{V_t\}$ is driven by a trading strategy satisfying the drawdown constraint (2.2), we get the following two possibilities for each $t \in \mathbb{N}$ :

Case 1: $\alpha M_{t-1} \leq V_t < e^{\lambda}M_{t-1}.$ In this situation, the quotient process $\{W_t\}$ takes values in the interval $[\alpha e^{-\lambda},1[.$

Case 2: If $V_t \geq e^{\lambda}M_{t-1},$ then $V_t = M_t$ ; that is, $W_t=1.$

Therefore, the quotient process $\{W_t\}$ takes values in the interval $[\alpha e^{-\lambda},1].$

We define the function $S\; :\; [\alpha e^{-\lambda},1] \times \mathbb{R}^{m}\times \mathbb{R}^{m} \rightarrow \mathbb{R}$ by

(2.4) \begin{align}S(w,a,z)\;:\!=\;wF(a,z).\end{align}

Let us denote $S(W_t,a_t,Z_{t+1})$ by $S_{t+1}$ , with $\{a_t\}$ a trading strategy. Then the dynamics of the quotient process $\{W_t\}$ is the following:

\begin{align*}W_{t+1}&=W_{t+1}\mathbb{I}_{[V_{t+1}<e^{\lambda}M_{t}]}+W_{t+1}\mathbb{I}_{[V_{t+1} \geq e^{\lambda}M_{t}]}\\[5pt] &=\frac{V_{t+1}e^{-\lambda}}{M_t}\mathbb{I}_{[S_{t+1}<e^{\lambda}]}+\frac{V_{t+1}}{V_{t+1}}\mathbb{I}_{[S_{t+1} \geq e^{\lambda}]}\\[5pt] &=\frac{V_{t}e^{-\lambda}F_{t+1}}{M_t}\mathbb{I}_{[S_{t+1}<e^{\lambda}]}+\mathbb{I}_{[S_{t+1} \geq e^{\lambda}]}.\end{align*}

Hence, defining

(2.5) \begin{equation}G(w,a,z)\;:\!=\;wF(a,z)e^{-\lambda}\mathbb{I}_{[S(w,a,z)<e^{\lambda}]}+ \mathbb{I}_{[S(w,a,z)\geq e^{\lambda}]},\end{equation}

the dynamics of $W_t$ are described by

(2.6) \begin{align} W_{t+1}= G(W_t,a_t,Z_{t+1}),\;\;\;\text{for all}\;\; t \in \mathbb{N}, \end{align}

and following a backward induction argument, we conclude that

\begin{align*} W_{t+1}=W_0e^{-(t+1)\lambda}\prod_{k=1}^{t+1}F_{k}\mathbb{I}_{[S_{k} < e^{\lambda}]} + \sum_{k=1}^{t}\mathbb{I}_{[S_{k} \geq e^{\lambda}]}e^{(k-(t+1))\lambda}\prod_{i=k+1}^{t+1}F_i\mathbb{I}_{[S_{i} < e^{\lambda}]} + \mathbb{I}_{[S_{t+1} \geq e^{\lambda}]}.\end{align*}

We use this auxiliary process to define the investor’s long-term growth rate for the certainty equivalence, to control the value drop by a fixed fraction of the last recorded maximum, and maximize this rate in the long run.

Utility functions and the Arrow–Pratt function

In the remainder, it is assumed that risk preferences of the investor are determined by a utility function U, belonging to the family $\mathcal{U}$ defined below.

The optimization of the long-term growth rate of the certainty equivalence involves the analysis of the Arrow–Pratt function associated with a utility function U, as a measure of risk from the investor’s perspective; see Theorem 2.

Definition 2. (The Arrow–Pratt function.) For a utility function $U \in \mathcal{U},$ and $x \in (0,\infty),$ the corresponding Arrow–Pratt function $\Delta_U(x)$ is defined by

\begin{equation*}\Delta_U(x)\;:\!=\;-x\frac{U''(x)}{U'(x)}.\end{equation*}

Given a random variable X, its certainty equivalent is defined as

\begin{equation*}\Psi_{U}(X)\;:\!=\;U^{-1}(\mathbb{E}[U(X)]).\end{equation*}

Let us suppose that the investor has initial capital x and that a utility function $U \in \mathcal{U}$ is used to measure her risk preferences. If she must choose between a loss without uncertainty of size C and a random loss $\theta Y,$ with $\theta \in \mathbb{R}$ and $\mathbb{E}[U(Y)] < \infty,$ the investor prefers the loss C when $\mathbb{E}[U(x-\theta Y)]<U(x-C).$ However, she is indifferent choosing C and $\theta Y$ if $\mathbb{E}[U(x-\theta Y)]=U(x-C).$ It is clear that the amount $x-C$ for which the last equality holds will be, by definition, the certainty equivalent of $x-\theta Y$ , and applying the Taylor expansion around $\theta = 0,$ we deduce that

(2.7) \begin{align}\Psi_{U}(x+\theta Y) \approx x + \theta \mathbb{E}[Y]- \frac{1}{2}\theta^{2}x^{-1}\Delta_U(x)\text{Var} (Y).\end{align}

Performance index

Let us fix the initial conditions for the factor and quotient processes $X_0=x$ and $W_0=w$ . For a utility function $U\in \mathcal{U}$ and a trading strategy $\pi=\{a_t\}$ , we define the long-term exponential growth rate of the expected utility (or objective function) of the final wealth as

(2.8) \begin{equation}\widehat{J}_{U}(x,w,\pi)\;:\!=\;\liminf_{t \rightarrow \infty} \frac{1}{t}\ln \Psi_{U}^{\pi,x,w}(V_t).\end{equation}

The function $\widehat{J}_{U}(x,w,\pi)$ quantifies the exponential growth of the certainty equivalence associated with the strategy $\pi$ . When the utility function $U_{\gamma}(x)$ is used, we denote the objective function by $\widehat{J}_{\gamma};$ applying Taylor expansion around $\gamma = 0$ , we get the approximation

(2.9) \begin{align}\widehat{J}_{\gamma}(x,w,\pi) \approx \liminf_{t \rightarrow \infty}\frac{1}{t}\left(\mathbb{E}_{x,w}^{\pi}[\ln V_t]+\frac{\gamma}{2}\text{Var}_{x,w}^{\pi}(\ln V_t)\right).\end{align}

Hence, $\widehat{J}_{\gamma}$ can be interpreted as the long-term expected growth rate per unit time plus a penalty proportional to the variance of the portfolio’s value and the risk coefficient $\gamma.$ Note the similarities between the expressions (2.9) and (2.7).

The investor aims to maximize, within the set of admissible trading strategies $\Gamma_{\alpha}$ , the long-term exponential growth rate of the certainty equivalent value $\widehat{J}_{U}(x,w,\pi)$ , and to find a trading strategy $\pi^*$ where the maximum of the objective function is achieved. That is, we are interested in the analysis of the value function

(2.10) \begin{equation}\widehat{J}_{U}(x,w,\pi^*)=\sup_{\pi \in \Gamma_{\alpha}}\widehat{J}_{U}(x,w,\pi)\;=\!:\;\,\widehat{J}_{{U}}^{*}( x,w ).\end{equation}

The set $ \Gamma_{\alpha}$ describing the admissible trading strategies $\pi$ is defined precisely below. When U is a CRRA utility function with parameter $\gamma\in \, ]0,1[$ , the corresponding value function is denoted by $\widehat{J}_{\gamma}^*(x,w).$

The main results presented in this paper can be summarized as follows. Let $U\in \mathcal{U}$ be a regular utility function with asymptotic risk-sensitive coefficient $\lambda_{U} =1-\gamma.$

• We characterize $\widehat{J}_{{U}}^{*}( x,w )$ and provide conditions under which an optimal trading strategy $\pi^*$ can be proposed.

• For a CRRA utility function $U_{\gamma}$ , with $\gamma\in \, ]0,1[$ , it is proved that $\widehat{J}_{{U}}^{*}( x,w )=\widehat{J}_{\gamma}^*(x,w)$ , and that the optimal strategy $\pi^*$ , obtained from the previous step, is also optimal for $\widehat{J}_{\gamma}^*(x,w)$ .

Admissible strategies. In order to complete the description of the mathematical model, we proceed to define precisely the set of admissible strategies $ \Gamma_{\alpha}$ for the investor and the probability space where the stochastic processes involved in the description of the problem are defined. Let us introduce first the positive cone $\mathbb{R}^{m^+}$ in Euclidean space, defined by

\begin{equation*} \mathbb{R}^{m^+}\;:\!=\;\{ \textbf{x} \in \mathbb{R}^m \ : \ x_1>0,x_2>0, \dotsb, x_m >0\};\end{equation*}

the zero vector in $\mathbb{R}^{m}$ will be denoted by $ \textbf{0}$ .

Throughout the remainder the next hypotheses are assumed to hold even without explicit reference.

Let X and Y be Borel spaces and denote by $\mathcal{C}(Y)$ the family of nonempty compact subsets of Y. A compact-set-valued function $\psi\;:\; X \rightarrow \mathcal{C}(Y)$ is upper semi-continuous (respectively, lower semi-continuous) if and only if, for each open subset G (respectively, closed subset F) of Y, the set $\{x \in X \ : \ \psi(x) \subset G\}$ (respectively, $\{x \in X \ : \ \psi(x) \subset F \}$ ) is open (respectively, closed) in X. The function $\psi$ is continuous if it is both upper and lower semi-continuous.

Next, a concrete example describing set $A_w$ is presented, in order to illustrate possible ways to verify Assumption 1(iii).

Example 2. Fix $\alpha \in \, ]0,1[,$ $r>0,$ $\lambda \in \, ]0,r]$ and take $m=1.$ Define $B^*=[0,b],$ with $b>e^r$ , and $A^*=[-1,1].$ Consider now the functions

\begin{equation*}\widehat{\zeta}(w)\;:\!=\;\frac{\alpha w^{-1}-e^r}{b-e^r}, \qquad \zeta(w)\;:\!=\;\max\{-1,\widehat{\zeta}(w)\}, \qquad \xi(w)\;:\!=\;1-\frac{\alpha}{w}e^{-r},\end{equation*}

with $w\in [\alpha e^{-\lambda},1]$ , and define $A^*_w\;:\!=\;[\zeta(w),\xi(w)].$ We claim that the action set $A^*_w$ can be characterized as $\{a \in A^* \ : \ \frac{\alpha}{w} \leq F(a,z), \; \forall \; z \in B^*\}$ and, moreover, that the set-valued function $w\longmapsto A^*_w$ is continuous on $[\alpha e^{-\lambda},1]$ , as required in (2.11). The verification of these claims is postponed to the appendix.

The control model. The set of possible market regimes affecting the asset prices is denoted by $S=\{1,\ldots,N\}$ , endowed with the discrete topology, and $Q_{xy}$ is the homogeneous transition probability on this set. Define the space $\overline{\mathbb{H}}_t$ of regime-quotient histories up to time t by

\begin{align*}\overline{\mathbb{H}}_t\;:\!=\;\bigg(\prod_{k=0}^{t-1}(S \times [\alpha e^{-\lambda},1] \times A)_k \bigg) \times (S\times [\alpha e^{-\lambda}, 1]),\;\;\text{for}\;\;t=1,2,\ldots,\end{align*}

with $\overline{\mathbb{H}}_0=S\times [\alpha e^{-\lambda}, 1]$ . A generic vector of $\overline{\mathbb{H}}_t$ is denoted by

\begin{equation*}\overline{h}_t=(x,w,a_0,x_1,w_1, \ldots , x_{t-1},w_{t-1},a_{t-1},x_t,w_t).\end{equation*}

Given $\alpha \in \, ]0,1[$ fixed, an admissible trading strategy $\pi$ is a sequence $\pi\;:\!=\;\{\pi_t\}_{t=0}^{\infty}$ of stochastic kernels $\pi_t$ on A given $\overline{\mathbb{H}}_t,$ such that for $ t=0,1\ldots,$

\begin{equation*}\pi_t(A_{w_t}|\overline{h}_t)=1, \ \ \textrm{with} \ \overline{h}_t \in \overline{\mathbb{H}}_t.\end{equation*}

The set of admissible strategies is denoted by $\Gamma_{\alpha}$ . Observe that this set is nonempty, and the value process $\{V_t\}$ driven by $\pi \in \Gamma_{\alpha}$ satisfies the drawdown constraint (2.2). Indeed, the trading strategy $\pi_0$ with the property that the total amount of wealth is invested in the riskless asset (that is, $a_t= \textbf{0}$ for all t) satisfies

\begin{align*}M_{t-1}^{\pi_0} &= \max \{ M_0e^{\lambda(t-1)}, V_0e^{\lambda(t-1)}, V_0e^{\lambda(t-1)+(r-\lambda)},\cdots, V_0e^{\lambda(t-1)+(t-2)(r-\lambda)}, V_0e^{r(t-1)}\}\\[5pt]&=\max \{ M_0e^{\lambda(t-1)},V_0e^{r(t-1)}\},\end{align*}

and $V_t^{\pi_0}=V_0e^{rt}.$ Moreover, $V_t^{\pi_0} \geq \alpha M_{t-1}^{\pi_0}$ if, and only if, either $V_0e^{rt} \geq \alpha V_0e^{r(t-1)} $ or $V_0 e^{(r-\lambda)t} \geq M_0 \alpha e^{-\lambda}$ holds, and recalling that $V_0$ and $M_0$ were chosen in such a way that $V_0 \geq M_0 \alpha e^{-\lambda},$ it follows that $\pi_0 \in \Gamma_{\alpha}.$ Throughout, the family of stationary strategies $\overline{\mathbb{F}}_{\alpha} \subset \Gamma_{\alpha}$ is identified with the family of measurable functions $f\;:\; S \times [\alpha e^{-\lambda},1] \rightarrow A$ such that $f(x,w) \in A_w$ for all $(x,w) \in S \times [\alpha e^{-\lambda},1].$

Let $(\Omega,\mathcal{F})$ be the measurable space defined as $\Omega\;:\!=\;\prod_{k=1}^{\infty}(A \times S \times \mathbb{R}^m)_k$ and $\mathcal{F}$ the corresponding product $\sigma$ -algebra. By the Ionescu–Tulcea theorem (see [Reference Ash2, Teorema: 2.7.2]), given the initial states $x_0=x \in S$ and $w_0=w \in [\alpha e^{-\lambda},1]$ and $\pi \in \Gamma_{\alpha},$ there exists a unique probability measure $\mathbb{P}_{x,w}^{\pi}$ on $(\Omega,\mathcal{F})$ given by

\begin{align*}\mathbb{P}_{x,w}^{\pi} &(da_0,x_1,dz_1,da_1,x_2\cdots) \\[5pt] & \, =\, \pi_0(da_0|x,w)Q_{xx_1}v(dz_1|x,x_1)\pi_1(da_1|x,w,a_0,x_1,w_1(w,a_o,z_1))Q_{x_1x_2}\cdots.\end{align*}

The expectation operator with respect to the probability measure $\mathbb{P}_{x,w}^{\pi}$ is denoted by $\mathbb{E}_{x,w}^{\pi}$ , and we define $ (\Omega,\mathcal{F}, \mathbb{P}_{x,w}^{\pi}, X_{\cdot}, Z_{\cdot}, V_{\cdot}^{\pi}, W_{\cdot}^{\pi})$ as an admissible control system, for $\pi\in \Gamma_{\alpha}$ . Observe that once the trading strategy $\pi$ is chosen, together with the initial wealth $V_0$ , the evolution in time of the value of the portfolio $\{V_t^\pi\}$ and the quotient process $\{W_t^\pi\}$ are given, respectively, by (2.1) and (2.6); here we make explicit the dependence of these processes on $\pi$ , but we omit it when the context is clear. The description of the control model is completed by specifying the payoff function to be optimized (cf. (2.8) and (2.10)), defined as

\begin{equation*}\widehat{J}_{U}(x,w,\pi)=\liminf_{t \rightarrow \infty} \frac{1}{t}\ln \Psi_{U}^{\pi,x,w}(V_t^\pi),\end{equation*}

and the value function is given by

(2.12) \begin{equation}\widehat{J}_{{U}}^{*}( x,w )=\sup_{\pi \in \Gamma_{\alpha}}\widehat{J}_{U}(x,w,\pi).\end{equation}

3.1. Contractive operators

First of all, recall that the function G involved in the dynamics of the quotient process $\{W_t\}$ was defined in (2.5) as

\begin{equation*}G(w,a,z)=wF(a,z)e^{-\lambda}\mathbb{I}_{\{S(w,a,z)<e^{\lambda}\}}+ \mathbb{I}_{\{S(w,a,z)\geq e^{\lambda}\}}=\min\{S(w,a,z) e^{-\lambda},1\}.\end{equation*}

Throughout, we keep $\alpha\in \, ]0,1[$ fixed. The proof of the following result is straightforward.

Let $C(S\times [\alpha e^{-\lambda},1])$ be the space of real-valued continuous functions V defined on $ S\times [\alpha e^{-\lambda},1]$ , endowed with the supremum norm $\|\cdot\|$ . For $\gamma,$ $\beta \in \, ]0,1],$ $V \in C(S\times [\alpha e^{-\lambda},1])$ , define the functions $l_V^{\beta}$ and $\tilde{l}_V^{\beta}$ by

(3.4) \begin{align}l_V^{\beta}(x,w,a)\;:\!=\;\sum_{y \in S} \int &e^{\beta V(y,G(w,a,z))}F^{\gamma}(a,z)v(dz|x,y)Q_{xy}, \nonumber\\[5pt]& \quad \text{for}\ (x,w,a) \in S\times [\alpha e^{-\lambda},1]\times A,\end{align}

and

(3.5) \begin{align} \tilde{l}_V^{\beta}(x,w)\;:\!=\;\sup_{a \in A_w}l_V^{\beta}(x,w,a),\;\;(x,w) \in S \times [\alpha e^{-\lambda},1],\end{align}

respectively. The following lemma allows us to ensure the existence of a mapping that maximizes the function $l_V^{\beta}$ , and will be helpful in appropriately defining the family of contractive operators in which we are interested.

Lemma 2. Let $\beta\in \, ]0,1[$ and $V \in C(S\times [\alpha e^{-\lambda},1])$ . Then there exists a function $f_V^{\beta} \in \overline{\mathbb{F}}_{\alpha}$ such that

\begin{equation*}\tilde{l}_V^{\beta}(x,w)=l_V^{\beta}(x,w,f_V^{\beta}(x,w)),\;\;(x,w)\in S \times [\alpha e^{-\lambda},1].\end{equation*}

Moreover, $\tilde{l}_V^{\beta}(x,w)$ is continuous in $S\times [\alpha e^{-\lambda},1].$

Proof. This result is a consequence of the measurable selection theorem; see the results by Himmelberg et al. in [Reference Himmelberg, Partasarathy and Van Vleck19, Theorem 2]. In order to apply that theorem to our case, we need to prove that the function $l_V^{\beta}(x,w,a)$ is continuous. Since the state space S is endowed with the discrete topology, we only need to verify the continuity of $l_V^{\beta}$ in the variables (w, a). Note that, from Assumption 1, it follows that

(3.6) \begin{align}\alpha^{\gamma} e^{-\beta||V||} \leq e^{\beta V(y,G(w,a,z))}F^{\gamma}(a,z)\leq e^{\beta ||V||}\overline{K}^{\gamma}.\end{align}

Now, for any sequence $(w_n,a_n)\to(w,a)$ , the dominated convergence theorem and Lemma 1 yield

\begin{align*}\lim_{n\to\infty} l_V^{\beta}(x,w_n,a_n)&=\sum_{y \in S}\int e^{\beta \lim_{n\to \infty}V(y,G(w_n,a_n,z))}F^{\gamma}(a,z)v(dz|x,y)Q_{xy}\\[5pt]&=\sum_{y \in S}\int e^{\beta V(y,\lim_{n\to \infty}G(w_n,a_n,z))}F^{\gamma}(a,z)v(dz|x,y)Q_{xy}\\[5pt]&=\sum_{y \in S}\int e^{\beta V(y,G(w,a,z))}F^{\gamma}(a,z)v(dz|x,y)Q_{xy}\\[5pt]&=l_V^{\beta}(x,w,a);\end{align*}

therefore $l_V^{\beta}(x,w,a)$ is a continuous function. Since $A_w$ is a continuous set-valued function by Assumption 1(iii), the measurable selection theorem [Reference Himmelberg, Partasarathy and Van Vleck19] yields the result.

Definition 4. For $\gamma,$ $\beta \in \, ]0,1[$ fixed, we define the operator $T_{\beta}\;:\;C(S \times [\alpha e^{-\lambda},1 ])\rightarrow C(S \times [\alpha e^{-\lambda},1])$ by

\begin{align*}T_{\beta}[V](x,w)\;:\!=\; \ln \tilde{l}_V^{\beta}(x,w),\end{align*}

with $V \in C(S\times [\alpha e^{-\lambda},1])$ .

The operator $T_{\beta}$ is well-defined in view of Lemma 2, and by the identity

(3.7) \begin{align}e^{T_{\beta}[V](x,w)}=\tilde{l}_V^{\beta}(x,w),\end{align}

the following two properties are satisfied:

1. 1. Monotonicity: For each $V, \ W \in C(S \times [\alpha e^{-\lambda},1])$ with $V \leq W,$ $T_{\beta}[V] \leq T_{\beta}[W].$

2. 2. $\beta$ -homogeneity: For $V \in C(S \times [\alpha e^{-\lambda},1])$ and $r \in \mathbb{R}$ , $T_{\beta}[V+r]=T_{\beta}[V]+\beta r.$

The constant $\overline{K}$ was introduced in Remark 1. Notice that if we write the identity $T_{\beta}[V_{\beta}]=V_{\beta}$ equivalently as $e^{V_{\beta}(x,w)}=\tilde{l}_{V_{\beta}}^{\beta}(x,w)$ and use Lemma 2, it follows that there exists a measurable function $f_{\beta}(x,w)$ such that

(3.9) \begin{align}e^{V_{\beta}(x,w)}=l_{V_{\beta}}^{\beta}(x,w,f_{\beta}(x,w)), \ \ \forall (x,w).\end{align}

Proof. Fix $\beta \in \, ]0,1[,$ and let V and W be arbitrary functions in $C(S \times [\alpha e^{-\lambda},1]).$ Then, for each $(x,w)\in S \times [\alpha e^{-\lambda},1]$ ,

\begin{equation*}W(x,w)-||V-W|| \leq V(x,w) \leq W(x,w) + ||V-W||,\end{equation*}

or, in terms of the absolute value $|\cdot|$ , $|V(x,w)-W(x,w)| \leq || V-W ||.$

The properties of monotonicity and $\beta$ -homogeneity, together with the above display, imply

\begin{equation*} T_{\beta}[W]- \beta||V-W|| \leq T_{\beta}[V] \leq T_{\beta}[W] + \beta ||V-W||,\end{equation*}

that is

\begin{equation*}||T_{\beta}[V]-T_{\beta}[W]|| \leq \beta ||V-W||.\end{equation*}

So, from Banach’s fixed point theorem (see [Reference Kirk23, Theorem 2.2]), there exists a unique function $V_{\beta} \in C(S \times [\alpha e^{-\lambda},1])$ such that $T_{\beta}[V_{\beta}]=V_{\beta}.$

Moreover, an easy computation shows that $|| T_{\beta}[0]|| \leq \ln \overline{K}^{\gamma}$ , and hence

\begin{align*}||V_{\beta}||-||T_{\beta}[0]|| &\leq ||V_{\beta}-T_{\beta}[0] ||\\[5pt]&=||T_{\beta}[V_{\beta}]-T_{\beta}[0] ||\\[5pt]& \leq \beta || V_{\beta}||,\end{align*}

from which we conclude that

\begin{align*}(1-\beta)||V_{\beta}|| &\leq || T_{\beta}[0] ||\\[5pt] &\leq \ln \overline{K}^{\gamma}.\end{align*}

3.2. Existence of a solution to the optimality inequality

Now, the discounted approach technique (see [Reference Cavazos-Cadena and Hernández-Hernández8] and [Reference Cavazos-Cadena and Salem-Silva9]) will be used to build a solution to the optimality inequality (3.1). The construction adopted in this paper is inspired by the one introduced by Cavazos-Cadena and Salem [Reference Cavazos-Cadena and Salem-Silva9] within the context of risk-sensitive optimal control. We begin by introducing the generalized superior limit of a sequence of functions.

Definition 5. For a sequence of functions $h_n\;:\; S \times [\alpha e^{-\lambda},1] \rightarrow [ - \infty,0],$ the generalized superior limit is defined as

\begin{align*}\tilde{h}(x,w)\;:\!=\;\sup\{r \;:\; r=\limsup_{n\to \infty} h_n(x,w_n), \ for \ some \ sequence \ w_n\ such \ that \ w_n \rightarrow w \},\end{align*}

and it is denoted by $\tilde{h}(x,w)=\mathfrak{g}\limsup h_n(x,w).$

The following property of $\tilde{h}$ will be needed in the proof of Proposition 1. Its proof can be obtained by adapting the arguments given in [Reference Jaśkiewicz and Nowak21, Lemma 3.1]; see also [Reference Cavazos-Cadena and Salem-Silva9, Lemma 4.2].

The family of fixed points obtained in Lemma 3, namely $\{V_{\beta}\;|\;\beta \in (0,1)\}$ , will be used to generate a real number $\widehat{g}$ and an upper semi-continuous function $\widehat{h},$ which together are natural candidates to be a solution to the optimality inequality (3.1).

Definition 6. For each $\beta \in (0,1),$ let $m_{\beta}\;:\!=\;\sup_{(x,w) \in S\times [\alpha e ^{-\lambda},1] }V_{\beta}(x,w).$ The relative rate $g_{\beta} \in [ - \ln \overline{K}, \ln \overline{K}]$ and the relative function $h_{\beta}\;:\; S\times [\alpha e ^{-\lambda},1] \rightarrow ]- \infty,0]$ are specified by

(3.10) \begin{align}g_{\beta}\;:\!=\;(1-\beta)m_{\beta} \ \ \textrm{and} \ \ h_{\beta}(x,w)\;:\!=\;V_{\beta}(x,w)-m_{\beta}, \ \text{for}\;\; (x,w).\end{align}

Note that from (3.10), Equation (3.9) can be written as

(3.11) \begin{align} e^{g_{\beta}+h_{\beta}(x,w)}=l_{h_{\beta}}^{\beta}(x,w,f_{\beta}(x,w)), \ \ \forall (x,w) \in S\times [\alpha e^{-\lambda},1].\end{align}

Now, select a sequence $\{\beta_n\}_{n=0}^{\infty} \,\subset ]0,1[$ such that the following two conditions are satisfied:

(3.12) \begin{align} \lim_{n\rightarrow \infty}\beta_{n}=1,\end{align}

and the corresponding sequence of relative rates $\{g_{\beta_{n}}\}_{n \in \mathbb{N}}$ converges, that is,

(3.13) \begin{align} \lim_{n \rightarrow \infty}g_{\beta_n}=\widehat{g}. \end{align}

The sequence $\{\beta_n\}_{n=0}^{\infty}$ satisfying (3.12) and (3.13) will be kept fixed unless otherwise stated. Thus, for the sequence $\{h_{\beta_n}\}_{n=1}^{\infty},$ its corresponding generalized superior limit is denoted by $\widehat{h}\;:\;S\times [\alpha e^{-\lambda},1] \rightarrow [ - \infty,0]$ ; that is,

(3.14) \begin{align}\widehat{h}(x,w)\;:\!=\;\mathfrak{g}\limsup h_{\beta_{n}}(x,w).\end{align}

In order to simplify the notation, we write $(g_{n}, h_n)$ instead of $(g_{\beta_n},h_{\beta_n})$ , and if $f_{\beta_n}$ satisfies (3.11), we use $f_n$ to denote $f_{\beta_n}.$

The following proposition states the existence of a solution to the optimality inequality and the existence of a measurable selector such that (3.2) holds. Hence, the first part of Theorem 1 is obtained from this result. For similar results for Markov decision processes, see [Reference Cavazos-Cadena and Salem-Silva9, Lemma 4.3], [Reference Schal30, Theorem 3.8], and [Reference Jaśkiewicz20, Theorem 1].

Proof. First, let us fix a point $(x,w) \in S \times [\alpha e^{-\lambda},1]$ . Following Definition 5, an arbitrary sequence $\{w_n\}_{n=1}^{\infty} \subset [\alpha e^{-\lambda},1]$ such that $\lim_{n \rightarrow \infty}w_n=w$ is chosen. From Equation (3.11), we can see that for each $n \in \mathbb{N},$

(3.15) \begin{align}e^{g_n+h_n(x,w_n)}=l_{h_n}^{\beta_n}(x,w_n,f_n(x,w_n)).\end{align}

Now we define the following compact set: $C\;:\!=\; \{w\} \cup \{w_n\}_{n=1}^{\infty}$ in $[\alpha e^{-\lambda},1].$ From continuity (or upper semi-continuity) of the function $w \mapsto A_{w},$ compactness of every $A_{w}$ , and Berge’s theorem (see [Reference Berge4] or [Reference Klein and Thomson24]), it follows that $\bigcup_{w \in C} A_{w}$ is a compact set in A. Then the sequence $\{f_n(x,w_n)\}_{n=1}^{\infty},$ has a sub-sequence which converges to $a^* \in A,$ and since the set-valued function $A_{w}$ is continuous (or upper semi-continuous), we have $a^* \in A_w.$ Without loss of generality, it can be assumed that

(3.16) \begin{align} \lim_{n\rightarrow \infty}f_n(x,w_n)=a^*.\end{align}

Using the estimate in (3.6), Fatou’s lemma and Lemma 1, together with (3.14) and (3.16), yield

\begin{align*} \limsup_{n\to\infty} l_{h_n}^{\beta_n}(x,w_n,f_n(x,w_n))&\leq \sum_{y \in S}\int_{B} e^{\limsup_{n\to \infty} h_n(y,G(w_n,f_n(x,w_n),z))}F^{\gamma}(a^*,z)v(dz|x,y)Q_{xy}\\[5pt]&\leq \sum_{y \in S}\int_{B} e^{\widehat{ h}(y,G(w,a^*,z))}F^{\gamma}(a^*,z)v(dz|x,y)Q_{xy}\\[5pt]&= l_{\widehat{h}}^1(x,w,a^*),\end{align*}

and hence

(3.17) \begin{align}\limsup_{n\to \infty} l_{h_n}^{\beta_n}(x,w_n,f_n(x,w_n)) \leq \tilde{l}_{\widehat{h}}^1(x,w).\end{align}

On the other hand, from (3.15) we see that

\begin{equation*}\limsup_{n\to \infty} l_{h_n}^{\beta_n}(x,w_n,f_n(x,w_n))=e^{\widehat{g}+\limsup_{n\to \infty} h_n(x,w_n)}.\end{equation*}

Moreover, by replacing this identity in (3.17), we can rewrite it as

\begin{equation*} e^{\widehat{g}+\limsup_{n\to \infty} h_n(x,w_n)} \leq \tilde{l}_{\widehat{h}}^1(x,w).\end{equation*}

Since the above display holds for any sequence $\{(x,w_n)\}_{n=1}^{\infty}$ converging to (x, w) for each $x\in S$ , by definition of $\widehat{h}$ in (3.14), and recalling Definition 5, it follows that there exists a pair $(\widehat{g},\widehat{h})$ such that

\begin{equation*}e^{\widehat{g}+\widehat{h}(x,w)} \leq \tilde{l}_{\widehat{h}}^1(x,w).\end{equation*}

Also, since $\widehat{h}$ is upper semi-continuous, by Lemma 4, and the set-valued function $A_w$ is continuous, the measurable selection theorem [Reference Himmelberg, Partasarathy and Van Vleck19, Theorem 2] implies that there exists a function $\widehat{f} \in \overline{\mathbb{F}}_{\alpha}$ such that

\begin{align*} \tilde{l}_{\widehat{h}}^1(x,w)=l_{\widehat{h}}^1(x,w,\widehat{f}(x,w)),\end{align*}

for $(x,w) \in S\times [\alpha e^{-\lambda},1].$

3.3. Verification results

The following two results complement the preliminaries needed to complete the proof of Theorem 1. Recall that one of the conclusions of that result shows that the optimal value function is constant on $\mathcal{H}=\{(x,w) \in S \times [\alpha e^{-\lambda},1] \ : \ h^{*}(x,w)>-\infty\}$ .

Lemma 5. Let $g^*,$ $h^*$ , and $f^*$ be as in (3.1)–(3.2). Given $(x,w)\in S \times [\alpha e^{-\lambda},1]$ , for each $t \in \mathbb{N},$

(3.18) \begin{align} V_0^{\gamma}e^{t g^*+h^*(x,w)}&\leq E_{x,w}^{f^*}[V_t^{\gamma}e^{h^*(X_t,W_t)}], \end{align}

where $\{V_{t}\}$ and $\{W_{t}\}$ are the processes associated to the stationary strategy $f^*\in \overline{\mathbb{F}}_{\alpha} $ .

As a consequence, if $(x,w) \in \mathcal{H},$ it follows that

\begin{equation*} \frac{g^*}{\gamma} \leq \widehat{J}_{\gamma}(x,w,f^{*}) \leq \widehat{J}_{\gamma}^{*}( x,w ).\end{equation*}

Proof. Given $(x,w) \in S \times [\alpha e^{-\lambda},1]$ arbitrary, from (3.2) we get that

\begin{align*}V_0^{\gamma}e^{g^*+h^*(x,w)}&\leq \sum_{y \in S}\int_{B}e^{h^*(y,G(w,f^*(x,w),z))}V_0^{\gamma}F^{\gamma}(f^*(x,w),z)v(dz|x,y)Q_{xy}\\[5pt]&= E_{x,w}^{f^*}[V_1^{\gamma}e^{h^*(X_1,W_1)}],\end{align*}

and therefore (3.18) holds for $t=1.$ Assuming that (3.18) holds for a positive integer t, again applying (3.2), it follows that

\begin{align*}V_0^{\gamma}e^{(t+1)g^*+h^*(x,w)} &\leq e^{g^*}E_{x,w}^{f^*}[V_t^{\gamma}e^{h^*(X_t,W_t)}]\\[5pt]&=\sum_{r_1 \in S} \int_{B}\cdots\sum_{r_{t}\in S}\int_{B}e^{g^*+h^*(r_{t},G(W_{t-1},f^*(r_{t-1},W_{t-1}),z_t))}V_0^{\gamma}\\[5pt]& \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ F^{\gamma}(f^*(r_{t-1},W_{t-1}),z_t)\cdots F^{\gamma}(f^*(x,w),z_1)\\[5pt]& \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ v(dz_t|r_{t-1},r_{t})Q_{r_{t-1}r_{t}}\cdots v(dz|x,r_1)Q_{xr_1}\\[5pt]&\leq \sum_{r_1 \in S} \int_{B}\cdots \sum_{r_t \in S}\int_{B}\sum_{r_{t+1} \in S}\int_{B}e^{h^*(r_{t+1},G(W_t,f^*(r_t,W_t),z_{t+1}))}V_0^{\gamma} \\[5pt]& \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ F^{\gamma}(f^*(r_t,W_t),z_{t+1}) \cdots F^{\gamma}(f^*(x,w),z_1)\\[5pt]& \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ v(dz_{t+1}|r_t,r_{t+1})Q_{r_tr_{t+1}}\cdots v(dz|x,r_1)Q_{xr_1}\\[5pt]&=E_{x,w}^{f^*}[V_{t+1}^{\gamma}e^{h^*(X_{t+1},W_{t+1})}].\end{align*}

The inequality (3.18) implies that

\begin{align*} V_0^{\gamma}e^{tg^*+h^*(x,w)}&\leq E_{x,w}^{f^*}[V_t^{\gamma}],\end{align*}

and hence, when $(x,w) \in S \times [\alpha e^{-\lambda},1]$ is such that $-\infty < h^*(x,w),$

\begin{align*} \frac{\ln V_0}{t} + \frac{g^*}{\gamma}+ \frac{h^*(x,w)}{\gamma t}&\leq\frac{1}{\gamma t} \ln E_{x,w}^{f^*}[V_t^{\gamma}].\end{align*}

By taking liminf as $t\to \infty$ in both sides of the above inequality, we get

\begin{align*}\frac{g^*}{\gamma} \leq \widehat{J}_{\gamma}(x,w,f^{*}) \leq \widehat{J}_{\gamma}^{*}( x,w ).\end{align*}

Lemma 6. Let $\beta \in \, ]0,1[,$ $\pi \in \Gamma_{\alpha}$ and $(x,w) \in S \times [\alpha e^{-\lambda},1]$ be arbitrary but fixed. Then, for each $t \in \mathbb{N},$

(3.19) \begin{align}e^{tg_{\beta}+\beta h_{\beta}(x,w)} &\geq E_{x,w}^{\pi}[V_t^{\gamma}e^{\beta h_{\beta}(X_t,W_t)}].\end{align}

Proof. The proof is by induction in the time variable t. First, note that Lemma 3(i) yields

(3.20) \begin{align} e^{g_{\beta}+\beta h_{\beta}(x,w)} \geq e^{g_{\beta}+h_{\beta}(x,w)} \end{align}

(3.21) \begin{align} =\tilde{l}_{h_{\beta}}^{\beta}(x,w),\end{align}

and hence (3.19) holds for $t=1.$ Assume now that (3.19) is satisfied for t, so we can proceed to verifying it for $t+1.$ Let $H_t=(x,w,a_1,X_1,W_1, \dots , W_{t-1},a_{t-1},X_t,W_t)$ be the regime-quotient history vector, for $\pi \in \Gamma_{\alpha}$ and $(x,w) \in S \times [\alpha e^{-\lambda},1]$ . Then, from (3.20), it follows that

\begin{align*}E_{x,w}^{\pi}[V_{t+1}^{\gamma}e^{\beta h_{\beta}(X_{t+1},W_{t+1})}|H_t,a_t]&=V_t^{\gamma}\sum_y\int_Be^{\beta h_{\beta}(y,G(W_t,a_t,z))}F(a_t,z)v(dz|X_t,y)Q_{X_ty}\\[5pt]&\leq V_t^{\gamma}e^{g_{\beta}+\beta h_{\beta}(X_t,W_t)}.\end{align*}

By the induction hypothesis, integrating both sides gives

\begin{align*}E_{x,w}^{\pi}[V_{t+1}^{\gamma}e^{\beta h_{\beta}(X_{t+1},W_{t+1})}]&=E_{x,w}^{\pi}[E_{x,w}^{\pi}[V_{t+1}^{\gamma}e^{\beta h_{\beta}(X_{t+1},W_{t+1})}|H_t,a_t]]\\[5pt]&\leq E_{x,w}^{\pi}[V_t^{\gamma}e^{g_{\beta}+\beta h_{\beta}(X_t,W_t)}]\\[5pt]&\leq e^{g_{\beta}}e^{tg_{\beta}+\beta h_{\beta}(x,w)}\\[5pt]&=e^{(t+1)g_{\beta}+\beta h_{\beta}(x,w)}.\end{align*}

We conclude this section by putting together the previous results to complete the proof of our main result.

Proof of Theorem 1. First, notice that Proposition 1 yields that the inequality (3.1) and the equality (3.2) hold if we take $\widehat{h}=h^*,$ $\widehat{g}=g^*$ , and $\widehat{f}=f^*.$

On the other hand, Lemma 6 implies that for each $\pi \in \Gamma_{\alpha}$ and $\beta \in \, ]0,1[$ ,

\begin{align*}\frac{g_{\beta}}{\gamma} + \frac{\beta (h_{\beta}(x,w)+||h_{\beta}||)}{t \gamma} \geq \frac{1}{t \gamma}\ln E_{xw}^{\pi}[V_t^{\gamma}],\end{align*}

and then, taking liminf as $t\to \infty$ in both sides, it follows that

\begin{align*} \frac{g_{\beta}}{\gamma} \geq \widehat{J}_{\gamma}^{*}( x,w )\end{align*}

for all $\beta \in \, ]0,1[$ . Recalling the definition of $\widehat{g}$ in (3.13), we conclude that the above inequality is valid when $g_{\beta}$ is replaced by $\widehat{g}$ . Now, when $(x,w) \in S \times [\alpha e^{-\lambda},1]$ is such that $-\infty < h^*(x,w),$ Lemma 5 and the above display imply

\begin{align*} \frac{g^*}{\gamma} \leq \widehat{J}_{\gamma}(x,w,f^{*}) \leq \widehat{J}_{\gamma}^{*}( x,w ) \leq \frac{g^*}{\gamma},\end{align*}

which completes the proof.

4.1. Optimal strategies for regular utility functions

The following proposition provides an inverse order relation on the family of certainty equivalent values; its proof can be found in [Reference Cavazos-Cadena and Hernández-Hernández7]. This property, together with Lemma 7 and Corollary 1, allows us to relate the optimal value functions $\widehat{J}_U^*$ and $\widehat{J}^*_{1-\lambda_{U}}$ ; the last one corresponds to the value function for the CRRA utility with parameter $\gamma\;:\!=\;1-\lambda_{U}$ studied in Section 3.

Proposition 2. Let $I=(a,b)$ be a nonempty interval contained in $(0,\infty),$ and let $U, V \in \mathcal{U}$ be arbitrary utility functions such that their corresponding Arrow–Pratt functions satisfy the relation $\Delta_U(x)\leq \Delta_V(x),$ for $x \in I.$ Further, consider a random variable Y taking values in I almost surely, such that U(Y) and V(Y) have finite expectations. Then

\begin{equation*} \Psi_{V}(Y) \leq \Psi_{U}(Y).\end{equation*}

Notice that for each $\pi \in \Gamma_{\alpha}$ and $\gamma \in \, ]0,1[$ , Assumption 1 implies that

(4.1) \begin{align}(\alpha M_0e^{\lambda (t-1)})^{\gamma} \leq (V_t^{\pi})^{\gamma}\leq (V_0 \overline{K}^t)^{\gamma},\end{align}

and hence, for $(x,w) \in S \times [\alpha e^{-\lambda},1]$ , it follows that $\lambda \leq \widehat{J}_{\gamma}^{*}( x,w ) \leq \ln \overline{K}.$

Proof. Given $(x,w) \in S \times [\alpha e^{-\lambda},1]$ and $\pi \in \Gamma_{\alpha},$ we choose $\gamma, \beta \in \, ]0,1[$ such that $\beta < \gamma$ . Then, using (4.1), we have that

(4.3) \begin{align}\mathbb{E}_{xw}^{\pi}[V_t^{\beta}](\alpha M_0e^{\lambda (t-1)})^{\gamma-\beta} \leq \mathbb{E}_{xw}^{\pi}[V_t^{\gamma}] \leq \mathbb{E}_{xw}^{\pi}[V_t^{\beta}](V_0\overline{K}^{ t})^{\gamma -\beta}.\end{align}

The left-hand side of the above display implies that

(4.4) \begin{align}(\gamma-\beta)\lambda + \beta \liminf_{t \rightarrow \infty} \frac{1}{t \beta } \ln \mathbb{E}_{xw}^{\pi}[V_t^{\beta }] & \leq \gamma \liminf_{t \rightarrow \infty} \frac{1}{t \gamma }\ln \mathbb{E}_{xw}^{\pi}[V_t^{\gamma}],\end{align}

and since $(\gamma-\beta)\lambda $ is positive, we have that $\beta\widehat{J}_{\beta }^{*}( x,w ) < \gamma \widehat{J}_{\gamma}^{*}( x,w ).$

In the same manner, from (4.3) we conclude that

(4.5) \begin{align} \gamma \liminf_{t \rightarrow \infty} \frac{1}{t \gamma } \ln \mathbb{E}_{xw}^{\pi}[V_t^{\gamma}] & \leq \beta \liminf_{t \rightarrow \infty} \frac{1}{t \beta }\ln \mathbb{E}_{xw}^{\pi}[V_t^{\beta}] + (\gamma-\beta)\ln \overline{K}.\end{align}

Hence (4.2) is obtained from (4.5), and finally from (4.2) it follows that

\begin{equation*}|\gamma\widehat{J}_{\gamma}^{*}( x,w ) - \beta\widehat{J}_{\beta }^{*}( x,w )| \leq |\gamma - \beta| \ln K.\end{equation*}

Notice that it is straightforward to verify the following claim, from (4.4) and (4.5).

Now we are ready to present the proof of the main result of this section.

Proof of Theorem 2. First, we take $\gamma\;:\!=\;1-\lambda_{U} .$ Hence, given $\varepsilon > 0$ small enough, from the definition of $\lambda_{U}$ , there exists $z>0$ such that

\begin{align*}1-(\gamma+\varepsilon) \leq \Delta_U(y)\leq 1-(\gamma - \varepsilon) \ \ \ \forall y > z,\end{align*}

which yields

\begin{align*}\Delta_{U_{\gamma +\varepsilon}}(y) \leq \Delta_U(y)\leq \Delta_{U_{\gamma - \varepsilon}}(y), \ \ \ \forall y > z.\end{align*}

Now, given $(x,w) \in S \times [\alpha e^{-\lambda},1]$ and $\pi \in \Gamma_{\alpha},$ we choose a positive integer $t^*$ such that $\alpha M_0e^{\lambda (t^*-1)} > z.$ Thus, from the drawdown constraint (2.2), it follows that $V_t^{\pi} > z$ for $t \geq t^*$ almost surely, and hence

\begin{align*}\Delta_{U_{\gamma+\varepsilon}}(V_t^{\pi}) \leq \Delta_U(V_t^{\pi})\leq \Delta_{U_{\gamma -\varepsilon}}(V_t^{\pi}).\end{align*}

Using Proposition 2, we get

(4.6) \begin{align}\frac{1}{t}\Psi^{\pi,x,w}_{U_{\gamma-\varepsilon}}(V_t^\pi) \leq \frac{1}{t}\Psi^{\pi,x,w}_U(V_t^\pi)\leq \frac{1}{t}\Psi^{\pi,x,w}_{U_{\gamma + \varepsilon}}(V_t^\pi),\end{align}

and therefore $\widehat{J}_{\gamma - \varepsilon}^{*}( x,w ) \leq \widehat{J}_{{U}}^{*}( x,w )$ and $\widehat{J}_{{U}}^{*}( x,w ) \leq \widehat{J}_{\gamma + \varepsilon}^{*}( x,w ).$ Letting $\varepsilon\to 0$ in the above inequality, Corollary 1 implies that

(4.7) \begin{align}\widehat{J}_{{U}}^{*}( x,w )=\widehat{J}_{\gamma}^{*}( x,w ).\end{align}

On the other hand, let $\pi^* \in \Gamma_{\alpha}$ be a strategy such that $\widehat{J}_{\gamma}^{*}( x,w )=\widehat{J}_{\gamma}(x,w,\pi^{*})$ . Replacing $\pi$ by $\pi^*$ in (4.6) and letting $\varepsilon\to0,$ from Corollary 1 and (4.7) we have that

\begin{align*}\widehat{J}_{U}(x,w,\pi^{*}) &= \widehat{J}_{\gamma}(x,w,\pi^{*})\\[5pt] &= \widehat{J}_{\gamma}^{*}( x,w ) \\[5pt] &= \widehat{J}_{{U}}^{*}( x,w ).\end{align*}

Now, if $\pi^{\star} \in \Gamma_{\alpha}$ is a strategy such that $\widehat{J}_{{U}}^{*}( x,w )=\widehat{J}_{U}(x,w,\pi^{\star}),$ following the same arguments given above, we conclude the proof.