We consider a production-inventory model operating in a stochastic environment that is modulated by a finite state continuous-time Markov chain. When the inventory level reaches zero, an order is placed from an external supplier. The costs (purchasing and holding costs) are modulated by the state at the order epoch time. Applying a matrix analytic approach, fluid flow techniques, and martingales, we develop methods to obtain explicit equations for these cost functionals in the discounted case and under the long-run average criterion. Finally, we extend the model to allow backlogging.

The state of a patient is an important concept in biomedical sciences. While analytical methods for predicting and exploring treatment strategies of disease dynamics have proven to have useful applications in public health policy and planning, the state of a patient has attracted less attention, at least mathematically. As a result, models constructed in relation to treatment strategies may not be very informative. We derive a patient-dependent parameter from an age-physiology dependent population model, and show that a single treatment strategy is not always optimal. Also, we derive a function which increases with the patient dependence parameter and describes the effort expended to be in good health.

The resolvent operators of the following two processes are obtained: (a) the bivariate Markov process W = (X, Y), where Y(t) is an irreducible Markov chain and X(t) is its integral, and (b) the geometric Wiener process G(t) = exp{B(t} where B(t) is a Wiener process with non-negative drift μ and variance parameter σ2. These results are then used via a limiting procedure to determine the long-run average cost per unit time of operating a finite dam where the input process is either X(t) or G(t). The system is controlled by a policy (Attia [1], Lam [6]).

The problem of optimal control of a finite dam in the class of policies has been considered by Lam Yeh [6], [7]. In this paper, by using the first Dynkin formula, the same problems of specifying an optimal policy in the class of the policies to minimize the expected total discounted cost as well as the long-run average cost are considered. Both the expected total discounted cost and long-run average cost are determined explicitly, and then the optimal policy can be found numerically, Also, we obtain the transition density function and the resolvent operator of a reflecting Wiener process.