Hostname: page-component-78c5997874-s2hrs Total loading time: 0 Render date: 2024-11-19T05:27:46.785Z Has data issue: false hasContentIssue false
  • English
  • Français

Controlled Markov set-chains with discounting

Published online by Cambridge University Press:  14 July 2016

Masami Kurano ,
Jinjie Song ,
Masanori Hosaka  and
Youqiang Huang
Masami Kurano*
Affiliation:
Chiba University
Jinjie Song*
Affiliation:
Chiba University
Masanori Hosaka*
Affiliation:
Chiba University
Youqiang Huang*
Affiliation:
Chiba University
*
∗Postal address: Faculty of Education, Chiba University, Yayoi-cho, Inage-ku, Chiba, Japan, 263.
∗Postal address: Faculty of Education, Chiba University, Yayoi-cho, Inage-ku, Chiba, Japan, 263.
∗Postal address: Faculty of Education, Chiba University, Yayoi-cho, Inage-ku, Chiba, Japan, 263.
∗Postal address: Faculty of Education, Chiba University, Yayoi-cho, Inage-ku, Chiba, Japan, 263.
Get access Rights & Permissions [Opens in a new window]

Abstract

In the framework of discounted Markov decision processes, we consider the case that the transition probability varies in some given domain at each time and its variation is unknown or unobservable.

To this end we introduce a new model, named controlled Markov set-chains, based on Markov set-chains, and discuss its optimization under some partial order.

Also, a numerical example is given to explain the theoretical results and the computation.

Keywords

Controlled Markov set-chainsdiscounted return criterioninterval arithmetic

MSC classification

Primary: 90C40: Markov and semi-Markov decision processes
Secondary: 90C39: Dynamic programming
Type
Research Papers
Information
Journal of Applied Probability , Volume 35 , Issue 2 , June 1998 , pp. 293 - 302
Copyright
Copyright © Applied Probability Trust 1998 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bertsekas, D.P., and Sheve, S.E. (1978). Stochastic Optimal Control: The Discrete Time Case. Academic Press, New York.Google Scholar
Blackwell, D. (1962). Discrete dynamic programming. Ann. Math. Statist. 33, 719–726.Google Scholar
Hartfiel, D.J. (1981). On the limiting set of stochastic products x A 1 … An . Proc. Amer. Math. Soc. 81, 201–206.Google Scholar
Hartfiel, D.J. (1991). Component bounds on Markov set-chain limiting sets. J. Statist. Comput. Simul. 38, 15–24.Google Scholar
Hartfiel, D.J. (1993). Cyclic Markov set-chains. J. Statist. Comput. Simul. 46, 145–167.Google Scholar
Hartfiel, D.J., and Seneta, E. (1994). On the theory of Markov set-chains. Adv. Appl. Prob. 26, 947–964.CrossRefGoogle Scholar
Hernandez-Lerma, O. (1989). Adaptive Markov Control Processes. Springer-Verlag, New York.Google Scholar
Hinderer, K. (1970). Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter. Lecture Notes. Oper. Res. Math. Systems 33. Springer-Verlag, New York.Google Scholar
Howard, R.A. (1960). Dynamic Programming and Markov Processes. MIT Press, Cambridge, MA.Google Scholar
Kumar, P.R., and Varaiya, P. (1986). Stochastic Systems: Estimation, Identification and Adaptive Control. Prentice-Hall, Englewood Cliffs, New Jersey.Google Scholar
Kuratowski, K. (1966). Topology. Academic Press, New York.Google Scholar
Neumaier, A. (1984). New techniques for the analysis of linear interval equations. Linear Algebra Appl. 58, 273–325.Google Scholar
Puterman, M.L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc.Google Scholar
White, D.J. (1993). Markov Decision Processes. Wiley, Chichester.Google Scholar