Hostname: page-component-745bb68f8f-d8cs5 Total loading time: 0 Render date: 2025-01-09T14:34:25.005Z Has data issue: false hasContentIssue false
  • English
  • Français

Optimal Occupation in the Complete Graph

Published online by Cambridge University Press:  27 July 2009

Kyle Siegrist
Kyle Siegrist
Affiliation:
Department of Mathematical Sciences, University of Alabama in HuntsvilleHuntsville, Alabama 35899
Get access Rights & Permissions [Opens in a new window]

Abstract

We consider N sites (N ≤ ∞), each of which may be either occupied or unoccupied. Time is discrete, and at each time unit a set of occupied sites may attempt to capture a previously unoccupied site. The attempt will be successful with a probability that depends on the number of sites making the attempt, in which case the new site will also be occupied. A benefit is gained when new sites are occupied, but capture attempts are costly. The problem of optimal occupation is formulated as a Markov decision process in which the admissible actions are occupation strategies and the cost is a function of the strategy and the number of occupied sites. A partial order on the state-action pairs is used to obtain a comparison result for stationary policies and qualitative results concerning monotonicity of the value function for the n-stage problem (n ≤ ∞). The optimal policies are partially characterized when the cost depends on the action only through the total number of occupation attempts made.

Type
Research Article
Information
Probability in the Engineering and Informational Sciences , Volume 7 , Issue 3 , July 1993 , pp. 369 - 385
Copyright
Copyright © Cambridge University Press 1993

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1.Andrews, G. (1976). The theory of partitions. Reading, MA: Addison-Wesley.Google Scholar
2.Freige, U., Peleg, D., Raghavan, P., & Upfal, E. (1990). Randomized broadcast in networks. Random Structures & Algorithms 1: 447–460.Google Scholar
3.Hedetniemi, S.M., Hedetniemi, S.T., & Liestman, A.L. (1988). A survey of gossiping and broadcasting in communication networks. Networks 18: 319–349.CrossRefGoogle Scholar
4.Hinderer, K. (1982). On the structure of solutions of stochastic dynamic programs. Bericht Nr. 20, Universität Karlsruhe, Fakultät für Mathematik.Google Scholar
5.Kamae, T., Krengel, U., & O'Brien, G. (1977). Stochastic inequalities on partially ordered spaces. Annals of Probability 5: 899–912.CrossRefGoogle Scholar
6.Marshall, A.W. & Olkin, I. (1979). Inequalities: Theory of majorization and its applications. New York: Academic Press.Google Scholar
7.Ross, S. (1983). Introduction to stochastic dynamic programming. San Diego: Academic Press.Google Scholar
8.Serfozo, R. (1981). Optimal control of random walks, birth and death processes, and queues. Advances in Applied Probability 13: 61–83.CrossRefGoogle Scholar
9.Topkis, D. (1978). Minimizing a submodular function on a lattice. Operations Research 2: 305–321.CrossRefGoogle Scholar
10.White, C.C. (1980). The optimality of isotone strategies for Markov decision problems with utility criterion. In Hartley, R., Thomas, L.C., & White, D.J. (eds.), Recent developments in Markov decision processes. San Diego: Academic Press, pp. 261–277.Google Scholar
11.White, D.J. (1981). Isotone optimal policies for structured Markov decision processes. European Journal of Operations Research 7: 396–402.CrossRefGoogle Scholar
12.White, D.J. (1982). Negatively isotone optimal policies for random walk type Markov decision processes. O.R. Spektrum 4: 41–45.CrossRefGoogle Scholar
13.White, D.J. (1984). Isotone policies for the value iteration method for Markov decision processes. OR Spektrum 6: 223–227.CrossRefGoogle Scholar