Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-10T22:50:40.593Z Has data issue: false hasContentIssue false
  • English
  • Français

The value of information and efficient switching in channel selection

Published online by Cambridge University Press:  25 August 2022

Jiesen Wang Open the ORCID record for Jiesen Wang [Opens in a new window] ,
Yoni Nazarathy  and
Thomas Taimre
Jiesen Wang
Affiliation:
School of Mathematics and Statistics, The University of Melbourne, Melbourne, VIC, Australia. E-mail: [email protected]
Yoni Nazarathy
Affiliation:
School of Mathematics and Physics, The University of Queensland, Saint Lucia, QLD, Australia
Thomas Taimre
Affiliation:
School of Mathematics and Physics, The University of Queensland, Saint Lucia, QLD, Australia
Get access Rights & Permissions [Opens in a new window]

Abstract

We consider a collection of statistically identical two-state continuous time Markov chains (channels). A controller continuously selects a channel with the view of maximizing infinite horizon average reward. A switching cost is paid upon channel changes. We consider two cases: full observation (all channels observed simultaneously) and partial observation (only the current channel observed). We analyze the difference in performance between these cases for various policies. For the partial observation case with two channels or an infinite number of channels, we explicitly characterize an optimal threshold for two sensible policies which we name “call-gapping” and “cool-off.” Our results present a qualitative view on the interaction of the number of channels, the available information, and the switching costs.

Keywords

Applied probabilityOperations researchStochastic modeling
Type
Research Article
Information
Probability in the Engineering and Informational Sciences , Volume 37 , Issue 4 , October 2023 , pp. 907 - 927
Copyright
© The University of Melbourne and The University of Queensland, 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aalto, S., Lassila, P., & Taboada, I. (2019). Indexability of an opportunistic scheduling problem with partial channel information. In Proceedings of the 12th EAI International Conference on Performance Evaluation Methodologies and Tools. Palma, Spain: Association for Computing Machinery, pp. 95–102.CrossRefGoogle Scholar
Aalto, S., Lassila, P., & Taboada, I. (2019). Whittle index approach to opportunistic scheduling with partial channel information. Performance Evaluation 136: 102052.CrossRefGoogle Scholar
Agrawal, R., Hegde, M., & Teneketzis, D. (1990). Multi-armed bandit problems with multiple plays and switching cost. Stochastics and Stochastic Reports 29(4): 437–459.Google Scholar
Asmussen, S. (2008). Applied probability and queues, vol. 51. New York, NY, US: Springer Science & Business Media.Google Scholar
Ayesta, U., Gupta, M.K., & Verloop, I.M. (2021). On the computation of Whittle's index for Markovian restless bandits. Mathematical Methods of Operations Research 93(1): 179–208.CrossRefGoogle Scholar
Banks, J.S. & Sundaram, R.K. (1994). Switching costs and the Gittins index. Econometrica: Journal of the Econometric Society 62(3): 687–694.CrossRefGoogle Scholar
Dusonchet, F. & Hongler, M.O. (2003). Optimal hysteresis for a class of deterministic deteriorating two-armed bandit problem with switching costs. Automatica 39(11): 1947–1955.CrossRefGoogle Scholar
Gittins, J., Glazebrook, K., & Weber, R. (2011). Multi-armed bandit allocation indices. New York, NY, US: John Wiley & Sons.CrossRefGoogle Scholar
Jacko, P. & Villar, S.S (2012). Opportunistic schedulers for optimal scheduling of flows in wireless systems with ARQ feedback. In 24th International Teletraffic Congress (ITC 24), Krakow, Poland: International Teletraffic Congress, pp. 1–8.Google Scholar
Kaza, K., Meshram, R., Mehta, V., & Merchant, S.N. (2019). Sequential decision making with limited observation capability: Application to wireless networks. IEEE Transactions on Cognitive Communications and Networking 5(2): 237–251.CrossRefGoogle Scholar
Kuhn, J. & Nazarathy, Y. (2015). Wireless channel selection with reward-observing restless multi-armed bandits. In R. Boucherie & N. van Dijk (eds), Markov decision processes in practice. Cham: Springer.Google Scholar
Larrañaga, M., Assaad, M., Destounis, A., & Paschos, G.S. (2017). Asymptotically optimal pilot allocation over Markovian fading channels. IEEE Transactions on Information Theory 64(7): 5395–5418.CrossRefGoogle Scholar
Larrnaaga, M., Ayesta, U., & Verloop, I.M. (2016). Dynamic control of birth-and-death restless bandits: Application to resource-allocation problems. IEEE/ACM Transactions on Networking 24(6): 3812–3825.CrossRefGoogle Scholar
Lin, K.Y. & Ross, S.M. (2004). Optimal admission control for a single-server loss queue. Journal of Applied Probability 41(2): 535–546.CrossRefGoogle Scholar
Maatouk, A., Kriouile, S., Assad, M., & Ephremides, A. (2020). On the optimality of the Whittle's index policy for minimizing the age of information. IEEE Transactions on Wireless Communications 20(2): 1263–1277.CrossRefGoogle Scholar
Meshram, R. & Kaza, K. (2021). Indexability and rollout policy for multi-state partially observable restless bandits. In 60th IEEE Conference on Decision and Control (CDC). Athens, Greece: Institute of Electrical and Electronics Engineers (IEEE), pp. 2342–2347.CrossRefGoogle Scholar
Mezzavilla, M., Goyal, S., Panwar, S., Rangan, S., & Zorzi, M. (2016). An MDP model for optimal handover decisions in mmWave cellular networks. In European Conference on Networks and Communications (EuCNC). Austin, Texas, USA: Institute of Electrical and Electronics Engineers (IEEE), pp. 100–105.CrossRefGoogle Scholar
Wang, M., Chen, J., Aryafar, E., & Chiang, M. (2016). A survey of client-controlled HetNets for 5G. IEEE Access 5: 2842–2854.CrossRefGoogle Scholar
Wang, J., Nazarathy, Y., & Taimre, T. (2020). The value of information and efficient switching in channel selection – Github. https://github.com/yoninazarathy/ValueOfInformationAndEfficientSwitchingGoogle Scholar
Weber, R.R. & Weiss, G. (1990). On an index policy for restless bandits. Journal of Applied Probability 27(3): 637–648.CrossRefGoogle Scholar
Whittle, P. (1988). Restless bandits: Activity allocation in a changing world. Journal of Applied Probability 25A: 287–298.CrossRefGoogle Scholar