Hostname: page-component-745bb68f8f-kw2vx Total loading time: 0 Render date: 2025-01-11T03:32:45.989Z Has data issue: false hasContentIssue false
  • English
  • Français

Optimal group testing

Part of: Theory of data Graph theory Probability theory on algebraic and topological structures

Published online by Cambridge University Press:  28 January 2021

Amin Coja-Oghlan ,
Oliver Gebhard ,
Max Hahn-Klimroth  and
Philipp Loick
Amin Coja-Oghlan*
Affiliation:
Goethe University Frankfurt, Robert-Mayer-Strasse 6–10, 60325 Frankfurt, Germany
Oliver Gebhard
Affiliation:
Goethe University Frankfurt, Robert-Mayer-Strasse 6–10, 60325 Frankfurt, Germany
Max Hahn-Klimroth
Affiliation:
Goethe University Frankfurt, Robert-Mayer-Strasse 6–10, 60325 Frankfurt, Germany
Philipp Loick
Affiliation:
Goethe University Frankfurt, Robert-Mayer-Strasse 6–10, 60325 Frankfurt, Germany
*
*Corresponding author. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

In the group testing problem the aim is to identify a small set of k ⁓ nθ infected individuals out of a population size n, 0 < θ < 1. We avail ourselves of a test procedure capable of testing groups of individuals, with the test returning a positive result if and only if at least one individual in the group is infected. The aim is to devise a test design with as few tests as possible so that the set of infected individuals can be identified correctly with high probability. We establish an explicit sharp information-theoretic/algorithmic phase transition minf for non-adaptive group testing, where all tests are conducted in parallel. Thus with more than minf tests the infected individuals can be identified in polynomial time with high probability, while learning the set of infected individuals is information-theoretically impossible with fewer tests. In addition, we develop an optimal adaptive scheme where the tests are conducted in two stages.

MSC classification

Primary: 05C80: Random graphs
Secondary: 60B20: Random matrices (probabilistic aspects; for algebraic aspects see ) 68P30: Coding and information theory (compaction, compression, models of communication, encoding schemes, etc.)
Type
Paper
Information
Combinatorics, Probability and Computing , Volume 30 , Issue 6 , November 2021 , pp. 811 - 848
Check for updates
Copyright
© The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

†

Supported by DFG CO 646/3 and Stiftung Polytechnische Gesellschaft. An extended abstract version of this work appeared in the proceedings of the COLT 2020 conference (Proc. Mach. Learning Res. 125 (2020) 1374–1388).

References

Abbe, E. (2017) Community detection and stochastic block models: recent developments. J. Mach. Learning Res. 18 6446–6531.Google Scholar
Alaoui, A., Ramdas, A., Krzakala, F., Zdeborová, L. and Jordan, M. (2019) Decoding from pooled data: phase transitions of message passing. IEEE Trans. Inform. Theory 65 572–585.Google Scholar
Alaoui, A., Ramdas, A., Krzakala, F., Zdeborová, L. and Jordan, M. (2019) Decoding from pooled data: sharp information-theoretic bounds. SIAM J. Math. Data Sci. 1 161–188.CrossRefGoogle Scholar
Aldridge, M. (2019) Individual testing is optimal for nonadaptive group testing in the linear regime. IEEE Trans. Inform. Theory 65 2058–2061.Google Scholar
Aldridge, M., Baldassini, L. and Johnson, O. (2014) Group testing algorithms: bounds and simulations. IEEE Trans. Inform. Theory 60 3671–3687.CrossRefGoogle Scholar
Aldridge, M., Johnson, O. and Scarlett, J. (2019) Group testing: an information theory perspective. Found. Trends Commun. Inform. Theory 15 196–392.CrossRefGoogle Scholar
Alon, N., Krivelevich, M. and Sudakov, B. (1998) Finding a large hidden clique in a random graph. In Proceedings of the Ninth Annual ACM–SIAM Symposium on Discrete Algorithms (SODA ’98), pp. 594–598.3.0.CO;2-W>CrossRefGoogle Scholar
Arıkan, E. (2009) Channel polarization: a method for constructing capacity-achieving codes for symmetric binary-input memoryless channels. IEEE Trans. Inform. Theory 55 3051–3073CrossRefGoogle Scholar
Berger, T. and Levenshtein, V. (2002) Asymptotic efficiency of two-stage disjunctive testing. IEEE Trans. Inform. Theory 48 1741–1749.CrossRefGoogle Scholar
Brennan, M. and Bresler, G. (2019) Optimal average-case reductions to sparse PCA: from weak assumptions to strong hardness. Proc. Mach. Learning Res. 99 469–470Google Scholar
Chen, H. and Hwang, F. (2008) A survey on nonadaptive group testing algorithms through the angle of decoding. J. Combin. Optim. 15 49–59.CrossRefGoogle Scholar
Coja-Oghlan, A., Gebhard, O., Hahn-Klimroth, M. and Loick, P. (2019) Information-theoretic and algorithmic thresholds for group testing. In 46th International Colloquium on Automata, Languages, and Programming (ICALP 2019), #43. Schloss Dagstuhl–Leibniz-Zentrum für Informatik.Google Scholar
Decelle, A., Krzakala, F., Moore, C. and Zdeborová, L. (2011) Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Phys. Rev. E 84 066106.CrossRefGoogle ScholarPubMed
Donoho, D. (2006) Compressed sensing. IEEE Trans. Inform. Theory 52 1289–1306.CrossRefGoogle Scholar
Donoho, D., Javanmard, A. and Montanari, A. (2013) Information-theoretically optimal compressed sensing via spatial coupling and approximate message passing. IEEE Trans. Inform. Theory 59 7434–7464.CrossRefGoogle Scholar
Dorfman, R. (1943) The detection of defective members of large populations. Ann. Math. Statist. 14 436–440.CrossRefGoogle Scholar
D’yachkov, A. and Rykov, V. (1982) Bounds on the length of disjunctive codes. Problemy Peredachi Informatsii 18 166–171.Google Scholar
Erdös, P. and Rényi, A. (1963) On two problems of information theory. Magyar Tud. Akad. Mat. Kutató Int. Közl 8 229–243.Google Scholar
Felstrom, A. and Zigangirov, K. (1999) Time-varying periodic convolutional codes with low-density parity-check matrix. IEEE Trans. Inform. Theory 45 2181–2191.CrossRefGoogle Scholar
Grötschel, M., Lovász, L. and Schrijver, A. (1988) The Ellipsoid Method and Combinatorial Optimization. Springer.Google Scholar
Hoeffding, W. (1994) Probability inequalities for sums of bounded random variables. In The Collected Works of Wassily Hoeffding (Fisher, N. and Sen, P., eds), Springer Series in Statistics (Perspectives in Statistics), pp. 409–426. Springer.Google Scholar
Hwang, F. (1972) A method for detecting all defective members in a population by group testing. J. Amer. Statist. Assoc. 67 605–608.CrossRefGoogle Scholar
Janson, S., Łuczak, T. and Ruciński, A. (2011) Random Graphs. Wiley.Google Scholar
Johnson, O., Aldridge, M. and Scarlett, J. (2018) Performance of group testing algorithms with near-constant tests per item. IEEE Trans. Inform. Theory 65 707–723.Google Scholar
Kautz, W. and Singleton, R. (1964) Nonrandom binary superimposed codes. IEEE Trans. Inform. Theory 10 363–377.CrossRefGoogle Scholar
Krzakala, F., Mézard, M., Sausset, F., Sun, Y. and Zdeborová, L. (2012) Statistical-physics-based reconstruction in compressed sensing. Phys. Rev. X 2 021005.Google Scholar
Kudekar, S. and Pfister, H. D. (2010) The effect of spatial coupling on compressive sensing. In 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Allerton, IL, 2010, pp. 347–353.CrossRefGoogle Scholar
Kudekar, S., Richardson, T. and Urbanke, R. (2011) Threshold saturation via spatial coupling: why convolutional LDPC ensembles perform so well over the BEC. IEEE Trans. Inform. Theory 57 803–834.CrossRefGoogle Scholar
Kudekar, S., Richardson, T. and Urbanke, R. (2013) Spatially coupled ensembles universally achieve capacity under belief propagation. IEEE Trans. Inform. Theory 59 7761–7813.CrossRefGoogle Scholar
Kwang-Ming, H. and Ding-Zhu, D. (2006) Pooling Designs and Nonadaptive Group Testing: Important Tools for DNA Sequencing. World Scientific.Google Scholar
Mézard, M., Tarzia, M. and Toninelli, C. (2008) Group testing with random pools: phase transitions and optimal strategy. J. Statist. Phys. 131 783–801.CrossRefGoogle Scholar
Moore, C. (2017) The computer science and physics of community detection: landscapes, phase transitions, and hardness. Bull. EATCS 121.Google Scholar
Reeves, G. and Pfister, H. (2019) Understanding phase transitions via mutual information and MMSE. arXiv:1907.02095Google Scholar
Scarlett, J. (2018) Noisy adaptive group testing: bounds and algorithms. IEEE Trans. Inform. Theory 65 3646–3661.Google Scholar
Scarlett, J. (2019) An efficient algorithm for capacity-approaching noisy adaptive group testing. In 2019 IEEE International Symposium on Information Theory (ISIT), pp. 2679–2683. IEEE.CrossRefGoogle Scholar
Sharma, A. and Salim, M. (2017) Polar code: the channel code contender for 5G scenarios. In 2017 International Conference on Computer, Communications and Electronics (Comptelix), pp. 676–682. IEEE.CrossRefGoogle Scholar
Takeuchi, K., Tanaka, T. and Kawabata, T. (2011) Improvement of BP-based CDMA multiuser detection by spatial coupling. In 2011 IEEE International Symposium on Information Theory, pp. 1489–1493. IEEE.Google Scholar
Ungar, P. (1960) The cutoff point for group testing. Commun. Pure Appl. Math. 13 49–54.Google Scholar
Wang, L., Li, X., Zhang, Y. and Zhang, K. (2011) Evolution of scaling emergence in large-scale spatial epidemic spreading. PLoS ONE 6 e21197.CrossRefGoogle Scholar
Wu, Y. and Verdú, S. (2010) Rényi information dimension: fundamental limits of almost lossless analog compression. IEEE Trans. Inform. Theory 56 3721–3748.Google Scholar
Zdeborová, L. and Krzakala, F. (2016) Statistical physics of inference: thresholds and algorithms. Adv. Phys. 65 453–552.Google Scholar