Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-6bf8c574d5-9f2xs Total loading time: 0 Render date: 2025-03-09T22:57:16.201Z Has data issue: false hasContentIssue false

19 - Probability sampling methods for hard-to-sample populations

Published online by Cambridge University Press:  05 September 2014

By
Graham Kalton
Edited by
Roger Tourangeau ,
Brad Edwards ,
Timothy P. Johnson ,
Kirk M. Wolter  and
Nancy Bates
Graham Kalton
Affiliation:
Westat
Roger Tourangeau
Affiliation:
Westat Research Organisation, Maryland
Brad Edwards
Affiliation:
Westat Research Organisation, Maryland
Timothy P. Johnson
Affiliation:
University of Illinois, Chicago
Kirk M. Wolter
Affiliation:
University of Chicago
Nancy Bates
Affiliation:
US Census Bureau
Get access

Summary

Introduction

There are no universally accepted definitions of such terms as “hard-to-reach,” “elusive,” and even “mobile” populations, although all are well recognized as populations that are difficult to sample (see Kish, 1991, for a taxonomy of elusive populations). A common but not universal characteristic of these populations is that they are rare populations for which no separate sampling frames exist. The prime focus of this chapter is on sampling these types of populations using probability sampling methods. Rather than attempt a formal classification, I will present a range of illustrative examples:

  • Blacks, Hispanics, American Indians/Alaska Natives, Vietnamese immigrants;

  • Religious groups;

  • Abused children, abused adults;

  • Persons with AIDS, rare forms of cancer, dementia;

  • Street prostitutes;

  • Men who have sex with men (MSM), lesbians;

  • Users of illicit drugs;

  • Disabled scientists;

  • Pregnant women;

  • Children in poverty, the very rich;

  • Homeless persons, nomads;

  • International travelers, visitors to museums or shopping malls; and

  • Drivers and passengers of cars in transit.

These examples are mostly drawn from other reviews such as Christman (2009), Kalton (1991, 2009), Kalton and Anderson (1986), Kish (1965a, 1991), Sudman and Freeman (1988), and Sudman and Kalton (1986). Flores Cervantes and Kalton (2008) present a review of the sampling methods for such populations in the context of telephone surveys and Elliott, Finch, Klein, Ma, Do, Beckett et al. (2008) review sampling methods for the widely encountered need to sample rare race/ethnic populations.

There are different analytic objectives for surveys of this diverse set of populations. With static populations, a main distinction is between surveys aiming to estimate the size of the population and its prevalence in the general population and surveys aiming to study the characteristics of the population. For example, is the main survey objective to estimate the proportion of children in poverty or the health, education, and housing conditions of poor children? With mobile populations, a key issue is whether the population of inference is the “visit” or the “visitor.” For example, is the population of inference the trips made by international travelers or the persons who engage in international travel?

Type
Chapter
Information
Hard-to-Survey Populations , pp. 401 - 423
Publisher: Cambridge University Press
Print publication year: 2014

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Anderson, D. W., & Kalton, G. (1990). Case-finding strategies for studying rare chronic diseases. Statistica Applicata, 2, 309–21.Google Scholar
Ardilly, P., & Le Blanc, D. (2001). Sampling and weighting a survey of homeless persons: a French example. Survey Methodology, 27, 109–18.Google Scholar
Bennett, S. (1993). Cluster sampling to assess immunization: a critical appraisal. Bulletin of the International Statistical Institute, 49th Session, 55(2), 21–35.Google Scholar
Bolling, K., Grant, C., & Sinclair, P. (2008). 2006–07 British Crime Survey (England and Wales). Technical Report. Volume 1. London: TNS BMRB. Retrieved from .Google Scholar
Brock-Roth, S., Han, D., & Montaquila, J. M. (2012). The ABS frame: quality and considerations. In Joint Statistical Meetings Proceedings, Survey Research Methods Section [pp. 3779–93]. Alexandria, VA: American Statistical Association.Google Scholar
Callegaro, M., & DiSogra, C. (2008). Computing response metrics for online panels. Public Opinion Quarterly, 72, 1008–32.CrossRefGoogle Scholar
Camburn, D. P., & Wright, R. A. (1996). Predicting Eligibility Rates for Rare Populations in RDD Screening Surveys. Paper presented at the American Association for Public Opinion Research Conference, Salt Lake City, UT. Retrieved from .
CentERdata (2013). Longitudinal Internet Studies for the Social Sciences (LISS) panel. Retrieved from .
Christman, M. C. (2009). Sampling of rare populations. In Pfeffermann, D. & Rao, C. R. (eds.), Handbook of Statistics (Vol. 29A). Sample Surveys: Design, Methods and Applications (pp. 109–24). Amsterdam: Elsevier.CrossRefGoogle Scholar
Citro, C. F., & Kalton, G. (eds.) (2007). Using the American Community Survey: Benefits and Challenges. Washington, DC: National Academies Press.
Clark, R. G. (2009). Sampling of subpopulations in two-stage surveys. Statistics in Medicine, 28, 3697–717.CrossRefGoogle ScholarPubMed
Cohen, S. B., DiGaetano, R., & Waksberg, J. (1988). Sample design of the NMES Survey of American Indians and Alaska Natives. In Joint Statistical Meetings Proceedings, Survey Research Methods Section (pp. 740–45). Alexandria, VA: American Statistical Association.Google Scholar
Cox, D. R. (1969). Some sampling problems in technology. In Johnson, N. L. & Smith, H. (eds.), New Developments in Survey Sampling (pp. 506–27). New York: John Wiley & Sons.Google Scholar
Deming, W. E. (1977). An essay on screening, or on two-phase sampling, applied to surveys of a community. International Statistical Review, 45, 29–37.CrossRefGoogle Scholar
DiSogra, C., Dennis, J. M., & Fahimi, M. (2010). On the quality of ancilliary data available for address-based sampling. In Joint Statistical Meetings Proceedings, Survey Research Methods Section (pp. 4174–83). Alexandria, VA: American Statistical Association.Google Scholar
Elliott, M. N., Finch, B. K., Klein, D., Ma, S., Do, D. P., Beckett, M. K., et al. (2008). Sample designs for measuring the health of small racial/ethnic subgroups. Statistics in Medicine, 27, 4016–29.CrossRefGoogle ScholarPubMed
Elliott, M. N., McCaffrey, D., Perlman, J., Marshall, G. N., & Hambarsoomians, K. (2009b). Use of expert ratings as sampling strata for a more cost-effective probability sample of a rare population. Public Opinion Quarterly, 73, 56–73.CrossRefGoogle ScholarPubMed
Elliott, M. N., Morrison, P. A., Fremont, A., McCaffrey, D. F., Pantoja, P., & Lurie, N. (2009a). Using the Census Bureau’s surname list to improve estimates of race/ethnicity and associated disparities. Health Services Outcomes Research Methods, 9, 69–83.CrossRefGoogle Scholar
Fecso, R. S., Baskin, R., Chu, A., Gray, C., Kalton, G., & Phelps, R. (2007). Design Options for SESTAT for the Current Decade: Statistical Issues. Working Paper SRS 07–201. Arlington, VA: National Science Foundation, Division of Science Resources Statistics.Google Scholar
Fiscella, K., & Fremont, A. M. (2006). Use of geocoding and surname analysis to estimate race and ethnicity. Health Services Research, 41, 1482–500.Google ScholarPubMed
Flores Cervantes, I., & Kalton, G. (2008). Methods for sampling rare populations in telephone surveys. In Lepkowski, J. M., Tucker, C., Brick, J. M., de Leeuw, E. D., Japec, L., Lavrakas, P. et al. (eds.), Advances in Telephone Survey Methodology (pp. 113–32). Hoboken, NJ: John Wiley & Sons.Google Scholar
Fluss, R., Mandel, M., Freedman, L. S., Weiss, I. S., Zohar, A. E., Haklai, Z., et al. (2013). Correcting of sampling bias in a cross-sectional study of post-surgical complications. [Published online September 4, 2012]. Statistics in Medicine, 32, 2467–78.CrossRefGoogle Scholar
Gile, K. J., & Handcock, M. S. (2010). Respondent-driven sampling: an assessment of current methodology. Sociological Methodology, 40, 285–327.CrossRefGoogle ScholarPubMed
Gonzalez, J. F., Ezzati, T. M., White, A. A., Massey, J. T., Lago, J., & Waksberg, J. (1985). Sample design and estimation procedures. In Maurer, K. R. (ed.), Plan and Operation of the Hispanic Health and Nutrition Examination Survey, 1982–84 (pp. 23–32). Vital and Health Statistics 1(19). Washington, DC: US Government Printing Office.Google Scholar
Haerer, A. F., Anderson, D. W., & Schoenberg, B. S. (1986). Prevalence and clinical features of epilepsy in a biracial United States population. Epilepsia, 27, 66–75.CrossRefGoogle Scholar
Hartley, H. O. (1974). Multiple frame methodology and selected applications. Sankhyã, 36, 99–118.Google Scholar
Heckathorn, D. D. (1997). Respondent driven sampling: a new approach to the study of hidden populations. Social Problems, 44, 174–99.CrossRefGoogle Scholar
Horrigan, M., Moore, W., Pedlow, S., & Wolter, K. (1999). Undercoverage in a large national screening survey for youths. In Joint Statistical Meetings Proceedings, Survey Research Methods Section (pp. 570–75). Alexandria, VA: American Statistical Association.Google Scholar
Hubble, D. L. & Brenner, R. A. (2012). Sample design issues in the National Children’s Study. In Joint Statistical Meetings Proceedings, Survey Research Methods Section [CD-ROM]. Alexandria, VA: American Statistical Association.Google Scholar
Judkins, D., DiGaetano, R., Chu, A., & Shapiro, G. (1999). Coverage in screening surveys at Westat. In Joint Statistical Meetings Proceedings, Survey Research Methods Section (pp. 581–86). Alexandria, VA: American Statistical Association.Google Scholar
Kalsbeek, W. D. (2003). Sampling minority groups in health surveys. Statistics in Medicine, 22, 1527–49.CrossRefGoogle ScholarPubMed
Kalsbeek, W. D., & Cross, A. R. (1982). Problems in sampling nomadic populations. In Joint Statistical Meetings Proceedings, Survey Research Methods Section (pp. 398–402). Alexandria, VA: American Statistical Association.Google Scholar
Kalton, G. (1991). Sampling flows of mobile populations. Survey Methodology, 17, 183–94.Google Scholar
Kalton, G. (1993a). Sampling Rare and Elusive Populations. New York: Department for Economic and Social Information and Policy Analysis, United Nations.Google Scholar
Kalton, G.(1993b). Sampling considerations in research on HIV risk and illness. In Ostrow, D. G. & Kessler, R. C. (eds.), Methodological Issues in AIDS Behavioral Research (pp. 53–74). New York: Plenum Press.Google Scholar
Kalton, G. (2003). Practical methods for sampling rare and mobile populations. Statistics in Transition, 6, 491–501.Google Scholar
Kalton, G. (2009). Methods for oversampling rare subpopulations in social surveys. Survey Methodology, 35, 125–41.Google Scholar
Kalton, G., & Anderson, D. W. (1986). Sampling rare populations. Journal of the Royal Statistical Society: Series A (General), 149, 65–82.CrossRefGoogle Scholar
Kish, L. (1965a). Selection techniques for rare traits. In Neel, J. V., Shaw, M. W., & Schull, W. J. (eds.), Genetics and the Epidemiology of Chronic Diseases. Public Health Service Publication No. 1163. Washington, DC: US Government Printing Office.Google Scholar
Kish, L. (1965b). Survey sampling. New York: John Wiley & Sons.Google Scholar
Kish, L. (1991). Taxonomy of elusive populations. Journal of Official Statistics, 7, 339–47.Google Scholar
Kish, L. (1999). Cumulating/combining population surveys. Survey Methodology, 25, 129–38.Google Scholar
Knowledge Networks. (2013). KnowledgePanel Latino. Retrieved from .
Körner, T., & Nimmergut, A. (2004). Using an access panel as a sampling frame for voluntary household surveys. Statistical Journal of the United Nations Economic Commission for Europe, 21, 33–52.Google Scholar
Langa, K. M., Plassman, B. L., Wallace, R. B., Herzog, A. R., Heeringa, S. G., Ofstedal, M. B., et al. (2005). The Aging, Demographics, and Memory Study: study design and methods. Neuroepidemiology, 25, 181–91.CrossRefGoogle ScholarPubMed
Lohr, S. L. (2009). Multiple-frame surveys. In Pfeffermann, D. & Rao, C. R. (eds.), Handbook of Statistics (Vol. 29A). Sample Surveys: Design, Methods, and Applications (pp. 71–88). Amsterdam: Elsevier.CrossRefGoogle Scholar
MacKellar, D., Valleroy, L., Karon, J., Lemp, G., & Janssen, R. (1996). The Young Men’s Survey: methods for estimating HIV seroprevalence and risk factors among young men who have sex with men. Public Health Reports, 111(Suppl. 1), 138–44.Google Scholar
McKenzie, D. J., & Mistiaen, J. (2009). Surveying migrant households: a comparison of census-based, snowball and intercept point surveys. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172, 339–60.CrossRefGoogle Scholar
Meccati, F. (2004). Center sampling: a strategy for sampling difficult-to-sample populations. In Proceedings of Statistics Canada Symposium: Innovative Methods for Surveying Difficult-to Reach Populations. Retrieved from .
Montaquila, J. M., Brick, J. M., & Curtin, L. R. (2010). Statistical and practical issues in the design of a national probability sample of births for the Vanguard Study of the National Children’s Study. Statistics in Medicine, 29, 1368–76.CrossRefGoogle ScholarPubMed
Rothbart, G. S., Fine, M., & Sudman, S. (1982). On finding and interviewing the needles in the haystack: the use of multiplicity sampling. Public Opinion Quarterly, 46, 408–21.CrossRefGoogle Scholar
Salganik, M. J., & Heckathorn, D. D. (2004). Sampling and estimation in hidden populations using respondent-driven sampling. Sociological Methodology, 34, 193–239.CrossRefGoogle Scholar
Silva, R., Klinger, J., & Weikart, S. (2010). Measuring lethal counterinsurgency violence in Amritsar District, India using a referral-based sampling technique. In Joint Statistical Meetings Proceedings, Survey Research Methods Section (pp. 552–80). Alexandria, VA: American Statistical Association.Google Scholar
Sirken, M. G. (2004). Network sample surveys of rare and elusive populations: a historical review. In Proceedings of Statistics Canada Symposium: Innovative Methods for Surveying Difficult-to Reach Populations. Available from .
Sirken, M. G. (2005). Network sampling developments in survey research during the past 40+ years. Survey Research, 36 (1), 1–5. Retrieved from .Google Scholar
Sudman, S., & Freeman, H. E. (1988). The use of network sampling for locating the seriously ill. Medical Care, 26, 992–99.CrossRefGoogle ScholarPubMed
Sudman, S., & Kalton, G. (1986). New developments in the sampling of special populations. Annual Review of Sociology, 12, 401–29.CrossRefGoogle Scholar
Sudman, S., Sirken, M. G., & Cowan, C. D. (1988). Sampling rare and elusive populations. Science, 240, 991–96.CrossRefGoogle ScholarPubMed
Tourangeau, R., Kreuter, F., & Eckman, S. (2012). Motivated underreporting in screening interviews. Public Opinion Quarterly, 76, 453–69.CrossRefGoogle Scholar
Turner, A. G., Magnani, R. J., & Shuaib, M. (1996). A not quite as quick but much cleaner alternative to the Expanded Programme on Immunization (EPI) cluster survey design. International Journal of Epidemiology, 25, 198–203.CrossRefGoogle ScholarPubMed
US Centers for Disease Control and Prevention. (2012). National HIV Behavioral Surveillance (NHBS). Retrieved from .
US National Center for Health Statistics. (2009a). National Health Interview Survey (NHIS). Retrieved from .
US National Center for Health Statistics (2009b). The National Immunization Survey (NIS). Retrieved from .
U.S. National Center for Health Statistics (2009c). State and Local Area Integrated Telephone Survey (SLAITS). Retrieved from .
U.S. National Center for Health Statistics (2012). About the Ambulatory Health Care Surveys. Retrieved from .
Waksberg, J. (1973). The effect of stratification with differential sampling rates on attributes of subsets of the population. In Joint Statistical Meetings Proceedings, Social Statistics Section (pp. 429–34). Alexandria, VA: American Statistical Association.Google Scholar
Waksberg, J., Judkins, D., & Massey, J. T. (1997). Geographic-based oversampling in demographic surveys of the United States. Survey Methodology, 23, 61–71.Google Scholar
Word, D. L., & Perkins, R. C. (1996). Building a Spanish Surname List for the 1990s – A New Approach to an Old Problem. Population Division Technical Working Paper No. 13. Washington, DC: US Census Bureau.Google Scholar
Xia, Q., Tholandi, M., Osmond, D. H., Pollack, L. M., Zhou, W., Ruiz, J. D., & Catania, J. A. (2006). The effect of venue sampling on estimates of HIV prevalence and sexual risk behaviors in men who have sex with men. Sexually Transmitted Diseases, 33, 545–50.CrossRefGoogle ScholarPubMed

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×