Skip to main content Accessibility help
×
Hostname: page-component-78c5997874-j824f Total loading time: 0 Render date: 2024-11-17T18:09:36.290Z Has data issue: false hasContentIssue false

4 - Evaluating Surveys and Questionnaires

Published online by Cambridge University Press:  05 June 2012

Norbert Schwarz
Affiliation:
University of Michigan
Robert J. Sternberg
Affiliation:
Yale University, Connecticut
Henry L. Roediger III
Affiliation:
Washington University, St Louis
Diane F. Halpern
Affiliation:
Claremont McKenna College, California
Get access

Summary

Much of what we know about human behavior is based on self-reports. When we want to learn about individuals' health behaviors, consumer habits, family problems, media consumption, values, or political beliefs, we ask appropriate questions. The answers provided to these questions serve as input into scientific analyses and provide the basis of statistical indicators used to describe the state of a society. Obviously, these data are only as meaningful as the questions we ask and the answers we receive. Moreover, whom we ask is of crucial importance to our ability to draw conclusions that extend beyond the particular people who answered our questions. Accordingly, the processes underlying question answering and the appropriate selection of respondents are of great importance to many areas of social research.

This chapter provides an introduction to these issues for consumers of published survey results. The first part introduces the concept of a survey, describes elements of survey design, and addresses who to ask. The second part addresses how to ask. It reviews key components of the question-answering process that apply to all self-reports, whether collected in a survey or in the psychological laboratory. Other elements of survey research, like interviewer behavior and training or the questionnaire pretesting, will only be touched upon in passing. Most reports of survey results provide little information about these components, rendering it impossible for readers to assess their quality.

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2006

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Belli, R. (1998). The structure of autobiographical memory and the event history calendar: Potential improvements in the quality of retrospective reports in surveys. Memory, 6, 383–406.CrossRefGoogle ScholarPubMed
Belson, W. A. (1981). The design and understanding of survey questions. Aldershot, England: Gower.Google Scholar
Bodenhausen, G. V., & Wyer, R. S. (1987). Social cognition and social reality: Information acquisition and use in the laboratory and the real world. In Hippler, H. J., Schwarz, N., & Sudman, S. (Eds.), Social information processing and survey methodology (pp. 6–41). New York: Springer-Verlag.CrossRefGoogle Scholar
Bradburn, N. M., & Sudman, S. (1988). Polls and surveys: Understanding what they tell us. San Francisco: Jossey-Bass.Google Scholar
Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions (2nd ed.). San Francisco: Jossey-Bass.Google Scholar
Brown, N. R. (2002). Encoding, representing, and estimating event frequencies: Multiple strategy perspective. In Sedlmeier, P. & Betsch, T. (Eds.), Etc. Frequency processing and cognition (pp. 37–54). New York: Oxford University Press.CrossRefGoogle Scholar
Campbell, A. (1981). The sense of well-being in America. New York: McGraw-Hill.Google Scholar
Cannell, C. F., Fisher, G., & Bakker, T. (1965). Reporting on hospitalization in the Health Interview Survey. Vital and Health Statistics (PHS Publication No. 1000, Series 2, No. 6). Washington, DC: U.S. Government Printing Office.Google Scholar
Clark, H. H., & Schober, M. F. (1992). Asking questions and influencing answers. In Tanur, J. M. (Ed.), Questions about questions (pp. 15–48). New York: Russell Sage Foundation.Google Scholar
DeMaio, T. J. (1984). Social desirability and survey measurement: A review. In Turner, C. F. & Martin, E. (Eds.), Surveying subjective phenomena (Vol. 2, pp. 257–281). New York: Russell Sage Foundation.Google Scholar
Groves, R. M., Cialdini, R. B., & Couper, M. P. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56, 475–495.CrossRefGoogle Scholar
Herbers, J. (1982, February 14). Polls find conflict in views on aid and public welfare. New York Times, p. 19.Google Scholar
Krosnick, J. A., & Alwin, D. F. (1987). An evaluation of a cognitive theory of response order effects in survey measurement. Public Opinion Quarterly, 51, 201–219.CrossRefGoogle Scholar
Markus, G. B. (1986). Stability and change in political attitudes: Observed, recalled, and explained. Political Behavior, 8, 21–44.CrossRefGoogle Scholar
Pepper, S. C. (1981). Problems in the quantification of frequency expressions. In Fiske, D. W. (Vol. Ed.), New Directions for Methodology of Social and Behavioral Science, Vol. 9. Problems with language imprecision (pp. 0–0). San Francisco: Jossey- Bass.Google Scholar
Ross, M. (1989). The relation of implicit theories to the construction of personal histories. Psychological Review, 96, 341–357.CrossRefGoogle Scholar
Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys. New York: Academic Press.Google Scholar
Schwarz, N. (1996). Cognition and communication: Judgmental biases, research methods, and the logic of conversation. Mahwah, NJ: Erlbaum.Google Scholar
Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93–105.CrossRefGoogle Scholar
Schwarz, N., Groves, R., & Schuman, H. (1998). Survey methods. In Gilbert, D., Fiske, S., & Lindzey, G. (Eds.), Handbook of social psychology (4th ed., Vol. 1, pp. 143–179). New York: McGraw-Hill.Google Scholar
Schwarz, N., & Hippler, H. J. (1991). Response alternatives: The impact of their choice and ordering. In Biemer, P., Groves, R., Mathiowetz, N., & Sudman, S. (Eds.), Measurement error in surveys (pp. 41–56). Chichester, England: Wiley.Google Scholar
Schwarz, N., Knäuper, B., Hippler, H. J., Noelle-Neumann, E., & Clark, F. (1991). Rating scales: Numeric values may change the meaning of scale labels. Public Opinion Quarterly, 55, 570–582.CrossRefGoogle Scholar
Schwarz, N., & Oyserman, D. (2001). Asking questions about behavior: Cognition, communication and questionnaire construction. American Journal of Evaluation, 22, 127–160.CrossRefGoogle Scholar
Schwarz, N., & Scheuring, B. (1992). Selbstberichtete Verhaltens- und Symptomhäufigkeiten: Was Befragte aus Anwortvorgaben des Fragebogens lernen [Frequency reports of psychosomatic symptoms: What respondents learn from response alternatives]. Zeitschrift für Klinische Psychologie, 22, 197–208.Google Scholar
Schwarz, N., Strack, F., & Mai, H. P. (1991). Assimilation and contrast effects in part–whole question sequences: A conversational logic analysis. Public Opinion Quarterly, 55, 3–23.CrossRefGoogle Scholar
Smith, T. W. (1987). That which we call welfare would smell sweeter by any other name. Public Opinion Quarterly, 51, 75–83.CrossRefGoogle Scholar
Sniderman, P. M., & Tetlock, P. E. (1986). Symbolic racism. Journal of Social Issues, 42, 129–150.CrossRefGoogle Scholar
Sirken, M., Hermann, D., Schechter, S., Schwarz, N., Tanur, J., & Tourangeau, R. (Eds.). (1999). Cognition and survey research. New York: Wiley.Google Scholar
Steeh, C. (1981). Trends in nonresponse rates. Public Opinion Quarterly, 45, 40–57.CrossRefGoogle Scholar
Strack, F., Schwarz, N., & Wänke, M. (1991). Semantic and pragmatic aspects of context effects in social and psychological research. Social Cognition, 9, 111–125.CrossRefGoogle Scholar
Strube, G. (1987). Answering survey questions: The role of memory. In Hippler, H. J., Schwarz, N., & Sudman, S. (Eds.), Social information processing and survey methodology (pp. 86–101). New York: Springer-Verlag.CrossRefGoogle Scholar
Sudman, S., Bradburn, N., & Schwarz, N. (1996). Thinking about answers: The application of cognitive processes to survey methodology. San Francisco: Jossey-Bass.Google Scholar
Thornberry, O. T., & Massey, J. T. (1988). Trends in United States telephone coverage across time and subgroups. In Groves, R. M., Biemer, P. P., Lyberg, L. E., Massey, J. T., Nichols, W. L., & Waksberg, J. (Eds.), Telephone survey methodology (pp. 25–50). New York: Wiley.Google Scholar
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. New York: Cambridge University Press.CrossRefGoogle Scholar
Traugott, M. W. (1987). The importance of persistence in respondent selection for preelection surveys. Public Opinion Quarterly, 51, 48–57.CrossRefGoogle Scholar
Visser, P. S., Krosnick, J. A., & Lavrakas, P. J. (2000). Survey research. In Reis, H. T. & Judd, C. M. (Eds.), Handbook of research methods in social and personality psychology (pp. 223–252). New York: Cambridge University Press.Google Scholar
Weisberg, H. F., Krosnick, J. A., & Bowen, B. D. (1989). An introduction to survey research and data analysis (2nd ed.). Glenview, IL: Scott, Foresman.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×