Article contents
The Stability and Reliability of Political Efficacy: Using Path Analysis to Test Alternative Models*
Published online by Cambridge University Press: 01 August 2014
Abstract
The reliability and stability of survey items designed to measure political attitudes are important to the study of political behavior. Several past studies have examined the reliability and stability of items measuring one construct, that of political efficacy. The results of this prior research have been contradictory, in part because of the limitations of the methodologies used. In this article, the authors employ path analysis to examine more closely the stability and reliability of the four SRC items commonly used to measure political efficacy. The American Panel Study (1956–1960), in which efficacy is measured at two points in time, is used as the data base. The authors conclude that two of the four items (NO CARE and NO SAY) seem to measure best what is meant by political efficacy. These two items are more stable and reliable than previously thought, while the other items are relatively unstable and unreliable, and they display systematic differences from each other and from the NO CARE and NO SAY items.
- Type
- Articles
- Information
- Copyright
- Copyright © American Political Science Association 1977
Footnotes
The data for this paper were provided by the Inter-University Consortium for Political Research. Neither the Consortium nor the original collectors of the data are responsible for the analysis and conclusions presented here.
References
1 Most observers agree about the general nature of political efficacy. It is a feeling that individual political action does have an impact upon the political process. See Campbell, Angus, Gurin, Gerald, and Miller, Warren, The Voter Decides (Evanston, Illinois: Row, Peterson, 1954), p. 187 Google Scholar. The authors of the American Voter used the concept to explain voting rates, but efficacy has been used since to try to explain a wide variety of kinds of political participation. Operationalizations have been diverse, but the four items developed by the Survey Research Center that are examined here are certainly the most widely used. See also Campbell, Angus, Converse, Philip E., Miller, Warren and Stokes, Donald, The American Voter (New York: John Wiley and Sons, 1964)Google Scholar; Matthews, Donald and Prothro, James, Negroes and the New Southern Politics (New York: Harcourt Brace and World, 1966)Google Scholar; Almond, Gabriel and Verba, Sidney, The Civic Culture (Princeton: Princeton University Press, 1963)CrossRefGoogle Scholar; Paige, Jeffery M., “Political Orientation and Riot Participation,” American Sociological Review 36 (October, 1971), 810–820 CrossRefGoogle ScholarPubMed. For an extensive bibliography on political efficacy see Easton, David and Dennis, Jack, “The Child's Acquisition of Regime Norms: Political Efficacy,” American Political Science Review, 61 (March, 1967), 25–38 CrossRefGoogle Scholar.
2 Campbell, et al., The American Voter, p. 516 Google Scholar. See also Lane, Robert E., Political Life (New York: The Free Press, 1959), pp. 149–151 Google Scholar. The child socialization literature also contributes to this belief by suggesting that attitudes about efficacy formed in childhood carry over into adult life. See Easton and Dennis, p. 33–38; and also Langton, Kenneth and Jennings, M. Kent, “Political Socialization and the High School Civics Curriculum in the United States,” American Political Science Review, 62 (September, 1968), 852–867 CrossRefGoogle Scholar. Some of the assumptions of this kind of socialization research have been challenged on grounds relevant to the problems under consideration here. Donald D. Searing, Joel J. Schwartz and Alden E. Lind show, for example, that attitudes of political efficacy as well as other “political orientations” have hardly any relationship at all to issue beliefs in adults. “The Structuring Principle: Political Socialization and Belief Systems”, American Political Science Review, 67 (June, 1973), 415–432 CrossRefGoogle Scholar.
3 Converse found that three of the four items in the SRC efficacy scale significantly increased during the 1950s then significantly decreased in the 1960s, while the other showed a continuous increase between 1952 and 1968. More detailed analysis indicated that these aggregate statistics substantially understate the instability of efficacy feelings because individual changes cancelled each other out to a significant degree. See Converse, Philip E., “Change in the American Electorate,” in The Human Meaning of Social Change, ed. Campbell, Angus and Converse, Philip E. (New York: Russell Sage Foundation, 1972), pp. 327–332 Google Scholar.
Using data from an SRC panel study, Asher found that for the 1956–1960 period, when only very marginal changes occurred in the aggregate efficacy scores, correlations between an individual's response to each question at the beginning and end of the study showed much greater inconsistencies as their temporal correlations averaged only .36. Even for the short period between the pre- and postelection survey of 1968 the temporal stability of efficacy feelings is hardly greater than for the four-year span. See Asher, Herbert B., “The Reliability of the Political Efficacy Items,” Political Methodology, 1 (Spring, 1974), 45–72 Google Scholar. Welch and Clark, investigating the relationships among the efficacy items and their changes over time, found a multidimensionality as well as substantial instability. Stability was only slightly greater for scores on separate dimensions as for individual items. See Welch, Susan and Clark, Cal, “Political Efficacy as an Underlying Political Orientation: The Problem of Attitudinal Instability” (paper presented at the Midwest Political Science Association Meeting, April, 1974)Google Scholar.
The high temporal variability of the efficacy items is underlined by a comparison with other political attitudinal sets and opinions about specific political issues. One would expect that deep-seated orientations should be more stable than positions on political issues. However, responses to the four efficacy items were on the average less consistent between 1956 and 1960 (Pearson's r averaging from .30 to .42) than attitudes toward a range of current issues (Pearson's r between .20 and .53) and much more variable than such a basic orientation as party identification (r = .90).
4 Welch and Clark.
5 Converse, “Change in the American Electorate;” Balch calls the dimensions “Internal” and “External” efficacy, as cited in Balch, George, “Multiple Indicators in Survey Research: The Concept ‘Sense of Political Efficacy,’” Political Methodology, 1 (Spring, 1974), 1–43 Google Scholar.
6 Welch and Clark, “Political Efficacy,” factor analyzed the four conventional efficacy items along with four questions about the importance of voting and found that the three items which Converse retained defined one of the two factors that emerged. These three variables also tended to have similar correlations with several indicators of political participation that were somewhat stronger than the ones with “voting only political influence,” but all four had approximately the same relationships to the political trust and cynicism questions.
7 Welch and Clark; Asher, “Reliability of the Political Efficacy Items.”
8 Balch, “Multiple Indicators.”
9 Welch and Clark.
10 Converse, “Change in the American Electorate.”
11 See Blalock, H. M. Jr., “A Causal Approach to Nonrandom Measurement Error,” American Political Science Review, 64 (December, 1970), 1099–1111 CrossRefGoogle Scholar. The major discussions of the use of causal modeling for this purpose include Costner, Herbert L., “Theory, Deduction, and the Rules of Correspondence,” American Journal of Sociology, 75 (September, 1969), 245–263 CrossRefGoogle Scholar; Heise, David R., “Separating Reliability and Stability in Test-Retest Correlations,” American Sociological Review, 34 (February, 1969), 93–101 CrossRefGoogle Scholar; Wiley, David E. and Wiley, James A., “The Estimation of Measurement Error in Panel Data,” American Sociological Review, 35 (February, 1970), 112–117 CrossRefGoogle Scholar; and Blalock, H. M. Jr., “Multiple Indicators and the Causal Approach to Measurement Error,” American Journal of Sociology, 75 (September, 1969), 264–272 CrossRefGoogle Scholar.
12 Costner. Here we will use standardized regression coefficients (beta weights); unstandardized coefficients can also be used. At this point in the paper we make no distinction between sources of error variance in the items due to specification error and sources in error variance due to measurement error.
13 “True” in this sense meaning an efficacy score that could be perfectly measured. The following discussion is drawn largely from Heise and from Costner.
14 The assumption that the error terms are uncorrelated is equivalent to the assumption that no major omitted variables cause common variation in the items except through the efficacy construct, or other explicit unmeasured variables. This assertion is the ceteris paribus assumption for causal models. It should be emphasized that this method does not deal with the semantic aspects of indicator validity. “True” efficacy may or may not be what most observers mean by efficacy. High reliabilities of the indicators of efficacy with the efficacy construct mean only that the indicators are validated for that particular model; different models might yield different indicator reliabilities.
15 Wright, Sewell, “The Method of Path Coefficients,” Annals of Mathematical Statistics, 5 (September, 1934), 161–215 CrossRefGoogle Scholar; Duncan, O. D., “Path Analysis: Sociological Examples,” American Journal of Sociology, 72 (July, 1966), 1–16 CrossRefGoogle Scholar.
16 Overidentification in general denotes an excess of known quantities in relation to the number of unknowns to be estimated. See Duncan.
17 Jöreskog, K. G., “A General Method for the Analysis of Covariance Structures,” Biometrika 57, #2, (1970), 239–251 CrossRefGoogle Scholar. The computer program designed to analyze data according to this covariance structure procedure is documented in Jöreskog, K. G., Gruvaeus, Gunnar T. and Van Thillo, Marielle, ACOVS: A General Computer Program for the Analysis of Covariance Structures (Princeton: Educational Testing Service, 1970)Google Scholar. A technical appendix outlining the application of the Jöreskog procedure to the four-item two-wave model considered here may be obtained from the authors by request.
18 See Hauser, R. M. and Goldberger, A. S., “The Treatment of Unobservable Variables in Path Analysis,” in Sociological Methodology, ed. Costner, H. L. (San Francisco: Jossey-Bass, 1971), pp. 81–118 Google Scholar; for a more technical discussion, see Goldberger, A. S., “Efficient Estimation in Overidentified Models: An Interpretive Analysis” in Structural Equation Models in the Social Sciences, ed. Goldberger, A. S. and Duncan, O. D. (New York: Seminar Press, 1973), pp. 131–152 Google Scholar.
19 Note that this a priori knowledge (for instance that certain factor loadings are zero) is what distinguishes confirmatory factor analysis from the more generally known exploratory factor analysis. As its name suggests, confirmatory factor analysis is designed to test hypotheses rather than suggest them.
20 More precisely, the method minimizes a scalar function of the differences between the variance-covariance matrix of the sample and that matrix produced by the causal structure of the model and the estimates of the parameters. For a more technical discussion, see Jöreskog, “A General Method for the Analysis of Covariance Structures; for extensions of the technique, see Joreskog, K. G., “A General Model for Estimating a Linear Structural Equation System” in Goldberger, and Duncan, , Structural Equation Models in the Social Sciences, pp. 85–107 Google Scholar.
21 The assumptions are essentially those of correlation analysis, including the assumption of multivariate normality. See references cited in the previous foot-note.
22 See Jöreskog et al., ACOVS.
23 Welch and Clark, “Political Efficacy as an Underlying Political Orientation.”
24 House, James S. and Mason, William, “Trends in Some Survey Measures of Political Alienation in America,” paper presented at the meeting of the American Sociological Association, New York, August, 1973 Google Scholar.
25 Converse, , “Change in the American Electorate,” p. 328 Google Scholar.
26 Ibid., pp. 337–338.
27 Balch, “Multiple Indications in Survey Research.”
28 In fact, if the analogy of the interpretation of the estimate e as a path coefficient can be maintained, the fact that e is 1.03 would simply mean that a score in 1956 which is one standard deviation above the mean becomes a score in 1960 which is 1.03 standard deviations above the mean. This finding is still rather unlikely. We note also that one could formulate the problem in unstandardized terms (i.e., use covariances instead of correlations, and examine unstandardized coefficients instead of path coefficients) however, a parallel unstandardized analysis using Jöreskog's procedure revealed no substantively different conclusions from the ones presented here. We present the standardized results for the purposes of clarity and simplicity.
29 In general, we expect to find relatively large χ2 values with large samples such as the present one. The χ2 statistic will be most useful in such cases to compare different models, rather than to evaluate the absolute fit of any single model. We will use the χ2 values produced by the different models generated in this section for this comparative purpose.
30 The use of chi square in this case is heuristic, since the model of Figure 3 is not an exact statistical alternative to the model of Figure 2.
31 Others might prefer to call the COMPLEX or VOTING items the “real” efficacy while giving a different name to NO CARE and NO SAY. Since all four items were initially designed to measure efficacy and since we have discovered that only two of them measure substantially the same thing, we think it is most reasonable to call what the two items measure in common “efficacy.”
32 The hypothesized single source of autocorrelation for VOTING and COMPLEX is simply a third unmeasured variable which is constrained to have a zero correlation with the two efficacy constructs. The paths linking this construct with the measured items are estimated in the same manner as the paths linking efficacy to the items.
33 The chi-square statistic is, of course, directly proportional to the sample size. Note that the two values have different degrees of freedom.
34 See Verba, Sidney and Nie, Norman, Participation in America (New York: Harper and Row, 1972)Google Scholar. The evidence on the curvilinearity based on a simple analysis of variance is mixed. The eta2 between the voting item and the responsiveness dimension is 6 per cent in 1956 and 4 per cent in 1960, surely a small relationship. Of that, the linear relationship accounts for an R 2 of 5 per cent and 3 per cent respectively. The simple relationship between voting and degrees of political participation is practically nil, although those who participate in a wide variety of political acts are slightly more likely to respond “no” to the item than either voting specialists or nonparticipants.
35 In fact, the full generality of the Jöreskog procedures has not even been hinted at in this paper. For instance, in the context of a more elaborate analysis, additional variables representing outside influences upon efficacy or individual items could be introduced. The technique can also be used to address more sophisticated measurement issues such as whether tests are parallel, congeneric, or tau-equivalent ( Jöreskog, K. G., “Statistical Analysis of Sets of Congeneric Tests,” Psychometrika, 36 (June 1971), 109–133 CrossRefGoogle Scholar. For a fuller discussion of the flexibility of the technique, see especially Jöreskog, “A General Method,” and Jöreskog, “A General Model.”
- 21
- Cited by
Comments
No Comments have been published for this article.