Hostname: page-component-745bb68f8f-f46jp Total loading time: 0 Render date: 2025-01-08T10:07:25.621Z Has data issue: false hasContentIssue false
  • English
  • Français

Harmonic Regression and Scale Stability

Published online by Cambridge University Press:  01 January 2025

Yi-Hsuan Lee  and
Shelby J. Haberman
Yi-Hsuan Lee*
Affiliation:
Educational Testing Service
Shelby J. Haberman
Affiliation:
Educational Testing Service
*
Requests for reprints should be sent to Yi-Hsuan Lee, Educational Testing Service, Princeton, NJ, USA. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Monitoring a very frequently administered educational test with a relatively short history of stable operation imposes a number of challenges. Test scores usually vary by season, and the frequency of administration of such educational tests is also seasonal. Although it is important to react to unreasonable changes in the distributions of test scores in a timely fashion, it is not a simple matter to ascertain what sort of distribution is really unusual. Many commonly used approaches for seasonal adjustment are designed for time series with evenly spaced observations that span many years and, therefore, are inappropriate for data from such educational tests. Harmonic regression, a seasonal-adjustment method, can be useful in monitoring scale stability when the number of years available is limited and when the observations are unevenly spaced. Additional forms of adjustments can be included to account for variability in test scores due to different sources of population variations. To illustrate, real data are considered from an international language assessment.

Keywords

seasonal adjustmentquality controlseasonality
Type
Original Paper
Information
Psychometrika , Volume 78 , Issue 4 , October 2013 , pp. 815 - 829
Copyright
Copyright © 2013 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723.CrossRefGoogle Scholar
Anscombe, F.J., & Tukey, J.W. (1963). The examination and analysis of residuals. Technometrics, 5, 141–160.CrossRefGoogle Scholar
Astronomical Applications Department (2011). http://aa.usno.navy.mil/data/docs/RS_OneYear.php. United States Naval Observatory.Google Scholar
Bloomfield, P. (2000). Fourier analysis of time series: an introduction (2nd ed.). New York: Wiley.CrossRefGoogle Scholar
Brockwell, P.J., & Davis, R.A. (2002). Introduction to time series and forecasting (2nd ed.). New York: Springer.CrossRefGoogle Scholar
Casella, G., & Berger, R.L. (2002). Statistical inference (2nd ed.). Pacific Grove: Duxbury Press.Google Scholar
Courant, R. (1937). Differential and integral calculus (Vol. 1, 2nd ed.). New York: Interscience. Translated by E.J. McShane.Google Scholar
Draper, N.R., & Smith, H. (1998). Applied regression analysis (3rd ed.). New York: Wiley.CrossRefGoogle Scholar
Findley, D.F., Monsell, B.C., Bell, W.R., Otto, M.C., & Chen, B.C. (1998). New capabilities and methods of the X-12-ARIMA seasonal-adjustment program. Journal of Business & Economic Statistics, 16, 127–152.CrossRefGoogle Scholar
Haberman, S.J. (2008). Outliers in assessments (ETS Research Report No. RR-08-41). ETS, Princeton, NJ.Google Scholar
Haberman, S.J., Guo, H., Liu, J., & Dorans, N.J. (2008). Consistency of SAT ®I reasoning test score conversions (ETS Research Report No. RR-08-67). ETS, Princeton, NJ.Google Scholar
Hutchinson, R.J., & Wong, E. (2009). The rewards and challenges of seasonally adjusting a short series: seasonal adjustment research for the U.S. Census Bureau’s Quarterly Services Survey. http://www.census.gov/ts/papers/2009jsmrjh.pdf. Presented at the Joint statistical meetings, 2009, Washington, DC.Google Scholar
Lee, Y.-H., & von Davier, A.A. (2013). Monitoring scale scores over time via quality control charts, model-based approaches, and time series techniques. Psychometrika, .CrossRefGoogle ScholarPubMed
Lindsey, D.T., Holzman, P.S., Haberman, S., & Yasillo, N.J. (1978). Smooth-pursuit eye movements: a comparison of two measurement techniques for studying schizophrenia. Journal of Abnormal Psychology, 87, 491–496.CrossRefGoogle ScholarPubMed
Miller, R.G.J. (1991). Simultaneous statistical inference. New York: Springer.Google Scholar
Montgomery, D.C. (2009). Introduction to statistical quality control (6th ed.). New York: Wiley.Google Scholar
Neuenschwander, A.L., & Crews, K.A. (2008). Disturbance, management, and landscape dynamics: harmonic regression of vegetation indices in the lower Okavango Delta, Botswana. Photogrammetric Engineering & Remote Sensing, 76, 753–764.CrossRefGoogle Scholar
SAS Institute Inc. (2008). SAS/STAT® 9.2 user’s guide. Cary: SAS Institute Inc.Google Scholar
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464.CrossRefGoogle Scholar
Shiskin, J., Young, A.H., & Musgrave, J.C. (1967). The X-11 variant of the Census Method II seasonal adjustment program (Technical Paper No. 15). Bureau of the Census, Washington, DC.Google Scholar
U.S. Department of Labor (2011). Unemployment weekly claims data. http://www.ows.doleta.gov/unemploy/wkclaims/report.asp.Google Scholar
Young, P.C., Pedregal, D.J., & Tych, W. (1999). Dynamic harmonic regression. Journal of Forecasting, 18, 369–394.3.0.CO;2-K>CrossRefGoogle Scholar